Milestones: Are You Ready for July 1 PowerPoint Presentation, PPT - DocSlides

Milestones: Are You Ready for July 1st ?

Kirk Lalwani, MD, FRCA, MCR

Associate Professor,

Chair,

Resident Evaluation and Clinical Competence Committee,

Anesthesiology and Perioperative Medicine.

On behalf of the AMIGOS

(Anesthesiology Milestones Implementation Group Operatives)

Disclosure

No conflict of interest

Some of my slides are borrowed from the faculty educational resources on the ACGME website

Implementing Milestones

Objectives:

Outline the Next Accreditation System (NAS)

Define a Milestone

Review our Milestones strategy

Discuss assessment of Anesthesiology Milestones

Abbreviations

NAS- Next Accreditation System

CLER- Clinical Learning Environment

ACGME- Accreditation Council for Graduate Medical Education

RRC- Residency Review Committee

The Next Accreditation System

Why ‘Next Accreditation System’ (NAS)?

“Self-regulation is a fundamental professional responsibility, and the system for educating physicians answers to the public for the graduates it produces.”

“The Next GME Accreditation System – Rationale and Benefits”

Nasca

 T.J., 

Philibert

I., Brigham T., Flynn T.C.

N 

Engl

J Med 2012; 366:1051-1056

Goals of NAS

Strengthen resident development in

Professionalism

Communication Skills

Systems Based Practice

Practice Based Learning

Enhance public accountability

More explicit definition of a good physician (Milestones)

Patient safety (Clinical Learning Environment Review Program)

Reduce burden required for accreditation

‘Next Accreditation System’ …in a Nutshell

Accreditation on the basis of educational

outcomes

NAS : Advance from an episodic `

biopsy ’ model to annual data collection or ` continuous oversight ’.

RRCs will measure compliance through the evaluation of annual program data elements.

The Next Accreditation System

Continuous

Observations

Identify Opportunities for Improvement

Program MakesImprovement(s)

AssessProgramImprovement(s)

PromoteInnovation

The Next Accreditation System

We will now have annual data collection

Trends in annual data

Milestones

Resident and Faculty Surveys

Resident and Faculty Scholarly activities

Case Log Data

ITE / ABA Examination pass rates

Clinical Learning Environment Review Visits

Oversight of Transitions of CareDuty Hours PolicyFatigue Management and MitigationEducation and Monitoring of Professionalism Involvement in Institutional Quality Improvement and Safety initiativesSupervision policies

Additional information available at: http://www.acgme-nas.org/CLER

Milestones

What are Milestones?

Specific behaviors, attributes, or outcomes in the six general competency domains to be demonstrated by residents during residency.

(ACGME and Specialty Boards)

Skill and knowledge-based development that commonly occurs by a specific time.

Why ????

ACGME requires itProgram Accreditation

Who, When, and How????

All residents in all ACGME-accredited programs

Subspecialty fellows from July 2015

New assessment methodology as well as existing tools

6-monthly reporting of Milestones to ACGME in January and July

Reporting starts July 2014

Expected Benefits of Milestone Assessments

For Residents

Clarify

expectations

F

eedback

should

identify specific areas to work

on

Earlier identification of under-performers

A

spirational

goals for residents who exceed expectations

Expected Benefits of Milestone Assessment

For the ProgramCurriculum developmentAccreditation requirementsEarlier identification of under-performersFor the PublicBetter definition of what a physician can do at the completion of trainingProgram is accountable to a common standardPossible use for board certification

17

Minimalist

CCC assigns milestone levels using existing tools

ComprehensiveProgram changes assessment system, invests time in faculty development and education to improve rater expertise and CCC validity

PragmaticProgram implements new milestone aligned tools, which help residents understand expectations

Milestones Development Strategy

Process: June 2013

General Strategy“Good now is better than perfect later….”

Priorities

Ease of use

Should not be burdensome for residents, faculty, RECCC or education office staff

No PAPER !

App for ‘on-the-go’ assessments - procedures, H&P

Implementation feasibility

Ready by July 2014

Integrate with existing system (

Evalue

)

Ensuring adequate data to assess Milestones comprehensively

A Milestone for every competency

17/25 Milestones chosen

Work in progress, as refinements will be made and new tools added over time

Which?

Which Milestones did the AMIGOS select?

Medical Knowledge (MK): 1

MK1

-

Knowledge of biomedical, clinical, epidemiological, and social sciences as outlined in the American Board of Anesthesiology Content Outline

ITE

AKT

ABA Basic Examination

ACLS certification

Direct Clinical Observation

Patient Care (PC): 8 of 10

PC1

-Preanesthetic Evaluation, Assessment, and Preparation

PC2

- Anesthetic choice and conduct

PC3-

Periprocedural

pain management

PC4- Management of

perianesthetic

complications

PC5

- Crisis management

PC6

- Triage and management of critically ill patient in a non-operative setting

PC7

- Acute, chronic, and cancer related pain consultation and management

PC8

- Technical skills: Airway management

PC9

- Technical skills: Monitoring and Equipment

PC10

-Technical skills: Regional anesthesia

Example of PC-8 Assessment

PC-8 (Technical skills: Airway management)

Direct Clinical Observation (DCO)

Daily evaluation form

Airway management checklist

Objective Structured Clinical Examination (OSCE) / Objective Structured Assessment Test (OSAT)

Simulation / Standardized Patient (SP)

Professionalism (Prof): 3 of 5

Prof1

- Responsibility to patients, families, and society

Prof2

- Honesty, integrity, and ethical behavior

Prof 3- Commitment to institution, department and colleagues

Prof4

- Receiving and giving feedback

Prof 5- Responsibility to maintain personal, emotional, physical and mental health

Example of Prof-1 Assessment

Prof 1 (Responsibility to Patient, Family, and Society)

360 degree evaluations

Patients

Healthcare team members

Peer evaluations

Administrative staff

Direct Clinical Observation (DCO)

Daily evaluation form

Written comments

Interpersonal and Communication Skills (ICS): 2 of 3

ICS1

- Communication with patients and families

ICS2

- Communication with other professionals

ICS3- Team and Leadership skills

Practice Based Learning (PBL): 2 of 4

PBL1 Incorporates quality improvement and patient safety initiatives into personal practice

PBL2

- Analysis of practice to identify areas in need of improvement

PBL3

- Self-directed learning

PBL4- Education of patient, family, students, residents and other health professionals

Systems Based Practice (SBP) 1 of 2

SBP1

- Systems-based approaches to patient care

SBP2- Coordination of patient care within the healthcare system

How will we Assess Milestone Levels?

Milestones are a

summary

of

how

a resident is

progressing based on information from other sources

Milestones are

not assessment tools

to be used for gathering detailed information

They do not replace end-of-rotation forms, simulation, 360’s, Daily evaluations etc.

More pieces of data allow for more precision

There will be a

minimum number

of evaluations required for procedural assessments

Clinical Competency Committee (RECCC)

Clinical Competency Committee

End-of-Rotation Evaluations

Peer Evaluations

SelfEvaluations

Case Logs

Student Evaluations

Patient/Family Evaluations

ProfessionalismIndicators likepharmacy, billing, evals etc

Nursing and Ancillary Personnel Evaluations

Assessment of Milestones

Clinical Workplace

Evaluations

Mock Orals

OSCE

ITE

Sim

Lab

Unsolicited

Comments

Anesthesiology Milestones

Level 4

is the Graduation Target

Not every resident will achieve Level 4 in every Milestone

Residents required to substantially meet

most

Milestones at Level 4

Residents

are not expected to achieve

Level 5 during residency

Residents may achieve a level of competency in specific

Milestones

sooner than

expected

Milestone Template

PC1. Preanesthetic Evaluation, Preparation Level 1Level 2Level 3Level 4Level 5Performs comprehensive H&PIdentifies medical issues that may affect anesthesia careIdentifies elements and process of informed consentIdentifies medical issues relevant to anesthesia careOptimizes preparation of non-complex patientsObtains informed consent but recognizes when assistance is needed Identifies medical issues relevant to subspecialty anesthesia careOptimizes preparation of complex or subspecialty patientsObtains informed consent of patients with complex problems or requiring subspecialty anesthesia care  Performs assessment of complex or critically ill patients with conditional independence Obtains informed consent in complicated clinical situations with conditional independence  Independently performs assessment for all patients Independently serves as a consultant to other members of the healthcare teamConsistently ensures that informed consent is obtained by using all available resources

General Competency

Developmental Progression or Set of Milestones

Subcompetency

Milestone

Instruments to Measure Milestones

Daily Evaluation Form for OR (

Evalue

)

Daily Evaluation Form for ICU (

Evalue

)

Daily Evaluation Form for Pain (

Evalue

)

Technical Skills forms (

Evalue

) (‘App’?)

Arterial line

CVP Line

Nerve Block / Epidural / Caudal

Airway Management

Peer evaluations

Assessment of Patient Interaction

Regional assessments

Simulation assessments

‘QUASAR’ Project:

Quality in Anesthesiology,

Systems Assessment and Research

Resident ‘QUASAR’ Project: Quality in Anesthesiology, Systems Assessment and Research

Quality Improvement

Systems Based Practice

Practice-Based Learning

CA1

- Background research, identify area of study

CA2

- Identify quality measure to be studied and change to be implemented, study effects, refine

CA3

- Write summary, submit abstract to national conference, or write a manuscript for publication

Summary

Next logical extension of competency-based evaluation

Mostly familiar, not burdensome

Levels 1 and 2 do NOT signify a ‘Fail’ grade

Evaluations will be more specific and focused

Direct clinical observation to evaluate procedures and patient interactions

We Need Your Data !

APOM NEEDS YOU

Acknowledgements

AMIGOSNicole ConradOumou DialloJudy FreemanJulio GonzalezKaren HandIzumi HarukuniAmy Miller JuveDiana KimEd KahlDean LaoDavid LarsenKim MauerMichele NolesAnnie RileyPeter SchulmanKatie SeligmanChris SwideLinda WylieDavid WilsonGlenn Woodworth