04
Clinical Competency CommitteePurpose “The Clinical Competency Committee (CCC) serves several purposes for multiple stakeholders: the program; the program director; faculty members; program coordinators; residents/fellows/those in graduate medical education (GME) programs; the Sponsoring Institution; and the ACGME. Ultimately, it serves the public by ensuring program graduates provide high quality, safe, reliable patient care.”<br>
05
A program’s creation of a CCC is, in itself, a “developmental” process. https://www.acgme.org/milestones/resources/; accessed 4.20.2024<br>
06
This Photo by Unknown Author is licensed under CC BY-SA Why CCCs? Evidence supports decision making by groups > individuals
Thomas et al, JGIM 2011
Gaglione et al, Academic Medicine 2005
Schwind et al, Academic Medicine 2004
The “Wisdom of the Crowd”, James Surowiecki<br>
07
Reproduced with permission of author<br>
08
Clinical Competency CommitteeRole Primary
Trainee performance on milestones
Problem identification
Developmental Hauer et al. Acad Med 2015<br>
09
Clinical Competency CommitteeRole Feedback to--
Trainees
Faculty
Program
Faculty development
Quality improvement of the program, assessment system Andolsek et al, ACGME CCC Guidebook<br>
10
Improving assessment Critique assessment tools
Offer suggestions to revise/simplify
Recognize barriers to effective use of tools by faculty
Make recommendations as to “where and when” milestones performance should be assessed
Not everything everywhere
“…the true assessment instrument is NOT the tool or the form. It is the faculty member(s) or others using it” Andolsek et al, ACGME CCC Guidebook<br>
11
How is our program of assessment? Evaluation completion reports
Data collected per milestone
Gaps in data
Quality of faculty comments
Mean evaluation ‘scores’ for faculty members
Curricular gaps
What kinds of reports can be generated? What info do we have?
Do you know where to find this information?<br>
12
Exercise: CCC Role What role does your CCC play in your program?
What other roles might the CCC play?<br>
13
Learning objectives Describe the role of the CCC
List best practices for CCC membership, structure, group process
Including addressing potential bias
Identify resources for CCC member faculty development
Set goals for continuous improvement for home CCC<br>
14
Clinical Competency CommitteeMembership PD appoints, but does not have to participate
At least three program faculty
At least one core faculty
Other faculty or health professionals
Must have extensive contact with trainees
Can be from other programs
Chief residents (if completed training)
Program administrators/coordinators?<br>
15
Clinical Competency CommitteeMembership - considerations<br>
16
Clinical Competency CommitteeChair Some RC restrictions
Selection
Who will best solicit broad input
Who will ensure voices heard
Guidelines (French et al, 2014)
Milestones “expert”
Encourage confidential positive environment
Ensure members know roles
Expert in CCC process, requirements
Engage members
Understands assessment methods<br>
17
Clinical Competency CommitteeMembership – other possible members Program director?
New faculty
Program administrator
Bias coach – consideration?<br>
18
Influencing CCC discussions - some examples “Anchoring”
Maintaining initial impressions despite change in performance
”Confirmation”
Only paying attention to data that “confirms” or supports your opinions and disregarding data that does not
“Visceral”
Making decisions based on emotion instead of on performance data
“Availability”
More importance placed on recent/first-hand or memorable data
“Reliance on gist”
Not making decisions based on actual observations/specific data Dickey et al, JGME 2017<br>
19
Influences in CCC discussions Appreciation of potential bias in assessment
Awareness of bias in narrative evaluations
Use of biased language in discussions
Awareness of personal/group unconscious bias<br>
20
Exercise: CCC Membership Who are the members of your CCC?
Is there diversity of expertise, opinions?
What is the role of the PD?
Who is watching for bias and how?
How are you working to address bias?<br>
21
Learning objectives Describe the role of the CCC
List best practices for CCC membership, structure, group process
Including addressing potential bias
Identify resources for CCC member faculty development
Set goals for continuous improvement for home CCC<br>
22
Clinical Competency CommitteeFaculty Development Purpose
Requirements
Process
Assessment
Feedback
Bias
Group dynamics and decision-making
Tools
Milestones/competencies<br>
23
CCCFaculty DevelopmentRecommendations Should be “intentional”
Annual review of topics – including:
Purpose & structure of the CCC
Assessment methods
Development of a shared mental model of the milestones
Effective group practice
Others – e.g. recognizing bias, decision making, role of the CCC
Varying levels of faculty development to address needs of
CCC chairs/PDs/CCC members Heath et al, JGME 2021<br>
24
Targeted faculty development<br>
25
Targeted faculty development Turner et al, JGME 2021<br>
26
CCC Faculty Development – Resources ACGME milestones resources - Resources (acgme.org)
ACGME faculty development toolkit - ACGME Faculty Development Toolkit: Improving Assessment Using Direct Observation
STFM CBME resources
ACGME Resources – unconscious bias
Harvard Macy Institute – Leadership and assessment courses Heath et al, 2021 JGME; Kinnear et al, 2018 Med Teacher<br>
27
CCCFaculty DevelopmentGetting started Orientation
Guidebook/Charter
Journal club
Data and tool reviews
Evaluations
Data
Milestone map
“case” presentations
Bias training
Micro learnings
Coach – bias, assessment, group process<br>
30
CCCFaculty DevelopmentGetting started - questions Who
Targeted? – PD, Chair, PA, Members
New vs current
Content
Format
Retreat, micro-learning, required development sessions
When
Annually, every meeting, separate sessions, on the fly?
Resources
Publications
Websites
People<br>
31
Learning objectives Describe the role of the CCC
List best practices for CCC membership, structure, group process
Including addressing potential bias
Identify tools to foster resident engagement in the review process
Identify resources for CCC member faculty development
Set goals for continuous improvement for home CCC<br>