Achieving Added Accuracy: Improving Fidelity in the Curriculum - PowerPoint Presentation

Slide1

Achieving Added Accuracy: Improving Fidelity in the Curriculum Inventory MedBiq 2017

Walter Fitz-William, MPP

Sr. Program Operations Specialist

Slide2

Plan!

Talk about the plan!

Review the Curriculum Inventory as a concept and application

Explain the AAMC’s use of the Curriculum Inventory StandardsIdentify problems/issuesIdentify solutionsCurriculum Inventory locally and “globally”

Slide3

Objectives

Describe technical and non-technical challenges to capturing data about curricular structure.

Describe potential solutions to these challenges and identify relevant options for

your institution.Identify local and global challenges that curriculum management paired with the aggregation in the Curriculum Inventory can help an institution tackle.

Slide4

What is this Curriculum Inventory you speak of?MedBiquitous

Standard

Association of American Medical Colleges utilization of those standards

Specific languageSpecific business rulesReporting

Slide5

Terminology

Previous Academic Year

(LCME model – if current year is 2017-2018, previous academic year is 2016-2017)Academic Level (formerly called ‘year of curriculum’, a.k.a. phase)

Sequence Block /Nested Sequence Block (Course, Module, Unit, Block, Clerkship)Integration Block (Theme, Vertical / Horizontal Integration)Event (Instructional or Assessment Session)

(e.g., Lecture, Lab, Discussion, Assessment)Expectations / Competency Framework(Competency / Objective / Milestone / EPA)

Slide6

Curriculum Inventory

A

E

B

F

C

G

D

H

I

J

K

L

A

1

M

E

O

N

C

G

S

H

I

J

P

R

Q

2

3

1

2

4

5

3

6

8

9

7

10

11

Competency framework

Expectations

Events

Sequence blocks

Integration blocks

Slide7

4000 character limit

4000 character limit

4000 character limit

Event

AL 1

AL 2AL 3

AL 4

Count

141

137

124

96

25th

431

299

68.75

24

Median

554

436

178.5

66.5

75th

650

529

341.25

162.75

SB

AL 1

AL 2

AL 3

AL 4

Count

141

137

124

96

25th

7

6

7

4

Median

10

9

9

8

75th

14

13

14

27.25

Slide8

4000 character limit

4000 character limit

average, minimum, 25th percentile, and median number of events per sequence block, or more than

1000 events

more than 8 years apartMany/all sequence blocks are linked to the same eventsThe Timing sub-element of all SB tags must specify a Dates element. The Start Date must precede the End Date.

All Event tags must be referenced by at least one SequenceBlockReference tagwithin a SequenceBlock.

“These help bring in ne…”?

“medications carried on a paramedic u…”?

SB: Sequence Block

Slide9

Rules rules rules

Each of these is in some ways a barrier to entry, but they are also ensuring that what is provided to us will look sufficiently alike so as to be comparable. And we are talking about material and experiences that are inherently comparable. Like making a meal, the end state is meant to be used in a generally comparable

manner.

The ultimate point is to accurately represent your take on the meal.

Slide10

AAMC Business Rules

Curriculum Structure Rules (CS)

CS01: Unique Report ID

The <ReportID> sub-element must be unique for each of a school’s submissions. If a Curriculum Inventory Upload is rejected, the next uploaded file must not have the same ReportID.

The <ReportID> sub-element must be alphanumericThe Report ID must not include dashes, underscores or other special characters

Slide11

AAMC Business Rules

Curriculum Structure Rules (CS)

CS02: Previous Academic Year

Submissions are checked to ensure they contain information for only the previous academic year (AY). Schools must submit data with <ReportingStartDate> and <ReportingEndDate> between July (of the previous year) and June (of the current year).

Note: Dates at the sequence block or event level are not checked to ensure they are within the previous academic year. Dates specified at these levels that fall outside of the date range for a previous academic year will not cause an error or rejection.

Slide12

AAMC Business Rules

Curriculum Structure Rules (CS)

CS03: Number of Academic Levels

The number of academic levels, or phases, are defined by <LevelsInProgram>.Each academic level is defined as a <Level> within the <AcademicLevels> tag. CS05: Academic Level Numbers

<Level> sub-elements of the <AcademicLevels> tag must have sequential numbers (i.e., the number attribute) starting with 1.

Slide13

AAMC Business Rules

Curriculum Structure Rules (CS)

CS04: Academic Levels and

Sequence BlocksA <Level> that is not defined within the <AcademicLevels> tag cannot be referenced by a sequence block. CS06: Academic Levels and Sequence Blocks

Each academic level must be referenced by at least one sequence block.

Slide14

AAMC Business Rules

Sequence Blocks

CS07:

<SequenceBlock> tags must have unique identifiers (i.e., the id attribute). CS08: <SequenceBlock> tags must be associated to a defined academic level using the required <Level> sub-element.

CS09: The <Timing> sub-element of all <SequenceBlocks> tags with a ClerkshipModel attribute must specify <Dates> and <Duration>. Document start and end dates for entire iteration period (e.g., 07/01/12 – 06/30/13).

CS16: If it is provided, the duration of a sequence block must be specified as a number of days. For clerkships, 5 days = 1 week

Slide15

AAMC Business Rules

Curriculum Structure Rules (CS)

CS10: Nested Sequence Blocks

When nesting sequence blocks, <SequenceBlockReference> must not result in circular references.See “Note about Hierarchical Conflict” section (pp. 33-36) of the Competency Framework Specification, version 1.0.

Slide16

AAMC Business Rules

Curriculum Structure Rules (CS)

Events

CS11: <Event> tags must have unique identifiers (i.e., the id attribute). CS12: Within <Event> tags, the <AssessmentMethod> and <

InstructionalMethod> tags must reference a unique ID from the Curriculum Inventory Standardized Vocabulary.CS13: At a minimum, each <Event> must reference at least one <AssessmentMethod> or <InstructionalMethod>.

CS14: If one or more <InstructionalMethod> tags are provided, one and only one must be denoted as primary. CS15: All <Event> tags must be referenced by at least one <SequenceBlockReference> tag within a <SequenceBlock>.

Slide17

Matching Local Terminology to CI Standardized Terminology

Local Terminology:

Instructional Methods

CI Standardized Terminology

Clinical SkillsClinical Experience - OutpatientProcedure TrainingClinical Experience - InpatientResource SessionLecture

Standardized PatientSimulation (with Resource: Standardized Patient)Wrap-Up SessionDiscussion, Small Group

The MedBiquitous Curriculum Inventory Standardized Terminology Subcommittee reviews the terminology each year, based on literature and terminology schools report as difficult to match (challenges)

Submit matching ‘challenges’ to

ci@aamc.org

or post comments in

iCollaborative

Slide18

Speaking the Same Language: Simple Improvements Clinic, Clinical, Clinician, Clinically

CLIN

CLINIAL

CLINICLAL

CLINC

CLINICALY

CLINICLLY

CLINCAL

CLINICAN

CLINICNAL

CLINCALL

CLINICIAL

CLINICSL

CLINCI

CLINICICALL

CLINIICAL

CLINCIAL

CLINICL

CLININCAL

CLINCICAL

CLINICLA

CLINIUCAL

CLINI

Slide19

Speaking the

Same

Language: Keywords

Epidemiology

Geriatrics

Pathology

Pharmacology

Obesity

Anatomy

Neurology

Genetics

Patient Safety

Hypertension

Biostatistics

Radiology

Physiology

Pediatrics

Histology

Pain Management

Pain

Biochemistry

Psychiatry

Evidence-Based Medicine

Asthma

Physical Examination

Headache

Diagnostic Imaging

Diabetes Mellitus

Communication

Ethics

Nutrition

Liver

Anemia

Pregnancy

Abdominal Pain

Dermatology

Diarrhea

Palliative Care

Inflammation

Respiratory System

Microbiology

Sepsis

Metabolism

Musculoskeletal System

Lung

Heart

Dyspnea

Pneumonia

Jaundice

Edema

Osteoporosis

Spleen

Emergency Medicine

Tuberculosis

Pharmacokinetics

Embryology

Public Health

Abdomen

Cultural Competency

Fever

Dementia

Spinal Cord

Neck

Meningitis

Hypothyroidism

Pelvis

Diuretics

Sexually Transmitted Diseases

Contraception

Pneumothorax

Parkinson Disease

Lymphoma

Hyperkalemia

Toxicology

32 Schools use “Epidemiology”

18 Schools use Pelvis, Fever, etc.

Slide20

Slide21

Slide22

CI Data Analysis

Issue 1: Copying one course over to all other courses

Issue 2: Keywords that have a non-content entry

Issue 3: Developing a Synonym-Matching ProcessIssue 4: Titles for Competency Objects (NA, Blank, Session Exp.)Issue 5: Uploading only One Event for All

Sequence BlocksIssue 6: Uploading only One Academic Level

Slide23

What are your issues?Multiple vendors for different levels?

Not enough faculty attention/time?

No clear list of uniform terminology to use?

No resources/process for content redesign?No resources/process for data review and analysis?

Slide24

Hear what I’ve heard…Consider creating multiple sessions, whether for ambulatory or out patient

Meet with clerkship director and go over your keywords

,

and disciplines, review the instructional methods and assessment methodsManage your curriculum AND your content, but recognize that those are independent but interrelated. Know the number of times each competency is mapped or maybe where it should be mapped.Think of new faculty as prime opportunities for exploring what you are teaching and what you now want to teach.

Slide25

Hear what I have heard…This is a piece by piece process – this is never a finished process.

This takes

time and as much as we’d like it all now, we appreciate whatever you can provide.

Perhaps start with keywords. People work well from first passes – get students to contribute to those first passes. Limit how much people can include in any session.Show the capitalization on the processes you need to have in place to keep your curriculum managed (graphs that show whether you are meeting the objectives you laid out for yourself, where you meet them, it doesn’t have to be yes/no, it can be degrees, but commit to documenting the assessments for those competencies

)

Slide26

Hear what I’ve heard…Get IT in the room at the curriculum Committee discussions so they can hear more of what you need, what you want, you want business architects or people with process skills in early so they can be part of the team from early on – they, or maybe this is you, are going to be having to support hard choices and the only good way to do that is to be informed.

When

you consider new technology, find out how it will integrate into your future. Know what you want before you ask to see it – nice features can be valuable, but are you getting the value you are looking for? Know what you want before you shop and, like food shopping, try to avoid doing it too hungry. Don’t wait for crises?

With curricular revision and design, advocate for designers to be a part of the process – they won’t know the content, but they will know how to express your message.

Slide27

What have you heard?

Slide28

What have we been doing about/with it all?

Slide29

Who Participates in the CI?

2012-2013

2013-2014

2014-2015

2015-2016

90 Schools

120 Schools

(82 of 90 retained; 28 new schools)

135 Schools

(114 retained;

21

new schools)

141 Verified

(10 new schools)

8 Schools that participated previously did not participate in 2015-2016 due to changes in curriculum management systems, changes in administration, and/or accreditation preparation.

 

2012-2013

2013-2014

2014-2015

2015

-2016

US Medical Schools

85

115

128

134

Canadian Medical Schools

5

5

5

5

US Osteopathic Schools (Pilot)

n/a

n/a

3

2

Slide30

60+ Reports –

Use search feature to find charts.

Detailed CI Reports are not included in this list – they are only available to Participating Schools. A list has been strategically hidden in the middle of this page (we are working on updates for the site) – send requests for the reports to ci@aamc.org.

https://www.aamc.org/initiatives/cir/curriculumreports

Slide31

Curriculum Inventory data will augment LCME Annual Questionnaire data with details such as:

Number of courses where content is covered (total, mean, median, range)

Number of events where content is covered (total, mean, median, range)

Instructional and assessment methods used in events where content is covered

Resources used in events where content is covered

Competencies linked to events and courses where content is covered

Curriculum Inventory Reports

Slide32

Options to include data from:

All Schools

US MD

CA MD

Drop-down includes ONLY participating schools

Email needs to be from your institution

Slide33

Address Your Interests/Needs

LGBT Report: Searched at least one of these:

 

Ambisexual Bisexual Bisexuality Bisexuals Gay Gays Glb Glbt Glbti Glbtq Homosexual Homosexuality Homosexuals Intersex Intersexual Intersexuality Lbg

Lbgt Lbgtq Lesbian Lesbians Lesbien Lgbq Lgbt

Lgbta Lgbthealth Lgbti Lgbtiq Lgbtq

Lgbtqa

Lgbtqi

Lgbtqia

Lgbtt

Transexual

Transexualism

Transgender Transgendered Transgenderism

Transgenderpre

Transsexual Transsexualism

Andgeneticinfluencesonsexualorientation

Homophobia Homophobic

Slide34

 OR searched for at least one of the first group within two words of the other group, order didn’t matter: Bi Tran Trans

Identificaiton

Identificatio Identification Identifications Identity Identity2 Identiy Identy Identigy HomoSex Sexâ Sexally Sexd Sexes Sexism Sexist Sexual Sexualactivity Sexualilty Sexualité Sexualities Sexuality Sexuality2 Sexualityâ

Sexualityc Sexualityh Sexuall Sexually Sexuallyâ Sexualy Sexula Sexulaity Sexutality Gender Gendered Genderexpecations Genderexpectations Genderl

Genders Genderspeci

Slide35

So…? Now what?

Audience Research?

Intent Data connections?

Analytic Literacy Your results?

Slide36

https://www.aamc.org/initiatives/cir/384428/ciincontexthome.html

Slide37

Stand up for YouMedical schools and physician training programs nationwide have been enhancing content on pain and addiction to reach practitioners early in their careers. “There has been a huge effort at the state, federal and local levels to better educate doctors [about opioids]. Doctors [in all specialties] need to have an understanding of the process of addiction and the risks of opioid prescribing,” said [Dr. Anna]

Lembke

[Stanford University Medical Center]

https://news.aamc.org/patient-care/article/stemming-opioid-epidemic/

Slide38

CI in Context: May 2017

Tackling the Opiate Epidemic Through Educational Innovation and Partnership

Melissa Fischer, MD Med, University of Massachusetts Medical School

https://www.aamc.org/initiatives/cir/384428/ciincontexthome.html

Slide39

Support Your Research, Curricular Redesign, Competency Mapping, Accreditation, CQI

Papers and posters from content search

SB length and event duration

PCRS as common competency languagehttps://www.aamc.org/initiatives/cir/423212/ciadmingroupwebinars.html CI Administrators' Group - 2017 Session 4

Slide40

Walter Fitz-Williamwfitzwilliam@aamc.orgci@aamc.org

August 1

st

, 2017!