Transcript Slide Template

56th ASH Annual Meeting
Disclosure Statement
Eric Holmboe, MD
• Employment: ACGME
Discussion of off-label drug use: not applicable
Accreditation Council for Graduate Medical Education
Realizing the Promise of
Competency-based Medical Education:
From Theory to Practice
© 2014 Accreditation Council for Graduate Medical Education
Disclosures
• Employed by the ACGME
• I receive royalties from Mosby-Elsevier for a
textbook on assessment
• I am a member of the board of NBME and
Medbiquitous
© 2014 Accreditation Council for Graduate Medical Education
Outline
 Why CBME and why now?
 External forces driving change
 Nostalgialitis Imperfecta
 CBME and educational theories
 CBME, NAS and professional selfregulation
© 2014 Accreditation Council for Graduate Medical Education
CBME: Start with System Needs
Frenk J, et al. Health professionals for a new century: transforming education to
strengthen health systems in an interdependent world. Lancet. 2010
© 2014 Accreditation Council for Graduate Medical Education
5
Early Principles: CBME
• World Health Organization (1978):
• “The intended output of a competencybased programme is a health
professional who can practise medicine
at a defined level of proficiency, in accord
with local conditions, to meet local
needs.”
McGaghie WC, Miller GE, Sajid AW, Telder TV.
Competency-based Curriculum Development in Medical
Education. World Health Organization, Switzerland, 1978.
© 2014 Accreditation Council for Graduate Medical Education
What Are The Outcomes?
• A competent (at a minimum) practitioner
aligned with:
CMS Triple Aim
© 2014 Accreditation Council for Graduate Medical Education
Current Realities:
Health System to
Training System Performance
© 2014 Accreditation Council for Graduate Medical Education
AHRQ Quality Report 2013
Measure
Focus
Aspirin Use
Measure
Name/Description
Outpatient visits at
which adults with
cardiovascular disease
are
prescribed/maintained
on aspirin
Blood
Adults with hypertension
Pressure
who have adequately
Control
controlled blood
pressure
Cholesterol
Adults with high
Management cholesterol who have
adequate control
Smoking
Outpatient visits at
Cessation
which current tobacco
users received tobacco
cessation counseling or
cessation medications
© 2014 Accreditation Council for Graduate Medical Education
Baseline
Rate
47%13
Most Recent
Rate
53%14
Aspirational
Target
Increase to 65%
by 2017
46%15
53%16
Increase to 65%
by 2017
33%17
32%18
Increase to 65%
by 2017
23%19
22%20
Increase to 65%
by 2017
Arnie Milstein 2010
• Since physician graduates of American medical
education organizations typically lead or heavily
influence US health care delivery, one source of
indirect, broad, outcome-based evidence [of the
effectiveness of the medical education enterprise] is
the overall performance of the US health care
system. The width of the performance gaps on the
aims of effectiveness, safety and efficiency
understandably reduces society’s confidence that
physicians are adequately honoring their Hippocratic
promises.
Milstein A. Trailing Winds and Personal Risk Tolerance:
An External Perspective on the Opportunity for Medical
Educators to Fulfill Their Social Contract Permanently.
Presented at ABIMF Summer Forum, August 2010
© 2014 Accreditation Council for Graduate Medical Education
Policymakers Raising Concern
 Institute of Medicine (2008)
 Resident Duty Hours: Enhancing Sleep, Supervision,
and Safety
 Retooling for an Aging America
 Congress (2011-12)
 Reductions in GME funding
 Request to IOM to review GME regulation
 MedPAC (2009-2010)
 June 2010 Report influenced reform legislation
 Institute of Medicine (2014)
 More accountability for GME funding
 Innovation fund
© 2014 Accreditation Council for Graduate Medical Education
Evaluating Residency Programs Using Patient Outcomes
JAMA 2009;302(12):1277-1283. Asch, DA, et.al.
Rate of Major Obstetric Complications
by Graduates (%)
14
12
Difference remains
after correction for
USMLE performance
10
8
Excess Risk ∆ 32%
Q1 vs Q5
6
4
2
0
Q5
Q4
Q3
Q2
Q1
∆ Q1-Q5
Residency Program of Origin, Ranked (Quintile) by Program Complication Rate
© 2014 Accreditation Council for Graduate Medical Education
The Medical Students “Strike Back”
Average # of physician visits in
last six months of life (teaching
hospitals in red)
From:
What Kind of Physician Will You
Be?
Variation in Health Care and Its
Importance for Residency Training
Dartmouth Institute for Health
Policy & Clinical Practice 2012
© 2014 Accreditation Council for Graduate Medical Education
Environment and Conservative Practice
Sirovich BE, Lipner RS, Johnston M, Holmboe ES. The Association Between
Residency Training and Internists’ Ability to Practice Conservatively. JAMA IM. 2014.
© 2014 Accreditation Council for Graduate Medical Education
Nostalgialitis Imperfecta
• Syndrome characterized by the following signs
and symptoms:
• “When I was an intern…<insert superlative>”
• “Medicine was so much better 25 years ago”
• Reality: Not really…
• “Younger physicians today are less professional,
skilled, etc. because of <insert favorite complaint>”
© 2014 Accreditation Council for Graduate Medical Education
Harvard Medical Practice Study
• Methods:
• Investigated prevalence of adverse events due to
medical management
• Review of 30,121 medical records from 51 randomly
selected acute care hospitals
• Results:
• Adverse events occurred in 3.7% of hospitalizations
• 27.6% due to medical negligence
• 13.6% resulted in death
© 2014 Accreditation Council for Graduate Medical Education
Harvard Medical Practice Study
• Study conducted in 1984 in the state of New
York
• My senior year (1984-85) as a medical student at the
University of Rochester
© 2014 Accreditation Council for Graduate Medical Education
Past, Present and Future
“Those who forget the past are
condemned to repeat it”
George Santayana
“The blind spot of contemporary
[education] is experience”
Francisco Varela
© 2014 Accreditation Council for Graduate Medical Education
The “Miracle” of Medical Education
© 2014 Accreditation Council for Graduate Medical Education
Competency-Based Medical Education
Part of the Solution (Camelot),
or Just a Fad (Trojan Horse)?
© 2014 Accreditation Council for Graduate Medical Education
Is CBME Just a “Fad”?
•
•
•
•
•
•
•
Pet rocks
Leisure suits
Streaking
Disco music
Yugos
Pokemon
Tickle me Elmo
…probably not…
© 2014 Accreditation Council for Graduate Medical Education
Is CBME a Paradigm Shift?
• Maybe…but perhaps that is not the
main point:
• CBME is yet another stage on what
should be the ongoing evolution and
improvement of medical education
• The focus on outcomes is worthy of our
attention
© 2014 Accreditation Council for Graduate Medical Education
What Exactly is CBME?
An outcomes-based approach to the
design, implementation, assessment
and evaluation of a medical education
program using an organizing framework
of competencies1
1Frank,
JR, Snell LS, ten Cate O, et. al. Competency-based medical
education: theory to practice. Med Teach. 2010; 32: 638–645
© 2014 Accreditation Council for Graduate Medical Education
Origins of CBET
Behaviorism
Thorndike
Scientific Management
Taylor
Progressive Education
Dewey
Objective-based
instruction
Operant
conditioning
Minimum
competency tests
Mastery-based
learning
Criterionreferenced tests
Instructional
design
McCowan; CDHS, 1998
© 2014 Accreditation Council for Graduate Medical Education
CBET
Experiential Learning: David Kolb
© 2014 Accreditation Council for Graduate Medical Education
Socio-cultural Theory: Key Principles
 Subject matter and learning processes not
uniform: diverse as the people
 Learning highly influenced by social milleau
 Learning mediated by artefacts and “sign”
systems (e.g. language)
 Learning situated within context where it occurs
 Subject matter, content and process inseparable
 Adversarial interactions (people or institutions)
produces different learning
Yardley S, Teunissen PM, Dornan T. Experiential learning: AMEE guide 63.
Med Teach. 2012; 34:e102-115.
© 2014 Accreditation Council for Graduate Medical Education
Experiential Learning
Predominant
Curriculum in
GME
Identity
key outcome
Yardley S, Teunissen PM, Dornan T. Experiential learning: AMEE guide 63.
Med Teach. 2012; 34:e102-115.
© 2014 Accreditation Council for Graduate Medical Education
Deliberate Practice
Ericsson & Lehmann, 1996:
 “Individualized training activities
especially designed by a coach or teacher
to improve specific aspects of an individual's
performance through repetition and
successive refinement.
– To receive maximal benefit from feedback,
individuals have to monitor their training
with full concentration, which is effortful
and limits the duration of daily training”.
© 2014 Accreditation Council for Graduate Medical Education
Deliberate Practice and Expertise
From Anders Ericsson: Used by Permission
© 2014 Accreditation Council for Graduate Medical Education
Design and Sequencing of Training Activities
*
* Monitor students’ development
Design and select training tasks for individual students
Professional teachers and coaches
From Anders Ericsson: Used by Permission
© 2014 Accreditation Council for Graduate Medical Education
Expert Performance vs. Everyday Skills
Ericsson KA. Acad Med. 2004
© 2014 Accreditation Council for Graduate Medical Education
Other Key Theories
Self-determination Theory
(Ryan and Deci)
 Intrinsic motivators much better than
extrinsic motivators
 Related to sense of autonomy and
independence
Self-efficacy Theory
(Bandura)
 Effectiveness at task leads to further
learning and development
 We don’t like being “bad” at things
© 2014 Accreditation Council for Graduate Medical Education
Effective Learning Strategies
 Practice testing
 Concept of retrieval practice
 Can be very simple – write down what you just learned
 Multiple low-stakes practice tests quite effective for
retention
 Interleaved practice
 Mix different concepts and competencies into the
learning
 Elaborative Interrogation and Self-Explanation
 Distributed practice
Willingham DT. http://www.ascd.org/publications/educationalleadership/oct14/vol72/num02/Strategies-That-Make-Learning-Last.aspx
© 2014 Accreditation Council for Graduate Medical Education
Getting to Expertise…
© 2014 Accreditation Council for Graduate Medical Education
Translating CBME into Action:
The Milestones, NAS and
Professional Self-Regulation
© 2014 Accreditation Council for Graduate Medical Education
The Milestones and NAS in a Nutshell
• A Continuous Accreditation Model based on assessment
of annual data – this list is not all encompassing and is
subject to change
• Annual program data (resident/faculty information, major
program changes, citation responses, program characteristics,
scholarly activity, curriculum)
• Aggregate board pass rate
• Resident clinical experience
• Resident survey and faculty survey (latter is new)
• Semi-annual resident Milestone evaluations
• 10 year Self-Study and Self-Study Visit
• Clinical Learning Environment Review (CLER)
Visits
© 2014 Accreditation Council for Graduate Medical Education
Dreyfus & Dreyfus Development Model
Expert/
Master
Proficient
Competent
Advanced Beginner
Novice
Time, Practice, Experience
Dreyfus SE and Dreyfus HL. 1980
Carraccio CL et al. Acad Med 2008;83:761-7
© 2014 Accreditation Council for Graduate Medical Education
Milestones
• By definition a milestone is a
significant point in development.
• Milestones should enable the
trainee and the program to know
an individual’s trajectory of
competency development.
© 2014 Accreditation Council for Graduate Medical Education
Defining Competency Based Education
Observable
&
Assessed
Defined outcome
Frank JR et al. Med Teach. 2010;32:631-7
© 2014 Accreditation Council for Graduate Medical Education
Milestones as Roadmap
Observations:
1) Journey not a
straight line
2) More than one
path (but not
infinite paths)
3) “If you don’t know
where you are
going, any road
will get you there”
© 2014 Accreditation Council for Graduate Medical Education
Dreyfus & Dreyfus Development Model
MILESTONES
Curriculum
Curriculum
Curriculum
Curriculum
Curriculum
Assessment
Assessment
Assessment
Assessment
Assessment
Expert/
Master
Proficient
Competent
Advanced Beginner
Novice
Time, Practice, Experience
Dreyfus SE and Dreyfus HL. 1980
Carraccio CL et al. Acad Med 2008;83:761-7
© 2014 Accreditation Council for Graduate Medical Education
What Milestones Are Not:
 A complete description of:




Clinical Competence of any individual
The elements of competence in a specialty/subspecialty
Promotion Criteria
Graduation Criteria
 The totality of a discipline
 The sole determinants to be used in
Competency Based Medical Education
 “Tools” to Close Programs
© 2014 Accreditation Council for Graduate Medical Education
Entrustable Professional Activities
• EPAs represent the routine professional-life
activities of physicians based on their specialty
and subspecialty
• The concept of “entrustable” means:
• ‘‘a practitioner has demonstrated the necessary
knowledge, skills and attitudes to be trusted to
perform this activity [unsupervised].’’1
1Ten
Cate O, Scheele F. Competency-based postgraduate
training: can we bridge the gap between theory and
clinical practice? Acad Med. 2007; 82(6):542–547.
© 2014 Accreditation Council for Graduate Medical Education
Entrustment in GME
• As faculty, we “entrust” trainees to do
many things without direct supervision
• Admit patients to hospital from the ED
• Night float
• Clinic preceptor sign-out (without seeing the
patient)
• What justifies these “entrustments”?
• How do we know when and if to make
such entrustments?
© 2014 Accreditation Council for Graduate Medical Education
Dyad Conversation
• What do you entrust your fellows to
do with only reactive (indirect)
supervision?
• How do you decide?
© 2014 Accreditation Council for Graduate Medical Education
There is No Holy Grail…
Holmboe’s
Cousin
CBME relies heavily on the judgments of humans.
The goal is to enhance the probability of making
better judgments for the benefit of both patients and
learners
© 2014 Accreditation Council for Graduate Medical Education
Professional Self-Regulatory System
Unit of Analysis:
Program
Residents
FB
Assessments within
Program:
• Direct observations Qual/Quant
• Audit and
“Data”
D
performance data
Synthesis:
• Multi-source FB
Committee
• Simulation
• ITExam
Faculty, PDs
and others
J
U
D
G
E
M
E
N
T
FB
Milestones and EPAs
as Guiding Framework and Blueprint
© 2014 Accreditation Council for Graduate Medical Education
Accreditation
D
FB
Certification and
Credentialing
Unit of Analysis:
Individual
P
U
B
L
I
C
Milestone Journey:
Revised Conceptual Model of Rapid Cycle Change
Tomolo A M et al. Qual Saf Health Care 2009;18:217-224
© 2014 Accreditation Council for Graduate Medical Education
Accreditation Council for Graduate Medical Education
Thank You and Questions
[email protected]
© 2014 Accreditation Council for Graduate Medical Education