Transcript USMLE Clinical Skills

Medical Licensure in the United States
Assessment of Clinical Skills
Ann C. Jobe, MD, MSN
Executive Director
Clinical Skills Evaluation Collaboration
(CSEC)
Raising the Bar in Geriatrics
Assessment
2010 Reynolds Grantee
9th Annual Meeting
October 25-27, 2010
St. Louis, Missouri
(sample of slides from
presentation)
Clinical Skills Evaluation
Collaboration
CSEC
• A Collaborative
Partnership,
established in
2003, between
the Educational
Commission for
Foreign Medical
Graduates
(ECFMG) and the
National Board of
Medical
Examiners
(NBME)
A Look Back in Time
• USMLE - introduced in early 1990’s
• Single examination pathway for initial medical
license (graduates of US and international
medical schools)
• A national standardized series of exams to assure
minimal competency
United States Medical Licensing
Examination (USMLE)
• Each of the three Steps of the USMLE
complements the others
•
• No Step can stand alone in the
assessment of readiness for medical
licensure.
United States Medical Licensing
Examination (USMLE)
• Step 1
• understanding and application of important
concepts of the sciences basic to the practice
of medicine
• Multiple choice exam
• Step 2
• application of medical knowledge, skills, and
understanding of clinical science essential for
the provision of patient care under supervision
• Clinical Knowledge (CK)
• Multiple choice exam
• Clinical Skills (CS)
United States Medical Licensing
Examination (USMLE)
•Step 2CS
•Standardized patients used to assess an
examinee’s ability to
• gather information from patients,
• perform physical examinations,
• communicate their findings to patients and
colleagues
United States Medical Licensing
Examination (USMLE)
• Step 3
• application of medical knowledge and
understanding of biomedical and clinical
science essential for the unsupervised
practice of medicine
• Multiple choice exam and computerized case
simulations (CCS)
Life Cycle of a Physician
in the United States
American Board of Medical Specialties (ABMS)
Specialty Boards
Board
Certification
Exams
NRMP Match
Medical School
Year 1
Year 2
Year 3
Year 4
Postgraduate
Training
3-7 yrs
Step 1 Step 2 Step 3
CK
Licensure
Step Medical
2
United States
Licensing Examination
CS
(USMLE)
Recertification
q 7-10 yrs
Maintenance of
Certification
Practice
Maintenance of
Licensure
Fellowship
Relicensure
q 1-3 yrs
State Licensing Authorities
Miller’s Pyramid
Action
DOES
Performance
SHOWS HOW
KNOWS HOW
KNOWS
Competence
Knowledge
Kirkpatrick Criteria
4. Results
Change in organizational practice
Benefits to patients/clients
3. Behavior
Transfer learning to workplace
Learners apply new knowledge and skills
2. Learning
Change attitudes/perceptions
Change knowledge/skills
1. Reaction
Customer satisfaction related to participation in
educational activities
COMPETENCY
• “Core Competencies”
Accreditation Council for Graduate Medical Education (ACGME)
American Board of Medical Specialties (ABMS)
•
•
•
•
•
•
Patient Care
Knowledge
Communication and Interpersonal Skills
Professionalism
Systems-Based Practice
Practice-Based Learning and Improvement
Correlations among Step
scores
Step l
Step 2 CK
Step 2 CS
Step 2 CK
~.65
---------
-----
Step 2 CS Data-gathering
Communication/IP skills
Spoken English proficiency
Patient note
.19
.09
.09
.20
.26
.16
.13
.30
-----
Step 3
~.50
~.70
N/A
USMLE Step 2 Clinical
Skills
• Mastery of clinical
and communication
skills, as well as
cognitive skills, by
individuals seeking
medical licensure is
important to the
protection of the
public. (from USMLE
Bulletin of Information)
Step 2 CS Examinees
YEAR
TOTAL
USMGs
IMGs
2009
34,837
18,983
15,854
2008
35,224
17,711
17,513
2007
33,832
17,711
16,121
2006
32,843
17,473
15,132
2005
31,939
17,671
14,268
Step 2 CS Fees
• USMG – increase of $100(10%) over
6 year period
– 2004-2006: $975
– 2007: $1,005
– 2008: $1,025
– 2009: $1,055
– 2010: $1,075
• IMG
– 2004-2009: $1,200
– 2010: $1,295
USMLE Step 2 Clinical
Skills
• The cases cover
common and
important situations
that a physician is
likely to encounter in
clinics, doctors’
offices, emergency
departments, and
hospital settings in
the United States.
Presentation categories
• Include, but are not limited to, cardiovascular,
constitutional, gastrointestinal, genitourinary,
musculoskeletal, neurological, psychiatric,
respiratory, and women's health.
• Examinees will see cases from some, but not all,
of these categories.
• The selection of cases is also guided by
specifications relating to acuity, age, gender, and
type of physical findings presented in each case.
• Each Step 2 CS “session” includes 12 twenty-five
minute patient encounters.
• 15 minutes with patient
• 10 minutes for patient note
• The examination lasts approximately 8 hours.
Two breaks are provided:
• 1st break is 30 minutes long (lunch)
• 2nd break is 15 minutes long (snack).
Step 2 CS Components
• Communication and
Interpersonal Skills (CIS)
• Spoken English
Proficiency (SEP)
• Integrated Clinical
Encounter (ICE)
• Data gathering (DG)
• History & PE
• Patient note (PN)
Step 2 CS Components
• USMLE Step 2 CS is a
Pass/Fail examination
• Each of the three
subcomponents (CIS,
SEP, ICE) must be
passed in a single
administration in order to
achieve a passing
performance on Step 2
CS
Assessment of Communication
and Interpersonal Skills (CIS)
• CIS performance is assessed by the standardized
patients
• a global rating of these skills using a series of generic
rating scales
• same CIS scale for all 12 encounters
• 3 sub-components:
• Information gathering (questioning skills)
• Information sharing
• Professional manner and rapport
Assessment of Communication
and Interpersonal Skills (CIS)
Questioning skills - examples include:
• use of open-ended questions, transitional statements,
facilitating remarks
• avoidance of leading or multiple questions, repeat
questions - unless for clarification, medical terms/jargon
unless immediately defined, interruptions when the
patient is talking
• accurately summarizing information from the patient
Assessment of Communication
and Interpersonal Skills (CIS)
Information-sharing skills - examples include:
• acknowledging patient issues/concerns and clearly
responding with information
• avoidance of medical terms/jargon unless immediately
defined
• clearly providing
• counseling when appropriate
• closure, including statements about what happens
next
Assessment of Communication
and Interpersonal Skills (CIS)
Professional manner and
rapport - examples include:
• asking about
• expectations, feelings, and
concerns of the patient
• support systems and impact of
illness, with attempts to explore
these areas
• showing
• consideration for patient
comfort during the physical
examination
• attention to cleanliness through
hand washing or use of gloves
Assessment of Communication
and Interpersonal Skills (CIS)
Professional manner and
rapport - examples
include:
• providing opportunity for the
patient to express feelings
and/or concerns
• encouraging additional
questions or discussion
• making
• empathetic remarks
concerning patient
issues/concerns
• patient feel comfortable
and respected during the
encounter
Assessment of Spoken English
Proficiency (SEP)
• SEP performance is assessed by the standardized
patients
• using rating scales; same scale for all 12 encounters
• based upon
• frequency of pronunciation or word choice errors that
affect comprehension
• amount of listener effort required to understand the
examinee's questions and responses
• clarity of spoken English communication within the
context of the doctor-patient encounter (e.g.,
pronunciation, word choice, and minimizing the need
to repeat questions or statements)
Scoring of the Step 2 Clinical
Skills Subcomponents
The ICE subcomponent includes assessment of:
• Data gathering (DG) - patient information
collected by history taking and physical
examination
• Documentation (PN) - completion of a patient note
summarizing the findings of the patient encounter,
diagnostic impression, and initial patient work-up
Scoring of the Step 2 Clinical
Skills Subcomponents
• Data gathering (DG) performance is assessed by
the standardized patients
• using checklists developed by committees of
clinicians and medical school clinical faculty
• checklists comprise the essential history and
physical examination elements for each specific
clinical encounter
Scoring of the Step 2 Clinical
Skills Subcomponents
• The patient note is rated/scored by trained
physician raters
• The patient note (PN) consists of three
areas
• Medical History and Physical Examination
• Differential Diagnosis
• Diagnostic Workup
Scoring of the Step 2 Clinical
Skills Subcomponents
• Scored holistically
• Relevant and correct information
• Congruency/consistency with specific case –
based scoring guidelines
• Integration/synthesis of information
• Organization, coherence, cohesiveness, flow,
legibility
Performance on Step 2 CS
• Failure rate for
USMGs
•
•
•
•
•
2004-2005: 4%
2005-2006: 2%
2006-2007: 3%
2007-2008: 3%
2008-2009: 3%
• Failure rate for IMGs
• 2004-2005: 17%
• US Citizens: 11%
• Foreign Citizens: 18%
•
•
•
•
2005-2006: 15%
2006-2007: 23%
2007-2008: 28%
2008-2009: 27%
Performance on Step 2 CS
• Passing rate for USMGs (first takers)
• 2005-2006: CIS >99%; SEP >99%; ICE 98%
• 2006-2007: CIS 99%; SEP 100%; ICE 97%
• 2007-2008: CIS 99%; SEP >99%; ICE 98%
• 2008-2009: CIS 99%; SEP >99%; ICE 98%
Performance on Step 2 CS
• Passing rate for IMGs (first takers)
• 2005-2006: CIS 93%; SEP 98%; ICE 89%
• 2006-2007: CIS 87%; SEP 99%; ICE 85%
• 2007-2008: CIS 81%; SEP 92%; ICE 86%
• 2008-2009: CIS 84%; SEP 94%; ICE 84%
• For more information about USMLE policies and
procedures, and about specific exams – please go to:
• www.usmle.org