Transcript First Annual National Advisory Board Meeting

Measuring the Impact of Resident
Work Hours Reform: Recent Findings
and Next Steps
Patrick S. Romano, MD MPH
Professor of Medicine and Pediatrics
Division of General Medicine
and
Center for Healthcare Policy and Research
University of California, Davis
May 21, 2009
Disclosure
• Financial support from NHLBI RO1 HL82637
•
(Kevin Volpp, University of Pennsylvania, PI)
Have you (or your spouse/partner) had a
personal financial relationship in the last 12
months with the manufacturer of the products
or services that will be discussed in this CME
activity?
NOT APPLICABLE
Acknowledgments
Investigative team
•
•
•
•
•
Kevin Volpp, MD PhD, University of Pennsylvania
Jeffrey H. Silber, MD PhD, Director, Center for Outcomes
Research, Children’s Hospital of Philadelphia
Amy K. Rosen, PhD, Bedford VA Center of Excellence in
HSR&D
Paul Rosenbaum, PhD (Statistician), Wharton School
Lisa Bellini, MD, Program Director, University of
Pennsylvania
Staff
•
Orit Even-Shoshan, MS, Anne Canamucio, MS, Tiffany
Behringer, MS, Yanli Wang, Hong Zhou, Liyi Cen, Mike
Halenar
Other sources of slides
•
•
John Welch, MD (former pediatric resident)
Garth Utter, MD, Department of Surgery
Learning Objectives
• To Explain Recent and Proposed Policy
Changes to Limit Resident Work Hours
• To Summarize Recent Evidence Regarding the
Impact of ACGME Work Hour Rules
Implemented in 2003
• To Discuss Ideas and Methods for Future
Research in this Area
The Birth of Residency
• 1889 with the opening of
•
•
•
The Johns Hopkins Hospital
Osler, Halsted, and Kelly
(bedside teaching)
Based on German model
Room, board, and laundry
provided; salary optional
until 1965
Core Concepts and Practices
• Graded responsibility,
•
•
•
especially for inpatients
Variable and lengthy
training period
Pyramidal system of
promotion
Restrictive lifestyle (100120 hrs/week, continuous
shifts up to 36 hours)
Libby Zion Case
• 18 year old college student, with known
•
•
history of depression, taking Nardil® (MAOI),
was brought into New York Hospital on
October 4, 1984
Presented with fever, agitation and strange
jerking motions of body, with occasional
disorientation
Admitted with diagnosis of “viral syndrome
with hysterical symptoms”
Libby Zion Case
• Ordered Demerol® to control her shaking
• Later in evening Libby became more agitated
• The intern was contacted at least twice,
•
•
•
•
ordered physical restraints and Haldol®
Patient finally fell asleep
At 6:30 a.m. her temperature was noted to be
107°F
Emergency measures were attempted
Patient suffered a cardiac arrest and died
Aftermath
• Sidney Zion’s efforts to change the system
• 1986 Grand Jury does not indict those
•
•
involved but strongly criticizes system
Bell Commission established to investigate
and make recommendations
July 1, 1989 Section 405.4 of Title 10 of the
New York Codes, Rules and Regulations of
the Department of Health go into effect
• Work hours: Maximum 80 hrs/week averaged over
•
4 weeks, 24 consecutive hours (12 in ED), at least
1 scheduled 24 hour break per week
On-site supervision 24 hrs/day, 7 days/week
The 80 hour work week
“The specific "80-hour week" was actually determined
by a colleague on my porch and was based on the
following informal reasoning:
(1) there are 168 hours in a week;
(2) it is reasonable for residents to work a 10-hour day
for 5 days a week;
(3) it is humane for people to work every fourth night;
(4) subtracting the 50-hour week (10 hours per day x 5
days) from 168 hours leaves 118 hours;
(5) divide 118 by 4 (every fourth night) and add to the 50
hours and, eureka, that equals an 80-hour week.”
Bell JAMA 2007: 298(24):2865-2866
Timeline
• June 1998 - New York State Department of
•
•
•
Health found work hours often exceeded
regulatory limits
April 2001 - OSHA is petitioned to regulate
work hours nationwide
November 2001 Representative John
Conyers (D-Mich) introduced federal
legislation to restrict resident work hours
In response ACGME announced its
guidelines effective July 1, 2003
ACGME Work Hour Restrictions
•
•
•
•
•
•
Principles, Supervision, Fatigue, Duty Hours, OnCall activities, and Moonlighting
Limited to 80 hours per week
One day in seven free of all responsibilities
10 hour off-duty period between work periods
In house call no more frequently than every 3rd night
In house call must not exceed 24 consecutive hours
(with up to 6 extra hours for transition of care)
Resident Education and QOL
• Meta-analysis of studies that assessed a system
change designed to counteract the effects of
resident work hours, fatigue, or sleep deprivation;
included an outcome directly related to residents;
and were conducted in the United States.
• 54 articles met inclusion criteria (12 IM, 6 ob/gyn,
7 pediatrics, 25 surgery, 4 other)
• Interventions included night and day float teams,
extra cross-coverage, and physician extenders.
Fletcher, JAMA, 2005
Resident Education and QOL
• Interventions to reduce resident work hours
resulted in mixed effects on both operative
experience and on perceived educational quality
but generally improved residents’ quality of life
(i.e., more sleep, better mood, better family
relationships, better satisfaction).
• Interpretation of the outcomes of these studies is
hampered by suboptimal study design and the
use of nonvalidated instruments. The long-term
impact of reducing resident work hours on
education remains unknown.
Fletcher, JAMA, 2005
N Engl J Med 2004;351:1838-1848
Effect of Reducing Interns’ Work Hours
• Investigated effect of reducing interns’ work
•
•
•
•
hours on serious medical errors in ICU
Randomized, prospective crossover trial
Conducted in MICU and CCU, 20 interns on 3
week rotations
Q3 day “traditional” call versus 4-day
schedule without extended shifts >16 hours
(7am-3pm on day 1, 7am-10pm "day call" on
day 2, 9pm-1pm “night call" on days 3-4)
Compared rates of serious medical errors (by
masked direct observation) made by interns
on traditional vs. intervention schedule
Effect of Reducing Interns’ Work Hours
• Traditional work week: 74-92 hours (mean 85)
• Intervention work week: 57-76 hours (mean 65)
• No change in staffing or other personnel
• Randomly assigned order, and spread
•
throughout year
Interns worked 19.5 hrs/week less (P<0.001),
slept 5.8 hrs/week more (P<0.001), and had
fewer attentional failures (EOG slow eye
movements) during on-call nights (0.33/hr=
2.6/overnight versus 0.69/hr=5.5/overnight;
P=0.02) on the intervention schedule
Relationship between work hours and
sleep duration on two schedules
Most but not all interns slept more on
intervention schedule
Results of RCT
• 2203 patient days, 5888 hours of observation
• During traditional schedule, interns made:
• 35.9% more serious medical errors
• 27.8% more intercepted serious medical errors
• 56.6% more nonintercepted serious medical errors
• 20.8% more serious medication errors
• 5.6 times as many serious diagnostic errors
• No difference in procedural errors
• No significant change in other staff errors
• No significant change in # of medications or
procedures, tests interpreted, LOS, mortality
Incidence of Serious Medical Errors
Landrigan C et al. N Engl J Med 2004;351:1838-1848
Evaluations of 1988 Libby Zion Laws
• No relative improvements in mortality for AMI,
•
•
CHF, pneumonia in teaching vs. non-teaching
hospitals1
Significant increases in proportion of patients
with at least one complication (35% vs. 22%,
p=.002) and in delays in diagnostic tests (17%
vs. 1.9%, P<.0001)2
Compliance poor (60% of surgical residents
exceeded 95 hours per week – 1997 audit)
1 Howard,
Silber, Jobes JGIM 2005
2 Laine JAMA 1993
Effect of work hours reform in NY teaching hospitals
on smoothed rates of Patient Safety Indicators
Poulose BK, et al., Ann Surg 2005;241:847-860
Effect of work hours reform in NY teaching hospitals
on smoothed rates of Patient Safety Indicators
Poulose BK, et al., Ann Surg 2005;241:847-860
Benefits versus Harms
• Benefits:
• More sleep, better sleep, better quality of life
• Better cognitive performance, fewer errors
• Caveat #1: while duty hour rules reduce total number
•
of hours work per week, 30 hour shifts allowed
Caveat #2: we don’t know how much more residents
are actually sleeping
• Harms:
• Less opportunity to observe trajectory of illness
• More frequent hand-offs
• Studies have shown higher rates of significant adverse
events when patients are “cross covered” (26% vs.
12%, OR=3.5)1
1Petersen
et al, Annals of Int Med 1994 121: 866-872.
First study of impact on mortality:
Inpatient only, different samples by year
0.25% absolute reduction; 3.75% relative reduction
Shetty, Ann Intern Med, 2004
Our Study Cohorts
Volpp K, et al. JAMA 2007;298(9):975-1001
•
All unique patients admitted between July 1, 2000 and
June 30, 2005 (3 yrs pre-reform, 2 yrs post-reform)
•
•
•
•
Principal diagnoses: AMI, CHF, GI bleed, or stroke
DRG classification of general, orthopedic, or vascular
surgery
VA
•
•
320,685 patients, 131 hospitals
Data from VA Patient Treatment File (PTF) and
Beneficiary Identification Record Locator System
(BIRLS), VA Office of Academic Affiliations
Medicare
•
•
8,529,595 patients, 3321 hospitals
Data from the Medicare Provider Analysis and Treatment
File (MEDPAR), denominator files, Medicare Cost
Reports
Effects measured by comparing pre- to postreform changes in mortality in hospitals of
differing teaching intensity
20
15
Hypothetical
Mortality 10
Data
Diff A1.
EFFECT =
Diff A2
Diff A – Diff
B
Diff B
5
(D-in-D)
ACGME
Reform
0
Pre
Non-teaching
Post
Teaching A
Teaching B
Logistic regression used to adjust for patient
comorbidities, secular trends, hospital site where
treated using “difference in differences”
Medicare - No significant relative change in
mortality according to teaching intensity
Volpp KG et al. JAMA; 2007: 298 (9): 975-983.
VA - Significant relative improvement in mortality
among medical patients in post-reform year 2
Volpp KG et al. JAMA; 2007: 298 (9): 984-992.
How big were these effects?
Medical patients: Improvement in mortality from pre-1 to post-2 of 0.70
percentage points (11.1%) for hospitals in 75th compared to 25th percentile
Volpp KG et al. JAMA; 2007: 298 (9): 984-992.
VA hospitals much more teaching intensive
•
Medicare
•
VA Hospitals
Volpp KG et al. JAMA; 2007: 298 (9): 975-983; Volpp KG et al. JAMA 2007; 298(9): 984-992.
Do effects of reform on mortality vary
across hospitals (Medicare)?
Why no improvement in quality among
Medicare patients?
• Design flaws
• 30 hour shifts allow acute sleep deprivation
• Current design does not respect circadian rhythms
• Sleep inertia at night when paged
• Implementation
• Compliance likely incomplete; may be worse than in
VA hospitals, given higher work intensity
• Offsetting factors
• Worsened continuity
• Higher work intensity
• Sicker patients
Why improvement in some
groups but not others?
• VA vs. Medicare
• VAs more teaching intensive (“dose response”)
• Better information systems may have mitigated
•
some of the continuity of care (hand-off) problems
Confounding due to other changes
• Medical vs. surgical
• Differences in balance between reduction in fatigue
•
•
and continuity?
Differences in compliance?
Differences in effort to address discontinuity
through structured sign-out, increased attending
involvement?
Failure to Rescue:
Death among surgical patients with
potentially treatable complications
Highest risk patients fared no differently than
lower risk patients - Medicare
Resident/bed ratio  post-reform
year 1 * top 10%/25% of severity
Odds ratio† (95% CI)
Resident/bed ratio  post-reform
year 2 * top 10%/25% of severity
P-value
Odds ratio† (95% CI)
P-value
0.86
0.90
(0.80, 1.02)
0.09
0.81
0.94
(0.85, 1.03)
0.17
0.18
1.01
(0.88, 1.15)
0.88
0.76
1.09
(0.95, 1.24)
0.21
0.40
1.01
(0.86, 1.18)
0.92
0.10
1.00
(0.88, 1.14)
0.98
Medicare
Combined medical
Highest 10%
(vs. bottom 90%)
1.01
(0.90, 1.13)
Highest 25%
(vs. bottom 75%)
0.99
(0.98, 1.08)
Combined surgical
Highest 10%
(vs. bottom 90%)
0.91
(0.80, 1.04)
Highest 25%
(vs. bottom 75%)
0.98
(0.86, 1.12)
Failure to rescue
Highest 10%
(vs. bottom 90%)
0.94
(0.80, 1.09)
Highest 25%
(vs. bottom 75%)
0.90
(0.79, 1.02)
Volpp KG et al. JGIM 2009. In Press.
Highest risk patients fared no differently than
lower risk patients - VA
Resident/bed ratio  post-reform
year 1 * top 10%/25% of severity
Odds ratio† (95% CI)
Resident/bed ratio  post-reform
year 2 * top 10%/25% of severity
P-value
Odds ratio† (95% CI)
P-value
0.02
1.35
(0.88, 2.07)
0.17
0.045
0.99
(0.67, 1.43)
0.93
0.19
0.80
(0.45, 1.43)
0.45
0.04
1.13
(0.59, 2.17)
0.71
0.23
0.64
(0.33, 1.24)
0.19
0.60
0.82
(0.46, 1.48)
0.51
Medicare
Combined medical
Highest 10%
(vs. bottom 90%)
1.63
(1.08, 2.46)
Highest 25%
(vs. bottom 75%)
1.44
(1.01, 2.05)
Combined surgical
Highest 10%
(vs. bottom 90%)
0.68
(0.39, 1.20)
Highest 25%
(vs. bottom 75%)
0.52
(0.29, 0.96)
Failure to rescue
Highest 10%
(vs. bottom 90%)
0.67
(0.35, 1.30)
Highest 25%
(vs. bottom 75%)
0.86
(0.49. 1.51)
Volpp KG et al. JGIM 2009. In Press.
The concept of prolonged stays
The rate of prolonged stays varies little over time
Medicare Combined Medical Patients
VA Combined Medical Patients
80%
Percent Prolonged
Percent Prolonged
80%
60%
60%
40%
40%
pre-3
pre-2
pre-1
post-1
pre-3
post-2
pre-2
post-1
post -2
Year
Year
VA Combined Surgical Patients
Medicare Combined Surgical Patients
80%
Percent Prolonged
80%
Percent Prolonged
pre-1
60%
60%
40%
40%
pre-3
pre-2
pre-1
post-1
pre-3
post-2
pre-2
pre-1
post-1
post -2
Year
Year
Non-teaching (0)
Very Minor/Minor (>0 & <.25)
Major (>0.25& <0.6)
Very Major (>0.6)
Silber et al. 2009.
Medical Care
Odds of prolonged stay change at similar rates in
more vs. less teaching intensive hospitals
Patient categories
(Number of Cases
Medicare/VA)
Medical Conditions
RB ratio  post-reform year 1
OR (95%CI)
RB ratio  post-reform year 2
OR (95% CI)
Medicare
VA
Medicare
VA
Stroke (933,225/25,385)
1.01 (0.92, 1.10)
0.92 (0.66, 1.27)
1.01 (0.92, 1.10)
0.95 (0.69, 1.31)
AMI (970,184/32,170)
1.01 (0.93, 1.10)
0.96 (0.72, 1.29)
1.06 (0.97, 1.15)
0.96 (0.72, 1.28)
GI bleed
(763,765/36,035)
1.06 (0.97, 1.16)
1.26 (1.00, 1.58) a
1.09 (1.00, 1.20)
1.08 (0.86, 1.36)
CHF (1,196,294/50,266)
0.99 (0.92, 1.06)
1.11 (0.92, 1.35)
1.02 (0.95, 1.10)
1.18 (0.97, 1.43)
Combined Medical
(3,863,468/143,856)
1.01 (0.97, 1.05)
1.07 (0.94, 1.20)
1.04 (0.99, 1.08)
1.05 (0.93, 1.19)
General Surgery
(651,515/22,482)
1.09 (0.99, 1.21)
1.07 (0.79, 1.43)
0.94 (0.85, 1.05)
1.02 (0.76, 1.36)
Orthopedic Surgery
(1,364,559/32,719)
1.03 (0.96, 1.10)
0.82 (0.61, 1.12)
0.94 (0.88, 1.01)
1.04 (0.77, 1.41)
Vascular Surgery
(179,473/11,219)
1.16 (1.00, 1.34)
1.08 (0.66, 1.77)
1.21 (1.04, 1.40) a
1.16 (0.71, 1.91)
Combined Surgical
(2,195,547/66,420)
1.04 (0.98, 1.09)
0.94 (0.78, 1.14)
0.96 (0.91, 1.01)
1.00 (0.83, 1.21)
Surgical Conditions
ap<0.05 bp<0.01 cp<0.001
Silber et al. Medical Care 2009.
AHRQ Patient Safety Indicators
•
•
•
Technical composite
•
•
•
•
•
iatrogenic pneumothorax
foreign body left in during procedure
postoperative wound dehiscence
accidental puncture or laceration
postoperative hemorrhage or hematoma
Continuity of Care composite
•
•
•
postoperative physiologic or metabolic derangement
postoperative pulmonary embolism or deep vein thrombosis
(PE/DVT)
postoperative sepsis
Collaborative Care composite
•
•
•
postoperative hip fracture
postoperative respiratory failure
selected infections due to medical care
PSI composite rates change at similar rates
in hospitals of different teaching intensity
2.5
MEDICARE CONTINUITY OF CARE PSI COMPOSITE UNADJUSTED RATES
BY ACACEMIC YEAR AND TEACHING INTENSITY
MEDICARE TECHNICAL PSI COMPOSITE UNADJUSTED RATES
BY ACACEMIC YEAR AND TEACHING INTENSITY
2.5
Rates (%)
2
1.5
1
0.5
1.5
1
0.5
0
0
Pre3
Pre2
Pre1
Post1
Post2
Pre3
Academic Year
Pre2
Pre1
Post1
Post2
Academic Year
MEDICARE COLLABORATIVE CARE PSI COMPOSITE UNADJUSTED
RATES BY ACADEMIC YEAR AND TEACHING INTENSITY
2.5
2
Rates (%)
Rates (%)
2
1.5
1
0.5
Rosen et al. Medical Care.
2009. In Press.
0
Pre3
Pre2
Pre1
Academic Year
Post1
Post2
Odds of experiencing a PSI generally
changed at similar rates in more vs.
less teaching intensive hospitals
Technical Care
Composite
Continuity of Care
Composite
Collaborative Care
Composite
VA
Odds Ratio
(95% CI)
Medicare
Odds Ratio
(95% CI)
VA
Odds Ratio
(95% CI)
Medicare
Odds Ratio
(95% CI)
VA
Odds Ratio
(95% CI)
Medicare
Odds Ratio
(95% CI)
Resident/bed
ratio*post1 a
1.09
(0.78 - 1.51)
P= 0.62
1.15
(1.04 – 1.27)
P=0.01
1.01
(0.70 - 1.46)
P= 0.95
1.02
( 0.94 -1.11)
P = 0.66
1.18
(0.75 - 1.85)
P= 0.48
0.98
( 0.87- 1.11)
P= 0.80
Resident/bed
ratio*post2 a
1.05
(0.75 -1.45)
P= 0.79
1.09
(0.99 – 1.21)
P=0.09
1.39
(0.97 - 1.99)
P= 0.08
1.08
( 0.99 - 1.17)
P = 0.08
1.60
(1.01 - 2.53)
P= 0.04
1.00
(0.89 -1.14)
P= 0.97
Number of
cases
795,306
12,426,475
339,504
7,669,946
653,270
11,295,527
Rosen et al. Medical Care. 2009. In Press.
Results Summary
• Good news?
• No evidence of worsening of outcomes for a broad
•
•
range of measures within either Medicare or VA
No evidence of harm (or benefit) for high-risk
patients
Question about prolonged length of stay for
vascular surgery patients
• Bad news?
• No evidence of significant relative improvements
in outcomes except for medical patients in VA in
post-reform year 2
Institute of Medicine 2008 report
The Institute of Medicine formed a consensus
committee to:
1) synthesize current evidence on medical resident
schedules and healthcare safety.
2) develop strategies to enable optimization of work
schedules to improve safety in the healthcare work
environment. The strategies recommended will take
into account the learning and experience that
residents must achieve during their training. The
recommendations will be structured to optimize both
the quality of care and the educational objectives.
What do we not know?
•
•
•
•
•
•
•
•
•
Hours per week residents are actually working
How much more sleep residents are actually getting
Impact on broader range of clinical outcomes
Longer-term impact on clinical outcomes
Impact on educational outcomes
How residents are spending their time
What approaches have helped programs successfully
adapt
Comparative effectiveness and cost effectiveness of
different approaches
Role of hospital finances
IOM Report on Resident Work Hours. 2008.
IOM Recommendations
•
•
“Safe transportation options”
•
“Adequate time to conduct thorough evaluations of
patients and for reflective learning…”
•
•
•
Minimize work “that is of limited or no educational value,
is extraneous to their program’s goals and objectives,
and can be done well by others”
Specialty-specific, RRC-set limits on caseload
Supervisory physician (resident) in house at all times
Schedule overlap time and facilitate safe handoffs
IOM Recommendations
• 80 hour week
• 5 hour mandatory nap between 10pm-8am if
overnight shifts used
• Averaging of days off not allowed; 1 day per
week and 5 days per month
• All moonlighting counted against limits
• 10 hours off after day shift; 12 hours off after
night shift; 14 hours off after extended shift
Design of R01 HL094593
New study focuses on educational
and clinical impact (NHLBI)
•
To describe the variety and frequency of program-level
behavioral responses to duty hour reform and resident work
conditions
•
•
•
•
•
qualitative field work at a sample of IM and GS residency
programs
Mixed-method approach of direct observation and interviews
12 hospitals placed in a 2x2 matrix of large versus small size
and good versus poor financial balance sheets in FY 2008 by
assessing operating margins and fund balances over the
previous 5 years
Direct observation of resident involvement in provision of
hospital care (especially rounds, hand-offs)
Semi-structured 1 on 1 interviews with open ended questions
involving residents, nurses, attending physicians, and
administrators
New study focuses on educational
and clinical impact (NHLBI)
•
To describe the variety and frequency of program-level
behavioral responses to duty hour reform and resident work
conditions
•
•
•
•
national surveys of program directors and residents in Internal
Medicine (IM) and General Surgery (GS)
Partner with the ABIM, ABS, APDIM, APDS, and ACP
Surveys of residents to focus on resident-specific issues such
as balance between service delivery and education,
assessment of work intensity, use of free time, how handoffs
are done, actual hours worked and days off, and hours slept.
Surveys of program directors to focus on use of non-teaching
services, hiring hospitalists or physician extenders, work
intensity (admissions per resident, # patients covered, hours
worked, days off), training (if any) in how to do ‘handoffs’,
helpful or problematic attributes of work environment.
New study focuses on educational
and clinical impact (NHLBI)
•
•
•
To assess how educational outcomes (board scores) have
changed with duty hour reform for residents in different
specialties.
To examine how clinical outcomes (mortality, FTR, PLOS,
PSIs) have changed beyond the first two years post-duty hour
reform.
To examine how pre-reform hospital financial health and
staffing levels predicted changes in staffing and educational
and clinical outcomes.
Variables
Comparison across specialties
Questions and discussion