Transcript July 2006: MS Sydney presentation:

Evidence into Practice
RCT: Effectiveness of rehabilitation in MS
Rehabilitation research going beyond the Holy
Grail of the RCT
Fary Khan
Department of Rehabilitation Medicine,
Royal Melbourne Hospital & The University of Melbourne
1
2
Outline
1.
RPC MS Rehabilitation program
2.
MS Complexity
3.
EBM - in MS Rehabilitation research
4.
RCT RPC
5.
New approaches to EBM - Observational studies
6.
Current research- PCT & Rehabilitation efficiency
7.
Future Direction
3
1. RPC- Centre for Excellence for MS
Rehabilitation
• 4000 pwMS in VIC, 1200 RMH MS Database, >250 RPC
• State wide service – Assessment & Referral
MD Rehabilitation IP 3-6 / 21 beds
Ambulatory services 3-7 slots
• Life Moves Program RPC- peer support group
• Wheelchair seating clinics RPC (3/12 commenced 2008)
• Close links - MSSA, pt advocacy gps & DHS programs, YACDAC
& equipment & accommodation
4
MS - Major health issue
• MS expected to grow 6.7% in the next 5 years, faster than
population growth due to demographic aging
• 3rd most common cause of disability in young adults (Dombovy
1991) - 74% women & 87% were of working age (MSSA 2005)
• Lifelong, fluctuating disability, usually progressive
• More frequent need for evaluation c/w other neurological
conditions (Brown 2005)
5
Major Health Issue - Costs
(Access Econ 2005, ABS 2004)
• Financial costs - $600 million (0.07% of the GDP) - $37,333/ pwMS
o
Informal care community - av 12.3 hours /week/person - $257.7m
o
Lost productive capacity (reduced work hours, early retirement) - $158m
o
Nursing home & related costs > $A25 million
• Financial & disease burden of MS expected cost $2 billion/
year
• The burden of disease suffering & premature death - estimated
cost an additional 8,968 DALYs - with 2/3 due to disability & 1/3
premature death
6
2. Complexity in MS
• Unpredictable course
• Heterogeneous patient population
• Many combinations of impairments, activity limitations &
participation
• Currently no tools fully capture these complex constructs
7
Complexity in MS Rehabilitation
• Intensity, setting & type of rehabilitation treatment
• Combination of modalities, therapist interaction
• Timing of rehabilitation
8
9
3. Evidence base for MS Rehabilitation
Main tools for EBM
• RCTs
• Meta- analysis
10
Multidisciplinary Rehabilitation for adults
with Multiple Sclerosis (Review)
Khan F,Turner Stokes L, Ng L,Kilpatrick T
The Cochrane Database of Systematic Reviews 2007
Issue 2. Art No: CD006036.DOI: 10.1002/14651858
11
Main results
7 RCTs scored well & 1 CCT scored poorly on methodological quality
assessment
• Strong evidence - IP: LL mobility, transfers & participation
• Moderate evidence - OP: mobility & transfers, self-care,
sphincter control, Qol, SE, HP behaviours & employment
• Moderate evidence - RITH: QoL (some domains)
12
Main results
• Not possible to suggest best 'dose' of therapy or supremacy of
one therapy over another
• No evidence for long term cost effectiveness of these programs
13
Future recommendations
• High quality RCTs & other designs
• Effectiveness of specific rehabilitation interventions
o Components
o Intensity
o Settings
o Cost effectiveness
14
Limitations of RCTs in Rehabilitation
• Lack of description & standardization of input
• Variation in location & duration of rehabilitation input
• Reluctance to use a control group
• Absence of blinding
• Lack of independent assessors
• Limited, inappropriate outcome measures
15
Methodological difficulties
included RCTs /CCTs
• 7 of 8 studies used randomization procedures
• Only 1 study - concealed allocation & blinded outcome
assessors
• All studies - inadequate patient blinding & blinded care
providers
16
Study Storr et al 2006
Double blind design to assess short term MD IP rehab c/w control
group - but reported no significant difference
• Under powered
• Confounding factors included variation in indication for
treatment
• Reliability ? & responsive outcome measures
17
18
4. MS RCT
Effectiveness of Rehabilitation intervention in
persons with Multiple sclerosis: A
randomized controlled study
F Khan, J Pallant, C Brand, TJ Kilpatrick
Journal of Neurology, Neurosurgery and Psychiatry
June 5 2008, DOI:10:1136.133777
19
RCT (n=101)
Randomized, stratified - waitlist control,
treatment groups
Inclusion
•
•
•
age 18 -65 years
definite MS (McDonald criteria)
known limitations in neurological status (EDSS - mobility 2-8, & cog KFS 0-2)
•
those active & mobile in the community
Exclusion
•
greater cognitive deficits (> than 2 on KFS)
•
relapse in the 3 months prior to recruitment
•
or had rehabilitation in the six months before admission
•
those bed bound and/ or institutionalized
20
RCT (n=101)
Randomized, stratified - waitlist control,
treatment groups
Concealed allocation, blinded treatment providers & outcome
assessors - with follow up after 12 months
Initially double blinded but participants unblinded at 8 months in the study!!
Outcomes based on the WHO & domains of ICF
• Disability (FIM)
• Participation (MSIS29, GHQ28, AQoL)
21
Study power
• Primary outcome - defined as the impact of the rehabilitation
program on disability - FIM (m)
• For an 80% chance to detect a 5 point difference in FIM from
baseline to 12 month in intervention versus control groups
(assuming SD for change is 8.5 & similar in both gps)
• 46 patients in each group were needed for recruitment
(estimate based on 2-sided alpha = 0.05)
22
Study Flow chart
Assessed for eligibility (n=204)
Excluded (n = 103)
Allocated to intervention (control)
(n = 52)
Received allocation intervention
(n = 40)
Did not receive allocated intervention
(n = 12)
Deteriorated requiring treatment (n=12)
Reasons:
Worsening spasticity=4
Severe pain=2
Incontinence management=4
Psychosocial reasons=2
Enrolment (n=101)
Randomised
Allocation
Not meeting inclusion criteria
(n= 1)
Refused to participate
(n = 91)
Other reasons:
Responded late n=2
Away n=4
Acute exac n=3
Relocated n=2
Allocated to intervention
(n = 49)
Received allocation intervention
(n =49)
Did not receive allocated intervention
(n = 1) withdrew
Reasons: ‘did not believe he had MS’
Lost to follow–up (n = 2)
Reason: Died (of malignancy) =1
Discontinued intervention (n = 1)
Reasons: Refused to participate ‘see no
benefit’ =1
Analysed (n = 50)
Excluded from analysis (n =0)
Follow-up
Analysis
Lost to follow – up (n = 0)
Analysed (n = 48)
Excluded from analysis (n = 0)
23
RCT (n=101)
Statistics
• ANCOVA - compare post treatment scores for control & treatment
gps
• MANCOVA for 2 analysis - individual FIM (m) scores - inclusion
of multiple dependent variables
• Change scores for both gps (t tests), ES (Cohen’s d) (Kazis 1989)
• % both gps - improved, remain same or deteriorated (comparison
- χ 2)
24
Results
Comparable baseline characteristics
Variable
Randomized to
Treatment
group (n=49)
Randomized to
Control group
(n=52)
Received
treatment
(N=61)
Did not receive
treatment
(N=40)
N (%) female
31 (63.3%)
41 (78.8%)
40 (65.6%)
32 (80%)
Mean age in yrs (SD)
49.5 (8.64)
51.1 (9.66)
49.7 (8.96)
51.2 (9.51)
Relapsing Remitting
13 (26.5%)
18 (34.6%)
17 (27.9%)
14 (35%)
Secondary Progressive
29 (59.2%)
27 (51.9%)
36 (59%)
20 (50%)
Primary Progressive
7 (14.3%)
7 (13.5%)
8 (13.1%)
6 (15%)
10.69 (6.33)
9.73 (7.99)
10.52 (6.61)
9.7 (8.11)
7 (14.3%)
12 (23.1%)
8 (13.1%)
11 (27.5%)
3.5 to 6.0
27 (55.1%)
32 (61.5%)
36 (59%)
23 (57.5%)
6.5+
15 (30.6%)
8 (15.4%)
17 (27.9%)
6 (15%)
Disease
Years since diagnosis:
Mean (SD)
EDSS
0 to 3
25
Results
• One way ANCOVA comparing post t/m FIM (m) scores for both
groups (with baseline score as covariate)
Highly significant [F(1,91)=10.95, p<.001]
Large ES = 0 .17
(Cohens 1988)
• MANCOVA - FIM (m) scores
Significant differences b/w control & t/m (p<.01) - locomotion,
transfers, self care & sphincter control
26
Change scores for treatment & control groups for
activity & participation
Scale
Treatment
Control
n
M (SD)
n
M (SD)
Mean
difference
95% CI for
mean diff
P
ES
FIM Motor
48
-3.0 (4.23)
46
1.78 (4.25)
4.78
3.04 to 6.52
<.001
1.13
Transfers
48
-.75 (1.28)
48
.50 (1.13)
1.25
.76 to 1.74
<.001
1.04
Locomotion
48
-.63 (1.28)
47
.21 (1.16)
.25
.34 to 1.34
.001
.69
Sphincter
48
-.33 (.91)
47
.02 (1.28)
.36
-.10 to .81
.12
.32
Self care
48
-1.29 (2.05)
47
.92 (2.56)
2.21
1.27 to 3.15
<.001
.95
Psychological
46
.11 (7.7)
46
-1.09 (6.5)
-1.20
-4.16 to 1.77
.43
.44
Physical
46
-1.13 (16.9)
46
2.30 (12.68)
3.44
-2.76 to 9.63
.27
.23
MSIS
27
RCT Results
Improved
Deteriorated
Treatment
(n=48)
34 (70%)
8 (16%)
Control
(n=46)
6 (13%)
27 (58%)
χ 2 = 32.5, df=2,
p<.001
28
29
5. New approaches EB in Rehabilitation
observational studies
Going beyond the Holy Grail of the RCT
RCTs - Evidence for overall effectiveness of rehabilitation but they
cannot answer all the questions that need to be answered (Whyte 2002)
•
Confirm new &/or current Interventions & practices
•
Not to the discovery of more effective/efficient interventions & practices
•
Address:
Does investigational t/m cause an effect?
How and why does the intervention work?
Designed to maximize the chance that some effect of new or existing
t/m will be revealed by the study
30
Practice Based Evidence (PBE)
• Prospective data collected systematically in routine clinical
practice - additional information (Horn 2007)
o
assist in understanding the nature of services provided
o
outcomes & service implications for pwMS
• PBE can address critical questions (De Jong 2005)
o
? which patients have the most to gain
o
? models & intensity of rehabilitation input are likely to be most effective
31
Practice Clinical Trials (PCTs)
(Tunis JAMA 2004, Berguer 2004)
• Select clinically relevant alternative interventions to compare
• Include diverse population of study participants
• Recruit patients from heterogeneous practice settings
• Collect data on a broad range of health outcomes
32
Variables (Comparison) RCT
CPI
Patient Variable
Patient eligibility and stratification
factors. Eliminate patients who could bias
results: comorbidities, more serious
disease, etc
About 10 – 15% of patients qualify
Patient eligibility and stratification
factors. Use severity of illness to measure
comorbidities and disease severity.
All patient quality by measuring patient
differences; none excluded
Process Variables
Treatment protocol
Specify explicitly every important
element of the process of care for both
treatment and control arms. Informed
consent
Measure or record all treatments and
interventions.
Abstract information from charts based
on existing practice. Informed consent
often not needed*
Outcome Variables
Powered for primary outcome
Change based on evidence
Many outcomes assessed
Improvement based on evidence
Measurements /
documentation
Limited number of patient variables,
treatments, outcomes measured. Variables
specified precisely for all patient,
treatment, and outcome measures
Comprehensive holistic framework.
Variables specified precisely for all
patient, treatment, and outcome measures
Database Result
Limited to the variables needed. Efficacy.
Assigned causality.
Comprehensive and detailed.
Effectiveness. Associated and assumed
causality.
Hypotheses
Typical 1 hypothesis. Clearly defined at
start
Narrow and focused
Typically many hypotheses. Many and
board at the start. Refined and new
hypothesis generated by analytic findings.
Local Knowledge
Not dependent on local knowledge
Depends on local knowledge; entails
participation by practicing clinicians
Confounders
Assumed not relevant to study or
outcome
Affect outcomes and are relevant to
include
33
Clinical Practice Improvement (CPI)
method - variant of PCT
CPI address practical questions about
• Risks, benefits & costs of an intervention as they would occur in
routine clinical practice
• Harnesses pt complexity & t/m differences - offers a view of what
actually happens in the care process
• Does t/m work in the real world of everyday practice?
• For whom does the intervention work best?
CPI method may establish standards for best practice (Horn 2005)
34
CPI observational study design
(Horn 2005)
Measurement encompasses a comprehensive view of
the care management process:
• Key patient characteristics
• All treatment & care processes
• Outcomes
35
Controlled Versus Naturalistic Studies (Horn 2005)
Improve or standardize
Process Factors
•Care management strategies
•Treatments & interventions
• Medications
Controlling for
Patient Factors
• Demographic & psychosocial characteristics
• Health conditions, impairments
Measure
Outcomes
•Health status
•Functional status
•Cost
•LOS
•Encounters
• Severity of illness or condition
•Physiologic signs & symptoms
•Functional Status
(Evaluate at multiple points in time)
36
37
6. Current Research
Efficiency of MS Rehabilitation
• CPI process -’pilot’ to quantify the ‘black box’ of IP Rehabilitation
Complexity, dependency & efficiency (submission November 2008)
Significant functional gains made in rehabilitation
• Report using the AROC – State of MS rehabilitation in Australia
(11,000 episodes of MS care) (October 2008)
Limited / under resourced/ disorganized Rehabilitation services
38
Functional improvement of the Australian
Health care system (New, MJA 2008)
• Current focus - strategies for managing  health system demand
acute sector & Ch disease Mx in the community
Little attention on the role of rehabilitation
• In 2006 53,000 IP rehab episodes in Australia
Improved patient flow through acute – if early involvement of
rehabilitation services
• Improved effectiveness of rehabilitation
Increased intensity of rehabilitation
Development of rehab models - provide alternatives to IP care
39
Factors that reduce efficiency of
rehabilitation services
(New, MJA 2008)
• Location of small stand alone facilities without acute support
• Lack of options for managing younger people with acquired
disability in the community
• Inadequate government programs for the supply of aids,
equipment & home modifications
40
Viewpoint
Improving the organization of rehabilitation services
should improve access to acute & rehabilitation IP beds,
improve patient outcomes & reduce costs
RMH Rehabilitation Redesign 2008

Evidence into practice for MS Rehabilitation
41
7. Future Direction
• Development of outcome measures using the WHO
ICF for MS rehabilitation (submission Nov 08)
• Aging in MS - Change in disability in pwMS: A
prospective population based analysis
42
Questions?
43