Transcript Mixing Efficacy and Safety Outcomes

Top Trial Cheats
Dr Rod Stables
The Cardiothoracic Centre
Liverpool UK
No Conflicts of Interest to Declare
Presentation Outline
• Examples without accusation
• Three topic areas
• Establishing the denominators
• Lost or not evaluated at follow-up
• Odds ratio v Risk ratios
• Composite outcome measures
• Mixing safety and efficacy measures
Identify the Denominator: Tracking Trial Subjects
• Need to follow the fate of ALL patients
• From inclusion to outcomes
• Warning signs !
• Results expressed as simple %age
• Smokescreen phrases :
• ‘In patients with outcome data …’
• ‘In evaluable (surviving) patients …’
• ‘When assessed at follow-up …’
Identify the Denominator: Examples
• Rickards and Stables CABRI long term follow-up
• Widely presented (but never published)
• Every result presented as a % value
• Masked incomplete follow-up
• No data on >120 Dutch patients
• More sinister impact - reported outcome measures
Not Assessed for the Outcome Measure ?
• Trial of drug A - exercise capacity in heart failure
• 100 patients randomised 1 : 1 drug v placebo
• Outcome measure
• Mean exercise duration at FUP
• Baseline exercise duration - equal
• Follow up Drug 3.2 mins
Placebo 2.4 mins
• Clinical and statistical significance
• Mechanism ?
Not Assessed for the Outcome Measure ?
• Widespread implications
• Incomplete QCA follow up in angiographic trials
• Numerical handling of extreme examples
• QCA - MLD of occluded vessel
? = 0
• QoL - score of dead patient = 0 (?excluded)
• Competing outcomes
• ↑ mortality = ↓ repeat revascularisation
• Event curve analysis - No of patients at risk
SoS Trial: Repeat Revasc Event Curve
BASIL Trial : Amputation - free Survival
Solutions ?
• Clear exposition of patient flows
Trial Profile and Patient Flow Diagram
Solutions ?
• Clear exposition of patient flows
• All event rates expressed: n / d (p%) (CIs)
• Describe status of patients not assessed for
outcome measures
• Express rates with correct denominators eg
• Repeat revasc rate per 100 patient years of Fup
• Full description of complex data handling methods
• QCA
QoL etc
Odds and Risk
Odds and Risk
• Throw one of a pair of dice - Aim - To get a 6
• Odds against = 5:1
(Odds)
• Event occurs 1:6 of all throws (Risk)
• Scope for confusion and manipulation of data
• ……. and audience
• Most observers interpret odds ratio as relative risk
Example: Relative Risk in a Classic RCT
• Randomised trial
Tx A v Tx B
• Each group comprises 100 patients
• Death rate on Tx A = 50/100
(= 0.5)
• Death rate on Tx B = 25/100
(= 0.25)
• Relative risk = 0.5 / 0.25 = 2
• (p values and confidence intervals)
Odds Ratio Calculation
• Tx A: No dead = 50 (a) No alive = 50 (c)
• Tx B: No dead = 25 (b) No alive = 75 (d)
• Odds ratio = ad/bc = (50 x 75)/(25 x 50)
• = 3
• Sounds more !!
From the Thesis of Dr R H Stables DM (Oxon) …..
Table 5.6 shows the proportion of PTCA patients in each of the two consultant
groupings. In the interventionist group, 15 / 27 (56%) patients were recommended
PTCA compared to 12 / 47 (25.5%) patients in the non-interventionist group. The
odds ratio for these proportions is 3.7 (95% CI, 1.3 – 9.9) : p = 0.01.
Table
5.6 Proportion of PTCA Patients in Each of the two Consultant Groupings
Tx With PTCA
I
NI
p Value
OR (95% CI)
15/27 (56%)
12/47 (25.5%)
0.01
3.7 (1.3 - 9.9)
Relative Risk and Odds Ratio
• If event rate is low OR is approx = RR
• If OR is interpreted as RR it will overstate the risk (benefit)
1
Odds Ratio
0.9
Risk Ratio
Reported Ratio
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
1
3
5
7
9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49
Event Rate %
Some Basic Rules
• Odds ratio correct in 2 x settings
• Backward investigation - Case control
Case Control Studies
• Breast cancer patients - series of 200
• History of OCP = 50 (a) No OCP = 150 (c)
• Odds = a/c = 0.33
• Age sex matched controls - series of 200
• History of OCP = 25 (b) No OCP = 175 (d)
• Odds = b/d = 0.14
• Odds Ratio = (a/c) / (b/d) = a/c x d/b
• Odds ratio = ad/bc = (50 x 175)/(150 x 25) = 2.3
Some Basic Rules
• Odds ratio correct in 2 x settings
• Backward investigation - Case control
• Multiple regression analyses
• Report independent association strength
• factor and outcome
• In all others - RCT and cohort studies
• Risk ratio applicable and best
Mixing Efficacy and Safety Outcomes
Mixing Efficacy and Safety Outcomes
• Developing area - Subject of continued debate
• CVA addition to MACE in revascularisation trials
• PCI trials - MACE outcomes
• ? Add in bleeding complications
• Usually PCI related (puncture ± adjunct drugs)
• Seems reasonable ?
• Value in trials of antithrombotic therapy ?
Mixing Efficacy and Safety Outcomes
• Cloud and confuse :
• Data analysis and reporting
• Procedural development
• BENESTENT 1
• Stent puncture site complications 7% !!
• Unrelated to coronary efficacy of stent
• Reduced to current levels - changes in
• Drugs devices techniques - experience
Mixing Efficacy and Safety Outcomes
• Efficacy and safety issues
• Parallel tracks - causation and solution
• Further complicate analysis and reporting
• Minefield of composite outcomes
• Perhaps best avoided
Questions and Discussion