Transcript Document

1 Study Design: experiments
Basic Medical Statistics Course
October 2010
W. Heemsbergen
Experiment
Experimental Research vs. Observational (Non-Experimental) Research:
manipulation / randomization versus observation.
In an experiment, the investigator controls conditions.
Treatment is allocated according to a procedure (randomization).
Experimental unit can be: cell, tissue, animal, test subject, patient.
1
Example: memory performance
Nature 447, 151-152 (10 May 2007)
2
Example: meta development
Clin Cancer Research 2009;15:22.
3
Example: vaccines against flu
Test vaccine A, B, C, against flu among childeren.
Experimental units: 600 childeren from 3 schools.
Bad experiment
A. September Vac A, November Vac B, January Vac C.
B. Grade 1-3 Vac A, Grade 4-6 Vac B, Grade 7-8 Vac C.
C. School 1 Vac A, School 2 Vac B, School 3 Vac C.
Good experiment ?
A. Vac A: School 1 Group 1-3, School 2 Group 4-6, School 3
Group 7-8. etc. for Vac B, C
B. Childeren in alphabetical order: first 200 Vac A, second 200
Vac B, last 200 Vac C.
4
Randomize !
The investigator can carefully balance the factors that seem
important. However, one can never be sure that the treatment
groups do not differ in some (unknown / not registered) factor
which is also important but ignored.
Therefore the allocation should incorporate
an element of randomization.
5
Randomized Clinical Trial (RCT)
RCT is a comparative, prospective study performed with human subjects to test
new drugs or new combinations of drugs, new approaches to surgery or
radiotherapy or new procedures to improve the outcome of a disease and/or the
quality of life of the patient.
An RCT is the only experiment with patients in medicine that is allowed, under
certain (ethical) conditions. It is only possible in a certain time window in which it is
not known (with reasonable certainty) which treatment option is optimal. One can
not randomize patients between medicine A or B, when medicine A or B is already
broadly recognized as the best option.
6
Patients who fulfill inclusion criteria are candidate for a trial
(e.g. T1-4N0M0 prostate cancer receiving primary radiotherapy)
Exclusion criteria are checked
(e.g. < 85 years, no prior malignancy, ECOG performance 0-2)
Patient is informed about the study and decides whether he will
participate. If yes, patient signs Informed Consent.
Patient is randomized for the trial.
7
Hypothesis
Possible hypothesis in a RCT (comparing experiment - standard):
Superiority trial
Goal (positive outcome): to show that the new treatment has a clear
beneficial effect and is superior to standard treatment.
When the new treatment is neither clearly superior nor clearly inferior to
standard treatment, the outcome is inconclusive.
Non-inferior trial / Equivalence trial
Goal: to show that the new treatment is equivalent to standard treatment.
Example: a new treatment is less toxic or cheaper.
8
Clinical trials
time-consuming, expensive, much effort
9
Terms in Clinical Trials
Blind (single-/double-) vs. open label. Blinding of patients, outcome
assessors, to the given treatment. It improves the validity of the research. It
is often not possible or feasible.
Randomization Blocks. Randomization is often in blocks, to make sure that
the no. of patients in each arm remains equal.
Stratification. Stratification makes sure that important (prognostic) factors are
balanced over the treatment groups. Also stratification for center is often
applied.
Declaration of Helsinki. A list of ethical principles for research in humans.
Good Clinical Practice. (Obligatory) guidelines to prepare & execute clinical
research.
Ethics Committee. Every study must be approved before initiation.
Data Safety and Monitoring Committee. For a confidential interim analysis: is
treatment safe. They can decide to stop the trial.
10
Drug development
Phase I
First test in human subjects (small group of healthy volunteers).
Main goal is to confirm the safety of the study drug / treatment.
Phase II
To determine toxicity / efficacy / optimal dose, in a limited patient group.
Design: case series or a limited comparative trial.
Phase III
Randomized controlled (multicenter) comparative trial in large patient groups.
Control group: standard treatment, placebo.
Main goal is to assess the efficacy of the drug / treatment.
Phase IV
Post Marketing Surveillance (observational). Main goal: safety surveillance
(long-term effects, interactions other drugs, larger patient groups).
11
http://www.consort-statement.org/
CONSORT: a group that tries to improve the quality of reports of clinical trials.
12
Example
13
Questions
Why exclude patients with bad health ?
Why exclude patients with prior tumors ? (both often the case in
clinical trials)
Do the results of the trial apply then as well to patients with prior
tumors, and bad health ?
Does it matter whether 10% or 40% of the initial population is
excluded by a health criterium ?
What should the doctor do in case of a patient with bad health,
should the new, promising medicine be prescribed?
14
Type of Comparisons
A control treatment is essential for comparison, to estimate the part of
the response that can be attributed to the new treatment.
Options are:
- Another treatment (standard treatment).
- A placebo. A placebo looks and tastes like the new drug but contains no
active
compound.
- No treatment. (e.g. physiotherapy exercises vs. no exercises).
Intention-to-Treat. When a patient is randomized to a treatment, it should be
analysed within this treatment group, even when the patient did not complete
treatment or switched to another treatment group.
Questions: - Should a placebo-controlled study be double-blind, by definition ?
- Can the intention-to-treat principle obscure treatment effects ?
15
Endpoints (Cancer) Trials
 A study endpoint is a clearly defined outcome for an individual.
 It should relate to the hypothesis and have clinical/biological relevance.
 A study protocol usually describes “primary” and “secondary” endpoints.
 Study endpoints can be clinical (death, onset fever) or based on surrogate
measurements (eg. tumor marker).
 An endpoint can be a composite endpoint. Example: the endpoint is
reached when one of the following occurs: myocardial infarction, stroke or
death related to cardiovascular disease.
 In cancer research, the (clinical) recurrence of disease: local, regional,
and/or distant (meta) is often a primary endpoint.
 Other relevant (secondary) endpoints are: death, toxicities, quality of life, ….
 Combining endpoints: locoregional control, disease free survival, …
16
Endpoints & Measurements
 Measurements are crucial and various: images, blood values, anamnesis,
questionnaires.
 The frequency and the number/type of measurements can influence the
number of failures found and/or the calculated time of failure.
 The frequency of follow-up visits, and the type and number of measurements
are usually well described in the study protocol.
 However, in clinical practice frequency and measurements will be different
from patient to patient, from hospital to hospital, from year 1 to year 5.
 A measurement can be 100% objective (e.g. blood value), or contain a
subjective component (e.g. “start hormonal treatment”, “bleeding needing
treatment”, “death (not) related to disease”).
 The endpoint “death” is free from any flaws in measurements and subjective
interpretations.
17
Statistics in clinical trials
How many subjects needed in each treatment group:
- How many events expected at 1 / 2 / 3 / 4 / 5 years of f-up ?
- Which difference is clinically relevant and should be stat. significant ?
“To demonstrate a significant difference of X or more, Y subjects
are needed in each treatment group.”
Sample size calculations are related to the primary endpoints of the trial.
The subjects Hypothesis Testing and Sample size calculations will be part of
course (Chapter “Considerations”)
this
In principle, the statistical analysis of a RCT is rather simple and straight forward
/ standard. It needs no further modeling. Data are however often used for
additional (subgroup) analyses and modeling.
18
Subgroup analysis
Reasons: exploratory analysis, known prognostic subgroups,
“me too” analysis (reports in literature).
Why not ?
- Multiple hypothesis testing.
- Reduced (study) power.
In principle, subgroups are only allowed when they are defined
in the protocol and the sample size was calculated accordingly.
19
Experiments with animals
For ethical principles as well as for economic reasons, an experiment should
be designed well. This applies also to experiments with animals* :
- valid objective,
- the correct number of subjects (not too many, not too few),
- protocol with detailed descriptions (in such a way it can be repeated),
- statistical methods and analysis should be designed in advance,
- blinding and randomization is also important in animal experiments.
- each experiment should be documented and reported.
* Festing MFW, Altman DG. Guidelines for the Design and Statistical
Analysis of Experiments using Laboratory Animals. ILAR Journal.
2002;43: 244-58.
20
Example: atherosclerosis after RT in mice
Mice were given 8 Gy, or 14 Gy, or 20x2 Gy (fractionation)
- random allocation
- age-matched controls
- control group: sham treatment
IJROBP 2008; 71: 848-857
21
Example: prostate cancer trial
Phase III Trial comparing 68 Gy of Radiotherapy with 78 Gy
in Prostate Cancer Patients
JCO 2006; 24: 1990.
22
Example: prostate cancer trial
- Open label, randomized, multicenter, phase III trial.
- Primary endpoint: clinical failure or biochemical failure or start
hormonal treatment, whichever comes first.
- Sample size calculations were based on detecting a difference
of (at least) 10%. Therefore at least 300 patients in each arm
were needed.
- Measurements: PSA (blood value), bone scan (when suspected
for meta), clinical examination, ultrasound.
- Study protocol was altered during the study, due to new
developments in treatment (hormonal therapy) and in definition
of biochemical failure.
23
Example: prostate cancer trial
Primary endpoint: clinical or biochemical failure (whichever comes first).
Two definitions of biochemical failure where applied: protocol (graph A) and a
superior one (graph B) which was proposed in literature AFTER protocol was written.
24
Subgroup analysis
Question
Was this subgroup analysis valid ?
25
Crossover Design
- Subjects receive a sequence of different treatments.
- Usually Washout Period to prevent from carry-over effects.
- Crossover design: not by definition an experiment.
- Each subject its own control: balanced for subject-specific
differences.
- Crossover designs are statistically efficient.
- Many research settings are however not suitable.
Common design:
Group 1: Medicine – Washout – Placebo
Group 2: Placebo – Washout – Medicine
No crossover study: longitudinal study
26
Example
A crossover design is common in studies of pharmacokinetics (potential
interactions) in drugs. E.g. combination of drugs and chemotherapy.
Study:
Aprepritant when added to a standard
antiemetic regimen consisting of
Ondansetron and Dexamethasone
does not affect Vinorelbine
pharmacokinetics in cancer patients.
Open-label, two period, crossover study,
n=14 patients with metastatic solid tumors
Vinorelbine: anti-mitotic chemotherapy drug.
Antiemetic: against nausea and vomiting.
Aprepritant: blocks vomiting motion.
Cancer Chemother Pharmacol 59: 407-412
27
Factorial Design
- An experiment in which the effects of >1 factor are investigated simultaneously
- The treatments exist of all possible combinations. E.g. five two-level factors:
32 treatments.
- Exp. units are assigned randomly to a treatment, equal numbers in each group.
- The design provides much more information than 1 factor design, with limited
extra work.
Example, 2 by 2 full factorial design
Factor 1
Factor 2
Interaction
Result
Experiment 1
+
+
+
...
Experiment 2
+
-
-
...
Experiment 3
-
+
-
...
Experiment 4
-
-
+
...
28
Factorial Design:
example
Lancet 2003; 362:95.
Women with breast cancer and
complete surgical excision
RT: 50 Gy, no boost.
Tamoxifen: antagonist of the estrogen
receptor in breast tissue
29
Table 3.
New events in patients in the
radiotherapy comparison.
Conclusion ?
Table 4.
Events in patients randomised to tamoxifen
stratified by whether or not they had radiotherapy.
30
Lancet 2003; 362:95.
Findings
Ipsilateral invasive disease was not reduced by tamoxifen but recurrence of
overall ductal carcinoma in situ was decreased (hazard ratio 0·68 [0·49–0·96];
p=0·03). Radiotherapy reduced the incidence of ipsi-lateral invasive disease
(0·45 [0·24–0·85]; p=0·01) and ipsilateral ductal carcinoma in situ (0·36 [0·19–
0·66]; p=0·0004), but there was no effect on the occurrence of contra-lateral
disease. There was no evidence of interaction between radiotherapy and
tamoxifen.
Interpretation
Radiotherapy can be recommended for patients with ductal carcinoma
in situ treated by complete local excision; however, there is little
evidence for the use of tamoxifen in these women.
31
Online Data on Trial Databases
http://clinicaltrials.gov/ currently 94,946 trials in 173 countries.
http://www.cancer.gov/clinicaltrials
http://www.controlled-trials.com/
For several journals it is obligatory that the trial was registered
in a international trial database, otherwise it will not be accepted
for publication.
NKI: http://wa.nki.nl/trion3/
32
33
34
35