Transcript Study design

Teaching Registrars Research Methods
Study design
Landon Myer PhD
Senior Lecturer, Infectious Diseases Epidemiology
Unit, School of Public Health, UCT
[email protected]
Orientation to today’s session
• So far:
– Introduction
– Study protocol
– Reviewing the Literature
• Today: → Study design ←
• To come:
– Sampling
– Measurement
– Data analysis
– Ethics
All of one
interrelated
process
Overview
• How to start to select a study design
• Framework for considering different study
designs
– Key similarities & differences
• Introduction to each major category of
study design
– Focus on major functional features
– Strengths & limitations
– Examples
Ask questions throughout!
• Exercises
Note on terminology
• “Outcomes”
– Health outcome of interest in the study
– Disease, death, side effect, complication
• (stats: “dependent variables”)
• “Exposures”
– Measures that may be associated with the
outcome
– Possible “risk factors”, “causes”,
“determinants”
• (stats: “independent variables”)
I. Framework for thinking
about study designs
What are study designs?
• Structured approaches to address
specific research questions
• Provide general guidelines for thinking
about specific aspects of study conduct:
– sampling populations
– systematically collecting measurements
– analysing data
• Strengths & limitations of specific designs
are well-established
How to select a study design
• Start with a good study
question
– Relevant
• Addresses topic of significance to health of local
population / health care services
– Novel
• Makes meaningful contribution to existing
knowledge ~ new insights
– Feasible
• Not overly ambitious
Creativity
Different types of study questions lead
to different types of study designs
• Descriptive
– What is the prevalence of condition Z in a
specific population?
• Analytic
– What are the factors associated condition Z?
Is condition X a risk factor for condition Z?
• Diagnostic
– How good is test Q in detecting condition Z?
Selecting the right study design option
– Relevant
• Design allows you to answer your research
question
– Novel
• Design allows meaningful contribution to existing
knowledge ~ new insights
– Feasible
• Design allows study to be done within available
time and funds
– Simple
• ALWAYS avoid ALL unnecessary complexities
Types of study designs
• Many types
– Most are some variation on general themes
presented here
• All designs based on same basic
principles
– key differences in how study design samples
participants with respect to
• “exposures” (risk factors, patient characteristics)
• “outcomes” (diseases, conditions)
Choice of study design closely
related to other aspects of protocol
1. Study design choices inform how you will
sample a specific study population
•
in a way that its understood how the
participants in the study relate to the
population in general
2. Study design choices inform the most
appropriate measurements to collect
on participants in a standardised manner
(create data)
3. Study designs will point to the most
appropriate analysis of data to answer
study question:
– Descriptive
• Calculate the proportion of the study population with
condition Z (incident or prevalent)
– Analytic
• Compare the frequency of condition Z among groups
of the population
– Diagnostic
• Calculate the validity (sensitivity/specificity) or
reliability of test Q in detecting condition Z
Broad categories of options in
study design
• Cross-sectional
• Case report / case series
• Case-control
• Cohort
• Randomised Controlled Trial (RCT)
Broad categories of options in
study design
• Cross-sectional
Diagnostic
?
• Case series
Descriptive
• Case-control
• Cohort
• Randomised Controlled Trial
Analytic
Broad categories of options in
study design
• Cross-sectional
• Case report/series
Observational designs:
investigator is only
observing distribution of
variables (risk factors,
diseases, etc) ‘in nature’
• Case-control
• Cohort
• RCT
Experimental designs:
investigator assigns study
conditions ~ usually testing
an intervention
(many variations here)
Key differences between study designs
1. How participants are sampled
– Are participants sampled according to exposure
status, disease status, neither, both?
2. When measurements are taken
– Are some variables measured before others, or
are measurements taken all at once?
3. How outcome variables are measured
– Incident or prevalent outcomes (morb/mort)?
4. Are there comparison groups involved?
5. Is design observational or experimental?
Time marches on
• Onset of conditions takes place over time
in populations
• Different study designs deal with the onset
of conditions through time in different ways
• Critical to understand how your choice of
study design handles the timing of
– Identification of participants
– Measurement of variables (exposure,
disease)
1
X
X= onset of
condition of interest
Died
2
Died
3
4
5
X
Died
6
7
X
8
X
9
10
Died
X
X
Died
11
Time
II. Details on categories of
study designs
a.
b.
c.
d.
e.
Case report & case series
Case-control
Cross-sectional
Cohort
RCT & other experimental
designs
Case-report & case-series
• Cases
– people with health outcome
– depends on what is of interest
• Case report / series
– Describes
• characteristics of disease / condition
• characteristics of individual that may be associated
with the condition
Issues in case-only designs
• Useful for descriptive purposes only
• Implicit comparisions to what is ‘expected’
or ‘normal’
• Why might this be problematic?
vs
Case-control studies
• Set of cases (usually from health service)
• Comparable set of controls (various
sources)
• Both groups evaluated on characteristics /
‘exposures’ of interest
– Compare distribution of ‘exposure’ in cases
and controls
Exposed
Cases
Unexposed
Exposed
Controls
Unexposed
Exposed
Cases
Unexposed
Exposed
Controls
Unexposed
Time
Example: Does children’s inhalation of
hairspray facilitates development of asthma?
• 50 new cases of severe asthma identified at
RXH in 12-month period, all <5 yrs of age
• These cases are compared to
90 children <5 years attending
RXH for orthopedic surgery
(who do not have asthma)
• Cases and controls are
compared on maternal hairspray
use since child’s birth
Hairspray
No hairspray
Hairspray
No hairspray
Children
with asthma
Children
without asthma
Odds ratio in 2x2 table
Odds ratio = (A/C) / (B/D)
Cases
Controls
E+
A
B
E-
C
D
A
( C)
B
(D)
Strengths & limitations of casecontrol study
• Relatively simple & quick approach to
address analytic questions
• Ideal to study rare diseases (vs cohort)
• Cases & potential controls are accessible in
health care setting
• Choice of the wrong control group ~
selection bias
• Cases may over-report past exposures ~
information bias
Cross-sectional studies
• Most common form of research ~ “surveys”
• Measure all variables on participants at
same point in time (approximately)
• Measure prevalent disease (not incidence)
X Disease
X Exposure
Time
Defined population
Sampling
Collect data on outcome (disease)
and exposure (risk factors)
Exposed
Diseased
Not
exposed
Diseased
Exposed
Not
exposed
Not
diseased
Not
Diseased
Example: How severe is disease among
rheumatoid arthritis patients attending GSH?
• Study population: patients attending
rheumatology clinic at GSH during one month
period
• Measures: degree of disease severity (outcome);
demographics, disease history, treatment history
(exposures)
• Analysis: prevalence of severe disease in clinic
population; association between severity of
disease and different exposures
Benefits of cross-sectional study
• Feasibility → easy to do
– In health care setting, can work from existing
records (consent issues)
– Low cost, rapid
• Not waiting for incident outcomes to develop
• Can calculate prevalence
– Often most relevant measure for burden of
disease, informing health care strategies
– Measure of association: calculate Odds Ratio
for prevalent disease
Issues in cross-sectional studies
• Measuring prevalent disease only
– Prevalence incorporates incidence of disease
AND duration of disease
– Risk factors for prevalent disease often
different from risk factors for incident disease
• Issues of timing (temporality) are a problem
– Exactly when did disease develop?
– Did exposures come before or after onset of
disease?
Cohort studies
• Start with group of individuals without the
outcome of interest: “at risk”
• Follow forward in time to observe
incidence of disease (a rate)
• Can be descriptive or analytic
– If analytic question, then measure exposures
on cohort at the beginning of the study
Cohort studies can be purely descriptive
Eg: What is the rate of remission among
men treated for prostate cancer at GSH?
At risk
participants
(without
outcome)
Develop outcome of
interest
Do not develop
outcome of interest
Time
Analytic cohort study
Eg: Do β-blockers increase risk of renal
transplant rejection?
Develop the outcome
of interest
Exposure
Do not develop the
outcome of interest
Study
population
without the
outcome of
interest
Develop the outcome
of interest
No exposure
Do not develop outcome
of interest
Time
Types of cohort studies
• Prospective
– Following cohort forward through time from
present
– Most common approach
• Retrospective
– Assemble cohort from medical records,
– “follow” based on records
– Follow-up is in the past (can extend into
present)
Measure of association in a cohort study:
relative risk (aka risk ratio, rate ratio)
New
cases of
outcome
Participants who
do not develop
outcome
Exposed
A
B
Total number of exposed = A + B
Unexposed
C
D
Total number of unexposed = C + D
Total number of participants =
A + B +C + D
RR = [A/(A+B)] / [C/(C+D)]
Strengths + problems in cohort studies
• Strengths
– Can calcluate rates of new events~ valuable
– Timing of exposure before disease assured
– Good for studying health effects of rare
exposures (can select an exposed cohort)
• Weaknesses
– Participants ‘self-select’ their exposure status~
leads to confounding, bias
– Take time, resources (if prospective)
– Many subjects needed for rare outcomes
Randomised controlled trials
• Principal experimental design in medical
research
• Like a cohort study, except exposure
status is assigned by investigator
(randomly)– not just observed
• Complex, take time → costly
• RCTs are usually best design for testing
the impact of a specific intervention in
improving a specific health outcome
Have outcome
of interest
Exposure
Study
population
without the
outcome of
interest
Do not develop
outcome of interest
Randomisation
Have outcome of
interest
No exposure
Do not develop
outcome of interest
Time
Key features of RCT
• Randomisation
• Removes selection bias or confounding
• Alternation or other assignment schemes are bad
idea
• Use of concurrent control groups
• Vs Before/After studies
• Blinding whenever possible
• Blind investigators: prevents information biases
• Blind participants: prevents selective behaviour
change during the trial
• Not just in trials
• RCTs are important tools
• But can encounter major problems that
hinder interpretation of results
– Generalizability: Trial participants are highly
selected individuals
• often not representative of general population at
risk
– Complexity: Trials procedures can be
complex (and costly)
• when key design features breakdown, the
experiment is compromised
Other experimental designs
• For an experiment, need to compare 2
states: with intervention vs without
• Before/after studies
Introduction of Pfizer
fluconazole donation
programme at GFJ
Median survival of
cryptococcal
meningitis in HIV+
before
Median survival of
cryptococcal
meningitis in HIV+
after
• Controlled before/after studies
BEFORE
AFTER
New training in
sterile procedures
MMH
intervention
vs
Rate of postoperative
sepsis
Rate of postoperative
sepsis
No new training
NSH
control
Rate of postoperative
sepsis
Rate of postoperative
sepsis
• Time-series studies
25000
300
Rate of advanced cervical
cancer cases per 100,000
250
20000
200
15000
150
10000
100
5000
# of pap smears performed
in Western Cape
50
0
0
1980
1984
1990
1995
2000
III. Conclusion
The “hierarchy” of study designs
• Frequently see framework for comparing
evidence based on the study design used
‘better evidence’
~ more valid
‘worse evidence’
~ less valid
RCT / experiments
Cohort
Case-control
Cross-sectional
Case series/report
Not (nearly) so simple
• The study design alone does not make
the evidence from a study better or worse
• The details of how a study is conducted is
what matters
– Rigour in design, sampling, measurements,
analysis
• This is why the Methods section is
the most important part of
scientific papers
Wrap-up
• Framework for thinking about study designs
when developing research ideas for MMed
– Start with a good research question
– Understand different study design options
– Select the most feasible study design based on
the study question
• Balance time, funding, available data sources
– Understand the strengths and limitations of
your approach
– Be able to justify your choice of study designs
Resources to learn more
• Consultations re: study design, conduct,
analysis
– Ask for help before you start collecting data!!
– Email Dr Jim teWaterNaude (to ID appropriate
support within School of Public Health)
• Self-learning
–
–
–
–
Hulley SB, Cummings SR. Designing Clinical Research
Gordis L. Epidemiology
Szklo M, Nieto J. Epidemiology: Beyond the Basics
Friedman LM, Fundamentals of Clinical Trials
IV. Examples
#1
• An investigator is interested in studying
the association between schizophrenia
and measles vaccinations.
– Hypothesis: childhood vaccinations
predispose individuals to develop
schizophrenia in later life
– What study designs are possible?
– Which design would you recommend and
why?
#2
• A study among outpatients attending the GSH
diabetes clinic during 2004 collects data on
1432 patients.
• Each patient is included once only in the dataset (ie,
info from their first visit during 2004)
• Data are collected on patient & family history, past
treatment, knowledge of disease & its management,
disease severity (GTT)
• Medicine Registrar decides to examine whether
patients with more severe disease have better
knowledge of disease & management
• What kind of study of this? What measures can be
calculated?
#3
• You have collected records from your
weekly clinic with information on 152
patients– you have data on:
–
–
–
–
–
patient demographic characteristics
detailed clinical information on morbidity
medical history
risk behaviours (smoking, drinking)
medications
– You need to write an MMed. Quickly.
– Identify a research question, a study design,
and describe these briefly.
• What are the strengths & limitations of the study
design you have selected in answering your
specific question?
#4
•
A study of neural tube defects and antenatal
folate supplementation, lasting 10 years,
follows 10,000 pregnancies in which women
used folate supplements, and 10,000
pregnancies in which no supplements were
used.
– Among women taking folate supplements, 50
cases of neural tube defects were observed
– Among women not taking supplements, 150 cases
of neural tube defects were observed.
#4
– What type of study is this?
– What is the appropriate measure of
association?
– Draw up a 2x2 table, calculate the measure
and interpret in one sentence
#5
•
Another study of folate supplementation and
neural tube defects uses a hospital referral
system to identify all cases of neural tube
defects in the local population.
– 200 cases are identified over 10 years (50 among
women using supplements, 150 among women
not using supplements).
– For comparison, investigators select 800 control
pregnancies (where no neural tube defects were
observed) at random from the same population (of
whom 498 use folate supplements and 492 are
unexposed).
#5
• What type of study is this?
• What is the appropriate measure of
association?
• Draw up a 2x2 table, calculate the
measure and interpret in one sentence.
• Comparing #4 and #5, what is the principle
advantage of the case-control design to
cohort design?