Transcript Study Design 10/22/09 - St. Barnabas Hospital

DEPARTMENT OF PEDIATRICS
Research Seminar
STUDY DESIGN
10/22/09
David H. Rubin, MD
Chairman, Department of Pediatrics, St.
Barnabas Hospital
Professor of Clinical Pediatrics
Albert Einstein College of Medicine
STUDY
DESIGN
FEATURE
EXAMPLE
Descriptive
Reports
Recognize
new/atypical
characteristic of
disease
Case report – first
case(s) of pediatric
lyme disease
Cohort
1 group followed over Infants followed for
time
effects of smoke
exposure for 2 years
Cross-Sectional
A group examined at
1 point in time
Case-Control
Two groups, based on Aspirin and Reyes
outcome
Syndrome
Randomized
Trial
Two groups, randomly Effect of educational
created, blinded
intervention on
intervention
asthma morbidity
Psychometric testing
in homeless vs.
nonhomeless children
RESEARCH DESIGN
(Jekel, 2007)
STUDY DESIGN
ADVANTAGE
DISADVANTAGE
Cohort
Useful for measure of
risk and many disease
outcomes; restrosp or
prospect
Time consuming and costly;
can only study risk factors
identified at start of study
Cross-Sectional
Fairly easy to perform;
useful for hypothesis
generation
No evidence of temporal
relationship between risk
and disease
Case-Control
Easy to perform, can
study several risk
factors/rare diseases
without waiting for
disease
Can obtain only relative
measure of risk; recall bias
and selection of case and
control bias
Randomized Trial
“Gold standard” for
treatment interventions
Time consuming/costly;
problems with dropouts
ADDITIONAL STUDY DESIGNS
• Medical record review
• Survey study
CASE CONTROL
STUDIES
CASE CONTROL STUDY
EXPOSURE?
CONDITION OR
PROBLEM
YES
YES
NO
YES
RESEARCH
POPULATION
AT RISK
TIME
NO
NO
(Fletcher, 1996)
CASE CONTROL
STUDIES/RECENT LITERATURE
• Pubmed search of “case control studies”
• 356,299 studies identified
• Case control studies and emergency medicine
journals
• Ann Emerg Med: 602 studies (1980-2007)
• Jour Emerg Med: 197 studies (1984-2007)
• Acad Emerg Med: 357 studies (1994-2007)
• Ped Emerg Care: 288 studies (1985-2007)
NESTED CASE CONTROL STUDY
(Gordis, 2000)
• Case control study “nested” in cohort
study
• Population identified and followed over
time
• Disease develops in some members of
the population
• Case control study of
• Cases (disease develops) and
• Controls (disease does not develop)
ADVANTAGES OF CASE CONTROL
STUDIES
(Fletcher et al, 1996, Newman et al, 2001)
• Relatively easy to perform
• Can study several risk factors and rare diseases
without waiting for disease to occur
• Beneficial for diseases with long latency
• High yield of information with relatively few subjects
• Ability to examine a large number of predictor
variables makes case control studies useful for
generating hypotheses
DISADVANTAGES OF CASE CONTROL
STUDIES
(Fletcher et al, 1996, Newman et al, 2001)
• Information available may be limited
• No direct method to estimate incidence
or prevalence of a disease
• Only 1 main outcome can be studied
• In cohort and cross sectional studies,
several outcomes can be examined
• Biggest weakness is bias
BIAS IN CASE CONTROL
STUDIES
(Fletcher 1996)
• Investigators create the comparison groups –
there is no waiting to see who becomes a
case and who becomes a control
• Cases and controls are comparable if:
• Controls would have been defined as cases
if they developed the condition under study
•
Cases and controls need to be members of
the same “base population”
COHORT STUDIES
COHORT STUDY
T0
T1
•Population followed forward over
time
•Baseline: acute pharyngitis
•Outcome: Prevention of rheumatic
fever or glomerulonephritis
•Admission Criteria?: Evidence of ßhemolytic streptococcus vs
pharyngeal inflammation
COHORT STUDY
EXPOSURE?
POPULATION
YES
YES
TIME
SAMPLE
NO
CONDITION OR
PROBLEM
NO
YES
NO
(Fletcher, 1996)
CROSS SECTIONAL
STUDIES
CROSS SECTIONAL STUDY
T0
T1
•Collect data on 2 groups at 1 point in
time
•Compare group differences
•Cholesterol levels in athletes vs. non
athletes at a midwest university
CROSS SECTIONAL DESIGN
• Face to face interview
• Mailed questionnaire
• Emailed questionnaire
• Telephone interview
• Mailed interview with telephone F/U
• Interview and observation
RANDOMIZED
CLINICAL TRIALS
RANDOMIZED CONTROL
TRIAL
CONTROL
ENROLL
SUBJECTS
RANDOMIZATION
EXPERIMENTAL
TIME 0;
BASELINE
T1; FOLLOWUP
SURVEY STUDIES
SURVEY STUDIES
• Collect information about
people to describe,
compare, or explain
• Knowledge
• Attitude
• Behavior
SURVEY STUDIES
• Features of good survey studies
• Specific measurable objectives
• Solid research design
• Good choice of population or sample
• Reliable and valid instruments
• Comprehensive analysis
• Accurate reporting of results
MEASURABLE OBJECTIVES
• Define aim of the study
• Define hypothesis
• Define outcomes
• Define independent, dependant,
and confounding variables
SAMPLING OF POPULATION
• Many options
• Sample - subset of the population chosen for
study (characteristics similar to larger group)
• Representative sample – use an unbiased
method to choose survey participants
• All members of the pediatric clinic at SBH
who are between 2 and 3 years of age
• Children seen at the Pediatric Endocrine
Clinic for any illness related to diabetes
mellitus
DESCRIPTIVE
REPORTS
DESCRIPTIVE REPORTS
• Description of a new aspect or new
disease
• No comparison group needed
• Description is usually a basic
statistic summary or profile of the
group of cases
• Mean, SD, range, confidence
intervals, correlation between
variables
MEDICAL RECORD
REVIEW
MEDICAL RECORD REVIEW
• Uses pre-recorded patient focused
data as the primary source of
information in a research study
• Physician, nurses notes
• Ambulance call reports
• Diagnostic tests
• Clinic, administrative, government records
• Computerized databases
WHY SELECT THIS DESIGN?
• Addresses issues that cannot be addressed
with prospective studies
• Effect of harmful exposures (no randomization
•
•
•
•
•
possible)
Effect of potentially beneficial exposures
Occurrence of rare events
Studies of patterns of disease or behavior
Quality assurance studies
Studies where cases may be shared (trauma
database)
• Pilot studies for prospective studies
DATA QUALITY
• “Free form” quality of medical records
may increase missing and/or
erroneous data
• Handwriting may be illegible or
uninterruptible
• May miss examining potential cases
• Computer vs paper records
• Data abstraction techniques require
standardization
SAMPLE SIZE
• Usually determined based on the
summary measure and the
size/width of the confidence
interval desired
• An interval with a greater CI (eg 99%
CI v 95% CI) is wider and more likely
includes the true population value
• The width of the CI depends on sample
size
SAMPLING
• Select all cases within a given time frame
• For nonconsecutive sampling it is best to
choose probability sampling
• Provides equal opportunity for each eligible
case to be selected
• Use random number generator
• Triage level
• Incidental sampling – choosing most
easily accessible cases
• Systematic sampling – choosing every xth
case
RELIABILITY
• Very important
• Any differences in data extraction by 2
different people?
• Kappa
• Value ranges from -1 (perfect disagreement) to
1 (perfect agreement)
• K = [observed agreement (%) – expected
agreement (%) / [100% - expected agreement
(%)]
• Try to achieve kappa of 0.6 or better (60%
agreement)
MINIMUM REQUIREMENTS FOR
MEDICAL RECORD REVIEWS
(Lowenstein, 2005)
1) Explicit protocols for case
selection/exclusion
2) Abstractor training
3) Precise definitions of key variables
4) Use of standardized abstraction and
coding forms
5) Monitoring of abstractor performance
6) Blinding of abstractors to study
hypothesis and patient groups
7) Testing of interrater reliability
QUALITY OF MEDICAL RECORD
REVIEWS
(Badcock, 2005)
• Observational study of medical record
reviews published in several emergency
medicine journals
• 107 articles analyzed
• Clear aim reported in 93%
• Standard abstraction forms: 51%
• Interrater reliability: 25%
• Ethics approval: 68%
• Sample size/power: 10%
METHODOLOGY FOR RETROSPECTIVE
REVIEWS IN CHILD PSYCHIATRY
Conceive question
Literature review
Proposal methods
Create data abstraction instrument
and manual
• Sample size
• Obtain IRB approval
• Pilot study
•
•
•
•
SAMPLE SIZE
(Gearing 2006)
• Estimate 10 charts per variable
•
•
•
•
(Sackett, 1991)
Others estimate 5-7 charts/variable
Convenience sampling – select cases
over specific time period
Quota sampling – predetermined
number sampled
Systematic sampling – every “nth”
case chosen
PRACTICAL ISSUES
• Check all possible CPT codes for
diagnosis or procedure code
• Febrile seizure may have been coded as
seizure
• Gastroenteritis may have been coded as
viral syndrome
• Pilot your Data Abstraction Form
• Create detailed “Codebook” for your
study
• Especially critical if > 1 researcher on
study
REFERENCES
• Fink A. How to design survey studies. Sage
Publications, Thousand Oaks, CA. 2003.
• Kline TJB. Psychological testing –a
practical approach to design and
evaluation. 2005. Thousand Oaks. Sage.
• Friedman JN. Development of a clinical
dehydration scale for use in children
between 1 and 36 months of age. J Pediatr
2004:145:201-207.
REFERENCES
• Cumming RG, Le Couteur D. Benzodiazepines and risk
of hip fractures in older people; a review of the
evidence. CNS Drugs. 2003;17(11):825-837.
• Ding R, McCarthy M, Li G. Patients who leave without
being seen: their characteristics and history of
emergency department use. Ann Emerg Med
2006;48:686-693.
• Feinstein AR. Clinical Epidemiology. Philadelphia. WB
Saunders. 1985
• Fletcher RH, Fletcher SW, Wagner EH. Clinical
Epidemiology The Essentials. Philadelphia: Lippincott
Williams and Wilkins, 1996.
• Gordis L. Epidemiology 2nd edition. Philadelphia: WB
Saunders, 2000.
REFERENCES
• Worster A, Haines T. Advanced statistics:
understanding medical record review (MRR) studies.
Acad Emerg Med 2004;11:187-192.
• Lowenstein SR. Medical record reviews in emergency
medicine: the blessing and the cure. Annals Emerg
Med April 2005;45(4):452-455.
• Babcock D et al. The quality of medical record review
studies in the international emergency medicine
literature. 2005;45(4):444-447.
• Worster A. et al. Reassessing the methods of medical
record review studies 2005;45:448-451.
• Gearing et al. Methodology for Retrospective chart
review in child adolescent psychiatry. J Can Acad
Child Adoles Psychiatry 15:3:2006
REFERENCES
• Hellems MA, Kramer MS, Hayden G. Case control
confusion. Ambulatory Pediatrics 2006;6:96-99.
Altzema C, Ann Emerg Med 2004;44:169-174
• Horwitz RI, Feinstein AR. Methodologic standards and
contradictory results in case control studies. Amer Jour
of Med 1979;66:556-564.
• Hurwitz ES, Barrett MJ, Bregman D et al. Public Health
Service study of Reye’s syndrome and medications.
Report of the main study. JAMA 1987;257:1905-11.
• Jekel JF, Katz DL, Elmore JG, Wild DMG. Epidemiology,
Biostatistics and Preventive Medicine 3rd edition.
Philadelphia: Saunders, 2007.
REFERENCES
• Katz AR. Selection of cases and controls. Additional
information is needed. Int J Cardiology
2007;doi:10.1016/j.ijcard206.12.047
• Levy JA, Bachur RG. Intravenous dextrose during
outpatient rehydration in pediatric gastroenteritis.
Acad Emerg Med 2007;14:324-331.
• Newman TB, Browner W, Cummings SR, Hulley SB in
Hulley SB et al. Designing Clinical Research. 2nd
edition. Philadelphia: Lippincott, 2001.
• Neuman MI, Kelley M, Harper MB et al. Factors
associated with antimicrobial resistance and mortality
in pneumococcal bacteremia. Jour Emerg Med
2006;32(4):349-357.
REFERENCES
• O’Brien KL, Selanikio JD, Hecdivert C et al. Epidemic of
pediatric deaths from acute renal failure caused by
diethylene glycol poisoning. Acute Renal Failure
Investigating Team. JAMA 1998;279:1175-1180.
• Panagiotakos DB, Rallidis LS, Pitsavos C et al. Cigarette
smoking and myocardial infarction in young men and
women: A case-control study
International Journal of Cardiology, Volume 116, Issue
3, 4 April 2007: 371-375.
• Pierfitte C, Macouillard G, Thicoipe M et al.
Benzodiazepines and hip fractures in elderly people.
BMJ 2001;322:704-708.
• Schultz CH, Koenig KL, Lewis RJ. Decisionmaking in
hospital earthquake evacuation: does distance from the
epicenter matter? Ann Emerg Med 2007;50:320-326.