Transcript Slide 1

Educational Research 102:
Selecting the Best Study Design for your
Research Question
Francis S. Nuthalapaty, MD
2010 APGO Faculty Development Seminar
Disclosures
No relevant financial
disclosures to declare
Learning Objectives
• Describe types of research and study designs
• Understand the characteristics of a good
research question
• Understand how to convert the research
question into a hypothesis
• Understand how to select the most
appropriate study design to test the
hypothesis
What do we already know?
Educational Concept
True or False?
1. High-fidelity medical simulations facilitate learning.
TRUE
2. Restriction in resident duty hours improves the quality of
patient care.
FALSE
1. Issenberg SB, McGaghie WC, Petrusa ER, Gordon DL and Scalese RJ (2005) Features and uses of high-fidelity medical simulations that lead to
effective learning: a BEME systematic review. Medical Teacher 27, 2, pp 10-28.
2. Fletcher KE, Davis SQ, Underwood W, Mangrulkar RS, McMahon LF Jr, Saint S. Systematic review: effects of resident work hours on patient
safety. Ann Intern Med. 2004 Dec 7;141(11):851-7.
Write an educational
research question in
which you have
interest
Types of Research
Empirical vs. Non-empirical
Basic vs. Applied
Empirical vs. Non-Empirical Research
Empirical
• Involves collection of data
first hand
Non-Empirical
• No first hand data
collection
Example: Empirical or Non-empirical?
The Effect of House Staff Working Hours on
the Quality of Obstetric and Gynecologic
Care
Bailit, J et al, Obstet Gynecol 2009
• OBJECTIVE: To measure the effect of house
staff working hours reforms on the quality of
obstetric and gynecologic care.
Example: Empirical or Non-Empirical?
Features and uses of high-fidelity medical
simulations that lead to effective learning:
a BEME systematic review.
Issenberg SB et al, Med Teach 2005
• OBJECTIVE: Review and synthesize existing
evidence in educational science that addresses
the question, 'What are the features and uses of
high-fidelity medical simulations that lead to
most effective learning?'.
Types of Research
Empirical vs. Non-empirical
Basic vs. Applied
Basic vs. Applied Research
Basic
Applied
• Results apply to a great
many people and situations
• Results are related to
general theory or to a
general field of knowledge
• Results need not have
immediate or even clear
implications
• Results are applicable only
to a specific group of people
in a particular situation.
• Results are not necessarily
related to a broader field of
knowledge
• Results must have
immediate and clear
implications for practice
Example: Basic or Applied?
Assessing Vaginal Surgical Skills Using
Video Motion Analysis
Diwadkar G et al, Obstet Gynecol 2009
• OBJECTIVE: To demonstrate the feasibility of
using video motion analysis to quantitate a key
step of vaginal hysterectomy and define
measurable differences between novice and
experienced surgical trainees during vaginal
hysterectomy.
Example: Basic or Applied?
Effects of a Depression Education Program
on Residents’ Knowledge, Attitudes, and
Clinical Skills
Learman L et al, Obstet Gynecol 2003
• OBJECTIVE: To determine whether an
interactive educational program would improve
obstetrics and gynecology Residents' knowledge,
attitudes, confidence, and skills in caring for
depressed patients.
Research Methods
What is the difference
between Qualitative
Research and
Quantitative Research?
Qualitative vs. Quantitative Research
Qualitative
• Descriptive and
exploratory focus
• Used to gain insight into
attitudes, behaviors,
values
• Analysis of unstructured
information
• Narrative reporting
Quantitative
• Hypothesis driven
• Used to identify
association and/or
causation
• Analysis of discrete
variables
• Statistical reporting
Quantitative Research
• Experimental & Quasi-experimental
• Non-Experimental
–Causal-comparative
–Correlational
Quantitative Research
• Experimental & Quasi-experimental
– Assess effect of an independent variable on
dependent variables
– Comparison of 2 or more groups
– Control over ‘treatment’ & measurement
– Randomization
– Control group
Quantitative Research Designs
Treatment
Study Population
Measurement
Randomize
Control
Measurement
Quantitative Research Designs
Measurement
Study Population
Treatment
Measurement
Control
Measurement
Randomize
Measurement
Quantitative Research Designs
Measurement
Treatment
Measurement
Measurement
Control
Measurement
Treatment
Measurement
Control
Measurement
Study Population
Randomize
Quantitative Research Designs
Study
Population
Treatment
Measurement
Quantitative Research Designs
Study
Population
Measurement
Treatment
Measurement
Quantitative Research Designs
Study
Group 1
Treatment 1
Measurement
Study
Group 2
Treatment 2
Measurement
Matching
Quantitative Research Designs
Study
Group 1
Measurement
Treatment 1
Measurement
Study
Group 2
Measurement
Treatment 2
Measurement
Quantitative Research
• Non-Experimental
–Causal-comparative
–Correlational
Causal-Comparative
• Baseline differences are pre-existing
– Non-permutable: Ethnicity, Gender
– Permutable: Teaching style
• Determine the cause or consequences of
differences
• Associations can be identified
• Causality cannot be determined
Quantitative Research Designs
Exposures
Outcome
Control
Outcome
Quantitative Research Designs
Exposures
Outcome
Exposures
Control
Quantitative Research Designs
Group 1
Group 2
Exposure
Outcome
Outcome
Correlational Research
• Single group of subjects
• Describe degree to which 2 or more
quantitative variables are related
• Help explain important human behaviors
• Predict likely outcomes
• Identifies associations
• Causality cannot be determined
Correlational Research
Observations
Subjects
Student A
Student B
Student C
Student D
Student E
O1
O2
Correlational Research
• Correlation Coefficient
– Calculated by regression
•
•
•
•
•
< .35 = only a slight relationship
.40 - .60 = possible theoretical value
.50 = minimum for crude predictions
>.65 = reasonably accurate predictions
>.85 = close relationship
– Positive correlation = direct association
– Negative correlation = inverse association
Threats to Internal Validity
•
•
•
•
•
•
Subject characteristics
Loss of subjects
Maturation
Repeated measures
Statistical regression
Investigator bias
External Validity
• Can the findings from the study be
generalized to larger populations?
– Subject characteristics
– Environment (lab vs. natural setting)
– Did the subjects act differently b/c they were
enrolled in a study (Hawthorne effect)?
What are the steps
in the research
process?
The Research Process
Define
Research
Question
No hole
in the
literature
Disseminate
Results
Hole
in the
literature
Conduct
Literature
Review
Collect
Analyze Data
Refine
Research
Question
Obtain
IRB Approval
Design
Study
FINER Research Questions
• F – Feasible
• I – Interesting
• N – Novel
• E – Ethical
• R - Relevant
FINER Research Questions
• Feasible:
–Can be investigated with available:
• Time
• Money
• Space
• Other resources
FINER Research Questions
• Interesting:
–Interesting to you
–Interesting to others in the field
• Novel:
–Will answers to the question advance
knowledge in the field?
FINER Research Questions
• Ethical:
– Will any physical or psychological harm come
to anyone as a result of the research?
• Protect participants from harm
• Ensure confidentiality of data
• Avoid knowing deception of participants
• Relevant:
– How might answers to this question improve
educational practice?
What is a research
hypothesis and how is it
different from a research
question?
Research Hypothesis
A prediction regarding
the possible outcomes
of the study
Wallen & Fraenkel. Educational Research: A Guide to the Process. 2nd Ed.
Research Hypothesis
• A statement which:
–Summarizes the elements of the study
• The sample
• The design
• The predictor and outcome variables
Yoder E, MERC Workshop, 2009 AAMC Annual Meeting
Research Hypothesis
• Based upon a FINER research
question
• Simple – one difference or
relationship
• Specific – clearly stated, defined
• Stated in advance – before data
collection
Yoder E, MERC Workshop, 2009 AAMC Annual Meeting
Research Hypothesis
• May be stated:
– As the alternative hypothesis
• Non-directional
– The difference is unknown
• Directional hypothesis
– Existing studies/data suggest direction of relationships
– As the null hypothesis
Yoder E, MERC Workshop, 2009 AAMC Annual Meeting
Research Hypothesis
• Ha: There is a difference between groups
or variables
– Female medical students have a different IQ than
male medical students
– μf ≠ μm
or
μf > μm
• H0: There is no difference between groups or
variables
– Female medical students have a different IQ than
male medical students
– μf = μm
Yoder E, MERC Workshop, 2009 AAMC Annual Meeting
Example: Hypothesis
Improving Resident Competency in the Management of
Shoulder Dystocia With Simulation Training
Deering S, et al, Obstet Gynecol 2004
• METHODS: Residents from 2 training programs were
randomized by year-group to a training session on
shoulder dystocia (SD) management that used an
obstetric birthing simulator or to a control group with no
specific training. Both groups were subsequently tested
on a standardized SD scenario, and a physician grader
rated the resident's performance with a standardized
evaluation sheet.
Example: Hypothesis
• What is the research question?
• What is the null hypothesis?
• What is the research study design?
– Basic or Applied?
– Qualitative or Quantitative?
– Experimental or Non-Experimental
Quantitative Research Designs
Treatment
Study Population
Measurement
Randomize
Control
Measurement
Small Group Activity
• Share your research questions
• Select one research question to
convert to a hypothesis
• Select the most appropriate
study design
Give Your Research Impact
• Investigate important questions
• Connect your study to prior studies and
help build the body of work
• Uses appropriate research design
• Consult with a biostatistician in the design
phase
• Dissemenate your results
Session content, including narrated
MS Powerpoint slides available at:
http://www.obgynknowledgebank.net