Chapter 14

download report

Transcript Chapter 14

chapter
14
Research
Synthesis
(Meta-Analysis)
Chapter Outline
• Using meta-analysis to synthesize research
• Tutorial example of meta-analysis
Contrasting Meta-Analysis
With Literature Review
• How thorough was the literature search?
– Computer search and hand search
– Which journals?
– Theses and dissertations included?
• On what basis were studies included or
excluded?
– Perceived internal validity
– Sample size
– Design
(continued)
Contrasting Meta-Analysis
With Literature Review (continued)
• How did the author arrive at conclusions?
– Vote counting
– Differential weighting
Purpose of Meta-Analysis
• Answer an important question when
adequate literature is available.
• Report a definitive methodology for the
review.
• Quantify the results of studies using a
standard metric.
Steps in a Meta-Analysis
• Identification of a problem
• Literature searched by specified means
• Review of studies to determine inclusion or
exclusion
• Careful reading and evaluation to identify
and code important study characteristics
• Calculating effect size
• Applying appropriate statistical analyses
• Reporting all the steps and outcomes in a
paper
Calculating Effect Size
M1 – M2
ES =
s2
Pooled s
sP =
(
)
(
)
2
2
+
s1 N1 – 1 s2 N2 – 1
N1 + N2 – 2
Distribution Overlap When Group 1
and Group 2 Are 1s Apart (ES = 1.0)
Differences Between Girls and Boys
(Effect Sizes) by Age
From J.R. Thomas and K. E.
French, 1985, “Gender
differences across age in motor
performance,” Psychological
Bulletin 98(2): 260-282.
Copyright © by American
Psychological Association.
Reproduced by permission.
Differences Between Boys and Girls
(Effect Sizes) in Throwing
From J.R. Thomas and K. E. French, 1985, “Gender differences across age in motor performance,”
Psychological Bulletin 98(2): 260-282. Copyright © by American Psychological Association.
Reproduced by permission.
Methodological Considerations
in Meta-Analysis
• Deciding what to code: Using theory and
empirical studies to decide on
characteristics of importance
• Choosing the standard deviation for ES
– Experimental versus control
– Nonrandomly formed groups (sex, age)
– Pretest to posttest
• Correcting ES
• Weighting ES
Statistical Issues
for Analyzing Effect Size Data
• Are effect sizes normally distributed?
– If yes, standard parametric procedures
– If no, rank-order procedures
• Testing for homogeneity
– How to do it
– What it means
Practical Issues
• Tutorial example (pp. 258 – 267)
• Developing a code book (pp. 251 – 252)
• Meta-analysis assignment
– Select a problem for meta-analysis.
– Find five studies about the problem (must have
mean and standard deviation).
– Calculate effect sizes using appropriate formula.
– Determine M and s of the effect sizes.
– Describe what you think the results mean.