Transcript Chapter 14
chapter 14 Research Synthesis (Meta-Analysis) Chapter Outline • Using meta-analysis to synthesize research • Tutorial example of meta-analysis Contrasting Meta-Analysis With Literature Review • How thorough was the literature search? – Computer search and hand search – Which journals? – Theses and dissertations included? • On what basis were studies included or excluded? – Perceived internal validity – Sample size – Design (continued) Contrasting Meta-Analysis With Literature Review (continued) • How did the author arrive at conclusions? – Vote counting – Differential weighting Purpose of Meta-Analysis • Answer an important question when adequate literature is available. • Report a definitive methodology for the review. • Quantify the results of studies using a standard metric. Steps in a Meta-Analysis • Identification of a problem • Literature searched by specified means • Review of studies to determine inclusion or exclusion • Careful reading and evaluation to identify and code important study characteristics • Calculating effect size • Applying appropriate statistical analyses • Reporting all the steps and outcomes in a paper Calculating Effect Size M1 – M2 ES = s2 Pooled s sP = ( ) ( ) 2 2 + s1 N1 – 1 s2 N2 – 1 N1 + N2 – 2 Distribution Overlap When Group 1 and Group 2 Are 1s Apart (ES = 1.0) Differences Between Girls and Boys (Effect Sizes) by Age From J.R. Thomas and K. E. French, 1985, “Gender differences across age in motor performance,” Psychological Bulletin 98(2): 260-282. Copyright © by American Psychological Association. Reproduced by permission. Differences Between Boys and Girls (Effect Sizes) in Throwing From J.R. Thomas and K. E. French, 1985, “Gender differences across age in motor performance,” Psychological Bulletin 98(2): 260-282. Copyright © by American Psychological Association. Reproduced by permission. Methodological Considerations in Meta-Analysis • Deciding what to code: Using theory and empirical studies to decide on characteristics of importance • Choosing the standard deviation for ES – Experimental versus control – Nonrandomly formed groups (sex, age) – Pretest to posttest • Correcting ES • Weighting ES Statistical Issues for Analyzing Effect Size Data • Are effect sizes normally distributed? – If yes, standard parametric procedures – If no, rank-order procedures • Testing for homogeneity – How to do it – What it means Practical Issues • Tutorial example (pp. 258 – 267) • Developing a code book (pp. 251 – 252) • Meta-analysis assignment – Select a problem for meta-analysis. – Find five studies about the problem (must have mean and standard deviation). – Calculate effect sizes using appropriate formula. – Determine M and s of the effect sizes. – Describe what you think the results mean.