Transcript Meta-Analysis

Statistics for clinicians
Biostatistics course by Kevin E. Kip, Ph.D., FAHA
Professor and Executive Director, Research Center
University of South Florida, College of Nursing
Professor, College of Public Health
Department of Epidemiology and Biostatistics
Associate Member, Byrd Alzheimer’s Institute
Morsani College of Medicine
Tampa, FL, USA
1
SECTION 7.1
Introduction to
Meta-Analysis
Meta analysis and
measures of public
health impact
Learning Outcome:
Recognize the methodological basis for
conducting meta-analyses.
Key Questions
Question 1:
Why is it difficult to appropriately
evaluate/synthesize findings from multiple
sources of research?
Question 2:
Why do we often arrive at different or
incorrect conclusions in groups of studies,
and among individual studies?
Traditional Definitions
Meta-Analysis: A quantitative approach for
systematically combining the RESULTS of
previous research in order to arrive at
conclusions about the body of research.
(e.g. Determination of risk ratio and
confidence interval across studies – Is
Intervention A better than Intervention B?)
Traditional Definitions
Pooled-Analysis: Pooling of PRIMARY DATA
from multiple studies for the purpose of
conducting an analysis of the enlarged data
set.
(Often referred to as “meta-analysis using
individual patient data”).
National Health and Medical Research Council (NHMRC) levels of evidence
and corresponding National Heart, Lung, and Blood Institute categories
NHLBI
category
NHMRC
level
Basis of Evidence
A
I
Evidence obtained from a systematic review of all
relevant randomised controlled trials
B
II
Evidence obtained from at least one properly
designed randomised controlled trial
C
III - 1
Evidence obtained from well-designed
pseudorandomised controlled trials (alternate
allocation or some other method)
C
III - 2
Evidence obtained from comparative studies
(including systematic reviews of such studies) with
concurrent controls and allocation not randomised,
cohort studies, case-control studies, or interrupted
time series with a control group
C
III - 3
Evidence obtained from comparative studies with
historical control, two or more single arm studies, or
interrupted time series without a parallel group
C
IV
Evidence obtained from case series, either post-test
or pretest/ post-test
The Need for Meta-Analysis
1940:
2,300 Biomedical journals
2011:
39,529 journals listed in PubMed
2011:
More than 10,000 randomized
clinical trials conducted each year
Similar proliferation of journal
articles for social science disciplines
Range of Reaction to Meta-Analysis
Supportive:
Cook, Guyatt (1994)
The professional meta-analyst: an evolutionary
advantage
Rosendaal (1994)
The emergence of a new species: the professional
meta-analyst
Feinstein (1995)
Meta-analysis: statistical alchemy for the 21st
century
Range of Reaction to Meta-Analysis
Neutral:
Meinart (1989)
Meta-analysis: science or religion?
Spector, Thompson (1991)
The potential and limitations of meta-analysis.
Bailar (1997)
The promise and problems of meta-analysis.
Range of Reaction to Meta-Analysis
Critical:
Eysenck (1978)
An exercise in mega-silliness.
Chalmers (1991)
Problems induced by meta-analysis
Thompson, Pocock (1991)
Can meta-analysis be trusted?
Greenland (1994)
Can meta-analysis be salvaged?
Shapiro (1994)
Meta-analysis/schmeta-analysis
SECTION 7.2
Steps Involved
In Meta Analysis
Learning Outcome:
Demonstrate the primary steps involved
in conducting a meta-analysis
Steps in a Meta-Analysis
Step 1:
Identify studies with relevant data
Step 2:
Define inclusion and exclusion criteria
for component studies
Step 3:
Abstract the data
Step 4:
Analyze the data
Steps in a Meta-Analysis
Step 1:
Identify studies with relevant data
•
Completeness of information
(e.g. published and unpublished reports)
•
Specificity of hypothesis (e.g. similarity of
treatments and/or exposure)
•
Choice among multiple publications
Steps in a Meta-Analysis
Step 2:
Define inclusion and exclusion
criteria for component studies
•
Study designs (experimental, observational, etc.)
•
Years of publication of study conduct
•
Languages (e.g. English language only)
•
Incomplete data (e.g. loss to follow-up)
•
Quality (e.g. subject selection, blinding, treatment
compliance, statistical methods, etc.)
Step 2: Define inclusion and exclusion criteria for studies
Steps in a Meta-Analysis
Step 3:
Abstract the data
•
Select the desired measure of effect and
reported estimate (e.g. odds ratio,
standardized mean difference, correlation
coefficient)
--unadjusted
--adjusted for age only
--adjusted for multiple confounders, etc.
•
Are the data available for subgroup
analyses?
Selection of Component Studies
Concern: Publication Bias
Statistically significant results 3x more likely to be
published than papers affirming a null result.(1)
Most common reason for non-publication is
investigator declining to submit results(2) e.g.:
--loss of interest in topic
--expectation that others will not be
interested in null results
Also known as “file drawer” bias.
1. Dickersin K, Chan S, Chalmers TC, et al. Publication bias and clinical trials. Controlled Clin Trials 1987; 8: 343-53
2. Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet 1991;337:86772.
http://clinicaltrials.gov/ct2/home
ClinicalTrials.gov is a registry and results database of federally and
privately supported clinical trials conducted in the United States and
around the world. ClinicalTrials.gov gives you information about a
trial's purpose, who may participate, locations, and phone numbers
for more details. This information should be used in conjunction
with advice from health care professionals.
Who is responsible for registering the trial?
In most cases, the Sponsor of the trial as defined by FDA regulations
[21 CFR 50.3(e)] has the obligation to register the clinical trial with ClinicalTrials.gov.
Steps in a Meta-Analysis
Step 4:
Analyze the data (statistical methods)
•
Fixed effects
•
Random effects
•
Bayesian approach (not discussed)
SECTION 7.3
Statistical Methods
in Meta-Analysis
Learning Outcomes:
Recognize analytical considerations in
conducting a meta-analysis
Calculate and interpret a summary
odds ratio using the MantelHaenszel method
Step 4:
Analyze the data (statistical methods)
Fixed Effect Method: Meta-Analysis
---
The “within-study” variance is used as the
weighting factor for each study
Example: MH method to estimate the summary
odds ratio:
ORMH =
sum(weighti x ORi)
---------------------------sum(weighti)
where weighti
Note – variance is for log(OR)
=
1 / variance
Example: Mantel Haenszel Summary Odds Ratio
Study 1
Study 2
Study 3
Study 4
Study 5
Study 6
Study 7
Totals
CASES
Exposed Not exposed
49
67
44
64
102
126
32
38
85
52
246
219
1570
1720
2128
2286
CONTROLS
Exposed Not exposed
566
557
714
707
730
724
285
271
725
354
2021
2038
7017
6880
12058
OR
0.72
0.68
0.80
0.80
0.80
1.13
0.89
Variance Weight W x OR
0.0389
25.71
18.50
0.0412
24.29
16.54
0.0205
48.80
39.18
0.0648
15.44
12.36
0.0352
28.41
22.67
0.0096 103.99 117.79
0.0015 663.92 594.19
11531
910.56 821.24
Example Calculations for Study #1
OR = (Odds of exposure among cases / Odds of exposure among controls)
OR = (49 / 67) / (566 / 557) = 0.72
Variance - (log)OR = ((1/N1) + (1/N2) + (1/N3) + (1/N4))
Variance – log(OR) = ((1/49) + (1/67) + (1/566) + (1/557) = 0.0389
Weight = 1 / variance = 1 / 0.0389 = 25.71
Weight x OR = 25.71 x 0.72 = 18.50
ORMH
sum(weighti x ORi)
= ---------------------------sum(weighti)
821.24
OR = ---------------------910.56
= 0.90
Practice: Calculate the Mantel-Haenszel Summary Odds Ratio for the
Studies Listed Below
Study 1
Study 2
Study 3
Study 4
Study 5
Study 6
Totals
CASES
Exposed Not exposed
32
72
19
39
12
20
62
112
43
72
90
192
258
507
CONTROLS
Exposed Not exposed
111
445
54
132
16
44
48
110
51
74
74
202
354
1007
OR
Variance
Weight W x OR
OR = (Odds of exposure among cases / Odds of exposure among controls)
Variance – log(OR) = ((1/N1) + (1/N2) + (1/N3) + (1/N4))
Weight (W) = 1 / variance
ORMH
sum(weighti x ORi)
= ---------------------------sum(weighti)
OR = ----------------------
=
Practice: Calculate the Mantel Haenszel Summary Odds Ratio for the
Studies Listed Below
Study 1
Study 2
Study 3
Study 4
Study 5
Study 6
Totals
CASES
Exposed Not exposed
32
72
19
39
12
20
62
112
43
72
90
192
258
507
CONTROLS
Exposed Not exposed
111
445
54
132
16
44
48
110
51
74
74
202
354
1007
OR
1.78
1.19
1.65
1.27
0.87
1.28
Variance
0.0564
0.1044
0.2186
0.0550
0.0703
0.0348
Weight W x OR
17.73
31.59
9.58
11.41
4.58
7.55
18.19
23.07
14.23
12.33
28.75
36.79
93.06 122.75
OR = (Odds of exposure among cases / Odds of exposure among controls)
Variance – log(OR) = ((1/N1) + (1/N2) + (1/N3) + (1/N4))
Weight (W) = 1 / variance
ORMH
sum(weighti x ORi)
= ---------------------------sum(weighti)
122.75
OR = ---------------------93.06
Across the 6 studies, the summary odds ratio is 1.32
= 1.32
Fixed Effect Method: Meta-Analysis
Since larger studies are more precise the smaller
studies (hence smaller confidence intervals), the
fixed effect approach gives more weight in the
analysis to the larger studies.
The inference applies to only the studies
included in the meta-analysis.
Random Effects: Meta-Analysis
---
The “within-study” and “between-study”
variances are used as the weighting factor
for each study.
Example: Dersimonian and Laird method:
ln ORDL =
sum(w*i x ln ORi)
---------------------------sum w*i
where w*i =
1 / [D + (1 / wi)]
Random Effects: Meta-Analysis
For the term in w*i
wi = 1 / variance
(within-study variance)
[Q – (S – 1)] x sum wi
D = ------------------------------[(sum wi)2 – sum (wi2)]
Q = sum wi(ln ORi – ln ORMH)2
(betweenstudy
variance)
Random Effects: Meta-Analysis
The addition of the between-variance term
usually results in a more conservative estimate
(larger confidence interval) than the fixed effects
method.
Larger studies AND those with disparate results
are given more weight in the analysis.
The inference applies to the research question at
large, not just the studies included in the metaanalysis.
Fixed Effect
Note differences in Relative Weights
Random Effects
Note wider CI compared to fixed effect
Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Research Synthesis Methods 2010; 1:97-111.
SECTION 7.4
Heterogeneity among
studies in metaanalyses
Learning Outcome:
Explain the importance of investigating
heterogeneity among studies included
in a meta-analysis.
Heterogeneity Among Studies
Statistical heterogeneity: Disparate results
across studies comprising a common
research question.
1) Statistical question is whether there is greater
variation between the results of the studies
than is compatible with the play of chance.
2) Tests of heterogeneity typically lack statistical
power – hence they are often inconclusive.
More Recent Issues - Heterogeneity
Heterogeneity:
Petiti (2001)
Approaches to heterogeneity in meta-analysis
Higgins (2002)
Quantifying heterogeneity in a meta-analysis
Huedo-Medina (2006)
Assessing heterogeneity in meta-analysis
Lau (1998)
Summing up evidence: one answer is not
always enough
Investigating Heterogeneity
1) Qualitative: Compare effect estimates of
component studies by respective design and
methodological characteristics:
•
•
•
•
•
•
Year of publication
Type/dose of intervention
Outcome assessment
Length and number of follow-up intervals
Study inclusion/exclusion criteria
Study size
Investigating Heterogeneity
2) Quantitative: Investigate variation in effect
estimates by statistical means:
• “Meta” regression analysis
--- variation in parameter of interest serves as
the dependent variable
--- treatment intervention or factor of interest
is included in the model with other study
variables suspected to contributing of
heterogeneity
Scenario: Assume among 10 randomized
controlled trials, there is evidence of
statistical heterogeneity. The trials have
the following features:
1) All address the same basic research question
(e.g. is treatment A better than treatment B?)
2) They all have sufficiently large sample size.
3) They are all of reasonably good quality
(e.g. no obvious problems of validity).
What are some of the possible sources of
Heterogeneity seen between the 10 trials?
1. Patient Population Heterogeneity:
• Age, gender, race, etc.
• Baseline medical history
• Baseline inclusion/exclusion criteria
2. Intervention Heterogeneity:
• Therapeutic improvements over time
• Individual versus group therapy
3. Outcome Measurement
Heterogeneity:
• Study endpoints ascertained using
different criteria/measures
4. Study Protocol Heterogeneity:
• Requirements for patient follow-up
(e.g. length, number of protocol
specified visits)
Left Main
LAD
Right
Coronary
Artery
LCx
SOURCE OF STATISTICAL HETEROGENEITY - EXAMPLE
TRIALS COMPARING ANGIOPLASTY VS. BYPASS SURGERY - SELECTED CATEGORIES OF BASELINE EXCLUSION CRITERIA
Prior MI
Age
Unstable
Angina
< 35%
Yes - w/i 10
days/overt
cardiac failure
> 76
-----
Left Main
-----
-----
-----
-----
EAST
Single
Left Main w/
>30% stenosis
< 25%
Yes
w/i 5 days
-----
-----
GABI
Single
Left Main w/
>30% stenosis
-----
Yes
w/i 4 weeks
> 75
-----
MASS
All except single w/at
least 80% proximal
LAD stenosis
LV
dysfunction
Yes
-----
Yes
Lausanne
All except single w/at
least 50% proximal
LAD stenosis
< 50%
Yes
(Q-wave
anterior)
-----
Yes
< 35%
Yes
evolving acute
MI
-----
-----
-----
Yes - acute
req. emerg.
revasc.
> 80
Yes
TRIAL
Vessel Disease
CABRI
Single
Left Main
Severe Triple
RITA
ERACI
BARI
Single
Severe left main
Severe triple
Single
Left Main w/>50%
stenosis
Ejection
Fraction
Testing for Statistical Heterogeneity
Q Test (Cochran, 1954)
•
Sum squared deviations of each study effect estimate from overall
effect estimate, weighting contribution of each study by
inverse of variance
•
H0: Homogeneity of effect sizes; Q statistic follows χ2
distribution with k-1 (#studies) degrees of freedom
•
Q statistic poor power to detect true heterogeneity among studies
when meta-analysis includes small number of studies.
•
Non-significant result is often inconclusive
Q = wi (Ti – T)2
Testing for Statistical Heterogeneity
Q = wi (Ti – T)2
For Odds ratio, the effect size (T), is in the log scale – ln(OR)
Study 1
Study 2
Study 3
Study 4
Study 5
Totals
CASES
Exposed Not exposed
191
295
21
97
75
165
99
132
74
607
460
1296
CONTROLS
Exposed Not exposed
141
312
19
92
68
139
87
134
111
735
426
1412
Summary OR
OR
1.43
1.05
0.93
1.16
0.81
ln(OR)
0.3595
0.0472
-0.0735
0.1442
-0.2141
Variance Weight W x OR
0.0189
52.85
75.71
0.1214
8.24
8.63
0.0413
24.22
22.50
0.0366
27.30
31.53
0.0255
39.17
31.62
151.76
1.12
(Ti – T)2
0.0606
0.0044
0.0349
0.0009
0.1073
170.00
0.1135
OR = (Odds of exposure among cases / Odds of exposure among controls)
Ln(OR) = natural logarithm of odds ratio
Variance – log(OR) = ((1/N1) + (1/N2) + (1/N3) + (1/N4))
Weight (W) = 1 / variance
sum(weighti x ORi)
Summary OR(T) = ---------------------------sum(weighti)
Critical value for χ2 at p=0.05= 9.49 (5-1 = 4 d.f. = see table 3 of textbook)
Even though χ2 of 8.31 < 9.49, individual OR’s look different…….
Qi
3.2003
0.0362
0.8463
0.0259
4.2031
8.31
Practice: Testing for Statistical Heterogeneity
Q = wi (Ti – T)2
Study 1
Study 2
Study 3
Study 4
Totals
CASES
Exposed Not exposed
38
62
18
54
50
60
29
48
135
224
For Odds ratio, the effect size (T), is in the log scale – ln(OR)
CONTROLS
Exposed Not exposed
44
55
24
48
30
62
32
42
130
OR
ln(OR)
Variance
Weight W x OR
207
Summary OR
OR = (Odds of exposure among cases / Odds of exposure among controls)
Ln(OR) = natural logarithm of odds ratio
Variance – log(OR) = ((1/N1) + (1/N2) + (1/N3) + (1/N4))
Weight (W) = 1 / variance
sum(weighti x ORi)
Summary OR(T) = -------------------------sum(weighti)
=
-------
Q = _______
Critical value for χ2 at p=0.05= _________
Conclusion: ________________________________
=
(Ti – T)2
Qi
Practice: Testing for Statistical Heterogeneity
For Odds ratio, the effect size (T), is in the log scale – ln(OR)
Q = wi (Ti – T)2
Study 1
Study 2
Study 3
Study 4
CASES
Exposed Not exposed
38
62
18
54
50
60
29
48
Totals
135
224
CONTROLS
Exposed Not exposed
44
55
24
48
30
62
32
42
130
OR
0.77
0.67
1.72
0.79
207
Summary OR
ln(OR)
-0.2664
-0.4055
0.5436
-0.2320
Variance
0.0834
0.1366
0.0861
0.1104
Weight W x OR
12.00
9.19
7.32
4.88
11.61
20.00
9.06
7.18
39.99
1.03
(Ti – T)2
0.0885
0.1906
0.2627
0.0692
41.25
0.0311
OR = (Odds of exposure among cases / Odds of exposure among controls)
Ln(OR) = natural logarithm of odds ratio
Variance – log(OR) = ((1/N1) + (1/N2) + (1/N3) + (1/N4))
Weight (W) = 1 / variance
sum(weighti x ORi)
Summary OR(T) = -------------------------sum(weighti)
=
41.25
------39.99
= 1.03
Q = 6.13
Critical value for χ2 at p=0.05= 7.81 (4-1 = 4 d.f. = see table 3 of textbook)
χ2 of 6.13 < 7.81
Conclusion: Do not conclude the studies differ in effect size.
Qi
1.0618
1.3955
3.0498
0.6270
6.13
Testing for Statistical Heterogeneity
I2 Index (Higgins and Thompson, 2002)
•
Percentage of total variability in a set of effect sizes due to true
heterogeneity (i.e. between-study variability)
•
Example: I2 = 50 means half of total variability in effect sizes is
not due to sampling error, but by true heterogeneity between
studies.
•
classification of I2 values (Higgins and Thompson, 2002)
I2
~ 25%:
“Low” heterogeneity
I2
~ 50%:
“Medium” heterogeneity
I2
~ 75%:
“High” heterogeneity
I2 = ((Q - (n-1)) / Q) x 100
where n = number of studies (i.e. d.f for Q)
Testing for Statistical Heterogeneity
I2 Index (Higgins and Thompson, 2002)
I2 = ((Q - (n-1)) / Q) x 100
where n = number of studies (i.e. d.f for Q)
Study 1
Study 2
Study 3
Study 4
Totals
CASES
Exposed Not exposed
38
62
18
54
50
60
29
48
135
224
CONTROLS
Exposed Not exposed
44
55
24
48
30
62
32
42
130
207
Summary OR
OR
0.77
0.67
1.72
0.79
ln(OR)
-0.2664
-0.4055
0.5436
-0.2320
Variance
0.0834
0.1366
0.0861
0.1104
Weight W x OR
12.00
9.19
7.32
4.88
11.61
20.00
9.06
7.18
39.99
1.03
(Ti – T)2
0.0885
0.1906
0.2627
0.0692
41.25
0.0311
I2 = ((6.13 - (4-1)) / 6.13) x 100 =
“medium heterogeneity”
51.09
Qi
1.0618
1.3955
3.0498
0.6270
6.13
SECTION 7.5
Example of
Meta-Analysis
and Conclusions
Learning Outcome:
Recognize methodological considerations
in interpreting results of meta-analyses
Example: Meta-analysis of St. John’s
Wort for Depression (Kim et al. – 1999)
Criteria for Trial Inclusions:
1.
Any language
2.
Blinded controlled studies of St. John’s Wort versus
standard anti-depressant medications
3.
Subjects from similar SES backgrounds with depressive
disorders defined by either ICD 10, DSM-IIIR or DSMIV criteria
4.
Clinical outcomes measured with the Hamilton
Depression Scale
5.
Varying dosages of St. John’s Wort
Example: Meta-analysis of St. John’s Wort
for Depression (Kim et al. – 1999)
Study
1
2
3
4
N
165
102
139
80
RR
0.79
0.92
1.15
1.14
95% C.I.
0.63 – 1.00
0.67 – 1.26
0.87 – 1.53
0.90 – 1.45
MA – Fixed
MA - Random
486
486
1.11
0.98
0.92 – 1.29
0.67 – 1.28
Meta-analysis of St. John’s Wort
Four controlled trials versus anti-depressant treatment:
Response Rate
RR (95% CI)
Dropout Rate
RR (95% CI)
Side Effect Rate
RR (95% CI)
Study
N
1
165
0.79
(0.63, 1.00)
0.78
(0.48, 1.29)
0.57
(0.42, 0.79)
2
102
0.92
(0.67, 1.26)
0.78
(0.31, 1.93)
0.72
(0.40, 1.32)
3
139
1.15
(0.87, 1.53)
0.26
(0.03, 2.31)
0.51
(0.27, 0.96)
4
80
1.14
(0.90, 1.45)
1.00
(0.15, 6.76)
0.41
(0.21, 0.78)
All
486
Fixed
1.11
(0.92,
1.29)
Rand.
0.98
(0.67
1.28)
Fixed
0.65
(0.39,
0.94)
Rand.
-------
Fixed
0.58
(0.47,
0.77)
Rand.
-------
Example: Meta-analysis of St. John’s Wort
for Depression (Kim et al. – 1999)
In this example, both the fixed effect result and
random effects result suggest that St. John’s
Wort provides similar treatment effect to
conventional anti-depressant medication.
However, the meta-analysis also found that St.
John’s Wort was associated with a lower dropout
rate (RR = 0.65, 95% CI: 0.39 – 1.94) and a lower
side-effect rate (RR = 0.58, 95% CI: 0.47 – 0.77).
CTT meta-analysis: Effects of MORE vs. LESS STATIN
on MAJOR VASCULAR EVENTS
Events (%)
Treatment
Outcome
Non fatal MI
CHD death
Any major coronary event
CABG
PTCA
Unspecified
Any coronary revascularisation
Haemorrhagic stroke
Presumed ischaemic stroke
Any stroke
Any major vascular event
99% or
RR (CI)
Control
1175 (5.9%)
730 (3.7%)
1380 (7.0%)
804 (4.1%)
0.85 (0.76 - 0.94)
0.91 (0.80 - 1.03)
1804 (9.1%)
2072 (10.5%)
0.86 (0.81 - 0.92)
637 (3.2%)
731 (3.7%)
0.86 (0.75 - 0.99)
1167 (5.9%)
322 (1.6%)
1508 (7.6%)
382 (1.9%)
0.76 (0.69 - 0.84)
0.82 (0.68 - 1.00)
2126 (10.7%)
2621 (13.2%)
0.80 (0.75 - 0.85)
63 (0.3%)
509 (2.6%)
57 (0.3%)
606 (3.1%)
1.10 (0.69 - 1.77)
0.84 (0.72 - 0.98)
572 (2.9%)
663 (3.4%)
0.86 (0.77 - 0.96)
3777 (19.0%)
4376 (22.1%)
0.85 (0.81 - 0.89)
95% CI
0.5
Treatment better
0.75
1
1.25
1.5
Control better
“Beneficial” Effects of High Dose Statins
(likely overestimated for general population)
1. RCTs often enroll “vanilla” patients without high
comorbidities or other factors related to safety and
efficacy.
2. Compliance with statin therapy in the general
population is notoriously low.
3. Metabolic syndrome is key target today – statins
have little to no impact of HDL and triglycerides.
4. Life insurance companies would basically “care
less” about total or LDL cholesterol.
Standard figure used in meta-analyses:
(to depict size and relative risk
of individual studies)
Yang, 2004
Chung, 2008
Libby, 2009
Currie, 2009
Lee, 2011
Combined
0.5
Risk ratio
1
2
Analytical Recommendations
1) Consider multiple methods of analysis and
compare results across methods.
2) Conduct a sensitivity analysis to assess the
influence of each individual study on the
overall results.
3) Explore sources of heterogeneity both
qualitatively and quantitatively.
Meta-Analysis – Concluding Remarks
4) There is a lack of consensus on the appropriate
statistical techniques to use, particularly when
evidence of heterogeneity exists between
studies.
5) When heterogeneity is present, calculation of an
“average” effect may be of dubious validity (to
what study population do the results apply?)