Transcript Data Types and Statistical Analysis

Statistical Analysis
MBS-01
Nominal Data
•
•
•
•
•
Dichotomous data
Categorizes variables
Assigns names, letters or descriptors
Gender, race, yes or no
No rank or mathematical
relationship to each other
• Have equivalent weight
or value
Ordinal Data
•
•
•
•
Reflects an order to variables or data
Does not imply magnitude
Zero point is arbitrary
Pain Scale 0-10
Continuous Data
• Interval or Ratio
Data
• Can compare
absolute
magnitude
• Implies a
numeric
relationship
• Can
undergo
arithmetic
operations
• Data can be
averaged
and
manipulated
Measures of Disease Frequency
Incidence Rate
• Proportion of group
initially healthy that will
develop a disease within
a specified period of
time
• Expressed in persondays, person-months,
etc.
• Measures only new cases
Prevalence
• Proportion of people in
the population with
disease at a given time
• Measures all existing
cases
• Underestimates acute
or rapidly occurring
illnesses
Relative Risk and Odds Ratio
•
•
•
•
Measures of association
Measures of disease frequency
Expressed as a single value
Describes the strength of
association between the exposure
and outcome
• Does not imply any extent of
variation
• Often accompanied by a confidence
interval
Relative Risk
• How many times more likely an outcome is for
one group compared with another
• Ranges from 0 to infinity
– RR of 0 is no association
– RR of 1 is risk of acquiring disease same for
subjects with and without risk factor
– Association is stronger as RR increases
(>10 felt to be strong association)
– RR = 0.5 then initial risk is cut in half
– RR = 2 then initial risk is doubled
• Used in cohort (follow-up) design
Odds Ratio
• Estimator of relative risk
• Compares the prevalence of a disease
when a specific factor is present or absent
• Assumes cases & control gp
representative of general population with
respect to occurrence of risk factors
• Assumes the frequency of disease in
exposed or unexposed is small
Odds Ratio
• Cross-sectional & Case-control
design
• Uses single value to describe
strength of the association between
exposure and outcome
• OR <1 then risk as decreased
• OR = 1 no association between risk
factor and disease
• OR >1 then risk has increased
Number Needed to Treat
(NNT)
• How many patients must be treated
to get one good event (or prevent
one bad event)
• Applicable to groups of patients with
similar underlying risk
• Calculated from follow-up and
experimental design studies
Disease
Present
F
a
c
t
o
r
Absent
Exposed A
B
Not
C
Exposed
D
Relative Risk = A /(A+B)
Odds Ratio = A * D
C/(C+D)
B*C
Number Needed to Treat
_________1___________
[A/(A+B) ] - [C/(C+D)]
Research vs Null Hypothesis
Research Hypothesis:
• The hypothesis tested by the study
• Can be one tailed:a difference in only 1 direction
• Can be two tailed:a difference in two directions
Null Hypothesis:
• Opposite of the research hypothesis
• Hypothesis of no difference
• Statistics are applied to the null hypothesis
Interval and Ratio Data
Normal Distribution
Nominal and Ordinal Data
Nonnormal Distribution
Independent Measurements
• Seen in parallel design trials
• Data do not depend on each other
• Data do not reflect serial
measurements
Measure
Treatment
Population
Sample
Outcome
Variables
X
Treatment
Measure
Outcome
Variables
Dependent Measurements
• Seen in cross-over design studies
• Seen in studies using matched
groups
• Data depend on or reflect each other
Sample
X
Treatment
Measure
Outcome
Measure
Outcome
Measure
Outcome
Statistical Analysis
Data
Type
Samples are:
2 Indep
2 Related
3 or > Indep
3 or >
Related
Measure of
Correlation
Nominal
Chi-square
McNemar
Chi square
Cochran Q
Contingency
coefficient
Ordinal
MannWhitney U
or
Wilcoxon
Rank Sum
Sign test or
Wilcoxon
Signed Rank
Test
KruskalWallis oneway
analysis of
variance
(ANOVA)
Friedman 2
way
analysis of
variance
(ANOVA)
Spearman
rank correl
coefficient
Kendel rank
correlation
coefficient
Interval
Student’s t
test
MannWhitney U
test
Paired t test
One-way
analysis of
variance
(ANOVA)
2 way
(repeated
measures)
analysis of
variance
(ANOVA)
Pearson
Correlation
Coefficient
Errors Related to Hypothesis
Research
Conclusion
True Situation: Null Hypothesis is:
True
False
Do Not
Reject H0
(No Difference)
Correct Decision
Confidence Level
Probability =1-
Error: Type II
Probability= 
Reject H0
(Difference)
Error: Type I
Significance Level
Probability = 
Correct Decision
Power of test
Probability =1-
Alpha
• The probability of making a Type I
error
• Predetermined by the investigator
• Usual values 0.05 or 0.1 (1 in 10 or 1
in 20 chance of Type I error)
• P value: the numeric representation
of 
Beta and Power
• Power is the probability of avoiding
a Type II error.
• Or the chance of finding a difference
if it truly exists
• Power is 1-
• Increase power by increasing n,
increasing , or increasing the size
of difference accepted
Summary
• Statistics tell us about the role that sampling
variability plays in results
• Statistics make no claim about the validity of
a study
• Consider the impact of Type I and II Errors
• Results May be Statistically
Significant but
Clinically irrelevant.
Error, Validity and
Precision
Random Error
• Not constant error
• Due to chance
• Unknown sources of variation equally
likely to affect findings in either direction
• Seen as inconsistency in repeated or
equivalent measurements when made on
the same object or person
• Increase sample size to reduce random
error
Systematic Error
• Constant error
• Due to bias
• Sources of variation that affect findings in
one direction
• Improve study design to reduce
• Investigator should include explanation of
systematic error in publication
• Change of sample size will not affect
systematic error
Reliability vs Validity of Data
• Reliability:
reproducibility of measurement
• Validity:
extent to which differences in scores
reflect the true differences among
individuals on the characteristic we
are seeking to measure
Validity
Study Results
Internal Validity
Truth in Study Results
External Validity
Errors of
chance
and bias
Truth in the Universe
Threats to Internal Validity
• History: naturally occurring event
external to study but occurring
simultaneously
• Maturation: change in the study
subject occurring as a function of
the passage of time
• Instrumentation: changes or errors
in the measuring instrument or
observer
Threats to Internal Validity
• Selection: the way in which the subjects
were selected and assigned to treatment
groups
• Experimental Mortality: dropout,
nonresponse, or death
• Main testing effect: being tested may
bring about a change in behavior on a
second observation
• Statistical regression: tendency of
extremes to move toward the mean during
an experiment
Threats to Internal Validity
Hx Mat Ins Sel
Mor
One-Shot Case Study
-
-
-
-
-
One-Gp Pre & Post Tx
-
-
-
+/-
+
Static Group Comp
+
-
+
-
-
Pre & Post Tx Control Gp
+
+
+
+
+
Post Tx only Control Group
Design
+
+
+
+
-
Nonequivalent Pre-post Tx
Control Group Design
+
+
+
+/-
+
Threats to External Validity
• Interaction of Subject Variables and Tx:
tx has varied effects on subgroups
• Interactive Effect of Testing:
pretesting may sensitize subjects to variable
• Reactive Effects of Experimental Arrangements:
study setting may be atypical
• Multiple Treatment Interferences:
same subject given several treatments; effects of
earlier treatments not completely erasable
• Hawthorne Effect:
volunteers try to give “right” answer
Measures of Central Tendency
• Mean: average
• Median: midpoint where ½ of observations
fall above and ½ fall below the value
• Mode: most frequently encountered number
• In a normal, or Gaussian, distribution the
mean, median, and mode are identical
Precision
• How closely the estimates will tend
to cluster about the true value
• Larger the standard deviation or
standard error of the mean the less
precise the data
Normality
6
Mean,
median, &
mode
Patients
5
4
3
2
1
0
0
1
2
3
4
5
6
Pain Score
7
8
9
10
Skewed Distribution
7
Mode
6
Median
Mean
Patients
5
4
3
2
1
0
1
2
3
4
5
6
Pain Score
7
8
9
10
Measures of Variability
Standard Deviation
• Measures how close
the values cluster to
the sample mean
• Interval Data
• Square root of
variance
• Reported as +/• If normal distribution
1 S.D. = 68% data
2 S.D. = 95% data
3 S.D. = 99% data
Standard Error of the
Mean
• Estimates mean of
population from
sample meanˉх
• Equals S.D./ square
root of n
• Smaller number than
SD therefore often
reported as measure
of dispersion
Confidence Intervals
• Further defines the p value by giving a
range of values to describe the data
• An interval that will, with the probability of
a confidence level, contain the true
difference being investigated
• A confidence interval which includes “0”
does not permit rejection of the null
Sample Size, MetaAnalysis & Evidence
Based Medicine
MBS-01
Sample Size
• Determined before initiation of study
• Re-evaluated at conclusion of study due
tp dropouts or deaths
• Often not included in publication
• Nomograms, formulas, tables are
available to assist reader with sample size
calculation
• Studies of inadequate sample size: pilot
studies
Sample Size Determinants
•
•
•
•
•
The outcome being evaluated
Tolerable risk of error (α and β)
Clinically important difference ()
Variability of measurement (s and s2)
Ratio of experimental to control
subjects
Sample Size Determinants
• The outcome being evaluated
– Dichotomous outcome
– Continuous outcome
•
•
•
•
Tolerable risk of error (α and β)
Clinically important difference ()
Variability of measurement (s and s2)
Ratio of experimental to control
subjects
Sample Size Determinants
• The outcome being evaluated
• Tolerable risk of error (α and β)
– Type I (α)
• concludes there is a difference when in fact
there is not
• convention sets risk at 1-in-20 chance
or an α of 0.05
– Type II (β)
• concludes there is no difference when in fact one
exists
• Convention sets risk at 2-in-10 chance or a β of 0.2
• Clinically important difference ()
• Variability of measurement (s and s2)
• Ratio of experimental to control subjects
Sample Size Determinants
• The outcome being evaluated
• Tolerable risk of error (α and β)
• Difference between experimental and
control group ()
– What is clinically important
– Detecting a small difference between
groups requires larger sample size
• Variability of measurement (s and s2)
• Ratio of experimental to control subjects
Sample Size Determinants
•
•
•
•
The outcome being evaluated
Tolerable risk of error (α and β)
Clinically important difference ()
Variability of measurement (s and s2)
– Expressed as s for continuous variables
– Not specified for dichotomous variables
– Smaller spread around mean requires
smaller sample size
• Ratio of experimental to control subjects
Sample Size Determinants
•
•
•
•
•
The outcome being evaluated
Tolerable risk of error (α and β)
Clinically important difference ()
Variability of measurement (s and s2)
Ratio of experimental to control subjects
– One-to-one ratio minimizes sample size
– Assumed to be one-to-one
in Young Nomogram
Sample Size
• Equations & nomograms differ for type of
outcome measured; nominal vs
continuous
• Z (Standard Normal) distribution is used
for alpha and beta
– Mean of 0
– S.D. of 1
• Can solve for any portion of equations if
other factors are known
• Plan for drop-out
Sample Size Equation for Dichotomous
Variables
 zα/2 2P(1 P) zβ P1(1  P1)  P0(1 P0) 
n

P1  P0


2
P: proportion of responders in both groups
P1: proportion of responders in experimental
group
P0: proportion of responders in control groups
Use 2 sided alpha of 0.05
 Zα =1.96
Use one-sided beta of 0.20  Zβ = 0.84
Sample Size Equation for Continuous
Variables
2σ (zα/2  zβ)
n
2
δ
2
Use 2 sided alpha of 0.05
2
 Zα =1.96
Use one-sided beta of 0.20  Zβ = 0.84
 : difference in means for control & experimental group
σ: weighted average standard deviation in the control
group
Nomograms for Calculation of Sample
Size
• Assumes parallel study (no crossover)
with two groups
• Assumes 2 tailed alpha; if not,
overestimates needed sample size
• Assumes alpha 0.05 and beta 0.2
• Approximation used retrospectively to
critique another author’s conclusions
Number of Patients Nomogram
MBS-01
Sample Size and Study Outcome
• Difference Observed
– n is large enough to identify difference
– Difference may be overestimated if n is
smaller than would have been determined
using sample size calculations
– Extremely large n may find difference but
clinical significance of the difference lacking
• No difference Observed
– n may be too small
– Truly may be no difference
Meta-Analysis
A statistical analysis which combines or integrates
the results of several independent clinical trials
considered by the analyst to be combinable.
Goal is to discern a more objective and generalized
answer to a particular question or clinical dilemma
by combining results from different studies
that have similar research hypotheses.
The Meta-Analysis
• Tertiary literature
• Applied to experimental and
observational designs
• Useful when previous studies
inconclusive, contradicting, or sample
size is too small
• Can be used to increase power of a study
• Exercise caution and
skepticism in evaluating
the conclusion!
MBS-01
Quality Areas of Meta-Analysis
•
•
•
•
•
•
Study design
Ability of data to be combined
Control of bias
Statistical analysis
Sensitivity analysis
Application of results
Meta-Analysis: Study Design
• Must clearly define focused clinical
question
• Provide details of literature searches
• Searches should be comprehensive
– Published and unpublished data
– References within publications
– Foreign language literature
• Blind selection of articles
by >1 reviewer
Meta-Analysis: Selection
• Identify how articles selected for
inclusion prior to project initiation
• Identify variables pooled to answer
question prior to project initiation
• Identify quality of articles to be
selected prior to project initiation
• Weighting of articles?
Meta-Analysis: Combinability
• Determine range of variables to be
included
– Identify data at identical timepoints
– Identify baseline and therapeutic
intervention data to be included
• Evaluate homogeneity of the
outcome variables
Meta-Analysis: Bias
•
•
•
•
•
•
•
•
Publication bias
Language bias
Reference bias
Selection bias
Duplicate publication bias
Data extraction bias
Support bias
Bias within clinical trial
Meta-Analysis:Application of Results to
Practice
•
•
•
•
Relative Risk Ratio
Absolute Risk Reduction
Number Needed to Treat
Consideration to trials not included
Evidence Based Medicine
Explicit and judicious use of current best evidence
in making decisions about the care of individual
patients
Evidence Based Medicine
• Convert information needs into a clearly defined
answerable clinical question
• Conduct a systematic search for the best
available evidence for the problem
• Evaluate validity & applicability of evidence
• Prepare a synthesis or summary of the evidence
for decision making and
implement the decision in practice
• Evaluate performance & follow-up
on any areas for improvement