Transcript Chapter 13 Inferential Data Analysis

Chapter 14
Inferential Data Analysis
Inferential Statistics

Techniques that allow us to study samples and
then make generalizations about the population.
Inferential statistics are a very crucial part of
scientific research in that these techniques are
used to test hypotheses
Uses for Inferential Statistics



Statistics for determining differences between
experimental and control groups in experimental
research
Statistics used in descriptive research when
comparisons are made between different
groups
These statistics enable the researcher to
evaluate the effects of an independent variable
on a dependent variable
Sampling Error

Differences between a sample statistic and a
population parameter because the sample is not
perfectly representative of the population
Hypothesis Testing

The purpose of the statistical test is to evaluate
the null hypothesis (H0) at a specified level of
significance (e.g., p < .05)

In other words, do the treatment effects differ
significantly so that these differences would be
attributable to chance occurrence less than 5 times in
100?
Hypothesis Testing Procedures





State the hypothesis (H0)
Select the probability level (alpha)
Determine the value needed for significance
Calculate the test statistic
Accept or reject H0
Statistical Significance

A statement in the research literature that the
statistical test was significant indicates that the
value of the calculated statistic warranted
rejection of the null hypothesis

For a difference question, this suggests a real
difference and not one due to sampling error
Parametric Statistics

Techniques which require basic assumptions about
the data, for example:





normality of distribution
homogeneity of variance
requirement of interval or ratio data
Most prevalent in HHP
Many statistical techniques are considered robust to
violations of the assumptions, meaning that the
outcome of the statistical test should still be
considered valid
t-tests

Characteristics of t-tests




requires interval or ratio level scores
used to compare two mean scores
easy to compute
pretty good small sample statistic
Types of t-test

One-Group t-test


Independent Groups t-test


t-test between a sample and population mean
compares mean scores on two independent samples
Dependent Groups (Correlated) t-test


compares two mean scores from a repeated
measures or matched pairs design
most common situation is for comparison of pretest
with posttest scores from the same sample
Hypothesis Testing Errors



Hypothesis testing decisions are made without
direct knowledge of the true circumstance in the
population. As a result, the researcher’s
decision may or may not be correct
Type I Error
Type II Error
Type I Error

. . . is made when the researcher rejects the null
hypothesis when in fact the null hypothesis is true


probability of committing Type I error is equal to the
significance (alpha) level set by the researcher
thus, the smaller the alpha level the lower the chance
of committing a Type I error
Type II Error

. . . occurs when the researcher accepts the null
hypothesis, when in fact it should have been rejected

probability is equal to beta (B) which is influenced by
several factors
 inversely related to alpha level
 increasing sample size will reduce B

Statistical Power – the probability of rejecting a false
null hypothesis


Power = 1 – beta
Decreasing probability of making a Type II error
increases statistical power
Hypothesis Truth Table
NULL HYPOTHESIS
TRUE
FALSE
ACCEPT
CORRECT
DECISION
TYPE II
ERROR
REJECT
TYPE I
ERROR
CORRECT
DECISION
DECISION
ANOVA - Analysis of Variance

A commonly used family of statistical tests
that may be considered a logical extension of
the t-test




requires interval or ratio level scores
used for comparing 2 or more mean scores
maintains designated alpha level as compared to
experimentwise inflation of alpha level with multiple
t-tests
may also test more than 1 independent variable as
well as interaction effect
One-way ANOVA

Extension of independent groups t-test, but may
be used for evaluating differences among 2 or
more groups
Repeated Measures ANOVA

Extension of dependent groups t-test, where
each subject is measured on 2 or more
occasions


a.k.a “within subjects design”
Test of sphericity assumption is recommended
Random Blocks ANOVA


This is an extension of the matched pairs t-test
when there are three or more groups or the
same as the matched pairs t-test when there
are two groups
Participants similar in terms of a variable are
placed together in a block and then randomly
assigned to treatment groups
Factorial ANOVA

This is an extension of the one-way ANOVA for
testing the effects of 2 or more independent
variables as well as interaction effects


Two-way ANOVA (e.g., 3 X 2 ANOVA)
Three-way ANOVA (e.g., 3 X 3 X 2 ANOVA)
Assumptions of Statistical Tests

Parametric tests are based on a variety of
assumptions, such as



Interval or ratio level scores
Random sampling of participants
Scores are normally distributed





N = 30 considered minimum by some
Homogeneity of variance
Groups are independent of each other
Others
Researchers should try to satisfy assumptions
underlying the statistical test being used
Improving the Probability of Meeting
Assumptions



Utilize a sample that is truly representative of
the population of interest
Utilize large sample sizes
Utilize comparison groups that have about the
same number of participants
Two-Group Comparison Tests


a.k.a. Multiple Comparison or Post Hoc Tests
The various ANOVA tests are often referred to
as “omnibus” tests because they are used to
determine if the means are different but they
do not specify the location of the difference

if the null hypothesis is rejected, meaning that
there is a difference among the mean scores, then
the researcher needs to perform additional tests in
order to determine which means (groups) are
actually different
Common Post Hoc Tests

Multiple comparison (post hoc) tests are used to
make specific comparisons following a
significant finding from ANOVA in order to
determine the location of the difference




Duncan
Tukey
Bonferroni
Scheffe

Note that post hoc tests are only necessary if there are
more than two levels of the independent variable
Analysis of Covariance


ANOVA
ANOVA design which statistically adjusts the
difference among group means to allow for the
fact that the groups differ on some other
variable

frequently used to adjust for inequality of groups at
the start of a research study
Nonparametric Statistics



Considered assumption free statistics
Appropriate for nominal and ordinal data or in
situations where very small sample sizes (n < 10)
would probably not yield a normal distribution of
scores
Less statistical power than parametric statistics
Chi Square

A nonparametric test used with nominally scaled
data which are common with survey research

The statistic is used when the researcher is
interested in the number of responses, objects, or
people that fall in two or more categories
Single Sample Chi-Square


a.k.a one-way chi-square or goodness of fit chisquare
Used to test the hypothesis that the collected
data (observed scores) fits an expected
distribution

i.e. are the observed frequencies and expected
frequencies for a questionnaire item in agreement
with each other?
Independent Groups Chi-Square



a.k.a. two-way chi-square or contingency table
chi-square
Used to test if there is a significant relationship
(association) between two nominally scaled
variables
In this test we are comparing two or more
patterns of frequencies to see if they are
independent from each other
Overview of Multivariate Tests

Univariate statistic –


used in situations where each participant contributed
one score to the data analysis, or in the case of a
repeated measures design, one score per cell
Multivariate statistic –

used in situations where each participant contributes
multiple scores
Example Multivariate Tests




MANOVA
Canonical correlation
Discriminant analysis
Factor analysis
Multiple Analysis of Variance



MANOVA
Analogous to ANOVA except that there are
multiple dependent variables
Represents a type of multivariate test
Prediction and Regression Analysis


Correlational technique
Simple prediction



Predicting an unknown score (Y) based on a single
predictor variable (X)
Y’ = bX + c
Multiple prediction


Involves more than one predictor variable
Y’ = b1X1 + b2X2 + c
Multiple Regression/Prediction



a.k.a multiple correlation
Determines the relationship between one
dependent variable and 2 or more predictor
variables
Used to predict performance on one variable
from another


Y’ = b1X1 + b2X2 + c
Standard error of prediction is an index of
accuracy of the prediction
Statistical Power

The probability that the statistical test will
correctly reject a false null hypothesis


. . . it is effectively the probability of finding
significance, that the experimental treatment actually
does have an effect
a researcher would like to have a high level of power
Statistical Power

alpha = probability of a Type I error



beta = probability of a Type II error


rejecting a true null hypothesis
this is your significance level
failing to reject a false null hypothesis
Statistical power = 1 - beta
Factors Affecting Power




Alpha level
Sample size
Effect size
One-tailed or two-tailed test
Alpha level

Reducing the alpha level (moving from .05 to
.01) will reduce the power of a statistical test.
This makes it harder to reject the null
hypothesis
Sample size

In general, the larger the sample size the
greater the power. This is because the standard
error of the mean decreases as the sample size
increases
One-tailed versus two-tailed tests

It is easier to reject the null hypothesis using a
one-tailed test than a two-tailed test because
the critical region is larger
Effect size



This is an indication of the size of the treatment
effect, its meaningfulness
With a large effect size, it will be easy to detect
differences and statistical power will be high
But, if the treatment effect is small, it will be
difficult to detect differences and power will be
low
Effect Size


Numerous authors have indicated the need to
estimate the magnitude of differences between
groups as well as to report the significance of
the effects
One way to describe the strength of a treatment
effect, or meaningfulness of the findings, is the
computation of “effect size” (ES)
ES =
M1 - M2
SD
Note: SD represents the standard deviation of the control group or the pooled standard
deviation if there is no control group
Effect Size

Interpretation of ES by Cohen (1988)




0.2 represents a small ES
0.5 represents a moderate ES
0.8 represents a large ES
Researchers using experimental designs are
advised to provide post hoc estimates of ES for
any significant findings as a way to evaluate the
meaningfulness
A Priori Procedures

Calculate the power for each of the statistical
procedures to be applied


requires three indices - alpha, sample size, effect
size
Estimate the sample size needed to detect a
certain effect (ES) given a specific alpha and
power

may require an estimation of ES from previous
published studies or from a pilot study