Transcript Methods in Education (2) Correlational Approaches

Statistics for Education Research
Lecture 6
Introduction to ANOVA, One-Way
ANOVA, & ANCOVA
Instructor: Dr. Tung-hsien He
[email protected]
ANOVA: ANalysis Of VAriance (變異數分析)
1. Assumptions for ANOVA
a. Samples are independent samples (Independence)
b. Populations are normal distributed (Normality)
c. Variances in the distributions of populations are
equal: Homogeneity of Variance
d. Use K-S tests and Levene’s tests to test Normality
and Homogeneity (just like what we do for t tests).

2. Conditions for ANOVA:
a. Three or more independent means are tested
simultaneously.
b. These means are on one single dependent variable.
c. These means are from 3 or more independent
samples.
d. K populations or K samples when K > 2.
Note: K > 2 is only a necessary but not sufficient
condition.
(a) K may be 1 when repeated measures are used 3
times or more and yield 3 or more dependent
means (or termed as related means). This is called
ANOVA with repeated measures.
(b) K may be 2 when repeated measures or factorial
designs are used to test more than 2 means
simultaneously. This is called two-way mixed ANOVA
or two-way ANOVA.
e. Bonferroni Adjustment Techniques should be used
if a type of ANOVA is repeatedly used for a number of
times on different dependent variables,
respectively (i.e., the original  is divided by the
number of tests and use the divided  to decide if
Ho should be retained or rejected.).
3. Purpose for Performing ANOVA: To identify causality
between independent variables and the single
dependent variable.
Types of ANOVA: Depending on the number of
independent variables:
1. One-Way ANOVA: One Independent Variable (i.e., one
factor that is used to classify participants into 3
groups).
2. Two-Way ANOVA: Two Independent Variables (i.e.,
two factors that are used to classify participants
into 4 subgroups or more).
3. MANOVA: with more than one dependent variable
(i.e., multiple dependent variables) that are tested
simultaneously.

Note:
MANOVA is a more preferred technique by
researchers over ANOVA with Bonferroni
Adjustment Techniques.
Conditions for Using One-Way ANOVA (All conditions
must be met):
1. One independent variable is used to classify
participants into 3 group.
2. The 3 groups are measured to collect data of a
single dependent variable.
3. The three means of the three independent groups
are compared to one another to see if they are
different from one another (pairwise comparisons).
4. See example, Table 13.1, p. 317.

Rationale for Using One-Way ANOVA
1. Repeating t tests will increase probabilities of
making Type I errors (i.e., rejecting false Ho), that is,
 will be inflated (i.e., the value of  will increase).
2. E.g., if t tests are performed 4 times and the original
 is set at 0.05, the actual  value will be 0.05 * 4 =
0.2. This will increase the rejection area and make
it easer to reject Ho compared to the original value.
When Ho becomes easier to be rejected, the chance
to reject Ho increases, so the chance of rejecting a
false Ho increases. That is why the probability to
make Type I errors increases.

3. ANOVA will maintain the  level (because ANOVA will
be performed just one time).
4. An alternative is using Bonferroni adjustment
technique when t tests are repeatedly used. But,
because ANOVA that can precisely maintain the
desired  level, researchers prefer ANOVA to
Bonferroni adjustment technique.
5. To identify the causal relation between the
independent variable and dependent variable;
changes in the dependent variable are presumed to
the result of changes in the independent variable.
Hypotheses Testing for One-way ANOVA:
1. Ho: i = J= k = . . .
2. Ha: i  J  k  . . .
when i, l, k are three dependent samples
3. Computing mean differences across three groups
to see if the probability for Ho to be retained is less
than .

4. To test mean differences, Fisher developed a
formula that computes variances (i.e, SD2) between
the three groups and within the three groups. That
is why ANOVA wins its name because it is a
technique of “ANalysis Of VAriance.”
5. Fisher used a particular distribution to decide the
critical value to reject or retain Ho (just like Z
distribution or Student’s t distribution). To honor
Fisher, the value of this distribution is called F.
6. Fisher’s Formula: Sources of Total Variances
(a) within-group variances [組內差異]: the
differences in scores of subjects who are in the
same group; resulting from “random sampling
fluctuation” (sampling errors); denoted as sw2; also
named “error variance”
Note: In SPSS, you will see the term “within-subject,”
and this is completely compatible to “within-group.”
In other textbooks, you may also come cross a term
like “error term.” “Error terms” actually mean
“variances.”
(b) between-group variances [組間差異]: the
differences between group means and the grand
mean (the mean of all participants, no matter which
groups they are assigned to); resulting from two
sources:
a. “random sampling fluctuation” and
b. “treatment”;
c. between-group variances is denoted as sB2;
d. sB2 = sw2 + Treatment Effects
Note: SPSS uses “between-subject.”
Hypothesis Testing Procedures for Fisher’s Formula:
1. Provided there are two samples, i & j, and Ho: i = J
is retained, it means variances among two sample
means are error variance (sw2) and no treatment
effects, so sB2 = sw2  sB2 sw2  1
2. Provided that if sB2 sw2  1, Ho is rejected in favor of
Ha; treatment effects manifest.

3. Expand this concept into 3 groups in ANOVA:
a. F = MSB/MSW
when MSB = SSB/k-1 and MSw = SSw/N-K
(1) “MS” stands for “Mean Square”
(2) “Square” is short for “the sum of variance”
because variance is the square of standard
deviation”
(3) “Mean” is short for “the division of variance by
degree of freedom.”
(4) “MS” expresses the quotient of “the sum of
variance that is divided by degree of freedom.”
b. According to these formulas, F is a ratio, and it
should be expressed as:
F ratio: F [df]: F [(K-1, N-K)) = . . .
df for between-group = k-1
df for within-group = N-K
N = total number of subjects
K = total number of groups
c. F ratio can be expressed in a simpler formula:
F = sB2 sw2 (variance between-group/variance
within-group)
d. For references, see Table 13.3 on p. 331 and Table
13.5/13.6 on p. 334.
ANCOVA: ANalysis of COVAriance (共變數分析):
1. Types of ANCOVA:
a. One-Way ANCOVA:
(1) One Independent Variable (i.e., one factor that is
used to classify participants into 3 groups).
(2) One Dependent Variable
(3) One Covariate or More than One Covariate (note:
try to avoid more than one covariate in your study)
(4) Covariate(s) can be continuous (using interval or
ratio scales) or dichotomous (assigned values 1 or
0)

b. Two-Way ANCOVA:
(1) Two Independent Variables (i.e., two factors that
are used to classify participants into 4 subgroups
or more).
(2) One Dependent Variable
(3) One or More than One Covariate (note: try to avoid
more than one covariate in your study)
2. Covariates:
a. A variable that will contribute variances to the
effect of the independent variable on the
dependent variable.
b. “IQ” may be a covariate for the effect of “instruction”
(i.e., the independent variable) on “academic
achievement” (the dependent variable) when a
study explores “instructional effects on academic
achievement.”
c. Assumption of Linearity: Covariates must be related
to “the dependent variables” but unrelated to “the
independent variable.” Performing Pearson r
between covariates and the dependent variable to
see if r is at least 0.2.
d. Covariates must exercise effects on “the
dependent variables” but no effects on “the
independent variable.”
e. If you read reports like: “when controlling X
variable. . .”, usually it means X variable is the
covariate and ANCOVA is performed.
3. Two Reasons for Using Covariates:
a. ANCOVA will increase precision by taking out
variances caused by covariates :
F = sB2/sw2: ANCOVA takes out covariate’s variance
from sw2 (i.e., error variance) . This results in a
smaller error variance . The smaller the error
variance is, the higher precision statistical results
will be.
b. Covariates will have control function (statistical
control). When a covariate is chosen, it means
researchers do not control it in their study design.
But researcher suspect the covariate will
contribute to the effects of the independent
variables on the dependent variable. In order to test
this suspicion, researchers perform ANCOVA by
including this covariate that has not been
“controlled.” Because ANCOVA will take out
covariate’s variance, the remaining variance will
originate from the independent variable.
After an ANCOVA is run, an adjusted mean (i.e., after
the covariate’s effect is removed) will be analyzed
and should be reported in papers.
4. Limitations for ANCOVA :
a. A covariate will affect effects of the independent
variable on the dependent variable. For two
covariates, they will exercise main and interaction
effects. This will make it very difficult to interpret
ANCOVA results.
b. ANCOVA can not be used to replace random
selection.
Rationale for Post Hoc
1. If F ratio is significant, it becomes necessary to
determine which pairs of means contribute to this
difference.
2. Post Hoc maintains Type I error rate as
comparisons are made.
 Types of Post Hoc: At least 10 types of tests
available
1. Pairwise Contrasts:

a. With equal n: Tukey (Tukey’s HSD: Honestly
Significant Difference in SPSS) or NewMan-Keuls
(SNK in SPSS)
b. With unequal n: Tukey/Kramer (not available in
SPSS
c. Scheffe: Most conservative
d. LSD: Most liberal
2. Planned Comparisons: No tests are available in
SPSS.

Effect Size:
a. It is an index to indicate “practical significance”
after “statistical significance” has been identified.
b. Meaning: It is an index of the proportion (or
percentage) of variability in the dependent variable
that is associated with (or is explained by) the
grouping variable (independent variable).
c. It shows the strength of treatment on dependent
variables.
d. Example: Check the follow tables to see different
measures of effect size and their cutting scores
(SPSS only produces partial Eta Squared:ηp2)
Sample: p. 316
1. Scenario: A researcher is interested in studying the
effects of the reinforcement methods on the
numbers of subjects’ trials to complete a task. The
researcher designs five different types of
reinforcement methods (Method 1, 2, 3, 4, & 5). 31
subjects were randomly selected and assigned into
these methods unevenly. Their numbers of trials
are measured.

The researcher wants to know: (a) whether the five
methods of reinforcement will exercise significant
effects on the numbers of trials; and (b) which
methods can exercise stronger effects.
2. Conditions:
a. 31 subjects are assigned into five groups and
receive different treatment;
b. One independent variable -> reinforcement
methods;
c. One dependent variable -> numbers of trials
3. Appropriate Stat Technique: One-Way ANOVA
4. SPSS Procedures:
1. Scenario: A researcher is interested in studying the
effects of the reinforcement methods on the
numbers of subjects’ trials to complete a task. The
researcher designs five different types of
reinforcement methods (Method 1, 2, 3, 4, & 5). 31
subjects were randomly selected and assigned into
these methods unevenly. Their numbers of trials
are measured. The researcher suspects that
subjects’ motivation may influence their trials. Thus,
before experiment, the researcher measures
motivation on a 10-point scale, and treat it as a
covariate.
The researcher wants to know: (a) whether the five
methods of reinforcement will exercise significant
effects on the numbers of trials; and (b) which
methods can exercise stronger effects.
2. Conditions:
a. 31 subjects are assigned into five groups and
receive different treatment;
b. One independent variable -> reinforcement
methods;
c. One dependent variable -> numbers of trials
d. One covariate: motivation (a continuous variable)
3. Appropriate Stat Technique: One-Way ANCOVA
4. SPSS Procedures:
Sample: p. 359
1. Scenario: A researcher randomly selects 20
subjects and randomly and evenly assigned them
into 4 groups where they receive different
motivation instruction. Then the researcher tests
the numbers of trials used to complete their
learning tasks. The researcher is interested in
knowing: (a) whether these motivational
instruction will exercise significant effects on the
numbers of trials; and (b) which type of motivation
instruction will exercise stronger effects?

2. Conditions:
a. 20 subjects are assigned into four groups and
receive different treatment;
b. One independent variable -> motivational
instruction;
c. One dependent variable -> numbers of trials
3. Appropriate Stat Technique: One-Way ANOVA
4. SPSS Procedures:
3P
PP: Knowledge of phonological awareness would
influence second language learning.
IP: However, previous studies: (a) do not focus on the
internal elements of sound, that is, the onset of
syllable structures; (b) do not research EFL learners
at elementary school level; and (c) do not tackle L1
transfer
SP: Participants would be expected to demonstrate an
acquisition order based on the difficulty level of
syllable structures.

Literature Review:
1. Are the studies being reviewed related to 3P? Is it
appropriate to mention the inappropriateness of
Minimal Sonority Distance that would not be used
as the theoretical framework (see p. 379)?
2. Is the link between PP & IP mentioned (see 3rd line
from the bottom of 1st paragraph on p. 377)?
 Method:
1. Subjects: 9 fifth-graders who underwent different
tasks

2. Tasks: Four Onset Types
3. Procedures: Reading aloud wordlists
3. Analysis: One-Way ANOVA on three onset types: (See
Figure 1) but not quite clear about how the rest one
was analyzed.
Criticisms:
a. What are the independent variables? The four
onset types, or the syllable structure with four
levels?
b. What are the dependent variables? Frequency of
Correctness?
c. Is the sample size large enough to run statistics?
d. Why one-way ANOVA? Is it because there are three
groups, or there is only one group of subjects who
underwent three types of tasks? What does it mean
by “three groups” and “one group”?
e. What techniques should be used if the number of
groups is one?
f. Does this study design suit for exploring causalities?
What is the treatment effect? Are participants
randomly selected?
g. Does this study design suit for answering the
research questions? Can strategies be identified
from qualitative data?
h. Is degree of freedom of the F ratio equal to F (2, 24)?
(See Table 3 and F ratio on p. 382); What can be
done to better the table? Where are the data of post
hoc? Which pairs of means are significantly
different?
Discussion
1. The onset types of single representation caused
greater difficulties than the type of complex onsets
(5th line, p. 383)
2. Different onset structures of L1 and L2 actually
influence L2 learning.
Criticisms:
1. How are these two findings justified in terms of data
collected and analyzed?
2. Is it appropriate to claim any causality of L1?

Final Remarks:
1. What are the main purposes of this study? Does the
researcher intend to tackle L1 and L2 in her 3P and
literature review? Do the research questions
specify this relationship? Do the data imply any
causal relationships? If yes, what is the cause and
what is the result?
2. Is one-way ANOVA an appropriate technique to be
used in this study? If yes, why? If not, why not?