Transcript March 31, 2010 - Quantitative Research II

EDIT 6900: Research Methods in
Instructional Technology
Lloyd Rieber
Instructor
Eunjung Oh
Graduate Assistant
UGA, Instructional Technology
Spring, 2008
If you can hear audio, click
If you cannot hear audio, click
If you have a question, click
Two Topics for Today
• Continue Introduction to Quantitative
Research Methods
• Overview of a class activity on how to
compute a t statistic to determine if two
means (pretest and posttest) are
significantly different.
Not This Week
Informal Activity
SDC
Systematic Data Collection
• An informal, (hopefully) enjoyable activity
designed to give you first-hand
experience collecting research data
• Your Task: Go and research something of
interest to you!
• Report on it informally in writing
• Give 5 minute oral report
• 10%, Due: April 9
Remaining Course Calendar
March 26
Quantitative Research (con’t)
April 2
Quantitative Research
April 9
Preparing a Research Report
SDC Reports (in class)
Finish SDC Reports (if needed)
Research Project Presentations?
April 16
April 23
Research Project Presentations
Notes About the Next RDA
Notes About the Next RDA
Final Project Rubric
Look for Email with this.
EDIT 6900 Research in Instructional Technology
Part IV. Quantitative Research
Methodologies
Chapters 9-11
Dr. Lloyd Rieber
The University of Georgia
Department of Educational Psychology
& Instructional Technology
Athens, Georgia USA
Running an Olympic
Marathon:
No Significant Difference?
• 26 miles, 385 yards
• Times of top 2 runners at 2004 Olympics
in Athens, Greece:
– 1. Stefano Baldini ITA 2:10:55
– 2. Meb Keflezighi USA 2:11:29
• Is a difference of 34 seconds statistically
significant?
Total votes cast for
Bush or Gore in 2000:
No Significant Difference?
Experimental Designs
 Experimental design is used to identify cause-and-effect
relationships.
 The researcher considers many possible factors that
might cause or influence a particular
condition/phenomenon.
 The researcher controls for all influential factors except
those having possible effects.
Independent and Dependent Variables
 Variable: any quality or characteristic in a research
investigation that has two or more possible values.
 Independent variable: a possible cause of something
else (one that is manipulated)
 Dependent variable: a variable that is potentially
influenced by the independent variable.
The Importance of Control
 Control the confounding variables

Keep some things constant.

Include a control group.

Randomly assign people to groups.

Assess equivalence before the treatment with one ore more
pretests.

Expose participants to both or all experimental conditions.

Statistically control for confounding variables.
Types of Experimental Designs (1)
 Pre-experimental designs
 True experimental designs
 Quasi-experimental designs
Overview of Experimental Designs (2)
Group
Time
Group
1
Group
Tx: indicates that a treatment (reflecting independent variable) is presented.
2
Obs: Indicates that an observation (reflecting the dependent variable) is made.
: Indicates that nothing occurs during a particular time period.
Exp: Indicates a previous experience ( an independent variable) that some
participants have had and others have not; the experience has not
been one that the researcher could control.
Pre-Experimental Designs
 Design 1: One-shot experimental case study
Group
Group1
Time
Tx
Obs
 Design 2: One-group pretest-posttest design
Group
Group1
Time
Obs
Tx
Obs
True Experimental Designs (1)
 Design 4: Pretest-posttest control group design
Random
assignment
Group
Time
Group1
Obs
Group2
Obs
Tx
Obs
Obs
 Design 5: Solomon focus-group design
Random
assignment
Group
Time
Group1
Obs
Group2
Obs
Group3
Group4
Tx
Obs
Obs
Tx
Obs
Obs
True Experimental Designs (2)
 Design 6: Posttest-only control group design
Group
Random
assignment
Group1
Group2
Time
Tx
Obs
Obs
Quasi-Experimental Designs
 Design 8: Nonrandomized control group pretest-posttest design
Group
Time
Group1
Obs
Group2
Obs
Tx
Obs
Obs
Factorial Designs
 Design 15: Randomized two-factor design
Group
Time
Treatment related to the two
variables may occur
simultaneously or sequentially
Treatment
related to
Variable 2
Treatment
related to
Variable 2
Group1
Tx1
Tx2
Group2
Tx1
Group3
Group4
Obs
Obs
Tx2
Obs
Obs
Inferential Statistics (1)
 Estimating population parameters(1)
 Inferential statistics can show how closely the sample statistics
approximate parameters of the overall population.
 The sample is randomly chosen and representative of the total
population.
The means we might obtain from an infinite number of samples
form a normal distribution.
The mean of the distribution of the sample means is equal to the
mean of the population from which the sample shave been drawn.
The standard deviation of the distribution of sample means is
directly related to the standard deviation of the characteristic in
question for the overall population.
Inferential Statistics (2)
 Testing Hypotheses (1)
 Research hypothesis vs. statistical hypothesis
 Statistical hypothesis testing: comparing the
distribution of data collected by a researcher with an
ideal, or hypothetical distribution
- significance level/alpha (α): e.g., .05, .01
- statistically significant
- reject the null hypothesis
Inferential Statistics (3)
 Testing Hypotheses (2)
 Making errors in hypothesis testing
- Type 1 error: alpha error
- Type 2 error: beta error
Inferential Statistics (4)
 Testing Hypotheses (3)
 Making errors in hypothesis testing
-Increase the power of a statistical test
1) Use as large a sample size as is reasonably possible
2) Maximize the validity and reliability of your measures.
3) Use parametric rather than non parametric statistics
whenever possible.
- Whenever we test more than one statistical
hypothesis, we increase the probability of making
at least one Type 1 error.
Inferential Statistics (5)
 Examples of inferential statistical procedures
Parametric statistics
Nonparametric statistics
Students’ t test
Sign test
Analysis of variance
(ANOVA)
Mann-Whitney U
Regression
Kruskal-Wallis U
Factor analysis
Wilcoxon matched-pair
signed rank test
Structural equation
modeling (SEM)
Chi-square goodnessof-fit test
Odds ratio
Fisher’s exact test
Inferential Statistics (6)
 Example of reporting a test of a statistical hypothesis:
Percentage means and standard deviations are
contained in Table 1. A significant main effect
was found on the test of learning outcomes, F(1,
97) = 9.88, p <.05, MSerror = 190.51. Participants
given the educational game scored significantly
higher (mean =91.5%) than participants who
were not given the game (mean=71.2%).
Your Task
(This has already been emailed to you.)
1. Finish watching my pre-recorded
presentation introducing quantitative
research methods first.
2. Launch your Excel from last week. “Save
as” with a new title.
3. Compute a t statistic from the data set
emailed to you. Follow my video tutorial.
4. Email your spreadsheet to me as an
attachment. (You do not have to finish this
evening, but I expect most will.)
This is meant as a class activity.
It is not a graded activity.
If you get stuck and become
totally frustrated, stop and send
me what you have.
To do list
• Follow the Course Learning Plan!