Transcript An Application of the Action Research Model for Assessment

An Application of the Action
Research Model for Assessment
Preliminary Report
JSM, San Francisco
August 5, 2003
Tracy Goodson-Espy, University of AL, Huntsville
M. Leigh Lunsford, University of AL, Huntsville
Ginger Holmes Rowell, Middle TN State University
1
Overview
• Action Research Model
• In the Context of a Collaborative Project
• Showing Results from a Specific
Example
2
Action Research Model*
• What is the problem? I.e., what is not
working in the classroom?
• What technique can be used to address
the learning problem?
• What type of evidence can be gathered
to show whether the implementation is
effective?
• What should be done next, based on
what was learned?
*1999 - R. delMas, J. Garfield, B. Chance
3
Teaching Experiment Cycle
Teaching Hypotheses;
Curricular & Instructional
Choices
Class
Implementation
& Feedback
Instructors’ Reflections and
Curricular Modifications
4
Action Research Model
• What is the problem? I.e., what is not
working in the classroom?
• What technique can be used to address
the learning problem?
• What type of evidence can be gathered
to show whether the implementation is
effective?
• What should be done next, based on
what was learned?
5
What is the Problem?
• Math majors who take only one course in
“Probability and Statistics” are exposed to
very little statistics
• Student understanding of complicated
concepts
– CLT, CI’s, Combinatorics, Baye’s Theorem, ...
• Increase reasoning and thinking instead of
memorization
• Better preparation for careers
6
Action Research Model
• What is the problem? I.e., what is not
working in the classroom?
• What technique can be used to
address the learning problem?
• What type of evidence can be gathered
to show whether the implementation is
effective?
• What should be done next, based on
what was learned?
7
What Technique can be used to
Address the Learning Problem?
Big Picture
NSF DUE A/I Collaborative Research Award
– Adaptation & Implementation of Activity & WebBased Materials in Post-Calculus Introductory
Probability & Statistics Courses
– PI’s
• Tracy Goodson-Espy, University of AL, Huntsville
• M. Leigh Lunsford, University of AL, Huntsville
• Ginger Holmes Rowell, Middle Tennessee State University
*This project is partially support by the National Science Foundation. The project
started in June 2002 and continues through August 2004..
8
A Collaborative Approach
A&I Materials into Post Calculus Prob/Stat Courses
Athens State Univ.
Middle Tenn. St. Univ.
M. Leigh Lunsford
Ginger Holmes Rowell
Univ. of Alabama, Huntsville
Tracy Goodson-Espy
Provide Objective Independent Assessment of A&I
9
The Materials for A&I
• “A Data-Oriented, Active Learning, PostCalculus Introduction to Statistical
Concepts Methods, and Theory (SCMT)”
• A. Rossman, B. Chance, K. Ballman
• NSF DUE-9950476
• “Virtual Laboratories in Probability and
Statistics (VLPS)”
• K. Siegrist
• NSF DUE-9652870
10
What Technique can be used to
Address the Learning Problem?
Specific Techniques
• Increase statistical content
– Change course description & course number
• Understand complicated concepts and
increase “thinking” instead of memorization
– Change course materials used
• Better preparation for careers
– Integrate technology, group work, report writing
11
Action Research Model
• What is the problem? I.e., what is not
working in the classroom?
• What technique can be used to address
the learning problem?
• What type of evidence can be gathered
to show whether the implementation is
effective?
• What should be done next, based on what
was learned?
12
What Evidence can be Gathered to
Show Implementation is Effective?
• Student’s self perception of learning of
concept (survey results)
• Teacher perception of student learning
– In-class student feedback during the activity
(continuous monitoring)
– Follow-up in-class quiz
– Student reports
– Test/Exam questions
• Student attitude survey results
13
Evidence Continued:
One Example – Central Limit Theorem
“Sampling Distributions of Sample Means”
Computer Laboratory Simulation Activity
– A. Rossman, B. Chance, K. Ballman
• “A Data-Oriented, Active Learning, Post-Calculus
Introduction to Statistical Concepts Methods, and
Theory (SCMT)”
– Spring 2003: used 3 of 4 examples from this
activity, students wrote reports, question on
test
14
Evidence (continued)
CLT Example
• Students’ self perception of learning
– 80% (n=25) of students remembered it as an
“activity that aided learning”
(Fall 02 – instructor lecture & 1 SCMT example: 23%)
– “My understanding of the Central Limit Theorem”
• Survey question of their self reported knowledge on a
scale of 1 (low knowledge) to 5 (high knowledge)
• Mean response = 4.6, stdev = .64, n = 27
(Fall 02: mean = 3.8, stdev =.90, n = 25)
15
Evidence (continued)
CLT Example
• Teacher’s perception of student learning
• In-class monitoring
– Overall minimal difficulty with computer lab
– Needed feedback on their understanding of the
concept
• Gave in-class (no credit) unannounced quiz at
beginning of following class
• Not enough students had gotten far enough on the
activity to provide conclusive evidence
16
Evidence (continued)
CLT Example
•
•
Teacher’s perception of student learning
Reports
– Structure (could work in groups)
•
Introduction, Explanation of the 3 examples in the
lab activity, Explain method used, Conclusions
– Results (graded for big picture concepts and
conceptual details)
•
Three “wrong” responses (n=27)
1) “I did not know how to do that [example 4].”
2) Mean gets “smaller.”
3) An incomplete conclusion.
17
Evidence (continued)
CLT Example
•
•
Teacher’s perception of student learning
Exam questions
– Students completed Example 3 (uniform lunch
times) from the Course Pack activity on the
test
•
•
•
•
•
Not previously assigned
No indication that it would be on the test
The “Uniform” subcommand was new
This portion of the test was taken in the lab
Worked individually
18
Variable
N
Mean
StDev
Sample means n=2
1000
29.482
12.232
Sample means n=5
1000
29.926
7.977
Sample means n=20
1000
29.826
3.801
70
60
50
40
100
30
20
10
0
10
20
30
40
50
60
50
Sample means n=2
100
0
0
10
20
30
Sample means n=5
40
50
Frequency
0
Frequency
Frequency
Test Question Output
50
0
20
30
Sample means n=20
40
19
Evidence (continued)
CLT Example
• Teacher’s perception of student learning
• Results
– Gave “hint” (minus 1 point) with Uniform
command
• Some found Uniform in the pull down menu
• Biggest mistake – 1/3 of students used
Uniform 30 17.32 instead of Uniform 0 60
– Filled in a table like they had done previously
– Wrote a “paragraph” summarizing their findings.
• Mean score 2.6 out of 3 points (st dev = 0.7)
20
Action Research Model
• What is the problem? I.e., what is not
working in the classroom?
• What technique can be used to address
the learning problem?
• What type of evidence can be gathered
to show whether the implementation is
effective?
• What should be done next, based on
what was learned?
21
What should be done next,
based on what was learned?
• Improve my writing assignment
• Improve my evaluation of student learning
on this concept
– Pre/post test
– Use delMas/Garfield/Chance instrument (?)
– Coordinate with Lunsford on this unit
• Incorporate more report writing in my class
• Keep using and evaluating these activitybased, discovery learning materials.
22
Resources
• delMas, R., Garfield, J., and Chance B. (1999), A Model of
Classroom Research in Action: Developing Simulation Activities
to Improve Students' Statistical Reasoning, Journal of Statistics
Education v7, n3,
http://www.amstat.org/publications/jse/secure/v7n3/delmas.cfm.
• Hollins, E. R. (1999), “Becoming a Reflective Practitioner,” in
Pathways to Success in School: Culturally Responsive
Teaching, eds. ER Hollins and EI Oiver, Mahwah, NJ: Lawrence
Erlbaum Associates.
• Hopkins, D. (1993), A Teacher’s Guide to Classroom Research,
Buckingham: Open University Press.
• Noffke, S., and Stevenson, R. (eds.) (1995, Educational Action
Research, NY: Teachers College Press.
23
Contact Information
[email protected]
24