Transcript The Study - University of Central Florida

Students’ Ratings of University Teaching:
A Data Mining Analysis
Chuck Dziuban
Research Initiative for Teaching Effectiveness
Ida Cook
Department of Sociology and Anthropology
Faculty Senate
Morgan Wang
Department of Statistics
University of Central Florida
Rationale for the Study
• Teaching evaluation data for a 3-year period
were available to allow a comparison of two
different sets of items (UCF/BOR).
• Responding to faculty interest, the UCF
Faculty Senate requested that an evaluation
of the Student Evaluation of Instruction
measures be performed. (FS 1995-96-11)
The Instrument: UCF Items
• Feedback on your performance in this course
• The instructor’s interest in your learning
• Use of class time
• The instructor’s overall organization of the course
• Continuity from one class meeting to the next
• The pace of the course
• The instructor’s assessment of your progress
• The text and supplemental learning materials used
The Instrument:
Board of Regent Items
• Description of course objectives and assignments
• Communication of ideas and information
• Expression of expectations for performance
• Availability to assist students In or outside of class
• Respect and concern for students
• Stimulation and interest in the course
• Facilitation of learning
• Overall assessment of instructor
The Study Layout
Approximately 450,000 student responses
Five Colleges:
•
•
•
•
•
Arts and Sciences
Business Administration
Education
Engineering
Health and Public Affairs
Three Levels:
Three Years:
• Lower Undergraduate
• 1996-97
• Upper Undergraduate
• Graduate
• 1997-98
• 1998-99
Overall Rating of the Instructor
Binary Decision Tree
SAS Enterprise Miner
Predictors
Remaining
15 Items
Developmental Developmental
Sample 1
Sample 2
• Level
• Year
• College
Developmental
Sample 3
Validation
Sample
Decision Tree Example
Overall
85.9%
n=11,286
Arts & Sciences,
Business Admin.,
Hospitality Mgmt.
85.8%
n=6,460
F2F, E, M
86.5%
n=5,639
females
88.4%
n=3,263
Engineering
Education
72.7%
n=378
91.5%
n=2,079
W
74.8%
n=821
males
84.1%
n=2,376
F2F
94.1%
n=1,036
A&S
78.5%
n=526
E, M, W
89.1%
n=1,043
F2F
64.7%
n=148
BA & Hosp. mgmt
68.9%
n=298
Health
& Pub.
Affairs
86.7%
n=2,369
E, M
79.6%
n=230
Rules Leading to Overall Rating of
Excellent
Questions
Facilitation of Learning
Ex
●
Communication of Ideas
and Information
●
N=46,805
VG
G
F
Probability = .96
P
Rules Leading to Overall Rating of
Excellent
Questions
Facilitation of learning
Ex
●
G
F
●
Communication of ideas
and information
Organization of the course
●
Assessment of student
progress
●
N=3,462
VG
●
Probability = .85
P
Rules Leading to Overall Rating of
Excellent
Questions
Facilitation of learning
Ex
●
G
F
●
Communication of ideas
and information
Organization of the course
●
Instructor interested in your
learning
●
N=6,215
VG
Probability = .78
P
Overall Ratings Unadjusted and
Adjusted for ‘Excellent’ Decision
Rules
Adjusted for Rule
College
Unadjusted
1
2
3
CAS
42%
95%
85%
80%
CBA
36%
95%
83%
77%
COE
54%
97%
86%
75%
CECS
32%
96%
84%
78%
COHPA
48%
96%
86%
74%
For more information…
Research Initiative for Teaching Effectiveness
Dr. Chuck Dziuban
407-823-5478
[email protected]
Dr. Patsy Moskal
407-823-0283
[email protected]
http://rite.ucf.edu