Transcript Data on Student Learning

Data on Student Learning
Office of Assessment
University of Kentucky
Sources of Data on UK Learning
Perspective
Direct Measures
Program-level
Institutional-level
(External/Nationally
Comparable)
Indirect Measures
Program Reviews &
Accreditation Responses
CLA
CAAP
Wabash National Study
Wabash National Study
Wabash National Study, Spring 2007
Good Practice Scales
Scale 1 - Good Teaching and High Quality Interactions with Faculty
Strong
Moderate
Weak
Faculty interest in teaching and student development
41%
47%
13%
Prompt Feedback
20%
69%
12%
Quality of no classroom interactions with faculty
22%
69%
10%
Teaching clarity and organization
46%
54%
1%
Wabash National Study, Spring 2007
Good Practice Scales
Scale 2 - Academic Challenge and High Expectations
Strong
Moderate
Weak
Academic challenge and effort
29%
71%
0%
Frequency of higher-order exams and assignments
19%
73%
9%
Challenging classes and high faculty expectations
14%
77%
9%
Integrating ideas, information, and experiences
27%
73%
0%
Wabash National Study, Spring 2007
Good Practice Scales
Scale 3 – Diversity Experience Scale
Strong
Moderate
Weak
Diversity experiences
3%
78%
20%
Meaningful discussion with diverse peers
15%
39%
47%
Wabash National Study: Overall
Scale 1 – Good Teaching and High Quality Interactions with Faculty
Small Institutions
Large Institutions
University of Kentucky
Strong
Moderate
Weak
44%
56%
0%
28%
72%
1%
23%
75%
2%
26%
74%
0%
18%
82%
0%
13%
87%
0%
5%
84%
12%
3%
83%
14%
3%
73%
24%
Scale 2 – Academic Challenge and High Expectations
Small Institutions
Large Institutions
University of Kentucky
Scale 3 – Diversity Experience Scale
Small Institutions
Large Institutions
University of Kentucky
Small Institutions N = 17; Large Institutions N = 9
Narrative Summary of Wabash Results
National Results: Initial Participating Institutions (N=19)

Largest positive change was student growth in moral reasoning
◦ 10% over the first two semesters of college.

CAAP Critical Thinking test revealed less than 1% positive change

Small but consistent declines were observed in attitudes about diversity
◦ Results reveal a consistent downward pattern across all measures of diversity.

A moderate, consistent decline observed in:
◦ academic motivation and interest in academic subject matter
◦ community involvement
◦ professional success.

Student perspectives on teaching and instructional practices:
◦ Good Teaching questions had a significant positive impact on 24 of the 29
outcomes
◦ Academic Challenge questions had a significant positive effect on 25 of the 29
outcomes
◦ Diversity Experiences questions had a significant positive impact on 17 of the 29
outcomes
Narrative Summary of Wabash Results
Outcomes Measures
(fall 2006 to spring 2007)
◦ UK students registered minimal change on most Outcomes
Measures
◦ UK students did show a significant increase in moral reasoning
◦ UK students registered a sharp decrease in academic motivation,
a component of the Inclination to Inquire outcome
Collegiate Learning Assessment (CLA)
Skills Measured

Performance Task and Analytical Writing Tasks
◦ Critical thinking
◦ Problem solving
◦ Analytic reasoning
◦ Written communication skills
CLA Performance Task

Requires students to use an integrated set of critical
thinking, analytic reasoning, problem solving, and written
communication skills to answer several open-ended
questions about a hypothetical but realistic situation.

Provides a document library that includes a range of
information sources.

Students use the materials in preparing essay answers

Requires students to present their ideas clearly,
including justifying their points of view.
CLA Analytic Writing Task

Students write answers to two types of essay
prompts:
◦ “Make-an-Argument” question asks them to support
or reject a position on some issue
◦ “Critique-an-Argument” question asks them to
evaluate the validity of an argument made by
someone else

Both tasks measure a student’s skill in articulating
complex ideas, examining claims and evidence,
supporting ideas with relevant reasons and
examples, sustaining a coherent discussion, and
using standard written English
Collegiate Learning Assessment
(CLA)
Fall 2007
◦ Freshman tested (longitudinal cohort), n = 303
 Spring 2008
◦ 102 Seniors were tested
 Spring 2009
◦ Longitudinal cohort re-tested
◦ N = 174 students

176 institutions were included in the comparisons
Collegiate Learning Assessment (CLA)
Fall 2007
Performance
Task
Analytic
Writing Task
Make-anArgument
Critique-anArgument
Total Score
Spring 2009
Student
Count
Mean score
Std
Deviation
Mean score
Std
Deviation
Effect Size
Performance
Level
173
1209
202
1224
166
0.07
168
1129
144
1170
179
0.29
Below
Expected
At Expected
168
1122
173
1151
217
0.17
At Expected
171
1134
181
1186
193
0.29
At Expected
168
1172
145
1199
149
0.19
At Expected
Collegiate Learning Assessment (CLA)
0.29
0.3
0.29
Effect Size
0.25
0.17
0.2
0.15
0.1
0.07
0.05
0
Note: An effect size of 0.50 is considered “large.”
0.19
Collegiate Learning Assessment (CLA)
CLA Measure
Performance Task
Analytic Writing Task
Make an Argument
Critique an Argument
Total Score
Effect Size
Collegiate Assessment of Academic
Proficiency (CAAP) – Math
Descriptive Analysis





N = 308
Level:
◦ 79.41% Freshman
College:
◦ 33.27% College of Arts & Sciences
◦ 27.12% Undeclared
◦ 9.15% Engineering
Transfer vs. Traditional
◦ 14.05% Transfer
◦ 85.95% Traditional
Residence
◦ 82.03% In-state
◦ 17.97% Out-of-State
Collegiate Assessment of Academic
Proficiency (CAAP) – Math Results
Overall Mathematics Test Scores of UK Students – Spring 2009
UK
National
Mean
58.5
58.5
Standard Deviation
3.8
4
N
308
21,678
CAAP – Math Results
Basic Algebra Scores – Spring 2009
UK
National
Mean
15.7
15.6
Standard Deviation
2.4
2.4
N
308
21,678
UK
National
Mean
15.1
15.4
Standard Deviation
2.8
2.7
N
308
21,678
College Algebra Scores – Spring 2009
CAAP – Critical Thinking
Spring 2010 Summary Results
National
Average
Score
Count
Percentage
Average
Score
Freshman
128
22%
62
62
Sophomore
252
43%
63
62
Junior
124
21%
63
62
Senior
61
10%
63
62
Other/No Response
27
4%
63
62
UK Average
592
100%
63
62
National N= 9,531
CAAP – Critical Thinking
Spring 2010
UK Student Perceptions of Effort
Tried my
Best
Gave
Moderate
Effort
Gave Little
Effort
Gave No
Effort
No
Response
Freshman
17%
34%
10%
2%
36%
Sophomore
19%
35%
12%
7%
26%
Junior
23%
48%
9%
2%
22%
Senior
34%
28%
7%
5%
26%
UK Average
21%
36%
11%
4%
27%
Program Review/Accreditation
Analysis
Program Reviews and/or Accreditation Documents from 2002-present
Assessment Related
Faculty Training
31
17
Faculty/Student Interaction
31
Improve Practicum and/or Internship
31
Graduate Teaching Experience
15
Improve Facilities
140
Curriculum Improvement
232
Need Resources
Need Faculty
247
96