Transcript SUNY GENERAL EDUCATION ASSESSMENT CONFERENCE

USING THE COLLEGIATE
LEARNING ASSESSMENT:
IMPRESSIONS AND ADVICE
Presentation to the Association of
Institutional Research and
Planning Officers
Buffalo, New York
June 11, 2009
PRESENTERS
Patricia Francis
Associate Provost for Institutional
Assessment and Effectiveness
SUNY Oneonta
Rosalyn Lindner
Associate Vice President
Buffalo State College
SESSION TOPICS
 CLA
Structure, Scoring, and Reporting
 Mapping the CLA to SUNY GE Outcomes
 Advantages and Challenges in Using the CLA
 Institutional Strategies for Overcoming
Challenges
DESCRIBING THE CLA:
Structure, Scoring, and
Reporting
WHAT DOES THE CLA
MEASURE?
 Key
Higher Order Skills
Critical thinking
 Analytical reasoning
 Problem solving
 Written communication

 Types
of Tasks
Performance Task
 Analytical Writing

Make-an-Argument
 Critique-an-Argument

ADMINISTRATION DETAILS
 CLA
Administered Over Computers for a 6090 Minute Session
 Two Approaches
Cross-sectional, comparing 100 first-semester
students and 100 second-semester seniors in
same academic year
 Longitudinal, comparing same students as
freshmen (n=300) and seniors

ADMINISTRATION DETAILS
(CONT.)
 The
CLA is Not a Multiple Choice Test
 Students Respond to One of Three Prompts

Analyze Complex, Realistic Scenarios
Write Persuasive, Analytic Essay
 Critique Written Arguments

WHAT DO STUDENTS SEE?
 Split
Screen
Left side: Directions, questions and response
box
 Right side: Document library with pull down
menu

EXAMPLES
 Dyna
Tech
 “Truth” in Media
 Child Obesity Study
CLA SCORES
 Unadjusted
Performance Score for Absolute
Comparisons
 Deviation Score for Controlled Comparisons
EXPECTED SCORE
 Uses
SAT/ACT Scores for Measures of
Academic Ability of Students Prior to
Matriculation (Entering Academic Ability, or
EAA, Score)
 Estimates Linear Relationship between CLA
Scores and EAA
 Reports in Standard Error Units
MAPPING CLA SCORES
TO SUNY GE LEARNING
OUTCOMES:
Critical Thinking and
Written Communication
GEAR REVIEW, SPRING 2009
 Focused
Exclusively on SCBA Outcomes for
Written Communication and Critical
Thinking
 Primary Questions
Is there reasonable face validity between CLA
measures and SUNY outcomes?
 Does the CLA report provide campuses with
sub-scores for individual outcomes?

CORRESPONDENCE BETWEEN
CLA AND SUNY OUTCOMES
 CLA
Measures Map Well to (and Provide
Sub-scores for):
Critical Thinking Outcome #1 (“Identify,
analyze, and evaluate arguments….”
 Critical Thinking Outcome #2 (“Develop wellreasoned arguments”)
 Written Communication Outcome #1 (“Produce
coherent texts within common college-level
written forms”)

 Implications
for Information Management?
ADVANTAGES AND
CHALLENGES IN USING
THE CLA:
And, Campus Responses to
Challenges
ADVANTAGES
 Student
Artifacts Much Richer and More
Complex
 Inherently Value-Added Nature of CLA
 Controls for Students’ Incoming Ability
 Provides Information on Multiple Learning
Outcomes
 Implications for Faculty Workload (for
Administration and Scoring)
 Student Engagement/Interest
CHALLENGES
 Recruitment
of Students (and Implications
for Representativeness of Sample)
 Student Participation and Motivation
 Relevance to Classroom
 Scheduling of Computer Labs
 Staff Time and Effort
CAMPUS RESPONSES:
BUFFALO STATE COLLEGE
 Recruitment
of Students and Sampling
Freshmen and seniors
 Direct recruiting compared to faculty recruiting

 Student
Participation and Motivation
Scheduling
 $$$

 Relevance

to Classroom
Closing the loop
 Scheduling
of Computer Labs
CAMPUS RESPONSES: SUNY
ONEONTA
 Student


Preparation of sampling plan for CAE
Assuring comparability between sample and other
students
 Student


Recruitment and Sampling
Participation and Motivation
Reliance on academic departments
Emphasis to students on benefits to institution,
programs, and themselves
 Relevance


to Classroom
Provision of individual student results to programs
“CLA in the Classroom” initiative
USING THE COLLEGIATE
LEARNING ASSESSMENT:
IMPRESSIONS AND ADVICE
Presentation to the Association of
Institutional Research and
Planning Officers
Buffalo, New York
June 11, 2009