Transcript slides - martin @ spielauer . ca

Student success analysis and
prediction using the US community
college microsimulation model
MicroCC
IMA 2011
Martin Spielauer
Ron Anderson
This project was funded by the US National Science Foundation's Advanced Technological Education (ATE)
Program with a grant to Colorado University's DECA Project
Organization
• Context & Goals
• Why Microsimulation
• MicroCC
– General
– Data
– Behaviours
• Simulations results & Illustrations
– Overall fit & trends
– Compositional analysis: outline
– Compositional analysis: examples
• Discussion & Outlook
Spielauer & Anderson
2
Context & Goals
• Enhanced understanding of US Community College (CC)
student success pathways
• Many initiatives to improve completion success (< 40%)
• Initiatives triggered data collection / utilization
• Challenges
– Heterogeneity of programs
– Heterogeneity of students
– Demographic & economic change
– Success hard to define and to compare
• Microsimulation can complement statistical analysis
Spielauer & Anderson
3
Why Microsimulation
• Education research key engine in development of advanced
statistical methods, e.g. multilevel models
• Individual level study progression data available
• Microsimulation can complement statistical analysis
– Quantify individual level differences; decomposition
– Projections accounting for composition effects
– Policy analysis
– Momentum point analysis
– Capacity planning
– Data development
• Education part of most large scale MS models; underused in
education research
Spielauer & Anderson
4
MicroCC: Overview
• MicroCC (Micro-Community-College) is a proof of concept
model
– Simple but able to reproduce observed totals, pattern
and trends
– Based on real data
– Output to demonstrate power and flexibility of MS
• Proved useful as demonstrational tool
– Development and discussion of research proposals
– Potential partners and clients
– Data providers
• Used to assess data quality and needs
Spielauer & Anderson
5
MicroCC: Data
• Rhode Island: 2500 students per study cohort 2005+
• Connecticut: 200.000 students, cohorts 2000+
• Three populations:
– Rhode Island 2005
– Connecticut: “Advanced Technical programs” (ATE)
– Connecticut: Non-technical studies
• Variables
– Demographic: age (group), sex
– Race: (Non Latin) White, Black, Latin, Asian, Other
– Term by term: Number of courses enrolled and passed
Spielauer & Anderson
6
MicroCC: Model
• Synthetic starting population sampled from the initial
distribution of students by province/program, cohort, age
group, sex, race, and full-/part-time status
• Students followed over 4.5 years (9 terms)
• Four decisions per term
– (Re-)enrolment decision
– Fulltime / part-time decision
– Number of courses enrolled (1-3; 4-10)
– Courses passed
• Models estimated separately by sex and province/program:
42 logistic (& ordered logit) models
• Success: 12 courses passed (proxy for transfer-readiness)
Spielauer & Anderson
7
MicroCC: Technical implementation
• Implemented in the generic microsimulation language
Modgen developed and maintained at Statistics Canada
Spielauer & Anderson
8
Illustration: Overall fit and trend
Modeled and observed trends in Connecticut succes rates
0.5
0.45
0.4
0.35
0.3
NON-ATE DATA
NON-ATE SIMULATION
0.25
ATE DATA
ATE SIMULATION
0.2
0.15
0.1
0.05
Spielauer & Anderson
Summer
2009 - Fall
Summer
2008 - Fall
Summer
2007 - Fall
Summer
2006 - Fall
Summer
2005 - Fall
Summer
2004 - Fall
Summer
2003 - Fall
Summer
2002 - Fall
Summer
2001 - Fall
Summer
2000 - Fall
0
9
Illustration: Decomposition – Intro 1/4
Compositional analysis: Latin students compared to White students, RI
45.0%
Success Rate of White Students
40.0%
35.0%
30.0%
25.0%
Effect of different course success probability
Effect of different number of courses enrolled
20.0%
Effect of different probability to continue/switch to fulltime
15.0%
Effect of different re-enrolment probability
10.0%
5.0%
Success Rate of Latin Students
0.0%
-5.0%
Spielauer & Anderson
10
Illustration: Decomposition – Intro 2/4
Compositional analysis: Latin students compared to White students, RI
45.0%
Success Rate of White Students
40.0%
35.0%
30.0%
25.0%
Effect of different course success probability
Effect of different number of courses enrolled
20.0%
Effect of different probability to continue/switch to fulltime
15.0%
Effect of different re-enrolment probability
10.0%
5.0%
Success Rate of Latin Students
0.0%
-5.0%
Spielauer & Anderson
11
Illustration: Decomposition – Intro 3/4
Compositional analysis: Latin students compared to White students, RI
45.0%
Success Rate of White Students
40.0%
35.0%
30.0%
25.0%
Effect of different course success probability
Effect of different number of courses enrolled
20.0%
Effect of different probability to continue/switch to fulltime
15.0%
Effect of different re-enrolment probability
10.0%
5.0%
Success Rate of Latin Students
0.0%
-5.0%
Spielauer & Anderson
12
Illustration: Decomposition – Intro 4/4
Compositional analysis: Latin students compared to White students, RI
45.0%
40.0%
35.0%
Success Rate of White Students
Difference due to different population
composition at first enrolment
30.0%
25.0%
Effect of different course success probability
Effect of different number of courses enrolled
20.0%
Effect of different probability to continue/switch to fulltime
15.0%
Effect of different re-enrolment probability
10.0%
5.0%
Success Rate of Latin Students
0.0%
-5.0%
Spielauer & Anderson
13
Illustration: Rhode Island, Latin vs. White
Decomposition of differences in study success rates between Latin and
White students - RI 2005 cohort - main groups
60.0%
50.0%
40.0%
Effect of different course success probability
Effect of different number of courses enrolled
30.0%
Effect of different probability to continue/switch
to fulltime
20.0%
Effect of different re-enrolment probability
Success Rate of Latin Students
10.0%
Success rate of White students
0.0%
-10.0%
Male
Spielauer & Anderson
Female
Initially
fulltime
Age at
Initially partenrolment
time
<22
Age at
enrolment
22+
Total
14
Illustration: Rhode Island, Black vs. White
Decomposition of differences in study success rates between Black and
White students - RI 2005 cohort - main groups
60.0%
50.0%
40.0%
Effect of different course success probability
Effect of different number of courses enrolled
30.0%
Effect of different probability to continue/switch
to fulltime
20.0%
Effect of different re-enrolment probability
Success Rate of Black Students
10.0%
Success rate of White students
0.0%
-10.0%
Male
Spielauer & Anderson
Female
Initially
fulltime
Age at
Initially partenrolment
time
<22
Age at
enrolment
22+
Total
15
Illustration: Connecticut, Black vs. White
Decomposition of differences in study success rates between Black and
White students - CT-TECH 2005 cohort - main groups
70.0%
60.0%
50.0%
Effect of different course success probability
40.0%
Effect of different number of courses enrolled
30.0%
Effect of different probability to continue/switch
to fulltime
Effect of different re-enrolment probability
20.0%
Success Rate of Black Students
Success rate of White students
10.0%
0.0%
-10.0%
Male
Spielauer & Anderson
Female
Initially
fulltime
Age at
Initially partenrolment
time
<22
Age at
enrolment
22+
Total
16
Illustration: Connecticut, ATE vs. non-ATE
Decomposition of different study success rates between technical (ATE) and nontechnical students in Connecticut
70.0%
60.0%
Effect of different course success
50.0%
Effect of different number of courses enroled
40.0%
Effect of different probability to switch to /
continue fulltimeFulltime-Parttime
Effect of different re-enrollment probability
30.0%
NON-TECHNICAL
TECHNICAL (ATE)
20.0%
10.0%
0.0%
Initially
fulltime
student
Spielauer & Anderson
Initially
parttime
student
Male
Female
Age at
enrolment
<22
Age at
enrolment
22+
All
17
Outlook
• Organizational: New England Board of Higher Education
– Coordinating center, project management, training
– Development of projects & proposals / funding
• Planned enhancements & projects for college institutions in
New England
– Job Market and Transfer Success. A college conducts an
annual follow-up survey
– Evaluation of a Campus-Wide Intervention
– Enrollment forecasting and capacity planning on state
level
Spielauer & Anderson
18