Transcript CONTEXT: Bring Data!

THE CAMPUS
COMPUTING PROJECT
MEASURING WHAT MATTERS
Mapping the New World of Higher
Educ. Assessment and Outcomes
Kenneth C. Green
THE CAMPUS COMPUTING PROJECT
THE CAMPUS
COMPUTING PROJECT
Key Questions
• What is quality in higher education?
• How do we define it? Measure it?
• How do we explain it to ourselves? Others?
• How do we improve the quality of academic
programs and institutional outcomes?
THE CAMPUS
COMPUTING PROJECT
Complaints About Quality
• FACULTY: Today’s college students are not
well-prepared for college-level work.
• EMPLOYERS: College graduates come to us
lacking key skills.
• EXECUTIVES: US college graduates are not
prepared to compete in the global economy.
• GOV’T OFFICIALS: Universities expect money
without accountability.
THE CAMPUS
COMPUTING PROJECT
Plus ca Change!
THESE ARE NOT NEW COMPLAINTS:
• Endemic to higher education
• Recurring theme in commission reports over the past 40
years
WHY NOW?
• GW Bush administration has made educational
assessment and accountability a priority
• Accelerating concern about higher educ. assessment and
outcomes in the states
THE CAMPUS
COMPUTING PROJECT
CONTEXT: Bring Data!
“Back in Texas we like to say ‘In God
we trust; all others bring data.’ ”
Margaret Spellings, Sec. of Education
CONTEXT: Bring Data!
“Back in Texas we like to say ‘In God
we trust; all others bring data.’ ”
Margaret Spellings, Sec. of Education
TRUE SOURCE:
“In God we trust; all others bring data.”
W. Edwards Deming
THE CAMPUS
COMPUTING PROJECT
The Spellings Commission
• GOOD NEWS: not so dire
• BAD NEWS: we’ve heard
these critiques before
• REAL NEWS: pay attention,
as this report has “legs”
THE CAMPUS
COMPUTING PROJECT
Spellings: Key Messages
• ACCESS: “too few Americans prepare for, participate
in, and complete higher education”
• AFFORDABILITY:
“costs have outpaced inflation for
two decades…. Our higher education financing system
is increasingly dysfunctional”
• ACCOUNTABILITY: “there is inadequate transparency
and accountability for measuring institutional
performance”
THE CAMPUS
COMPUTING PROJECT
The Quest for Accountability
We are concerned about the widespread resistance to cost effectiveness
thinking in higher education because it is so profoundly anti-intellectual. It
rejects reason, and it puts a low value on the time of faculty trained to
reason well…
We must guard against the widespread tendency to trivialize the problem
of efficiency in higher education. It is not only a financial problem but an
intellectual one. The questions about efficiency lead to a host of questions
about teaching and learning, and the ultimate questions about the nature
and purpose of higher education. These are too important to the colleges
and universities – and too intellectually challenging – to be dismissed as
illegitimate.
President’s Task Force on Higher Education, 1971
THE CAMPUS
COMPUTING PROJECT
PROVOST
PERSPECTIVES
100%
4.3%
8.3%
13.4%
11.0%
Outcomes from OnLine learning
compared to Face-to-Face courses
80%
35.5%
60%
50.6%
HOW DO
THEY KNOW?
78.3%
• Are provosts in classes
and chatrooms?
62.0%
40%
• Are they looking at data?
20%
38.5%
24.5%
56.2%
17.3%
0%
All Institutions
Superior
Source: The Sloan Consortium (2004)
www.sloan-c.org
Public
Private
Same
For-Profit
Inferior
• Are they doing formal
assessments and
evaluations?
• Are they talking with
students, faculty &
employers?
THE CAMPUS
COMPUTING PROJECT
How Do We Measure Program
Quality and Institutional Outcomes?
Resources
Performance
• Student characteristics
• Retention and
graduation rates
• Faculty profile
• Financial resources
• Physical plant
• Retention in key
fields or among key
groups/populations
• Faculty research and
publications
• Testing
THE CAMPUS
COMPUTING PROJECT
Assessment Strategies
• SHADOW ISSUE: No Child Left Behind (NCLB) and
the rise of “high stakes testing”
• DATA MINING: exploiting the rich array of student
and institutional data already available
• TESTING: turn to third party instruments for
independent measures and metrics
HYBRID: Multiple Data Sources
• ACKNOWLEDGE: multiple outcomes, both
individual and institutional
• EXPLOIT: existing resources & data
• INVEST: in resources and strategies that
address real institutional needs
• RECOGNIZE: benign neglect is not an effective
institutional response
THE CAMPUS
COMPUTING PROJECT
The ERP Problem
THE CAMPUS
COMPUTING PROJECT
The ERP Silo
Adm Info Systems
Finance
Student
Info. Sys.
HR
Development
The ERP Silo Turtle
Course
Mgmt
eServices
Adm Info Systems
Finance
Student
Info. Sys.
HR
Development
Content
Portals
Source: Green, 2004
Alumni
Services
ePortfolio
THE CAMPUS
COMPUTING PROJECT
Icons of the New Internet Economy
The consumer and corporate experience drives expectations
about campus resources and services, including analytics.
Tracking transactions to provide timely data
about shoppers for managers & shareholders
Extracting value from transaction data to
provide feedback to clients
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
Investing in research to provide feedback,
improve programs and enhance client services
THE CAMPUS
COMPUTING PROJECT
The Spellings Commission
Message to Campus Leaders
• LEVERAGE IT: IT moves is now part of the
conversation about data, assessment, and
outcomes
• KEY TOOLS: data warehousing/mining & ERP
analytics
• RIP: The IR office as we know it
THE CAMPUS
COMPUTING PROJECT
Outcomes and ERP Analytical Tools
A key component of the outcomes and assessment solution [mandated by the
Spellings Commission] resides in the emerging analytical IT tools increasingly
deployed in the corporate sector and now coming to higher education. These tools
can, do, and should expand the mission of information technology in colleges and
universities to include assessment.
Bring Data: A New Role for IT After
the Spellings Commission
Kenneth C. Green
EDUCAUSE Review, Nov-Dec 2006
THE CAMPUS
COMPUTING PROJECT
Leadership
The issue before us in the wake of the Spellings Commission report concerns when college
and university IT leaders will assume an active role, a leadership role in these conversations
[about assessment and outcomes], bringing their IT resources and expertise - bringing data,
information, and insight - to the critical planning and policy discussions about institutional
assessment and outcomes that affect all sectors of US higher education.
Bring Data: A New Role for IT After
the Spellings Commission
Kenneth C. Green
EDUCAUSE Review, Nov-Dec 2006
THE CAMPUS
COMPUTING PROJECT
www.campuscomputing.net
Thank You For Your Participation
Please complete our online evaluation