Transcript Folie 1

Impact measures
for libraries and
information services
Roswitha Poll
Münster
Bielefeld
2006
The importance of libraries
What libraries ensure (IFLA)
Cultural
diversity
Democratic
values
The open
information
Libraries make
a difference society
Intellectual
freedom
But who will
believe it?
Economic
progress
Information
equality
Information
literacy
Reduction of
poverty
Bielefeld
2006
input
activities
a contribution of work, information, or material
activities change input into output
output
the amount produced, the results supplied
impact
the effect or influence of one person, thing, or
action, on another
outcome
the consequence, visible or practical result or
effect of an event or activity
value
the importance or preciousness of something,
the perception of actual or potential benefit
Bielefeld
2006
Bielefeld
2006
Different outcomes
—
negative
direct/immediate —
long-term
actual
—
potential
intended
—
unexpected
positive
Bielefeld
2006
Impact/outcome of cultural institutions
knowledge changes in skills,
competences,
information literacy
attitudes, behaviour
academic or professional success
social inclusion
individual well-being
Bielefeld
2006
Outcomes of universities
recruitment and retention of students and
excellent academic staff
Evidence needed:
effective teaching
The library supports
- high graduation rates
these goals
- high grades in examinations
- high employment rates after examination
effective research
- high renown and use of research results and
publications
- high amount of special grants
- awards, honours
Bielefeld
2006
Problems of assessing outcome
Value and outcome of a service can differ between
user groups
Data may not be available because of data
protection rules
Influences on
Results of projects
are often not
individuals
arecomparable
because of different
data collection methods
manifold
Long-term effects cannot be assessed as persons are
no more available
All tested methods are time-consuming
Bielefeld
2006
Methods for assessing impact
Quantitative measures
tests assessing user skills before and after training on a
service
"Surrogate
performance monitoring / data mining
measures" have
observation (unobtrusive)
to be used
analysis of documents and bibliographies compiled
by users
correlation between library use and success
Bielefeld
2006
Methods for assessing impact
Qualitative measures
Surveys (print, telephone, online)
Interviews
The
Focus "anecdotal
groups, discussion groups
evidence"
Self-assessment
of users
supports data
Facts and stories must be organized to show patterns
Bielefeld
2006
User satisfaction as outcome measure?
ACRL
A change of
behaviour?
Or a basis for
such change?
Jennifer Cram
Bielefeld
2006
Assessing impact
Correlation of library use
and academic success
Library use, expressed by
Frequency (loans, visits)
Range of services used (reference service, user training, ILL)
Range of collections used (undergraduate collection, E-journals, special
material)
Attendance at training sessions
Success, expressed by
Duration of studies
Grades in exams
Student retention
Employment rate after exam
Quality of publications (impact factor,
peer-review journals)
Bielefeld
2006
Assessing impact
The library's impact on information literacy
Outcome of a specific training or series of
trainings
•
surveys after instruction
•
pretest / posttest
•
self-assessment of users
•
behavioural observation
•
transaction logs
•
longitudinal studies
•
assessing changes in students' bibliographies
Bielefeld
2006
Assessing impact
3
Importance of the local library for research
1 Percentage of material cited in academic publications
that was (could have been) retrieved via the local library.
Citations in
• dissertations
• student papers
• publications of a faculty
2 Estimated percentage of the information needed
for a publication / paper / report found via the local library
• questionnaire/interview after publication
Bielefeld
2006
Assessing impact
4
Impact on social inclusion and community life
Does the existence of a specific library / of libraries further
• individual well-being
• the quality of life in a society
- democracy
- social inclusion
- cultural life
- local identity
- life-long learning
Method: Assessing public opinion (users and / or nonusers) as to the benefits of libraries and special library
services
• to the population (indirect benefit)
• to the individual user (direct benefit)
Bielefeld
2006
Assessing impact
5
The financial value of libraries
Assessing time costs (replacement value of a client's time)
Users invest time and effort in order to use library services. The
For
£1 of-public
value that they
- or every
their institution
place onfunding
that use must be at
least asthe
high British
as their „sacrifice“
of receives
time.
Library
each year,
(Costs are calculated by time and average salary)
£4.4 is generated for the economy
Using the contingent valuation method
Willingness-to-pay
What would users pay for the
maintenance of a service?
Willingness-to-accept
Which sum would users accept as
equivalent for the deletion of a service?
Bielefeld
2006
Using the results
accountability
Libraries on the agenda
decision making
(Claudia Lux)
resource management
improvement of services
promotion of the library's role
Bielefeld
2006
Outcome projects
eVALUEd: in "evidence base", University of Central
England: Toolkit for evaluating electronic information
services
http://www.evalued.uce.uk/index.htm
IMLS (Institute of Museum and Library Services):
outcome-based evaluation of projects
http://www.imls.gov/index.htm
ARL New Measures Initiative: several projects
- Learning outcomes
- Higher education outcomes research review
- MINES (Measuring the impact of networked electronic
services)
http://www.arl.org/stats/newmeas/index.html
Bielefeld
2006
Outcome projects
IBEC: Information School of the University of
Washington and University of Michigan School of
Information: Toolkit for assessing the impact of
information in communities
http://ibec.ischool.washington.edu/default1024.aspx
SCONUL and LIRG (Library and Information Research
Group): impact initiative and mailing list
http://www.sconul.ac.uk/activities/performance/impact.html
IFLA Section Statistics and Evaluation: working
group on outcome/impact; bibliography of literature and
projects worldwide
http://www.ulb.uni-muenster.de/outcome.html
Bielefeld
2006
(Goethe: Faust)