Transcript Folie 1
Impact measures for libraries and information services Roswitha Poll Münster Bielefeld 2006 The importance of libraries What libraries ensure (IFLA) Cultural diversity Democratic values The open information Libraries make a difference society Intellectual freedom But who will believe it? Economic progress Information equality Information literacy Reduction of poverty Bielefeld 2006 input activities a contribution of work, information, or material activities change input into output output the amount produced, the results supplied impact the effect or influence of one person, thing, or action, on another outcome the consequence, visible or practical result or effect of an event or activity value the importance or preciousness of something, the perception of actual or potential benefit Bielefeld 2006 Bielefeld 2006 Different outcomes — negative direct/immediate — long-term actual — potential intended — unexpected positive Bielefeld 2006 Impact/outcome of cultural institutions knowledge changes in skills, competences, information literacy attitudes, behaviour academic or professional success social inclusion individual well-being Bielefeld 2006 Outcomes of universities recruitment and retention of students and excellent academic staff Evidence needed: effective teaching The library supports - high graduation rates these goals - high grades in examinations - high employment rates after examination effective research - high renown and use of research results and publications - high amount of special grants - awards, honours Bielefeld 2006 Problems of assessing outcome Value and outcome of a service can differ between user groups Data may not be available because of data protection rules Influences on Results of projects are often not individuals arecomparable because of different data collection methods manifold Long-term effects cannot be assessed as persons are no more available All tested methods are time-consuming Bielefeld 2006 Methods for assessing impact Quantitative measures tests assessing user skills before and after training on a service "Surrogate performance monitoring / data mining measures" have observation (unobtrusive) to be used analysis of documents and bibliographies compiled by users correlation between library use and success Bielefeld 2006 Methods for assessing impact Qualitative measures Surveys (print, telephone, online) Interviews The Focus "anecdotal groups, discussion groups evidence" Self-assessment of users supports data Facts and stories must be organized to show patterns Bielefeld 2006 User satisfaction as outcome measure? ACRL A change of behaviour? Or a basis for such change? Jennifer Cram Bielefeld 2006 Assessing impact Correlation of library use and academic success Library use, expressed by Frequency (loans, visits) Range of services used (reference service, user training, ILL) Range of collections used (undergraduate collection, E-journals, special material) Attendance at training sessions Success, expressed by Duration of studies Grades in exams Student retention Employment rate after exam Quality of publications (impact factor, peer-review journals) Bielefeld 2006 Assessing impact The library's impact on information literacy Outcome of a specific training or series of trainings • surveys after instruction • pretest / posttest • self-assessment of users • behavioural observation • transaction logs • longitudinal studies • assessing changes in students' bibliographies Bielefeld 2006 Assessing impact 3 Importance of the local library for research 1 Percentage of material cited in academic publications that was (could have been) retrieved via the local library. Citations in • dissertations • student papers • publications of a faculty 2 Estimated percentage of the information needed for a publication / paper / report found via the local library • questionnaire/interview after publication Bielefeld 2006 Assessing impact 4 Impact on social inclusion and community life Does the existence of a specific library / of libraries further • individual well-being • the quality of life in a society - democracy - social inclusion - cultural life - local identity - life-long learning Method: Assessing public opinion (users and / or nonusers) as to the benefits of libraries and special library services • to the population (indirect benefit) • to the individual user (direct benefit) Bielefeld 2006 Assessing impact 5 The financial value of libraries Assessing time costs (replacement value of a client's time) Users invest time and effort in order to use library services. The For £1 of-public value that they - or every their institution place onfunding that use must be at least asthe high British as their „sacrifice“ of receives time. Library each year, (Costs are calculated by time and average salary) £4.4 is generated for the economy Using the contingent valuation method Willingness-to-pay What would users pay for the maintenance of a service? Willingness-to-accept Which sum would users accept as equivalent for the deletion of a service? Bielefeld 2006 Using the results accountability Libraries on the agenda decision making (Claudia Lux) resource management improvement of services promotion of the library's role Bielefeld 2006 Outcome projects eVALUEd: in "evidence base", University of Central England: Toolkit for evaluating electronic information services http://www.evalued.uce.uk/index.htm IMLS (Institute of Museum and Library Services): outcome-based evaluation of projects http://www.imls.gov/index.htm ARL New Measures Initiative: several projects - Learning outcomes - Higher education outcomes research review - MINES (Measuring the impact of networked electronic services) http://www.arl.org/stats/newmeas/index.html Bielefeld 2006 Outcome projects IBEC: Information School of the University of Washington and University of Michigan School of Information: Toolkit for assessing the impact of information in communities http://ibec.ischool.washington.edu/default1024.aspx SCONUL and LIRG (Library and Information Research Group): impact initiative and mailing list http://www.sconul.ac.uk/activities/performance/impact.html IFLA Section Statistics and Evaluation: working group on outcome/impact; bibliography of literature and projects worldwide http://www.ulb.uni-muenster.de/outcome.html Bielefeld 2006 (Goethe: Faust)