Transcript Data Matters for User Service: Acquiring & Assessing the Impact of

Data Matters for User Service:
Acquiring & Assessing the Impact of NYPL’s
Money Matters Grant-Funded Initiative
Marie L. Radford
Chair, Department of Library & Information Science
& Associate Professor
Rutgers University, NJ
Kristin McDonough
Director, Science, Industry, and Business Library,
The New York Public Library
Library Research Round Table
American Library Association Conference
Chicago, Illinois
June 27th-July 2nd, 2013
The New Normal
Agencies Mandated to Develop
Consumer Education Programs
Pre-Training Survey
of Branch Staff
 Why
 How
 What We Found
Grants: Acquisition & Execution
 Not about the money…but
the meaning
 Opportunity lookout
 Chat before submission
 Pitch & pivot
 Define scope
 Manage RFP process
 Negotiate and
collaborate with
contractor
 Engage internal
stakeholders
Evaluating Money Matters
 Goal: To create a corps of frontline staff equipped with
the skills & confidence to meet the growing need
among their patrons to access timely, accurate &
trustworthy information about personal finance.
 Evaluation
 IMLS requirement
 RFP by NYPL to recruit outside evaluator
 Measure MM’s success in meeting goals for:
 Individual workshops (content & format)
 Instructors
 Overall program
Money Matters Pro Program Staff
Training Portal
http://bit.ly/NYPLMoneyMatters
Evaluation Design
 Mixed-Methods
 Quantitative & Qualitative
 Instruments designed with NYPL input
 Surveys
 Pre-Training Survey
 Immediate Post-Training Survey
 Final Post-Training Survey
 Observation
 Focus Groups
 Website Analytics
Surveys
 Mix of paper & online (Survey Monkey)
 Staff cooperation vital
 Understand need & purpose
 Paper successful for individual session evals (100%
return rates)
 Online useful for Pre & Post surveys
 Incentive for Final Survey (gift card raffle)
Money Matters Workshops
Workshop Titles (3 core, 8 electives,
…..10 eLearning Modules)
Banking (core)
Retirement Planning (core)
Credit & Debt 1 (core)
Credit & Debt 2
Identity Theft
Investing 1
Investing 2
Paying for College
Income Tax
Insurance
Living on the Financial Edge
Immediate Post-Training
Survey Results
 Completed at end of each training workshop
 Nearly 100% return rate (administered onground)
 Uniformly excellent results (means 5.0 to 4.0)




Quality of instruction
Content/scope
Activities/scenarios
Resources/handouts
Likert scale = Strongly Agree (5) to Strongly Disagree (1)
Immediate Post-Training
Survey Results
“Before this workshop, how prepared were you
to answer patron questions?
 3.0 to 2.3 (mean, on several PF topics)
“After completing this workshop how well
prepared are you to answer patron questions?”
 5.0 to 4.0 (mean, on same PF topics)
Reported all workshops practical, relevant, enjoyable,
excellent instructors, & useful in daily interactions with
NYPL patrons.
Survey Results: Pre-Training (n= 90, 69%)
vs. Final Post-Training (n= 95, 73%)
Confidence in knowledge & understanding Personal
Finance
 2.6 (Pre-Training mean) rose to 3.9 (PostTraining mean)
Adequate training in locating quality PF resources
 2.7 (Pre) rose to 4.2 (Post)
Adequate training in evaluating PF resources
 2.7 (Pre) rose to 4.1 (Post)
Would welcome PF questions
 2.8 (Pre) rose to 3.8 (Post)
Staff Voices on
Money Matters Program
 “Suggested to a single mom that starting to save now
for her child's college education is beneficial because
of the time value of money.”
 “Helped a patron who was a grandfather find
information on how to pay his granddaughter's college
costs and how to ask financial questions.”
 “Helping immigrants at the adult learning center set up
bank accounts and helping them to improve their
financial knowledge.”
 “I understand investment company information and
reports better since I'm more comfortable with financial
terms.”
Conclusion
 Critical factors in successful MM program





Clear vision of need & goals
Talented/qualified instructors
NYPL involvement in content & design
Staff cooperation in evaluation
Feedback & adjustments
 Early observation
 Mid-point Focus Groups
 Interim reports
Acknowledgements & Contact
 This is one of the outcomes from the project Money
Matters Financial Education Program
 Funded by the Institute for Museum and Library
Services (IMLS) and NYPL
 Special thanks to: Marzena Ermler, T.J. Woods,
Gretchen Smith, Brandy McNeil, Barbara O’Neill, &
Carole Glade
 Staff training portal Money Matters Pro
http://bit.ly/NYPLMoneyMatters
 Money Matters Web Site:
http://www.nypl.org/moneymatters