Transcript Slide 1

EVALUATION PLANNING
Legal Services Corporation
May 19, 2004
Please standby while others connect
Goals for the Session
Review the LSC evaluation plan template
Learn how to formulate strong evaluation questions
Introduce LSC’s evaluation questions
Give you the skills to understand which data collection
techniques are best suited to answering what kinds of evaluation
questions
 Gain a better understanding of how program assessment can
help legal services organizations improve services




Innovation Network, Inc.
Slide 2
Objectives
 At the end of the training grantees will be able to:
 Complete the LSC evaluation template
 Integrate LSC’s required evaluation questions with questions you
develop on your own
 Deploy a data gathering strategy that helps you answer the
questions you created
Innovation Network, Inc.
Slide 3
Ground Rules
 Using Isoph Blue’s features
 We ask that you not use a speaker phone unless you have a
mute button to block external sounds.
 Please do not use the “Hold” function on your phone, as it can
induce a loud beeping noise for the rest of us on the call.
Innovation Network, Inc.
Slide 4
Questions During the Session
 If you can, please hold your questions till we get to a question
break
 If you can’t wait, please click the question mark and type your
question into the window, we’ll answer as soon as we can.
 If we do not get a chance to answer your question, please email
it to Ehren Reed at [email protected].
Innovation Network, Inc.
Slide 5
What is Evaluation?
The systematic collection of information about a
program in order to better understand it, improve
its effectiveness, replicate it and/or make
decisions about future programming.
Innovation Network, Inc.
Slide 6
Benefits of Evaluation
 Effectively communicates successes
 Builds common understanding about the program
 Helps you improve what you do
 Turns shortcomings into opportunities
 Helps staff overcome obstacles
 Positions your organization as a learning organization
Innovation Network, Inc.
Slide 7
10 Steps to Evaluation
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Develop program goals
Identify specific goal targets
Outline strategies and activities
Develop evaluation questions
Create data collection instruments
Collect data
Analyze and interpret information
Present findings
Reflect on findings
Revise program & institute changes
Innovation Network, Inc.
Slide 8
Evaluation Plan
Project Goal(s)
The overall aim(s)
or intended
impact(s) of your
project
Innovation Network, Inc.
Strategies/
Activities to
Achieve Goals
Evaluation
Questions
The actions that
The questions you
the project takes to want to answer
achieve desired
about your project
outcomes
Evaluation Data
The evidence or
information that
tells you whether
or not your project
is achieving its
outcomes
Slide 9
Project Goals
 Clear statements identifying the ultimate results
you want to achieve
Innovation Network, Inc.
Slide 10
Goals & Objectives
 Develop objectives or more specific targets for your goals.
 Consider the major steps will you need to accomplish in order
to achieve each of them.
 Make your objectives specific and achievable.
 Some goal words:



Enhance
Establish
Enable
 Tip: Review your grant proposal to see what you said you
would accomplish
Innovation Network, Inc.
Slide 11
Some Intake Examples
Intake
 Goal:
 Enable Asian-American clients in northern Virginia to seamlessly
access legal services through a single, integrated intake system
 Objectives:
 Design integrated system that allows partner organizations to
access critical client information
Innovation Network, Inc.
Slide 12
Some Infrastructure Examples
Infrastructure
 Goal:
Enhance the delivery of legal services to low-income families in rural
Mississippi via the website through high quality web applications
 Objectives:
 Develop partnerships with providers and case managers to identify critical
needs


Increase access to services by integrating communication systems
Innovation Network, Inc.
Slide 13
Some Pro Se Examples
Pro Se
 Goal:
 Enhance the ability of poor people in Ohio to access quality legal services
to assert their rights and interests through online and in-person training
and technical assistance
 Objectives:
 Identify patterns and causes of fundamental legal programs affecting our
client community
 Develop and test solutions to recurring legal problems faced by our client
community
 Ensure appropriate and sustainable web-based access to legal services
Innovation Network, Inc.
Slide 14
Strategies & Activities
 The major actions a program takes to achieve desired outcomes
 Use action words like:
 Conduct
 Train
 Mentor
 Provide
 Support
Innovation Network, Inc.
Slide 15
Some Intake Examples
Intake
 Develop database program that gathers case information
and assesses eligibility for legal aid
 Develop process for integrating systems and databases in
various offices that conduct intake, provide legal advice, and
represent clients
 Train case workers to use the system
 Identify interpreters for mono-lingual clients
Innovation Network, Inc.
Slide 16
Some Infrastructure Examples
Infrastructure
 Identify potential partners
 Hold regular meetings with partners and document community
needs
 Develop web-based services
 Train providers in how to use web-based services
Innovation Network, Inc.
Slide 17
Some Pro Se Examples
Pro Se
 Prepare training materials for template developers
 Participate in national template developers training
 Prepare logistics for Ohio template developers training
 Conduct Ohio template developers training
 Organize multi-state template developer work group
Innovation Network, Inc.
Slide 18
Questions
Innovation Network, Inc.
Slide 19
Work on Strategies and Activities
 Take a look at your grant proposal or work from memory and
complete the first two columns of the LSC Evaluation Plan grid.
 List your activities and organized by category or strategy.
Remember to use appropriate action words.
 When you’re done, click in the check box next to you name
Innovation Network, Inc.
Slide 20
Check in / Debrief / Discussion
Innovation Network, Inc.
Slide 21
Evaluation Questions
 Identify broad questions that will guide your evaluation report.
Process questions: What did you do and what were the
tangible results of your work?
 Outcome questions: What change occurred among your
constituents as a result of your activities?
 Try to keep the questions open-ended

Innovation Network, Inc.
Slide 22
Some Intake Examples
Intake
 How has the intake system assisted clients in accessing
seamless services?
 To what extent has the new system increased efficiency among
intake staff?
Innovation Network, Inc.
Slide 23
Some Infrastructure Examples
Infrastructure
 To what extent has the website helped strengthen client
education?
 How have the partnerships supported the development of quality
content for the website?
 How has the website increased access to legal services for rural
clients?
 In what ways has this project been supported by LSC and the
organizations partners?
Innovation Network, Inc.
Slide 24
Some Pro Se Examples
Pro Se
 To what extent did the training provide necessary skills and
knowledge to achieve intended outcomes?
 Was the training curriculum and delivery offered of high quality?
 How well have the online services met the needs of the
community and addressed the current trends in necessary legal
assistance?
 Has our online process and materials adequately prepared
clients to accurately complete their legal forms?
Innovation Network, Inc.
Slide 25
What are Your Questions?
 What information do you want to know about your program?
 Think about the assumptions that you have made about how
your program will be planned and implemented.
 Draft questions that you want to answer about your program.
 Tip: Avoid Yes/No questions. Instead, ask when, how, how
much or to what extent
Innovation Network, Inc.
Slide 26
Check in / Debrief / Discussion
Innovation Network, Inc.
Slide 27
Evaluation Data
 Talk to people

Interview, focus group, facilitated discussion
 Gather written responses

Surveys, questionnaires, paper tests
 Observe

Individuals, communities, group activities
 Review documents

Attendance records, progress reports, journals
Innovation Network, Inc.
Slide 28
….and the source
 Where will you find the information?
 What target group or documents will you need to gather the
necessary information?
Innovation Network, Inc.
Slide 29
Some Intake Examples
Intake
 Survey of intake staff
 Focus Group with mono-lingual clients
 Exit survey of mono-lingual clients
Innovation Network, Inc.
Slide 30
Some Infrastructure Examples
Infrastructure
 Monthly update form completed by project staff
 Survey of rural clients/users
 Survey of case workers
 Content review of website
Innovation Network, Inc.
Slide 31
Some Pro Se Examples
Pro Se
 Survey clients
 Document review/Content review of online process
Innovation Network, Inc.
Slide 32
Finish Your LSC Evaluation Plan
 List the data collection methods and source of information for
each one of your evaluation questions
 Remember to keep in mind the following questions:


Who or what has the answer to each evaluation question (partners,
clients, records, etc
What is the best way to access it (survey, focus group, document
review, etc)
 Tip: Think about what information you are already gathering and
try to integrate those tools into this data collection effort.
Innovation Network, Inc.
Slide 33
Evaluation Plan
Project
Goal(s)
Strategies/
Activities
Evaluation
Questions
Evaluation
Data
Innovation Network, Inc.
Slide 34
Check in / Debrief / Discussion
Innovation Network, Inc.
Slide 35
Questions
Innovation Network, Inc.
Slide 36
Next Steps
 Let us know where you are in the process
 Have you completed your plan?
 Do you want us to review it?
 Are you ready to create data collection instruments?
Innovation Network, Inc.
Slide 37
We can provide more assistance
 Asynchronous training
 Individual technical assistance
 Synchronous training
 Potential topics include:
 Survey
 Document review
Innovation Network, Inc.
Slide 38
For More Information
Innovation Network, Inc.
1001 Connecticut Ave., NW, Suite 900
Washington, DC 20036
(202) 728-0727
www.innonet.org
Innovation Network, Inc.
Slide 39