Transcript Evaluation - People Server at UNCW

REC 375—Leadership and
Management of Parks and
Recreation Services
Jim Herstine, Ph.D., CPRP
Assistant Professor
Parks and Recreation Management
UNC Wilmington
Evaluation
REC 375—Leadership and Management of
Parks and Recreation Services
Evaluation Defined
• Evaluation is the systematic process of
determining the effectiveness of current
practices, procedures, and plans
• Evaluation is a process that judges worth
Why Evaluate?
• Determine that funds are wisely spent
• Conduct services efficiently
• Reinforce staff efforts or recommend new
directions
• Assist policy makers in directing the
organization toward more productive
channels
• Evidence the strengths of services
Types of and Approaches to Evaluation
• Outcome Evaluation
• Process Evaluation
Outcome Evaluation
• Also called product evaluation, program
evaluation, impact evaluation or summative
evaluation
• Documents the effect of a service on a client
Process Evaluation
• Also called formative evaluation
• Determines whether the service has been
conducted in an efficient, legal, and ethical
manner
• Produces information during and after the
delivery of the program or service that
judges the worth of its development and
operation
Scope of Evaluation
•
•
•
•
•
•
•
•
•
Annual evaluation of goals and objectives
Annual review of policies
Needs assessments
Management information systems
Performance appraisal of personnel
Program effectiveness
Equipment cost-effectiveness
Risk management
Fiscal accountability
Steps in the Evaluation Process
1.
2.
3.
4.
5.
6.
Determine the evaluation issues
Select an evaluation design
Collect the information
Analyze the information
Prepare a report
Implement the findings
Maintaining Research Integrity
•
•
•
•
Be objective
Minimize bias
Approach the project fairly and honestly
Understand the principles and procedures of
evaluation
• Have exceptional mastery of the tools of research
• Be concerned with honest and careful evaluation
Sampling Techniques
• Probability Sampling
• Non-probability Sampling
Probability Sampling
• Probability sampling methods are based on randomness
• Desire is to be able to “generalize” to the entire population
from which the sample is drawn
• Most common method is simple random sampling—all
members of the study population have an equal
opportunity to be chosen
• Fishbowl Technique
• Random sampling with replacement
• Random sampling without replacement
• Also can use a table of random numbers or computer based
selection
Probability Sampling, continued
• Not always possible to select respondents
with complete randomness
• Stratified sampling
• Systematic sampling
• Deliberate sampling
Stratified Sampling
• The overall population of respondents is divided
into several different subgroups according to
common characteristics—age, gender, ethnic or
racial identity, socio-economic status, etc.
• 20 basketball teams, 30 soccer teams, 25
volleyball teams, 35 baseball teams
• Want a 25% survey sample
• Take 25% of each category as the stratified sample
Systematic Sampling
• This selection method occurs when some “system”
is applied to the subjects
• Interview every 5th person that comes by
• While systematic selection approximates a random
sample, it does not strictly satisfy the definition of
random sampling and, hence, is not entirely bias
free
• It is quick, efficient and saves time and energy
Deliberate/Purposive Sampling
• With this method, the researcher knows that
specific characteristics exist in a certain
segment of a population
• Since these traits are extremely critical to
the results of the investigation, the
researcher deliberately selects those
subjects who contain the characteristics
• This selection process is always biased
Non-Probability Sampling
• Samples not selected at random
• Intact classes, volunteers, a typical group or
individual
• Intact or available groups impose a serious
restriction on the researcher’s ability to
generalize the data obtained to the larger
population from which the sample was
drawn
Information Gathering Methods
•
•
•
•
•
Questionnaires and structured interviews
Unstructured interviews and focus groups
Observations
Document study/secondary data review
Non-reactive or unobtrusive measures
• Physical traces
Analysis of Information
• Information gathered or measured is
essentially nothing more than numbers or
words—”raw” data
• Need to analyze the “raw” data to determine
exactly what it means
Quantitative Data Analysis
• Describing Data
• Frequency distributions
• Measures of central tendency
• Mean, mode and median
• Measures of variability
• Standard deviation, spread of scores
• Measures of relationship
• Chi-square test (named variables) and Pearson
correlation (numerical variables)
Quantitative Data Analysis
• Inferring or inferential statistics
• Measure differences between variables
• T-test—is there a significant difference
between the scores of 2 groups (compares
means)
• Analysis of variance—is there a significant
difference between the scores of 3 or more
groups
Quantitative Data Analysis
• Always have issues regarding “Reliability”
and “Validity”
Reliability
• Refers to whether the repetition of a study
or measurement will result in the same or a
very similar answer
• Do I get the same answers again and again?
• If subjects stand on a scale and the subjects are
the same weight, does the scale record their
weight as equal
Validity
• Refers to whether the evaluation actually
measured what was intended to be
measured
• Is the weight of the subjects who stood on the
scale accurate? Do they really weigh what the
scale says?
Qualitative Data Analysis
• Not all data that is collected is quantitative
• Some of the data is qualitative
• Text (narrative), photographic or video form
• Look for themes and commonalities
Evaluation
Jim Herstine, Ph.D., CPRP
UNCW
[email protected]
910.962.3283