Transcript CHAPTER THREE - Keith Wilmot

CHAPTER THREE
Research Design: The
Experimental Model and Its
Variations
Research Design
Research design is the plan, the blueprint, or a
schematic for a study – it flows from
the problem formulation stage.
The who, what, where, when, why, and how
of an investigation.
The goal is how best to address the hypothesis.
Causality
The purpose of scientific investigation is:
to isolate, define, and explain the relationship between
key variables
in order to predict and understand
the underlying nature of reality.
(who, what, where, when, and how)
Causation lies at the basis of reality.
Questions
How would causation apply to the variables:
foot patrol and crime?
Does showing a relationship (correlation) between
two variables imply or demonstrate causation?
Steps for Resolving
the Causality Problem
•
Demonstration of a relationship between variables
(covariance). In other words, is there a predictable
relationship in the value of one variable to the value
of another variable.
•
Specifying or indicating the time sequence or time
order of the relationship. Which variable will be “X”
(predictor) and which variable will be “Y” (outcome).
•
Eliminate rival causal factors to avoid a spurious
relationship (a false relationship) or, avoid the
elimination of other variables that could conceivably
explain away the original relationship.
Classic Experimental Design
E O1 X O2
E O1
O2
E = equivalence
O = observation
X = treatment
1,2 = time
Examples: Candid Camera, Scared Straight,
Community Policing (Text, pages: 91-93).
Experimental Design
Terms:

X = treatment (independent variable)

Y = outcome (dependent variable)

Z = any rival causal factors

O = observation (some measurement or assessment of the
dependent variable)

E = equivalence (randomization or matching)

1,2 = number of times
Rival Causal Factors

Factors other than “X” (the treatment) that may be
responsible for the relationship.

Validity refers to accuracy or correctness in research,
i.e., internal and external.

Internal Validity is concerned with a variable other
than X that may have produced the change in Y.

External Validity is concerned with what other
variables may limit one’s ability to generalize the
findings in a study to larger groups or populations.
Internal Validity

History: Refers to other specific events that may
have occurred over the time of the study that may
have produced the results. Example: a new program
introduced such as “Crime Watch” or urban renewal.

Maturation: Biological or psychological changes in
respondents during the course of study that are not
due to the treatment variable. Example: age.

Testing: Pretest bias, bias and foreknowledge
introduced to respondents as a result of having been
pretested.
Internal Validity (cont’d)

Instrumentation: Changing the measurement
instrument from the beginning or the first period of
evaluation to the second or final evaluation. Ex.: the
method of recording citizen complaints changes.

Statistical Regression: the tendency of groups
selected for study on the basis of high or low scores
to regress or move toward the mean or the average
on second testing. Scores become more normal
upon retest. Ex.: First test may have been atypical.
Internal Validity (cont’d)

Selection Bias: Choosing nonequivalent groups for
testing. Ex: Selecting all volunteer prisoners or all
model prisoners.

Experimental Mortality: Expected loss of subjects in
the sample group over a period of time. Ex:
Following recidivism cohorts over long periods of
time.

Selection-Maturation Interaction: Selection bias
coupled with issues that emphasize biological or
psychological changes during the course of the
study or over time. Ex.: Age and Recidivism
External Validity

Testing Effects: Exposure to pretests by
respondents negates the generalizability of the
results to larger populations that have not been
pretested. Example: Pretest of community attitudes
toward the police prior to a foot patrol experiment
followed up by a posttest regarding community
attitudes toward the police after the experiment.

Selection Bias: Specific studies that are based on a
specific group may not be comparable to a larger
group that does not have the same specific
characteristics. Ex.: Local drug use studies.
External Validity (Cont’d)

Reactivity: Awareness of being studied tends to
produce atypical or unnatural behavior on the parts
of subjects. Example: Hawthorne Effect

Multiple-Treatment Interferences: Occurs when
more than one treatment or predictor variable is
used on the same subjects. Example: addition of
foot patrol plus foot patrol officers were also
unarmed and wore different uniforms.
Question
What might be a rival causal factor which may affect
the relationship between our
predictor variable foot patrol (X),
and our outcome variable crime (Y)?
Related Rival Causes
(be able discuss the following):

Hawthorne Effect

Halo Effect

Post Hoc Error

Placebo Effect

Diffusion of Treatment

Compensatory Equalization of Treatment

Local History
Experimental Design
Terms:

X = treatment (independent variable)

Y = outcome (dependent variable)

Z = any rival causal factors

O = observation (some measurement or assessment of the
dependent variable)

E = equivalence (randomization or matching)

1,2 = number of times
Classic Experimental Design
E O1 X O2
E O1
O2
E = equivalence
O = observation
X = treatment
1,2 = time
Examples: Candid Camera, Scared Straight,
Community Policing (Text, pages: 91-93).
THREE ELEMENTS OF
THE CLASSIC EXPERIMENTAL DESIGN
1) Equivalence is the assignment to comparison
groups in a manner in which the subjects are alike
in all major respects, i.e., randomization and
matching. (E)

Randomization is the random assignment of
subjects where all individuals have an equal
probability to being assigned to a particular group.

Matching is the selecting of subjects for
comparison groups based on key characteristics so
that the group is similar in respect to these
characteristics.
Three Elements
(Cont’d)
2) The classic experiment consists of pretests and
posttests.

A pretest is an observation prior to exposure to
treatment (O1).

A posttest is an observation and measurement
after treatment (O2).
Three Elements
(Cont’d)
3) The classic experiment consists of experimental
and control groups.

The experimental group is exposed to treatment
(X).

The control group is not exposed to the
treatment.
Solomon Four-Group Design
E O1 X O2
E O1
O2
E
X O2
E
02

Eliminates testing effects and reactivity (awareness
of being studied)
Cross-sectional and
Longitudinal Design

Often referred to as cross-sectional (one group at one time);
longitudinal (one group over time), i.e., time-series, cohort studies,
panel studies, and trend studies.

Time series: measuring a single variable at successive points in
time (E = matching)

Interrupted time-series: measurement before and after treatment
for an equivalent period of time

Trend studies: analyze different sample of the same population
longitudinally

Cohort studies: Analyze subgroups over time.

Panel studies: Analyze the same group over time.
Time Series Design

Interrupted Time-Series Designs
O O O X O O O

Multiple Time-Series Designs
O O O X O O O
O O O
O O O
Examples: Problem Oriented Policing (Spelman, et al.),
Monahan and Walker’s Study of mental health centers,
and Shneider and Smykla’s “War and Capital
Punishment” study (Text, pages: 102-103).
Advantages and
Disadvantage of Experiments

Advantages: 1) Control of rival factors (internal
validity); 2) Quick and inexpensive; 3)
Manageability; and, 4) Can be applied to natural
settings.

Disadvantages: 1) Artificiality (hinders
generalizability); and, 2) Difficult to apply
experiments to human subjects and situations in
criminal justice (i.e., ethical issues and experimenter
effects). Disadvantages in criminal justice subject
matter often outweigh the advantages.
Questions
What type of design was the
Kansas City Gun Experiment?
What were the major findings of the project?
What is a rival factor of concern
when evaluating shock incarceration programs?
Why are time-series designs particularly useful in
criminal justice studies?