Transcript Observation Techniques, Experiments and Test Markets 7

Chapter Seven
Causal Research Designs:
Observation Techniques,
Experiments and Test Markets
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-1
Learning Objectives
 Discuss the characteristics, benefits
and weaknesses of observational
techniques, and explain how these
techniques are used to collect primary
data
 Describe and explain the importance of
and differences between the variables
used in experimental research designs
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-2
Learning Objectives
 Explain the theoretical importance and
impact of internal, external and
construct validity measures in
experiments and interpreting functional
relationships
 Discuss the three major types of
experimental designs used in marketing
research
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-3
Learning Objectives
 Explain what test markets are and how
researchers and marketing practitioners
use the resulting data structures
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-4
Introduction
PHASE II:
Design the research
Marketing Research
Step 3:
Select the research design
 If the researcher wishes to develop clear
insights into why certain events occur and why
they happen under some conditions and not
others, such as:
 Predicting sales, uncovering valuable market
information, or anticipating the consequences of a
marketing program
 Determining customer attitudes
 Investigating cause–effect relationships
 Then the researcher should consider using a
causal research design, such as:
 Test marketing
 Experimental procedures
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-5
Phase II: Select the Research
Design—Causal Research Designs
 The need for causal research

The desire for substantial amounts of information from
enough members of the target population so that inductive
logic and probabilistic inferences can be drawn
 Causal research designs include a range of
methods:

Observational techniques, experimentation techniques
such as pre-experimental designs, true experimental
designs, quasi-experimental designs, field experiments and
test marketing
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-6
Observational Techniques
 These methods rely on observation
rather than actually communicating with
people to collect primary data:
 Much information about the behaviour
of people and objects can be observed,
such as:
 Physical actions
 Expressive behaviours
 Temporal behaviour patterns
 Verbal behaviour
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-7
Observational Techniques
 These methods consist of the
systematic activities of witnessing and
recording the behavioural patterns of
people or objects without actually
communicating with them
Condition
Brief description
Information
Current behaviour patterns must be part of the data requirements
Type of data
Necessary data must be observable
Time frame
Data patterns must meet repetitiveness, frequency and
predicability factors in a pre-specified time frame
Setting
Behaviour must be observable in some type of public or laboratory
setting
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-8
Unique Characteristics of
Observational Techniques
Characteristic
Description
Directness of
observation
The degree to which the researcher or trained
observer actually observes the behaviour or
event as it occurs.
Researchers can use either direct or indirect
observation techniques.
Subjects’
awareness of
being observed
The degree to which subjects consciously
know their behaviour is being observed and
recorded.
Researchers may use either
disguised or undisguised observation
techniques.
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-9
Unique Characteristics of
Observational Techniques
Characteristic
Description
Structuredness
of observation
The degree to which the behaviour activities
or events to be observed are specifically
known to the researcher before doing the
observations.
Structured and unstructured techniques are
available to collect primary behavioural data.
Type of observing
mechanism
How the behavioural activities or events will
be observed and recorded.
Basically researchers have the option either
of using a trained human observer or some
type of mechanical device.
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-10
Type of Observing Mechanism
 Human observation
 Mechanical observation
 Voice Pitch Analyser
 Pupilometer
 Eye Tracking Monitor
 Psychogalvanometer
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-11
Benefits and Limitation of
Observation Techniques
Major Benefits of Observation
 Accuracy of actual
behaviour
 Reduction of
confounding factors
 Detail of the
behavioural data
Limitations of Observation
 Lack of generalisability
of data
 Inability of explaining
behaviours or events
 Complexity of setting
and recording of
behaviour(s) or events
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-12
The Nature of Experimentation
 All marketing research practices
require either manipulation or the
measurement of variables
 Variables
 Any observable and measurable element
or attribute of an item or event
 Functional relationship
 An observable and measurable systematic
change in one variable as another variable
changes
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-13
Variables Used in
Experimental Designs
Type of Variable
Comments
Independent
variable
Also called predictor or treatment variable (X).
An attribute or element of an object, idea, or
event whose measurement values are directly
manipulated by the researcher. The independent
variable is assumed to be the causal factor of a
functional relationship with a dependent
variable.
Dependent
variable
Also called criterion variable (Y). A singular
observable attribute or element that is the
measured outcome or effect change on specified
test subjects that is derived from manipulating
the independent variable(s).
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-14
Variables Used in Experimental
Designs
Type of Variable
Comments
Control
variables
Variables that the researcher controls so that
they do not affect the functional relationship
between the independent and dependent
variables included in the experiment.
Extraneous
variables
Uncontrollable variables that should average
out over a series of experiments. If not
accounted for, they can have a confounding
impact on the dependent variable measures
that could weaken or invalidate the results of
an experiment.
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-15
Randomisation, the Role of Theory,
and Validity and Reliability Concerns
Concept
Comments
Randomisation
Using randomisation, researchers can assign
subjects to different treatment conditions,
resulting in each group averaging out any
systematic effect
Theory
A large body of interconnected propositions
about how some portion of a certain
phenomenon operates
Validity
The extent to which the conclusions drawn
from the experiment are true
Internal validity
The extent to which the research design
accurately identifies causal relationships
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-16
Randomisation, the Role of Theory,
and Validity and Reliability Concerns
Concept
Comments
External validity
The extent to which a causal relationship
found in a study can be expected to be true
for the entire target population
Construct validity The extent to which the variables under
investigation are completely and accurately
identified before hypothesising any functional
relationship
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-17
Threats to Internal Validity
History
When extraneous factors that enter the
experiment process between the first and
later manipulations affect measures of the
dependent variable.
Maturation
Changes in the dependent variable based on
the natural function of time and not
attributed to any specific event.
Testing
When learned understanding gained from
the first treatment and measures of the
dependent variable distort future treatments
and measurement activities.
Instrumentation
Contamination from changes in
measurement processes, observation
techniques and/or measuring instruments.
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-18
Threats to Internal Validity
Selection bias
Contamination created by inappropriate
selection and/or assignment processes of test
subjects to experimental treatment groups.
Statistical
regression
Contamination created when experimental
groups are selected on the basis of their
extreme responses or scores.
Mortality
Contamination due to changing the
composition of the test subjects in the
experiment.
Ambiguity
Contamination from unclear determination of
cause–effect relationship.
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-19
Threats to External Validity
Treatment vs
treatment
When test subjects in different treatment
groups are exposed to different amounts of
manipulations.
Treatment vs
testing
When the premeasurement process sensitises
test subjects to respond in an abnormal
manner to treatment manipulations.
Treatment vs
selection
Generalising the results to other categories of
people beyond those types used in the
experiment.
Treatment vs
setting
Generalising the results to other environments
beyond the one used in the experiment.
Treatment vs
history
Using the existing functional relationship to
predict future phenomenon outcomes.
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-20
Threats to Construct Validity
Inadequate
preoperationalisation
of variables
Contamination due to inadequate
understanding of the complete makeup of the
independent and dependent variables
included in the experimental design.
Mono-operation
bias
Contamination created by using only one
method to measure the outcomes of the
dependent variable.
Monomethod bias
Contamination due to assessing multiattribute
treatment manipulations (independent
variables) using single-item measuring
instruments.
Hypothesis-guessing
Contamination by test subjects believing they
know the desired functional relationship prior
to the manipulation treatment.
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-21
Construct Validity
Evaluation
apprehension
Contamination caused by test subjects being
fearful that their actions or responses will
become known to others.
Demand
characteristics
Contamination created by test subjects trying
to guess the true purpose behind the
experiment, thus giving abnormal socially
acceptable responses or behaviours.
Diffusion of
treatment
Contamination due to test subjects discussing
the treatment and measurement activities to
individuals yet to receive the treatment.
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-22
Improving Internal and
External Validity
Inclusion of control
groups
Represent the greatest strength of the
experiment and the best way to ensure internal
validity
Time order of the
manipulation
The researcher decides which variables will
occur first. Either though pre-experiment
measures or control groups, prevents
influencing the dependent variable before the
manipulation
Matching
extraneous
variables
By matching, the researcher measures certain
extraneous variables individual by individual,
and assigns those who respond similarly to the
experimental and control groups
Randomisation of
test subjects to
treatment groups
Researcher must ensure randomness in a
carefully controlled manner
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-23
Types of Experimental Designs:
Pre-experimental
One-shot study
A single group of test subjects is exposed to the
independent variable treatment X, and then a
single measurement of the dependent variable
is taken (o1).
One-group,
pretest–posttest
First a pretreatment measure of the dependent
variable is taken (o1), then the test subjects are
exposed to the independent treatment X, and then
a post-treatment measure of the dependent
variable is taken (o2).
Static group
comparison
There are two groups of test subjects: one group
is the experimental group (EG) and is exposed to
the independent treatment, and the second group
is the control group (CG) and is not given the
treatment. The dependent variable is measured in
both groups after the treatment.
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-24
Types of Experimental Designs:
True Experimental Designs
Pretest–posttest, control
group
Test subjects are randomly assigned to either the
experimental or control group, and each group
receives a pretreatment measure of the dependent
variable. Then the independent treatment is
exposed to the experimental group, after which
both groups receive a post-treatment measure of
the dependent variable.
Post-test-only,
control group
Test subjects are randomly assigned to either the
experimental or the control group. The
experimental group is then exposed to the
independent treatment, after which both groups
receive a post-treatment measure of the dependent
variable.
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-25
Types of Experimental Designs:
True Experimental Designs
Solomon
Four Group
This design combines the ‘pretest–post-test,
control group’ and ‘post-test-only, control group’
designs and provides both direct and reactive
effects of testing. Not used in marketing research
practices because of complexity and lengthy
time requirements.
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-26
Types of Experimental Designs:
Quasi-experimental Designs
Non-equivalent
control group
This design is a combination of the ‘static group
comparison’ and the ‘one-group, pretest–posttest’ pre-experimental designs.
Separate-sample,
pretest–post-test
Two different groups of test subjects are drawn;
neither group is directly exposed to the
independent treatment variable. One group
receives a pretest measure of the dependent
variable. Then after the insignificant independent
treatment occurs, the second group of test
subjects receives a post-test measure of the
dependent variable.
Field
experiment
This is a causal design that manipulates the
independent variables in order to measure the
dependent variable in the natural setting of the
event or test.
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-27
Test Marketing
Types of Test
Marketing
Comments
Traditional
test markets
Also referred to as ‘standard’ tests; these use
experimental design procedures to test a
product and/or a product’s marketing mix
variables through existing distribution
channels.
Controlled
test markets
Tests that are performed by an outside
research firm that guarantees distribution of
the test product through prespecified outlets in
selected cities.
Electronic
test markets
Tests that integrate the use of select panels of
consumers who use a special identification
card in recording their product-purchasing
data.
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-28
Test Marketing
Types of Test
Marketing
Comments
Simulated
test markets
Also referred to as ‘laboratory tests’ or ‘test
market simulations’. These are quasi experiments
where test subjects are preselected, then
interviewed and observed on their purchases and
attitudes towards the test product.
Web-based TV
test markets
Similar to electronic test markets; these use
broadband interactive TV (iTV) and advances in
interactive multimedia communication technologies
to conduct the field experiment. Preselected
respondents are shown various stimuli and asked
questions online through their iTV.
Virtual
test markets
Tests that are completely computerised, allowing
the test subjects to observe and interact with the
product as though they were actually in the test
store’s environment.
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-29
Advantages and Disadvantages of
Test Markets
Advantages of Test Markets
 Traditional tests
 Conducted in actual
distributions channels
 Can determine both
customer acceptance
and trade support
 Controlled test markets
 Distribution is assured
 Costs are lower
 Competitive monitoring
is difficult
Disadvantages of Test Markets
 Traditional tests
 Cost, time and exposure to
competition
 Competitors can ‘rush to
market’
 Controlled test markets
 Limited number of markets
 Trade support is unknown
when incentives are
removed
 The effect of advertising is
difficult to evaluate
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-30
Advantages and Disadvantages of
Test Markets
Advantages of Test Markets
 Simulated test markets
 Cost and time savings
 Can predict trial,
repurchase and
purchase cycle
accurately
 Minimise exposure to
competition
 Virtual test markets
 ‘Stores’ closely
resemble reality
 Can be changed quickly
 No exposure to
competition
Disadvantages of Test Markets
 Simulated test markets
 Isolation from real world
 Broad-based customer
reaction is difficult to
measure
 Virtual test markets
 Will consumers shop as in
real stores?
 Cost of equipment to
conduct tests
 Lack of ‘touch’
Copyright  2004 McGraw-Hill Pty Ltd
PPTs t/a Marketing Research by Lukas, Hair, Bush and Ortinau
Slides prepared by Tony Peloso
7-31