Transcript Public Relations

Primer of Public Relations
Research, Second Edition
(c) Guilford Press, 2011
1
Chapter 1
Understanding Research:
An Introduction with
Public Relations Implications
(c) Guilford Press, 2011
2
Role of Research
• Qualitative or Quantitative?
• Research Questions Asked?
(c) Guilford Press, 2011
3
Why Conduct Public Relations
Research?
• Essential to any campaign or publicity activity
• Provides feedback on the management of a
campaign
• Provides a baseline from which to judge
efficacy of campaigns
• Helps establish the bottom line and return on
investment (ROI)
(c) Guilford Press, 2011
4
Formal (Quantitative) vs. Informal
(Qualitative) Research
Quantitative Research Methods
Controlled
Objective
Systematic observation
Reliable
Valid
Deductive
Description
Understanding
Prediction
Control
Qualitative Research Methods
Data Collection
Uncontrolled
Subjective
Random observation
Data Assessment (Measurement)
Unreliable
Assumed validity
Inductive
Outcome(s)
Description
Understanding
From: Primer of Public Relations Research, Second Edition, by Don W. Stacks. Copyright 2011 by The Guilford Press. All rights reserved.
(c) Guilford Press, 2011
5
Timeline
Baseline
Secondary/
Benchmark
Informational/
Evaluation
Behavioral/
Evaluation
Motivational/
Evaluation
Planned benchmarked evaluations
Time
Development
(Evaluation)
Refinement
(Evaluation)
Final Evaluation
From: Primer of Public Relations Research, Second Edition, by Don W. Stacks. Copyright 2011 by The Guilford Press. All rights reserved.
(c) Guilford Press, 2011
6
Questions that Concern
Communication Researchers
• Questions of Definition
 Reportative
 Stipulative
• Questions of Fact




Descriptive
Empirical
Observational
Scientific
• Questions of Value
 Goodness/Badness
 “Ought to be”
 Qualitative/humanistic
• Questions of Policy
 How “should” we act?
 Both factual & value
 Used by policy makers
(c) Guilford Press, 2011
7
Advantages of Theory
• Tells us what to observe
• Tells us how to observe
• Is self-corrective
through the scientific process
• Addresses questions
primarily definition and fact
can be used for questions of value
Best Practices in Public Relations
• Research methods
and procedures
should:
– Be clear & have welldefined research
objectives
– Have a rigorous
research design
– Provide detailed
supporting
documentation
• Quality and substance
of findings should:
– Demonstrate
effectiveness
– Link outputs to outcomes
– Develop better
communication
programs
– Demonstrate impact on
business outcomes
– Demonstrate cost
effectiveness
– Be applicable to a broad
range of activities
(c) Guilford Press, 2011
9
Chapter 2
Management of Research
in Public Relations
(c) Guilford Press, 2011
10
Public Relations
• “The management of credibility.”
• “The management function that identifies,
establishes, and maintains mutually beneficial
relationships between an organization and the
various publics on whom its success or failure
depends.”
(c) Guilford Press, 2011
11
Approaches to PR Goals
Traditional
From: Primer of Public Relations Research, Second Edition, by Don W. Stacks. Copyright 2011 by The Guilford Press. All rights reserved.
(c) Guilford Press, 2011
12
Approaches to PR Goals, Cont’d.
Contemporary
From: Primer of Public Relations Research, Second Edition, by Don W. Stacks. Copyright 2011 by The Guilford Press. All rights reserved.
(c) Guilford Press, 2011
13
Approaches to PR Goals, Cont’d.
ROI Model
Outcome = B ± [Credibility ± Relationship ± Reputation ± Trust ] ± Confidence + Error
From: Primer of Public Relations Research, Second Edition, by Don W. Stacks. Copyright 2011 by The Guilford Press. All rights reserved.
(c) Guilford Press, 2011
14
Entities & Publics
• Publics
– External
– Internal
– Intervening
• Publics vs. Populations
– Target Public
– Target Audience
(c) Guilford Press, 2011
15
Research Assumptions
• Decision making is basically the same in all
organizations
• All communication should
– Set objectives
– Determine strategies to meet objectives
– Implement strategies to bring strategies to life
(c) Guilford Press, 2011
16
Research Assumptions, Cont’d.
• Research can be divided into three
general phases
– Developmental
– Refinement
– Evaluation
• Communication research is behaviordriven and knowledge-based
• PR research should be based on applied
research and informed by PR theory
(c) Guilford Press, 2011
17
Goals & Objectives
• Goal: General outcome expected by campaign
end
• Objective: Very specific projected outcome
• Outtake: Response from intended target
– Opinion leader
– Editor
• Output: Individual communication element
– Impact of specific tactics
– Written, visual, verbal
(c) Guilford Press, 2011
18
Writing Objectives
• Objectives are
• Writing
– Cause & Effect related
– Specific to the problem
• Types
– Informational
• General and specific
knowledge
– Motivational
• Attitudinal and belief
oriented
– Behavioral (most important!)
• Actual counts, $$$, and so
forth
– Written with both output and
outcome in mind
– Almost always start with “to”
– Must include some
measurable outcome — it
must be quantifiable
– In a campaign, begin with
informational, then
motivational, and finally
behavioral objectives
(c) Guilford Press, 2011
19
Stating Objectives
• Should be in the form:
To (verb) target audience (do) by (amount of change) by (target date for outcome)
• Should be results oriented in terms of
– Information (Knowledge)
– Motivational (Predispositions toward object)
– Behavior (Impact on business)
• Should specify a single outcome
– May require multiple objectives
– Knowledge  Predisposition  Behavior
(c) Guilford Press, 2011
20
Objectives, Cont’d.
Informational Objective(s)
Motivational Objective(s)
Behavioral Objectives
Business objective(s)
From: Primer of Public Relations Research, Second Edition, by Don W. Stacks. Copyright 2011 by The Guilford Press. All rights reserved.
(c) Guilford Press, 2011
21
Sample Objectives
• To increase the number of senior public relations professionals
who know how to write correct objectives from X to XX by
December 15. (Information Objective)
• To increase the percentage of senior public relations
professionals who think measurement is essential to PR
practice by 15-20% by July 1. (Motivational Objective)
• To increase the number of senior public relations professionals
who actually write measurable objectives from 5% to 25% by
June 1, 2010. (Behavior Objective)
(c) Guilford Press, 2011
22
Internal Objectives
• All campaigns take a strategy and
implement it via some form of output
• Internal objectives can be best looked at
as specific output objectives that will lead
to outtake and outcome objectives
• Add the “by” or “through” to the objective:
– To increase the number of senior public
relations professionals who know how to write
correct objectives from X to XX through press
releases to opinion leaders by December 15.
(c) Guilford Press, 2011
23
Objectives, Cont’d.
• Informational objectives fairly clear cut
• Motivational (attitudinal) objectives require
– Research
– Means to isolate effect provided by public
relations
• Behavioral objectives
– What you expect the public to do
(c) Guilford Press, 2011
24
Best Practices
• Understanding business goals and objectives
• PR objectives stated in measureable form with
planned benchmark evaluations
• PR research objectives state the methods to
be employed as well as tactics
(c) Guilford Press, 2011
25
Chapter 3
Measuring Outcomes
(c) Guilford Press, 2011
26
Measurement Theory
• Defined
• Measuring behaviors
• Measuring “Intervening Variables”
– Attitudes
– Beliefs
– Values
• Measuring via Scales
(c) Guilford Press, 2011
27
Measurement Applied
• Defined
• Levels
–
–
–
–
Nominal
Ordinal
Interval
Ratio
• Reliability
– Consistency?
• Validity
– Mirror behavior?
(c) Guilford Press, 2011
28
Measurement
• Assigning numbers to categories
• Four Levels
– Nominal (distinguishes only)
– Ordinal (distinguishes and orders)
– Interval (assumes an equal distancing between
categories)
– Ratio (assumes absolute distancing between
categories)
(c) Guilford Press, 2011
29
Measurement
• Categorical Data
– Places observations into classes
– Reported as counts, percentages, and proportions
• Continuous Data
– Places observations on a continuum
– Reported as averages, ranges, standard deviations
(c) Guilford Press, 2011
30
Measurement Examples
• Interval
– Strongly Agree, Agree, Disagree, Strongly Disagree
• Assumes that the distance between SA = A = D = SD (problem is that
SA = A, and D = SD, but D A)
• Data are interval, but not “scalar” in that there is no arbitrary zero point
• Ratio
– Actual date and time of birth
– Bank account balance
• Has a true zero point and can be expressed in the negative (i.e., –$500.00)
(c) Guilford Press, 2011
31
Measurement Validity
• Defined
• Types
– Face Validity
– Content Validity
– Construct Validity
– Criterion-Related
• Relationship to Reliability
(c) Guilford Press, 2011
32
Measurement Reliability
• Defined
• Influencing Factors
– Systematic Error
– Random Error
• Establishing
– Correlation
• Coefficient ; KR-20
• Investigations
(c) Guilford Press, 2011
33
Attitude Measurement
• Most Often Likert-Type Measurement
– Assumes interval data
– Respondents “react” to statements, typically by degree of
agreement
– MUST have a “zero” point — a midpoint
– MUST have an ODD number of responses (3, 5, 7)
– MUST consist of two or more statements
statement 1.
Strongly
Agree
statement 2.
Strongly
Agree
Agree
Agree
NA/ND
NA/ND
Disagree
Strongly
Disagree
Disagree
Strongly
Disagree
From: Primer of Public Relations Research, Second Edition, by Don W. Stacks. Copyright 2011 by The Guilford Press. All rights reserved.
(c) Guilford Press, 2011
34
Attitude Measurement
From: Primer of Public Relations Research, Second Edition, by Don W. Stacks. Copyright 2011 by The Guilford Press. All rights reserved.
(c) Guilford Press, 2011
35
Factor Analysis Example
From: Primer of Public Relations Research, Second Edition, by Don W. Stacks. Copyright 2011 by The Guilford Press. All rights reserved.
(c) Guilford Press, 2011
36
Semantic Differential
(c) Guilford Press, 2011
37
Chapter 4
Descriptive Statistical Reasoning
and Computer Analysis
(c) Guilford Press, 2011
38
Introduction
• Review of data
– Levels
– Scales
• Parametric/Nonparametric distinctions
• Populations versus samples
• Description versus inference
(c) Guilford Press, 2011
39
Computer Packages
•
•
•
•
•
IBM®SPSS®STATISTICS
SAS
BIOMED
MiniTab
SYSTAT
(c) Guilford Press, 2011
40
Nonparametric Statistics
• Introduction
• Counting
– Frequencies
– Percentages
– Proportions
• Displaying
• Contingency Tables
(c) Guilford Press, 2011
41
Parametric Statistics
• Introduction
• Measures of Central Tendency
• The “Normal Curve”
(c) Guilford Press, 2011
42
Normal Curve
From: Primer of Public Relations Research, Second Edition, by Don W. Stacks. Copyright 2011 by The Guilford Press. All rights reserved.
(c) Guilford Press, 2011
43
Measures of Central
Tendency
•
•
•
•
•
•
•
Mean (M or X with a bar over it)
Median (Md)
Mode (Mo)
Range (R)
Variance ( 2)
Standard Deviation ( )
Z-Scores (Z)
(c) Guilford Press, 2011
44
Correlation
• Use
• Parametric
– Pearson r
• Nonparametric
– Spearman Rho
• Evaluating
– Strength
– Variance Accounted for (R2)
(c) Guilford Press, 2011
45
Relationship Between Two
Variables
From: Primer of Public Relations Research, Second Edition, by Don W. Stacks. Copyright 2011 by The Guilford Press. All rights reserved.
(c) Guilford Press, 2011
46
Chapter 5
Ethical Concerns in
Public Relations Research
(c) Guilford Press, 2011
47
Public Relations Ethical Codes
• Public Relations Society of America (PRSA)
• International Association of Business
Communicators (IABC)
• International Public Relations Association
(IPRA)
• Arthur W. Page Society
• What is missing?
– Research Ethical Codification
(c) Guilford Press, 2011
48
Research Codes
• American Psychological Association
• Federal “human subject” guidelines
–
–
–
–
Agreement to participate
Ability to withdraw without penalty
Understanding what participants are getting into
No harm—psychologically or physically—must
come to participants
– All behavior/statements must remain confidential
and anonymous unless specifically released from
it
(c) Guilford Press, 2011
49
Ethical Best Practices for Public
Relations Researchers
• Treat people as you would like to be treated
• Keep your commitment to confidentiality and
anonymity of responses
• Report what you find, not what you would like
to have found, even when it is not what was
expected
• Keep promises to participants
• Know who owns the data
(c) Guilford Press, 2011
50
Chapter 6
Qualitative Research Methodology:
Content Analysis
(c) Guilford Press, 2011
51
Overview
• Uses
– A way to quantify qualitative data
– A way to examine messages objectively
– Both a measurement technique and a method
• Measure as a way of understanding how messages are
created
• Method as a way of interpreting those messages
– Particularly good for historical/critical method
(c) Guilford Press, 2011
52
Explanatory Power
• Can quantify content data as a dependent
(evaluated) variable
– Example: Type/Token Ratio
– Example: Corporate Image
• Can be used as an independent (manipulated)
variable
– Example: “Good” vs. “Bad” Coverage
– Example: Treatment of Organization by Media
(c) Guilford Press, 2011
53
Steps in Content Analysis
•
•
•
•
•
•
Review the Literature
Define the Population of Messages
Sample from that Population
Define Units of Analysis
Create Category System
Code the Data
(c) Guilford Press, 2011
54
What Is Content?
• Manifest Content
– That which is visible
– Easily observable or measurable
– Quantitative
• Latent Content
– That which is not obvious
– That which requires attitude measurement
– Qualitative
(c) Guilford Press, 2011
55
Conducting a Content Analysis
• Delimit the Population of Messages for study
• Select the Units of Analysis (can be more than one)
– Operationally define what you are looking for
– Berelson’s system:
•
•
•
•
Symbols
Themes
Characters
Space/Time Measures
(c) Guilford Press, 2011
56
Conducting, Cont’d.
– Units of Analysis:
• Symbols
–
–
–
–
–
–
names
words
phrases
utterances
pictures
Anything that can be seen and taken as is
(c) Guilford Press, 2011
57
Conducting, Cont’d.
– Themes
•
•
•
•
•
ideas
concepts
theses
Good for finding latent meaning
Examples: violence, sexual programming
(c) Guilford Press, 2011
58
Conducting, Cont’d.
– Characters
• individuals
• individual roles
• examples:
–
–
–
–
Archie Bunker types
Team Leaders
Playboy or Playgirl
Role models (could be + or –)
(c) Guilford Press, 2011
59
Conducting, Cont’d.
– Time–Space Measures
•
•
•
•
•
column inches
program or story air time
number of pages
size of story or photo or advertisement
physically measurable units
(c) Guilford Press, 2011
60
Conducting, Cont’d.
• Create Category System(s)
– Categories must reflect research purpose
– Categories must be exhaustive
– Categories must be mutually exclusive
– Categories must allow for independence
– Categories must reflect one and only one
classification principle
(c) Guilford Press, 2011
61
Conducting, Cont’d.
• Sampling
– Random
• Problems with periodicity
• Problems with special events
– Stratified
• Randomly across time periods
– Cluster
• Randomly by certain characteristics
– Newspaper: by circulation and whether daily or not
(c) Guilford Press, 2011
62
Conducting, Cont’d.
• Counting
– Nominal quantification (simple differences)
– Ordinal quantification (ordered differences)
– Interval quantification (degrees of agreement on
some metric)
(c) Guilford Press, 2011
63
Reliability & Validity
– Intercoder reliability
• 2 or more coders
• Special formula or simple
correlation
– Holsti’s reliability coefficient
Reliability =
2M
N1 + N2
Where M = total items agreed on, N1 = total items for Coder 1,
and N2 = total items for Coder 2.
– Scott’s pi index
% observed agreement – % expected agreement
Pi =
1 – % expected agreement
Where % expected agreement =
(% observed)2 + (% observed)2. . .
[e.g. (.45)2 + (.65)2 = .62].
– Krippendorf’s alpha
• Used with interval data (similar to Scott’s pi)
– Cohen’s kappa
• Used with more than 2 coders.
% observed – % expected agreement
(N × M) – % expected agreement
Where N = total number of objects
coded and M = number of coders.
From: Primer of Public Relations Research, Second Edition, by Don W. Stacks. Copyright 2011 by The Guilford Press. All rights reserved.
(c) Guilford Press, 2011
64
Reliability & Validity, Cont’d.
– Intracoder Reliability
•
•
•
•
1 coder
Usually the researcher
Codes the data twice after an appropriate time span
Uses special formula or simple correlation
– Holsti’s reliability coefficient
– Cohen’s kappa
– Scott’s pi
» Use when you have multiple categories in the unit of
analysis or want to control for chance
(c) Guilford Press, 2011
65
Chapter 7
Qualitative Research Methodology:
Historical and Secondary Methods
(c) Guilford Press, 2011
66
“All public relations actions, whether they be
adding a client or preparing for a campaign,
begin with an analysis of secondary and
historical data about the client and its
publics.”
John W. Hill
Founder, Hill & Knowlton
(c) Guilford Press, 2011
67
Getting Started
• Establish a Plan of Action
• Critical Analysis of Needs and Resources
• Orientation
(c) Guilford Press, 2011
68
Establishing Parameters
• Epistemology Enters in the Form of
– Tradition (Tenacity)
– Authority
• Basic Theoretical Assumptions
– What are you looking for (definition)?
– Where is it most likely to be found (fact)?
– What is missing?
(c) Guilford Press, 2011
69
Using the Document Repository
• Libraries
– Physical source of material
– Can access on-line searches
– Can access databases
– Library locations (where are the books, articles,
non-print materials, special collections?)
– Search strategies
(c) Guilford Press, 2011
70
Types of Libraries
From: Primer of Public Relations Research, Second Edition, by Don W. Stacks. Copyright 2011 by The Guilford Press. All rights reserved.
(c) Guilford Press, 2011
71
Sources
• Primary
– The actual document itself, in its original condition
• Secondary
– Someone else’s review and analysis of the original
material
• Tertiary
– A review of the review and analysis
(c) Guilford Press, 2011
72
Documents
• Types
– Books
– Periodicals
– Newspapers
• Electronic
– Databases
– Indexes
– Abstract Services
• Unpublished Papers
– Archives
• Personal
• Corporate
• Governmental
– Position Papers
– Convention Papers
– ERIC documents
(c) Guilford Press, 2011
73
Online Document Repositories
From: Primer of Public Relations Research, Second Edition, by Don W. Stacks. Copyright 2011 by The Guilford Press. All rights reserved.
(c) Guilford Press, 2011
74
Search Engines
•
•
•
•
•
•
Alta Vista
Bing
Google
Google Scholar
Lycos
Yahoo!
(c) Guilford Press, 2011
75
Appraising Documents
• By content
• By authority
• By critical standards
–
–
–
–
–
–
purpose clear?
central assumptions laid out?
predictions or questions clearly stated?
findings defended?
biases admitted?
written well?
(c) Guilford Press, 2011
76
Secondary Research
• Analysis of findings already published
• Sources
– Organizational Research
• Communication Audits
• Statistical Audits
– Industry Research
• From the trades through their publications
• Governmental sources (i.e., GAO/GPO)
• Third-party sources (customers, suppliers, competition)
(c) Guilford Press, 2011
77
Secondary Research, Cont’d.
• Sources, cont’d.
– Stakeholder Research
•
•
•
•
Prospective Employees
Shareholders
Governmental Agencies
Special Interest Groups
(c) Guilford Press, 2011
78
Data Analysis
• Quantitative Analysis
– Can be quite complex, if there are no
• Problems with comparability
• Problems with validity
– Includes all major statistical tests and especially metaanalysis
• Qualitative Analysis
– Content Analysis
– Historical Analysis (time, place, circumstances)
(c) Guilford Press, 2011
79
Chapter 8
Qualitative Research
Methodology:
Case Studies
(c) Guilford Press, 2011
80
Overview
• Descriptive in detail
• Describes and analyzes
– People
– Organizations
– Events
• Focus is on process
• Should take into account: time, place, &
circumstance
(c) Guilford Press, 2011
81
Case Studies
• Defined:
– In-depth studies of particular people,
organizations, events, or even processes
• Represents grounded theory, where the focus
is explaining the normal with an eye toward
creating theory based on:
– Patterns, themes, and common categories
– Concrete examples, not abstract principles
(c) Guilford Press, 2011
82
Research Questions
• Definition
• Value
– Required
– Sets the variables
• Fact
– Secondary
– Historical
– Have the advantage of
hindsight
– Historical
– Hindsight
– Appropriateness to
problem
• Policy
– Allowed due to the fact
that we know the
outcome
(c) Guilford Press, 2011
83
Approaches
• Linear (ROPE)
– Jerry A. Hendrix
• Process
– Allen H. Center & Patrick Jackson
• Grounded
– Stephen Greyser, James S. O’Rourke, IV,
& Paul Argenti
(c) Guilford Press, 2011
84
The Linear Case: ROPE
• Research
– Background
– SWOT analysis
– Publics
• Objectives
• Programming
– Theme
– Action
– Media
• Controlled
• Uncontrolled
– Impact
• Informational
• Attitudinal
• Behavioral
– Output
• Evaluation
– Impact objectives
– Output objectives
(c) Guilford Press, 2011
85
Process
• Adds the concept of “feedback” to the case
study
• Takes a more business approach
(c) Guilford Press, 2011
86
Grounded
1.
2.
3.
4.
5.
6.
7.
8.
Research & define the problem
Objectives listed & defined
Target publics identified & analyzed
Strategies employed
Tactics used
Timeline
Budgetary analysis
Evaluation of strategy & tactics in measurable
terms
(c) Guilford Press, 2011
87
Chapter 9
Qualitative Research Methods:
Methods of Observing People
(c) Guilford Press, 2011
88
Descriptive Methodologies
• In-Depth Interviews
• Focus Groups
• Participant-Observation
(c) Guilford Press, 2011
89
Overview
•
•
•
•
•
Exercising control in the natural world
Natural observation to controlled interview
Taking conversation into account
Analysis via content analysis
Analysis via prescribed roles, rules, routines
(c) Guilford Press, 2011
90
In-Depth Interviews
• Uses
– Wanting to know a lot about one individual’s
experiences
– Wanting to work with an “authority” on some
area
– Establishes “intimate familiarity” with “subjects”
of study
(c) Guilford Press, 2011
91
In-Depth Interviews
• A one-on-one conversation
• Controlled by the interviewer
• Allows the interviewee to tell his or her story
under the direction of the interviewer
• Most controlled of qualitative methods
• Most often uses an interview “schedule”
• Avoids “yes” and “no” questions
(c) Guilford Press, 2011
92
In-Depth Interviews
• Conducting
– State your research question(s) (definition, value,
policy)
– Identify interviewees
• personal contacts
• snowball technique
– Schedule the interview
• When?
• Where?
(c) Guilford Press, 2011
93
In-Depth Interviews
• Conducting, Cont’d.
– Write the schedule
• “funnel” format (general to specific)
• as open-ended as possible
– Conduct the interview
• using audio/visual equipment (permission required)
• taking notes
– what was said
– interviewee subjective feelings
(c) Guilford Press, 2011
94
In-Depth Interviews
• Data Analysis
– Note analysis
– Subjective feelings logged
– Transcription of audio/visual materials
– Content Analysis or Historical/Rhetorical-Critical
Analysis conducted
• Writing the final report
– Narrative like, it tells a story
(c) Guilford Press, 2011
95
Focus Groups
• A “controlled group discussion”
• Purposes
– Gather preliminary information
– Help develop questionnaire items for other
methods
– Better understand why people communicate or
react to communication in a controlled setting
(c) Guilford Press, 2011
96
Focus Groups, Cont’d.
• Questions best answered:
– Questions of definition (what is it that is being
communicated?)
– Questions of value (how well was something
communicated, how well could it be
communicated?)
– Questions of policy (what should be done?)
(c) Guilford Press, 2011
97
Conducting the Focus Group
• Define the Problem (Stating Research Question)
• Select the Sample
– Random selection
– Known Group Method
• Determine the number of groups necessary
– At least 2, more if splits are needed (e.g., males
and females = 4 groups)
– Allows for comparison of findings
(c) Guilford Press, 2011
98
Conducting the Focus Group, Cont’d.
• Preparation
– Costs run up to $5000 or more
– Recruitment of participants
• What is being offered?
• Compensation changes how people respond
• Need many more than will show up
– Facilities
• Comfortable, but not too comfortable
• Round table best, with moderator at 12 o’clock
(c) Guilford Press, 2011
99
The Focus Group Room
From: Primer of Public Relations Research, Second Edition, by Don W. Stacks. Copyright 2011 by The Guilford Press. All rights reserved.
(c) Guilford Press, 2011
100
Conducting the Focus Group, Cont’d.
• Preparation, Cont’d.
– Moderator selection
• Females best
• Can be costly — $500 to $4000 per session
– Materials
• Questionnaires (pre- and post-session)
• Stimulus materials
• Moderator Schedule
– Opening statements, probe questions
(c) Guilford Press, 2011
101
Conducting the Focus Group, Cont’d.
• Conducting the Study
– Recording equipment
• Audio (omnidirectional mic)
• Video (must take in all perspectives)
– 2-way mirror use
– Begin with a shared experience
– Rules for handling participants
• getting people to talk or quiet down
• removing uncooperative participants
(c) Guilford Press, 2011
102
Conducting the Focus Group, Cont’d.
• Data Analysis
– Simple
• Brief synopsis of session(s) by moderator
• Interpretation of participant responses
– Complex
• Begin with simple analysis, then have
– Transcripts made (expensive)
– Content Analysis conducted on the transcripts
(c) Guilford Press, 2011
103
Participant-Observation
• As a method
– Highly valid
• Reporting what was observed without manipulating the
actors
– Questionable reliability
• Still has a problem with researcher bias
– Takes a long time to conduct
(c) Guilford Press, 2011
104
Conducting the Participant-Observation
• How different is it from
other similar or dissimilar
groups?
• Asking the right
questions
– Questions of definition
– Questions of value
• Does this particular social
group exist?
• What are its
communication norms?
– Questions of fact
• How does this group
communicate?
(c) Guilford Press, 2011
• How well does the group
maintain its
communication norms?
• How well are the group’s
norms communicated?
To whom? With what
effect?
105
Conducting the Participant-Observation, Cont’d.
• Reviewing the Literature
– Where does the past research reside?
• Libraries
• Internet sites
• Interviews
– What does past research tell us?
• Group roles
• Group rules
• Group routines
(c) Guilford Press, 2011
106
Conducting the Participant-Observation, Cont’d.
• Developing a Theoretical Model
– Specifying what you believe are the group’s
communication norms
– Establishing typicality
– The expected
• Roles
• Rules
• Routines
(c) Guilford Press, 2011
107
Conducting the Participant-Observation, Cont’d.
• Collecting the Data
– Becoming a member of the group
– Collecting data unobtrusively
– Creating field notes – immediate notes
– Creating the reflective log – refined notes
– Expecting the unexpected
• change
• combinations of roles
(c) Guilford Press, 2011
108
Conducting the Participant-Observation, Cont’d.
• Data Analysis
– Becoming the story teller
– Selecting that which is similar and dissimilar to
what was expected
– Adding the researcher’s own feelings to the story
– Admitting biases and explaining them
(c) Guilford Press, 2011
109
Conducting the Participant-Observation, Cont’d.
• Developing the Empirical Model
– Summary of the observations
– Reflects the data observed in the naturalistic
setting
– Represents the world as it existed when the
observations were collected
(c) Guilford Press, 2011
110
Conducting the Participant-Observation, Cont’d.
• Model Comparison and Contrast
– Sums up what was expected and what was
observed
– What roles, rules, and routines were observed?
– What norms were created from the roles, rules,
and routines observed?
– When and why were norms violated? How?
– Which norms were upheld and not violated?
(c) Guilford Press, 2011
111
Advantages/Disadvantages
• In-depth Interviews
– Advantages
• Rich data
• Understand personalities
• Allows for introspection while controlled
– Disadvantages
• Expensive
• Time consuming
• Access
(c) Guilford Press, 2011
112
Advantages/Disadvantages, Cont’d.
• Focus Groups
– Advantages
• Quick to conduct
• Relatively inexpensive
• Allows for “tagging” of responses from other
members
– Disadvantages
• Reliance on moderator’s skill
• Time
• Voluntary nature of participants
(c) Guilford Press, 2011
113
Advantages/Disadvantages, Cont’d.
• Participant-Observation
– Advantages
• Occurs in the “real world”
• Sets the “roles, rules, and routines” of daily life
– Disadvantages
• Time
• Costs
• Getting others to act “naturally” around you
(c) Guilford Press, 2011
114
Chapter 10
Quantitative Research Methodology:
Sampling Messages and People
(c) Guilford Press, 2011
115
Census vs. Sampling
• Census
– Everyone in a population is counted or asked
questions
– You don’t miss anyone
• Sampling
– Obtaining a group of people from a population in
such a way as to be representative of that
population
(c) Guilford Press, 2011
116
Common Errors Found in Sampling
•
•
•
•
•
•
•
•
Universe
Population
Sampling Frame
Sample
Completed Sample
Coverage Error
Sampling Error
Measurement Error
(c) Guilford Press, 2011
117
Sampling Types
• Nonprobability Samples
– Not everyone in the population has an equal
chance of being chosen
– Sampling for a specific purpose
• Only those who fit certain criteria
– Often found when sampling children
(c) Guilford Press, 2011
118
Sampling Types, Cont’d.
• Nonprobability Samples, Cont’d.
– Types
• Convenience (“man on the street”)
• Purposive
• Quota (making sure you don’t exclude anyone, but not
allowing for everyone to be possibly in the sample)
• Snowball
• Volunteer (often paid or provided a special service)
(c) Guilford Press, 2011
119
Sampling Types, Cont’d.
• Probability Sampling
– Random Sampling
• Everyone in the sample has an equal chance of being
chosen and responding
• Allows researcher to infer beyond the sample to the
population by knowledge of
– Population parameters (characteristics)
– Estimating accuracy
– Estimating error of measurement
(c) Guilford Press, 2011
120
Probability Sampling, Cont’d.
• Random Sampling, Cont’d.
– Normal Curve
• Provides a way of establishing how much confidence
we can place in accuracy and measurement
• Larger the sample, the more “normal” the normal curve
is
• All samples have their own “normalcy”
• Provides a “confidence level” for sampling and for
response error
(c) Guilford Press, 2011
121
From: Primer of Public Relations Research, Second Edition, by Don W. Stacks. Copyright 2011 by The Guilford Press. All rights reserved.
(c) Guilford Press, 2011
122
Probability Sampling, Cont’d.
• Random Sampling, Cont’d.
– Sample size
• Determines how much error we are willing to allow
• Error is the maximum error that will be found in the
sample, the absolute error
• Can be calculated:
N=
(Py )(Pn)
Standard Error 2
(c) Guilford Press, 2011
123
Probability Sampling, Cont’d.
• Random Sampling, Cont’d.
• For infinite population magic number is 384:
.05 (acceptable error level )
N = 1.96 (area under normal curve for 95% confidence)
 Standard Error 2 
N 
 1..0596 2  .0006507
( .50) ( .50)
.0006507

.25
.0006507
 384
(c) Guilford Press, 2011
124
Probability Sampling, Cont’d.
• Random Sampling, Cont’d.
• Finite Population
( Py )( Pn )  Standard Error 2
N
( Py )( Pn )
Standard Error 2 
N
N=
(.50)(.50) + .0006507
.0006507 + .25
10,000
N = 371
(c) Guilford Press, 2011
125
Sample Calculation
• On-line calculation
• Enter either sampling error (bullets) or
measurement error you are willing to accept,
and then enter the population size from which
you are drawing your sample
• www.surveysystem.com/sscalc.htm
(c) Guilford Press, 2011
126
Probability Sampling, Cont’d.
• Types
– Simple Random Selection
• Chosen people are put back and next draw taken
– Systematic Random Selection
• Chosen people are removed and next draw taken
– Weighted Random Selection
• Chosen people represent a % of the population drawn
at random from that population
(c) Guilford Press, 2011
127
Probability Sampling, Cont’d.
• Sampling Types, Cont’d.
– Systematic Random Sampling
• Choosing via some systematic characteristic
• Uses a calculated Skip Interval:
Population Size
Sample Size
• Uses a randomly selected start number
• Stratifies by characteristic, such as sex or city
(c) Guilford Press, 2011
128
Probability Sampling, Cont’d.
• Sampling Types, Cont’d.
– Cluster Sampling
• Used when you need to get a certain percentage of
respondents from particular categories.
• Similar to weighted sampling
– You randomly select within clusters to get your total N
– For instance: Newspapers are weekly, daily, and weekday
only. To get a valid sample you would want to sample within
each “cluster.”
(c) Guilford Press, 2011
129
Chapter 11
Quantitative Research
Methodology:
Survey and Poll Methods
(c) Guilford Press, 2011
130
Polls vs. Surveys
• Polls
– Short
– More behavioral than attitudinal
– Mostly close-ended questions
• Surveys
– Longer
– Attempt to assess attitudes toward objects
– Consist of open- and closed-ended questions
(c) Guilford Press, 2011
131
Survey Design
• Types of Surveys
– Cross-Sectional
– Longitudinal
• Trend
• Panel
• Cohort
(c) Guilford Press, 2011
132
Survey Methods
•
•
•
•
Telephone
Mail
Internet
In-Person
– Random (probability)
– Random Intercept (nonprobability)
(c) Guilford Press, 2011
133
Questionnaire Structure
• Questionnaire
– Precise
– Pretested
– Types
• Closed-ended
• Open-ended
• Combination of closed- and open-ended
(c) Guilford Press, 2011
134
Questionnaire Structure, Cont’d.
• Design
– Introduction (credibility induction)
– Body (questions addressing research concerns)
– Demographics
• Statistical comparison
• Sampling adequacy
– Concluding statement
(c) Guilford Press, 2011
135
Questionnaire Structure, Cont’d.
• Question Types
– Filter Questions
– Order and Order Effects
– Design
•
•
•
•
Clear
Concise
Unambiguous
Easily stated for telephone use
(c) Guilford Press, 2011
136
Reporting Survey Results
• Minimal Reporting Considerations
– The question (as it was asked)
– Number of respondents
– How respondents were selected
– How respondents were contacted
– The error rate and sampling confidence
– Sponsors and cosponsors
– Any social desirability factors
(c) Guilford Press, 2011
137
New Technologies
•
•
•
•
Software
Formatting issues
Attitudinal scale issues
Length issues
(c) Guilford Press, 2011
138
Comparison of Survey Instruments
Paper
Computer Screen
From: Primer of Public Relations Research, Second Edition, by Don W. Stacks. Copyright 2011 by The Guilford Press. All rights reserved.
(c) Guilford Press, 2011
139
Chapter 12
Quantitative Research
Methodology:
Experimental Method
(c) Guilford Press, 2011
140
What Is an Experiment?
• Method that definitely tests for relationships
between one variable and another
– Independent variables
– Dependent variables
• Control over extraneous variables
• Explanation of variance due to variables
• Prediction
(c) Guilford Press, 2011
141
Importance of Experiments
• Advantages of the method:
– Establish if a variable exists
– Establish relationships between variables
– Ability to provide evidence for causation
• Disadvantages
– Lack of “reality”
– Lack of ability to cross the “ecological” barrier
(c) Guilford Press, 2011
142
Variable Relationships
• Ability to establish if one variable influences
another
• Requires a controlled environment
• Allows researcher to establish whether a
relationship is real or spurious
• Reduces impact of extraneous variables
(c) Guilford Press, 2011
143
Relationship Testing
• Laboratory Experiment
– Highly controlled
– Artificial
• Simulation Experiment
– Takes what is found in the lab and extends it
to other variables under controlled conditions
• Field Experiment
– Reduces control over variables to see how
different situations change relationships
(c) Guilford Press, 2011
144
Causal Relationships
• Determines whether one variable actually
causes another to change
• Three criteria necessary for causation:
– Variables must be empirically related
– The effect must follow the cause in time
– The relationship cannot be accounted for by a
third variable
(c) Guilford Press, 2011
145
Causation Criteria
• Covariation
• Dependent variable is systematically changed by the independent
variable in an empirically observed way
• Randomization
• All parties in the study have an equal chance of being in an
experimental or a control group
– experimental groups receive a manipulation that results in a change
– control groups receive no such manipulation
• Control
• All conditions are made as similar as possible
• Participants are assigned to conditions randomly
(c) Guilford Press, 2011
146
Threats to Internal Validity
•
•
•
•
•
•
•
History
Maturation
Testing
Instrumentation
Selection
Regression
Mortality
=
=
=
=
=
=
=
Participants’ background
Change over time
The situation
The manipulation
Selection criteria
Time or selection
Loss of participants
(c) Guilford Press, 2011
147
Controlling for Internal Validity
• Random Assignment
– Takes care of history, maturation, regression, and
selection
• Comparison of Conditions
– Takes care of testing, instrumentation, and
mortality
(c) Guilford Press, 2011
148
The “True” Experiment
•
•
•
•
Has an experimental and a control group
The experimental group is manipulated (X)
The control group is not manipulated
Design:
R
O1
R
O3
X
O2
O4
(c) Guilford Press, 2011
149
Other Designs
• Experimental
– Must have random assignment
– Must have a control group for comparison
R O1 X O2
R
O3
• Quasi-Experimental
– Has control group R O O X O O X
– May not have random assignment O X O X
• Becomes a question in Internal Validity
(c) Guilford Press, 2011
150
Experiments & Public Relations
• Used to test relationships
• Testing practice assumptions (e.g.,
“multiplier”)
• Pretesting messages
• Produce practical simulations to test
relationships under different conditions
(c) Guilford Press, 2011
151
Chapter 13
Quantitative Statistics:
Advanced Inferential Statistical
Reasoning and Computer
Analysis
(c) Guilford Press, 2011
152
Inferential Statistics
• Tests whether results are truly the result as
found or whether they occurred by chance
• Allows for testing of predictions
• Usually tests that account for 5% error or less
(you are 95% confident that the variable
differences due to an independent variable on
a dependent variable were not by chance)
(c) Guilford Press, 2011
153
Probability of Error
One-tailed test
Two-tailed test
From: Primer of Public Relations Research, Second Edition, by Don W. Stacks. Copyright 2011 by The Guilford Press. All rights reserved.
(c) Guilford Press, 2011
154
Statistical Significance
•
•
•
•
Testing hypotheses
Answering Questions
Specifying Significance Levels
Type I and Type II Error
– Related to sample size
– If we are conservative, then we want minimal
Type I error (false positive into the body of
knowledge)
(c) Guilford Press, 2011
155
Error Relationships
Relationships are really:
TRUE
You find the
Relationships:
FALSE
TRUE
No problem
Type I error
FALSE
Type II error
No problem
From: Primer of Public Relations Research, Second Edition, by Don W. Stacks. Copyright 2011 by The Guilford Press. All rights reserved.
(c) Guilford Press, 2011
156
Statistical Analyses
• Calculated Values
• Critical (Tabled) Values
• Degrees of Freedom
Degree(s)
of
Freedom
Statistic = ##; df = xx; p = ##
Calculate
d value
Probability of
error
(c) Guilford Press, 2011
157
Statistical Tests
• Nonparametric Tests
– Chi-Square
•
•
•
•
•
Goodness of Fit (off Crosstabs)
Independence (off Nonparametric)
Coefficient of Contingency (C)
Phi/Cramer’s V
Problems
– Effect size
– Yates Correction
(c) Guilford Press, 2011
158
Statistical Tests
• Parametric
– Variance
Accounted for (between groups)
Unaccounted for (within groups/individuals)
Tests
t-tests
Sample size restriction
IV and DV restrictions
(c) Guilford Press, 2011
159
Statistical Tests
• Parametric
– ANOVA
•
•
•
•
F-statistic
Simple ANOVA
Factoral ANOVA
One-way ANOVA
– Post hoc tests
» Significance requirements
» Test limitations
» Uses
(c) Guilford Press, 2011
160
Predictive vs. Descriptive Tests
• Descriptive tests look for differences
• Predictive tests look to predict outcomes
– Regression
– Path Analysis
– Structural Equation Models
(c) Guilford Press, 2011
161
Chapter 14:
Writing and Evaluating the Request
for Research Proposal
(c) Guilford Press, 2011
162
What Is It?
• Request for Proposals
• A way to generate research
• If you want a third party
• If you need objective research from outside the
organization
• A way to get research
• If you are a research firm or even an “internal client”
• Competition
(c) Guilford Press, 2011
163
Writing the RFP
• First identify what you want done and when
• Carefully state what the requirements are and
what parameters you need covered
• From previous experience (or from reading
other RFPs) identify who might or might not
reply.
(c) Guilford Press, 2011
164
From: Primer of Public Relations Research, Second Edition, by Don W. Stacks. Copyright 2011 by The Guilford Press. All rights reserved.
(c) Guilford Press, 2011
165
The RFP
I.
II.
Introduction
Research types
considered
A.
B.
C.
III.
IV.
V.
A.
B.
C.
D.
Quantitative
Qualitative
Triangulation
Forms
Special requirements
Budget
Timetable
VI.
Research preferences
Eligibility requirements
A.
B.
Proposal requirements
Expertise
Inclusion/exclusion
criteria
History of your
organization
VII. Review process &
important dates
VIII. How and where to
submit proposals
(c) Guilford Press, 2011
166
Chapter 15
Writing and Presenting
the Final Research Report
(c) Guilford Press, 2011
167
Writing the Report
• Executive Summary
– Quick overview of the study
– Major findings of interest
• Research Specifics
– Background (literature review)
– Method
– Results
– Discussion/Ramifications
– Appendices
(c) Guilford Press, 2011
168
Presenting the Report
•
•
•
•
Oral presentations have short “windows”
Make the presentation to the entire audience
Present in a conversational style (don’t read!)
Audiovisuals are not as important as
interpretations
• Keep the audiovisuals as simple as possible
• Be prepared to stop and answer questions
(c) Guilford Press, 2011
169
Things to Consider
• Executive Summary
– Not a full-fledged report
– Summarize findings
– Important facts and figures are imbedded in the
presentation
• Do not read the slides
– They are talking points
– They make the findings concrete
(c) Guilford Press, 2011
170
Sample Slides
• Charts
• Tables
• Graphics
(c) Guilford Press, 2011
171
Gantt Chart
Task
Begin Search
Write RFP
5
1
Advertise
3
Receive
1
Evaluate
5
Send Notifications
Jun 5
2
Jun 12
Jun 19
Jun 26
Time Frame
172
Percent Awareness by Sex
Official
Male
Female
Slesnick
50%
50%
Anderson
10%
30%
Cabrera
5%
5%
Kerdyck
20%
10%
Withers
15%
5%
173
Demographics: Sex of Respondent
males
47%
females
53%
174
Demographics
50
45
40
35
30
25
20
15
10
5
0
males
females
< 18
19-24
25-30
30+
175
Order of Presentation
• Title Slide (with credits)
• Quick summary of why
the study was done
• Any specific questions
addressed
• Your method
• Your findings
– Bulleted major points
– Supporting evidence
summarized in
• Tabular format
• Graphic format
• Your conclusions
(c) Guilford Press, 2011
176
Final Summary Slide
•
•
•
•
Simple review of findings
What they mean in general
Specific things that might require more study
Open for questions
(c) Guilford Press, 2011
177