Transcript moriarty8e_media_19

Advertising Principles
and Practices
Evaluation of
Effectiveness
Questions We’ll Answer
• How well do you understand why and
how advertising evaluation is
conducted?
• Can you list and explain the stages of
message evaluation?
• What are the key areas of media
evaluation?
• How are campaigns and IMC programs
evaluated?
Prentice Hall, © 2009
19-2
Outstanding in the Field
• New Holland had to reinvent and
reintroduce its tractor brand with
sliding market share, a shrinking
market, and limited budget.
\
• The campaign
positioned New
Holland as the better choice of an
empowered consumer, and drove
traffic to its Web site.
Visit the
Site
• Sales grew by 9%, market
share by 21% and 36% for
low and high horsepower
categories, Web site visits
were 68,000 over the
campaign period.
Prentice Hall, © 2009
19-3
Impact: How does it work?
• Many executive feel advertising is only
successful if it produces sales.
• Others feel advertising should emphasize
long-term brand building.
• If advertising delivers the desired
communication effects, but
sales don’t increase, was the
advertising ineffective?
Video Snippet
AFLAC’s initial campaign
created 89% brand
recognition.
Prentice Hall, © 2009
19-4
Evaluating Effectiveness
• Intuitive analysis is based on an experienced
managers judgment.
• Measurement tracks consumer responses with
structured feedback like response cards and calls.
• Formal evaluation is necessary:
– Financial stakes are high—production of :30 spot
averages $200,000; national media costs several
million.
– Advertising optimization—reducing the risk failure
through testing, analyzing, tracking performance, and
making changes to increase performance.
– Identify “best practices”—what works and what
doesn’t, so brand advertising continues to improve.
Prentice Hall, © 2009
19-5
Types of Evaluation
• Testing—to predict results
– Sample ads are tested before they run.
• Monitoring—to track performance
– Performance is tracked to see if anything needs
to be changed.
• Measurement—to evaluate the results
– The results, or actual effects, are measured after
the campaign runs.
Prentice Hall, © 2009
19-6
Stages of Evaluation
1. Developmental research
• Pretesting to see if an idea will work, or another is
better.
2. Concurrent research
• Tracking studies and test marketing to see how
campaign is unfolding and how messages and media
are working.
3. Posttesting research
• Comparing the impact of campaign after it’s over
against a benchmark, baseline, or other starting
point.
4. Diagnostic research
• Taking apart an ad to see what elements are working
and which aren’t; examine frame by frame or piece
by piece.
Prentice Hall, © 2009
19-7
Facets: Measuring Responses
• It’s difficult to measure advertising’s effect on sales:
– Other factors affect sales (e.g., pricing, distribution,
competition), making it hard to isolate impact.
– Effects are delayed; it’s hard to link sales to
advertising.
• Communication effects an be measured as surrogate
measures for sales impact:
– Awareness of the advertising, purchase intention,
preference, liking.
Principle:
Good evaluation plans, as well as effective
promotional work, are guided by a model of how
people respond to advertising.
Prentice Hall, © 2009
19-8
Table 19:1
Effect
Perception
Awareness/Noticed
Attention
Recognition
(Aided )
Relevance
Emotion
Liking/Disliking
Desire
Cognition
Interest
Comprehension/
Confusion
Recall (Unaided)
Brand Recall/
Linkage
Differentiation
Effectiveness Research Questions
Research Questions
What ads do you remember seeing?
Which ads were noted?
What caught your attention?
Did the ad stand out among the other ads and content around it?
What stood out in the ad?
Have you seen this ad/this campaign?
Sort elements into piles of remember/don’t remember.
How important is the product message to you? Does it speak to your interest and
aspirations?
What emotions did the ad stimulate?
How did it make you feel?
Do you like this brand? This story? The characters (and other ad elements)?
What did you like or dislike about the brand? The ad?
Do you want this product or brand?
Did you read/watch most of it? How much?
Did it engage your interest or curiosity?
Where did your interest shift away from the ad?
What thoughts came to you? Do you understand how it works? Is there anything
in the ad you don’t understand? Do the claims/product attributes/benefits make
sense? Do you have a need for this brand or can it fulfill a need for you?
What happened in the commercial? What is the main message? What is the
point of the ad?
What brand is being advertised in the ad? (In open-ended responses, was the
brand named?)
What’s the difference between Brand X and Y?
Prentice Hall, © 2009
19-9
Table 19:1
Effectiveness Research Questions
Effect
Research Questions
Persuasion
Attitude
Do you have a favorable or unfavorable opinion of the brand? The ad?
How excellent or weak is the brand? The ad?
Do you respect it?
In Category X (or product set), which brand would you choose? (Usually a pre- or
posttest question).
What brand do you prefer?
Do you want to try or buy this product/brand?
Would you put it on your shopping list?
What are your reasons for buying it? Or for not buying it—or its competing
brand(s)? How does it compare to competitor’s brands?
Do you argue back to the ad?
Do you believe the reasons, claims, or proof statements?
Are you convinced the message is true? The brand is best?
Do you have confidence in the brand?
When you think of this brand, what (products, qualities, attributes, people,
lifestyles, etc.) do you connect with it?
Do you link this brand to positive experiences?
What is the personality of the brand? Of whom does it remind you? Do you like
this person/brand personality?
What is the brand image? What does it symbolize or stand for?
Can you see yourself or your friends using this brand?
Do you connect personally with the brand image?
Preference
Intention
Argument/Counter
Argument
Believability/Conviction
Trust
Association
Personality/Image
Self Identification
Prentice Hall, © 2009
19-10
Message Evaluation
• Copy Testing
• Message
Development
Research
• During Execution:
Concurrent Testing
• Posttesting: After
Execution
Research
Prentice Hall, © 2009
19-11
Copy Testing
• Companies that conduct research and perform
diagnostic methods to identify an ad’s strong and
weak points:
– Ameritest: brand linkage, attention, motivation, communication, flow of
attention and emotion through the commercial
– ARS: persuasion, brand/ad recall, communication
– Diagnostic Research: brand recall, main idea, attribute statements
(importance, uniqueness, believability)
– IPSOS-ASI: recall, attention, brand linkage, persuasion, (brand switch,
purchase probability), communication
– Mapes & Ross: brand preference change, ad/brand recall, idea
communication, key message delivery, like/dislike, believability,
comprehension, desire to take action, attribute communication
– Millward Brown: branding, enjoyment, involvement, understanding, ad
flow, brand integration, feelings to ad, main stand-out idea, likes/dislikes,
impressions, persuasion, new news, believability, relevance
– RoperASW: overall reaction, strengths and weaknesses, understanding,
clutter-busting, attention, main message, relevance, appeal, persuasiveness,
motivate trial, purchase intent
Prentice Hall, © 2009
19-12
Ameritest
• Ameritest is an international advertising research
firm that studies thousands of ads for some of the
world’s largest advertisers to discern which ones
work, which ones don’t, and which are close—and
why.
Visit the
Site
Prentice Hall, © 2009
19-13
Message Development Research
• Concept Testing
– Compares the effectiveness of various message
strategies and their creative ideas (the big idea).
• Pre-testing
– Helps marketers make final go/no-go decisions about
finished/nearly finished ads using photoboards or
animatics.
• Diagnostics
– Designed to diagnose strengths and weaknesses of
ideas to improve work still in development or to learn
more in order to improve subsequent advertisements.
Principle:
Advertising effects are too complicated to be reduced to a
single score.
Prentice Hall, © 2009
19-14
During Execution:
Concurrent Testing
• Coincidental Surveys
– In broadcast media, random calls to target market determine
stations choices, ads they’ve seen/heard, brand perceptions.
• Tracking Studies
– Every 3 to 6 months, measure top-of-mind brand awareness.
– Brand tracking tracks the performance of the brand.
• Test Markets
– Evaluate product variations, campaign or media elements.
– Generally, two or more markets with markets as controls.
Prentice Hall, © 2009
19-15
Posttesting:
After Execution Research
• Breakthrough: Attention—interest, enjoyability, liking
• Engagement Tests—eye-tracking as readers scan ads
• Memory Tests—recognition test, recall tests, unaided
recall, aided recall
• Emotion Test—MRI measures brain activity
• Likeability Tests—relevant, important, enjoyable,
entertaining, fun
• Persuasion Tests—intention to buy, motivation
• Inquiry Tests—measures number of responses to an ad
• Scanner Research—tally up purchase and collect
consumer buying info
• Single-Source Research—advertising and brand purchase
data come from the same households, linking advertising
to sales
Prentice Hall, © 2009
19-16
Media Evaluation
• Evaluating Audience Exposure
– How did each media vehicle perform? Were reach and frequency
objectives met?
– Services include Simmons-Scarborough, Arbitron, MediaMark.
– For outdoor, traffic counts don’t equal exposure.
– For Web or Internet advertising, what is measured and how does it
compare to traditional media: hits, click-throughs, minutes spent?
– Alternative or guerilla marketing is even more difficult to equate to
traditional media.
Prentice Hall, © 2009
19-17
Arbitron
• Arbitron Inc. measures radio, TV, cable, online radio
and out-of-home media audiences; surveys the retail,
media and product patterns of local market
consumers; and provides software for analyzing
media audience and marketing information data.
Visit the
Site
Prentice Hall, © 2009
19-18
Media Evaluation
• Advertising ROI and Media Efficiency
– Return on investment (cost to sales ratio) is
hard to calculate because many factors affect
sales.
– How do you determine if you’re
overadvertising or underadvertising?
– Wearout—recall stabilizes or declines and
irritation increases until there’s no or less
response (can be a combo of creative impact
and media buying).
– Media optimization—the goal is optimum
media performance getting the most impact
for the investment.
Prentice Hall, © 2009
19-19
Evaluating Marketing
Communications Campaigns
• Last, and perhaps most important, stage in the
development of a campaign plan
• Determines whether the campaign’s message
and media were effective
• Measures the overall impact on the brand, but
the pieces are still evaluated to determine their
individual effectiveness.
Prentice Hall, © 2009
19-20
Marcom Tools
• Certain marketing communication functions such
as public relations and sales promotion, do some
things better than other areas.
• An integrated plan uses the best tools to
accomplish the desired effect.
Principle:
Advertising is particularly effective in
accomplishing such objectives as creating
exposure, awareness, and brand image, and
delivering brand reminders.
Prentice Hall, © 2009
19-21
Table 19:2
Message Effectiveness Factors
Key Message
Effects
Surrogate
Measures
Communication Tools
Perception
Exposure
Adv Media; PR, PoP
Attention
Adv; Sales Promo, Packaging; PoP
Interest
Adv; SP; PR, Direct; PoP
Relevance
Adv; PR; Direct; PoP
Recognition
Adv; PR, Pkging; PoP, Specialties
Understanding
Adv; PR; Sales; Direct
Recall
Adv; SP; PR, PoP, Specialties
Cognitive
Adv; PR; Pkging
Affective
Emotions and Liking
Adv; Sales Promo, Pkging; PoP
Appeals
Adv; PR; Sales; Events/Spnsrshps
Resonate
Adv; PR; Events/Spnsrshps
Attitudes
Preference/Intention
Adv; PR; Direct
Adv; PR; Sales; Sales Promo
Credibility
PR
Conviction
PR; Sales; Direct
Motivation
Adv; PR; Sales; Sales Promo
Brand Assoc
Brand Image
Adv; PR, Events/ Spnsrshps
Action
Trial
SP; Sales; Direct, PoP
Purchase
Repeat Purchase
SP; Sales; Direct
Adv; SP; Sales; Direct, Specialties
Persuasion
Prentice Hall, © 2009
19-22
Direct Response
• The objective is to generate
an immediate behavior
response (transaction, buy)
• Use toll-free numbers, mailin coupons, Web site or
email address, an offer in
the copy
• Response is easy to measure
in terms of effectiveness and
ROI.
– Total responses divided by
total mailed = Response per
thousand (RPM)
Prentice Hall, © 2009
19-23
Sales Promotion
• May be necessary to
evaluate both trade
and consumer
promotions
• Payout analysis
compares the costs of
a promotion to the
expected sales
• Breakeven
analysis—finds the
point at which the
total cost of the
promotion exceeds
the total revenues
Prentice Hall, © 2009
19-24
A Sales
Promotion
Breakeven
Analysis
At the breakeven point, where
30,000 premiums are
delivered at a cost of
$45,000, the sales revenues
exactly cover, but do not
exceed, total costs. Below
and to the left of the
breakeven point (in the
portion of the diagram
marked off by dashed lines)
the promotion operates at a
loss. Above and to the right of
the breakeven point, as more
premiums are sold and sales
revenues climb, the
promotion makes a profit.
Prentice Hall, © 2009
19-25
Public Relations
• Measure the success in getting out the
message in terms of output and outcomes
– Output: materials produced and distributed;
how many press releases ran
– Input: acceptance and impact of materials;
changes in public opinion
• Content analysis: Was coverage favorable?
• Public opinion studies: Have attitudes,
behaviors, or knowledge changed?
Prentice Hall, © 2009
19-26
Common Measures of Output and Outcomes in Public Relations
Output Objectives Achieved
Production. Number of PR messages, such as news releases or brochures, generated.
Distribution. Number of media outlets (TV stations, newspapers) receiving PR products.
Coverage. Number and size of clips, column inches, or minutes of time or space.
Impressions. Media placements multiplied by circulation or broadcast reach.
Advertising value. Equivalent ad costs for time or space.
Content Analysis. Positive or valence (whether the story or mention seems to be more
positive or negative), key messages (the idea in the story), sources, and prominence.
Outcome Objectives Achieved
Awareness. Aided and unaided recall.
Attitudes. Perceptions, preferences, and intent to buy.
Behavior. Did target do what you wanted them to do?
Prentice Hall, © 2009
19-27
Web Site Evaluation
• Traffic volume
– Page views
– Site visitors
• Click-through rates
– Ads sold as pay-per
click
• Cost per lead
– An attempt to measure
ROI using a conversion
rate (percent of visitors
who complete desired
action)
Prentice Hall, © 2009
19-28
Special Advertising Situations
• Retail
advertising
• B2B
advertising
• International
advertising
• Objective: Generate store
traffic
– Simple counts of people at
promotions and events
• Objective: Visibility
– Participation counts at
events, or “how-to”
classes
– Sign-up and fill-out forms
• Objective: Loyalty
– Participation in frequency
clubs or loyalty programs
Prentice Hall, © 2009
19-29
Special Advertising Situations
• Retail
advertising
• B2B
advertising
• International
advertising
• Objective: Generate
response/sales leads
– Lead count based on calls,
emails, and cards returned
to the advertiser
• Objective: Conversion
rates—number of leads
who make a purchase
Prentice Hall, © 2009
19-30
Special Advertising Situations
• Retail
advertising
• B2B
advertising
• International
advertising
• Difficult to evaluate
because of the number of
markets, distance, cost
and variety of cultures.
• Evaluation should focus
initially on pretesting to
help correct big problems
(due to unfamiliarity with
the culture, language or
consumer behavior)
before they occur.
Prentice Hall, © 2009
19-31
Campaign Evaluation
• It’s difficult to evaluate and
estimate the impact of synergy.
• Brand tracking can measure
campaign effectiveness by
adding and taking away
ingredients, and studying the
effects of those changes
• The challenge: look at the big
picture rather than individual
pieces and parts.
• Advertisers seek an evaluation
method that brings all the
individual metrics together to
efficiently and effectively
evaluate and predict
communication effectiveness.
Prentice Hall, © 2009
19-32
ARS Group
• ARS Group is a marketing research firm that helps
clients evaluate and optimize the effectiveness of
their advertising messages within individual
touchpoints and across integrated marketing
campaigns.
Visit the
Site
Prentice Hall, © 2009
19-33
Discussion Questions
Discussion Question 1
• Many creative people feel that formal copy
testing research doesn’t do justice to their
ideas.
• In particular, they feel that research results are
designed to reward cognitive approaches and
don’t do a good job of evaluation for brand
image ads and emotional ads.
• From what you have read in this chapter
about copy testing, why do they feel that
way?
• Do you believe that is a legitimate criticism of
copy testing?
Prentice Hall, © 2009
19-35
Discussion Question 2
• Most clients want a quick and easy answer
to the question of whether the ad works.
• Advertising professionals, however, tend to
believe that a one-score approach to copy
testing is not appropriate.
• Why would they feel that way?
• If you were helping an agency prepare for a
presentation on its copy testing results, what
would you suggest the agency say to explain
away the idea that you can evaluate an ad
with a single test?
Prentice Hall, © 2009
19-36
Discussion Question 3
• Three-minute debate: You are hiring a research
consulting company to help a client evaluate the
effectiveness of its advertisements. One of the
consultants recommends using focus groups to evaluate
their effectiveness. Another consultant suggests that
focus groups aren’t very effective for posttesting and
recommends other measures. Which viewpoint do you
believe is most insightful?
• In class, organize into small teams with pairs of teams
taking one side or the other. Set up a series of threeminute debates with each side having half of that time to
argue its position. Every team of debaters has to present
new points not covered in the previous teams’
presentations until there are no arguments left to present.
Then, the class votes as a group on the winning point of
view.
Prentice Hall, © 2009
19-37
All rights reserved. No part of this publication may be reproduced, stored in a
retrieval system, or transmitted, in any form or by any means, electronic,
mechanical, photocopying, recording, or otherwise, without the prior written
permission of the publisher. Printed in the United States of America.
Copyright © 2009 Pearson Education, Inc.
Publishing as Prentice Hall
Prentice Hall, © 2009
19-38