Transcript Evaluation of the NC HWTF TRU Media Campaign

Evaluation of
North Carolina
TRU Media Campaign
Sponsor:
The North Carolina Health & Wellness
Trust Fund (HWTF)
Media Vendor:
CapStrat
Evaluators:
University of North Carolina School of
Medicine Tobacco Prevention &
Evaluation Program
(UNC-TPEP)
Presenters:
Kelly Kandra, Shelley Summerlin-Long
Presentation Overview
 Background
 Media Campaign
 Evaluation Methodology
 Focus Group Results
 Telephone Survey Results
 Conclusions
Youth and Tobacco Use in NC
 25% of NC population under 18
 2 million youth (1.4 million in school)
 750,000 ages 11-17 (target of TRU campaign)
 Tobacco use:
 14.3% middle school students
 33.7% high school students
Sources: 2003 NC Census, 2003 NC Youth Tobacco Survey
NC’s Teen Tobacco Prevention Program
Health and Wellness Trust Fund
Created 2001
Receives 25% of NC’s MSA funds
Current annual budget of $15 million makes NC 21st
in country in funding for tobacco prevention
NC’s Teen Tobacco Prevention Program
Teen Tobacco Use Prevention and Cessation Initiative
68 grants: 47 community; 7 statewide – minority
youth; 14 TFS mini-grants
Statewide media campaign – funded at ~ $1.5 million
in 2004
$2 per 11-17 year old in NC
Campaign Background
 Brand: Tobacco.Reality.Unfiltered. or TRU
 Radio campaign: launched 2003
 Decision to switch to TV based on early evaluation
Campaign Background
UNC-TPEP produced report on best practices
for youth anti-tobacco media campaigns to guide
formation of TV ads.*
 Literature review
 Expert Interviews
 Stakeholder interviews
* The report is available at:
http://fammed.unc.edu/TPEP/hwtfceval/reports/media_analysis03.pdf
Campaign Background
 Made recommendations for TRU ads based on 4 point model:
 Content/Themes
 Serious health consequences, SHS, Industry, Addiction
 Format
 Testimonials, graphic images
 Emotional Tone
 Negative emotional tone – sadness, anger, fear, shock
 People in Ads
 Multicultural, real people
Campaign Background
 TRU campaign launched on TV in April 2004
 Ads ran April-early Nov. 2004
 Ads played more frequently in Charlotte media market to
test effect of different levels of exposure to the campaign
TRU Ads
Anna
Teen talks about grandmother smoking since
age 13, getting mouth cancer
Jacobi
Young man gives facts about smoking; says
smoking killing his 34-year-old uncle
Brad
Teen gives facts about smoking; says friend
smokes and cannot run across the street
Travelogue
“TRU Road Trip” with stories from a woman
with tracheotomy, addicted teen who has
smoked since age 8
Evaluation: Methodology
 Adolescent focus groups
 Longitudinal telephone survey
Focus Group Goals
 to understand how NC youth
seeing/interpreting TRU ads
 to gain insight on how ads might be
improved
The full focus group report is available at: http://fammed.unc.edu/TPEP/focus_group_report05.pdf
Focus Groups
 14 groups conducted statewide after campaign, Nov/Dec
2004
 Groups of varying age, gender, ethnicity, and smoking
status
 ~ 140 youth ages 13-17
 Tested TRU and non-TRU ads
Focus Group Results
 Youth response to most frequently shown ads (Anna,
Jacobi) mixed
 Youth felt these ads could have been more effective if:
 they had grabbed attention earlier;
 people were more emotional when telling their stories
 they had featured the actual people suffering health
consequences
 they had shown graphic images of the health consequences
Focus Group Results
The last ad produced for the campaign, Travelogue, was
chosen as the best/most effective ad (of TRU and nonTRU ads shown) by the majority of youth in 13 of 14
groups.
Focus Group Results
 The reasons youth gave for choosing Travelogue
included three major elements emphasized by groups as
effective or memorable.
 graphic display of serious health consequences of smoking,
which elicited strong reactions/emotions;
 the featured individuals telling stories about their own
experiences, rather than those of a friend or family member; and
 a diversity (ethnic, gender, and age) of people.
Telephone Survey
 Time 1: 637 NC youth (ages 11 to 17); Time 2: 604 NC
youth
 Longitudinal design (will continue to follow)
 Spring and Fall of 2004
 Assessed campaign awareness, comprehension, and reaction
 Short-term outcomes of smoking behaviors and intentions,
tobacco-related knowledge and attitudes
 Not expected that behaviors or attitudes would change in just 6
months
Sample Design
Dual frame approach
 Random digit dialing (probability of selection  0.00071)
 Age-targeted (probability of selection  0.04389)
 Specified 10:90 split for final sample
Stratified into two domain
Stratum
RDD
Targeted
Total
Charlotte
181
425
606
Remainder of
State
702
1316
2018
Total Used
883
1741
2624
Final Response Rates Time 1
 2624 calls placed into one of four outcomes
 Completed or partial interviews by eligible respondents (n = 637)
 Non-response or refusal to participate by eligible respondents (n
= 378)
 Ineligibility (n = 1176)
 Unknown eligibility (n = 433)
 48 RDD completed calls
 589 targeted completed calls
Unweighted Response Rates
Response rate = number of completed interviews / number of eligible
households in sample
Response Rate 4 (RR4)
 American Association for Public Opinion Research Standard Definitions
(2000)
IP
RR 4  (100)
I  P  NR  e(U )
Unweighted Response Rate
 Response Rate 4 (RR4)
 Must determine e, the estimated proportion of cases of unknown
eligibility that may be eligible
e
nU  U  NE 2624  433  1176

 0.463
nU  U
2624  433
RR4 (100)
637  0
 52.4%
637  0  378  .463(433)
Weighted Response Rate
Computed by multiplying unweighted counts for frames by adjusted
weights
RR  (100 )
[I14 wU14 )  ( P14 wU14) ]
[I14 wU14 )  ( P14 wU14) ]  [( NR14 wU14 )]  e[U14 ( wU14 )]
RR  54.1
e
[(nU14 wU14 ]  [(U14 wU14 )]  [NE14 wU14 )]
[(nU14 wU14 ]  [(U14 wU14 )]
e  11.1
Telephone Survey
 Campaign designed to measure a “dose effect”
 Minimum dose needed to show effect not well established
 Data weighted according to NC demographics
Procedures for Weighting Data
 Raw weight calculation
 Trimming of excessive or large weights
 Post-stratification adjustment
Telephone Survey Results
Campaign Awareness
 45% of youth reported seeing at least one ad
 Charlotte youth reported higher campaign awareness (52.9%) than rest of
state (41.5%)
 Younger youth reported greater awareness (47.4%) than older youth (41%).
 Non-white youth reported greater awareness (54.8%) than white youth
(40.5%).*
 No overall awareness differences between males and females.
* Statistically significant
Individual Ad Awareness
The Anna ad was the most
commonly seen of all the TRU
ads with 29.9% of the youth
reporting confirmed awareness of
that ad in comparison to 19.1%
for Jacobi , 5.2% for Brad, and
5.5% for Travelogue.
Percentage
Ad Reactions - Anna
100
90
80
70
60
50
40
30
20
10
0
Talked to friends about
ad
Reasons not to smoke
Grabbed my attention
Ad was convincing
Anna Awareness = 29.9%
Percentage
Ad Reactions - Jacobi
100
90
80
70
60
50
40
30
20
10
0
Talked to friends about
ad
Reasons not to smoke
Grabbed my attention
Ad was convincing
Jacobi Awareness = 19.1%
Percentage
Ad Reactions - Brad
100
90
80
70
60
50
40
30
20
10
0
Talked to friends about
ad
Reasons not to smoke
Grabbed my attention
Ad was convincing
Brad Awareness = 5.2%
Percentage
Ad Reactions - Travelogue
100
90
80
70
60
50
40
30
20
10
0
Talked to friends about
ad
Reasons not to smoke
Grabbed my attention
Ad was convincing
Travelogue Awareness = 5.5%
Brand Recognition
Tobacco.Reality.Unfiltered.
Time 1
Time 2
Charlotte at Time 2
TRU*
0
20
40
60
80
100
% of youth that recognized brand/logo
*The TRU brand was not tested in Time 1.
Attitudes Toward Smoking
Cigarettes are not addictive.*
Smoking shows you’re not
afraid to take risks.
Time 1
Smoking cigarettes makes
people look cool/fit in.
Time 2
Young people who smoke have
more friends.
It is safe to smoke for only a
year or two.*
0
2
4
6
8
10
% of youth who Strongly Agree/Agree with statement
12
14
Attitudes Toward Smoking
Secondhand smoke is
harmful to you.*
People who smoke have
a harder time keeping up
in sports
Time 1
Time 2
Not smoking is a way to
express your
independence.
0
20
40
60
80
% of youth who Strongly Agree/Agree with statement
100
Susceptibility to Smoking
 Susceptibility to smoking is a measure of “likelihood to
smoke” based on youth’s responses to several
hypothetical questions:
 Do you think you will smoke a cigarette in the next year?
 Do you think that you will ever smoke a cigarette in the future?
 If one of your best friends offered you a cigarette, would you
smoke it?
Susceptibility to Smoking
 Initial results show that youth who were
non-susceptible, non-smokers at Time 1 and
aware of the TRU campaign were less likely
to have experimented with cigarettes at Time
2 than their non-TRU aware counterparts.
 Other data on smoking behavior are
inconclusive, as expected at such an early
time in the campaign.
Limitations
 Had to design and begin pre-test before ads created
 Validity concerns with self-report data
 Higher-level analyses still in progress
I. Conclusions: Campaign Design
A. Continuation of an evidence-based mass media
campaign to reduce youth consumption of tobacco
products must be a high priority in NC.
B. The media campaign should maintain a continuous
(year-round) presence on television and in community
activities with NC youth.
II. Conclusions: Ad Design
A. Ads should include the following features where best
practices overlap with focus group suggestions:
 Serious health consequences with graphic images,
personal story
 Negative emotional tone – strong response
 Smokers/former smokers telling own stories
 Diversity of people
 Content of ad must appear real, genuine
III. Conclusions: Campaign Awareness
A. The 2004 TRU/Tobacco.Reality.Unfiltered. campaign
reached approximately 4.5 out of every 10 youth.
B. The campaign reached diverse youth (males and females,
whites and non-whites, younger and older youth).
C. Awareness of campaign slogan and brand increased
substantially over the last two years. The brand Tobacco.
Reality. Unfiltered. is now recognized by an estimated 55% of
NC youth.
UNC Tobacco Prevention & Evaluation Program
CB 7595, Manning Drive, UNC School of Medicine
Department of Family Medicine
Chapel Hill, NC 27599
T: 919-843-9751; F: 919-966-9435
Web: www.tpep.org
Email: [email protected]