Transcript E-Mail Attribution Rules

Catalog Matchbacks and
Advertising Attribution
How to Best Invest Advertising Dollars
in a Confusing Environment
Kevin Hillstrom
President, MineThatData
Multichannel Forensics
Most of my projects are called “Multichannel
Forensics” projects. I analyze customer data to
see how customers migrate across channels
and interact across channels … I’ve conducted
about 50 projects for catalog/retail/online
clients in the past 3+ years.
What we’ll learn today is an outcome of 50+
Multichannel Forensics projects!
2
Let's Talk Attribution!!!
Quiz: An existing customer receives a catalog
on September 1, receives e-mail marketing
campaigns on September 7 and September 9,
and purchases on your website on September
10, buying merchandise featured in your
catalog and merchandise available only online.
What % of the order do you allocate to
catalogs, e-mail, and to organic brand loyalty?
3
Let's Talk Attribution!!!
Quiz: An existing customer receives a catalog
on September 1. On September 2, the
customer uses Google to search for
merchandise, visits your website, and
purchases an item.
What % of the order do you allocate to
catalogs, to e-mail, to search, and to organic
brand loyalty?
4
Let's Talk Attribution!!!
Quiz: An existing customer receives a catalog
on August 1, and receives 17 e-mail marketing
messages between 8/1 and 10/1. On 10/4, the
customer uses Google to search for
merchandise, visits, and buys an item.
What % of the order do you allocate to
catalogs, to e-mail, to search, and to organic
brand loyalty?
5
Can Anybody Agree?
If you poll 100 marketers, asking them to
attribute orders to the marketing channel that
caused the order to happen, you’re likely to get
as many as 100 different methodologies!!
And if you ask a computer model to attribute
orders to the marketing channel that drove the
order, you’re likely to obtain an answer you
don’t agree with!
6
The Impact on Catalogs is Significant
A simple 10% change in catalog demand due to
attribution changes causes a significant change
in circulation strategy.
Base Strategy: Circ = 1,500, Demand = $3,237.
New Attribution: Circ = 2,050, Demand =
$4,430 (or demand really is $3,987, you just
over-attributed orders & over-circulated).
7
Why Guess?
The beautiful thing about catalog marketing
and e-mail marketing is that you can test not
sending contacts to customers.
In other words, you can see what happens to
other channels when you do not mail a catalog.
You can see what happens to other channels
when you do not send an e-mail campaign!
8
Test Design!
Sample 80,000 twelve-month buyers with a
valid e-mail address.
Group 1 = 20,000:
Group 2 = 20,000:
Group 3 = 20,000:
Group 4 = 20,000:
Catalogs = Yes, E-Mail = Yes.
Catalogs = Yes, E-Mail = No.
Catalogs = No, E-Mail = Yes.
Catalogs = No, E-Mail = No.
Execute for a month, quarter, season, or year!
9
Test Results = Attribution Rules
In a controlled experiment, the results of your
test tell you what impact catalog marketing and
e-mail marketing have on other channels
(search, mobile, social, display ads, affiliates).
When you know, via a controlled experiment,
what impact catalogs and e-mail truly have,
you can set up reasonable attribution rules!
10
Test: Overall Results, One Month Test
We have four
test panels in
this test. Our
company sent
one catalog and
nine e-mail
campaigns
during this onemonth
timeframe.
Wow!
11
Test: Compare Catalog Mail/No-Mail
Via classic matchback
or attribution rules,
the catalog would be
credited with
generating $10.70 per
customer.
Via mail/no-mail tests,
the catalog is truly
credited with $3.59.
Oh oh.
12
Catalog Attribution Rules
Attribute $1.00 of every $4.75 online to the
catalog (21% of online dollars).
Attribute ($0.50) of every $0.80 e-mail demand
to the catalog (-63% of e-mail dollars).
Attribute $0.78 of every $1.58 in search to the
catalog (48% of search dollars) --- and,
attribute search expense, too!
13
Catalog Attribution Rules
Attribute $0.00 of every $0.28 social media
demand to the catalog (0%).
Attribute $0.00 of every $0.38 mobile demand
to the catalog (0%).
Attribute $0.04 of every $0.11 (36%) affiliate
demand to the catalog, and ($0.07) of every
$0.17 display ad demand to the catalog (-41%).
14
Catalog And Search Secret
So often, I find that catalogs cause search to
happen.
Two problems: First, catalogers tend to match
the search order to a catalog, when in reality,
without search, the catalog order doesn’t
happen! Second, catalogers need to fully
allocate all search expenses (converted & nonconverted clicks) back to catalogs.
15
Catalog Attribution Rules
The results are very different from classic
matchback algorithms, and for good reason!
This is a controlled experiment. In this case,
customers continued to spend money, when
catalogs were not mailed.
This means that we are dramatically overcirculating catalogs to customers. Oh oh.
16
Test: Compare E-Mail Mail/No-Mail
We can run the very
same analysis with email marketing.
Average the results of
the two e-mail ‘mail’
segments, and average
the results of the two
e-mail ‘no-mail’
segments.
What do you observe?
17
E-Mail Attribution Rules
Attribute $0.00 of every $1.48 telephone
demand to e-mail marketing (0%).
Attribute ($0.50) of every $4.00 online demand
to e-mail marketing (-13%).
Attribute $0.28 of every $1.33 in search to the
e-mail marketing (21%). Remember to allocate
search expense to e-mail as well.
18
E-Mail Attribution Rules
Attribute $0.05 of every $0.30 social media
demand to e-mail marketing (17%).
Attribute $0.25 of every $0.50 (50%) mobile
demand to e-mail marketing (e-mail drives
mobile).
Attribute $0.04 of every $0.11 affiliate demand
to e-mail (36%), and ($0.03) of every $0.19
display ad demand e-mail (-16%).
19
E-Mail Attribution Rules
The results are very different from classic
open/click/conversion metrics.
The end result is nearly identical.
The results by channel, however, are very
different. E-Mail caused Search, Mobile, and
Affiliate orders to happen … without e-mail,
those channels wouldn’t perform as well.
20
Profit: Which Strategy Works Best?
Once catalog
expenses, e-mail
expenses, and
search expenses
are allocated
across test
groups, we can
identify the
strategy that
works best.
Take a look!!!!!
21
Which Strategy Works Best?
In this test, the most profitable strategy is
essentially a tie … catalogs + e-mail, or just email.
When a business has a high organic
percentage, matchbacks and attribution
algorithms consistently allocate too many
orders to marketing activities. Mail/Holdout
strategies yield different results.
22
What Is The "Organic Percentage"?
The organic percentage is possibly the most
important metric a direct marketer / catalog
brand can track.
The organic percentage is the percentage of
demand that will be generated if no marketing
exists.
What about our example?
23
The Organic Percentage: 51%????
Take the $5.80
generated in the
no catalogs / no
e-mail test
panel, and
divide it by the
$11.37
generated in the
catalogs + email test panel.
The result is
51%.
24
A 51% Organic Percentage
We must execute catalog and e-mail
mail/holdout test panels, in order to properly
estimate what our organic percentage is.
When the organic percentage is < 20%, your
matchback/allocation process is generally
accurate. When the organic percentage is >
40%, matchbacks and allocation programs
become increasingly inaccurate.
25
Does The Organic Percentage Vary?
If you follow my blog, you know that I am an
advocate of “Digital Profiles”, segments of
customers with unique buying behavior.
BIG SECRET: Some customers are “highly
organic”, while other customers require “large
amounts of marketing”. There are HUGE profit
opportunities in knowing this difference!
26
Digital Profile Secret!
27
Digital Profile Secret!
Customers who mail orders to a company or
use the telephone to order require advertising.
Customers who combine catalogs and online
channels are a “hybrid”, requiring much less
advertising.
Customers who order online or in stores are
highly organic, you can reduce advertising!
28
Optimal # Of Contacts
I work with many catalogers that execute 3month and 12-month holdout tests.
This is where all of the magic happens!
You don’t lose demand … you learn the right
number of annual contacts!!!! You test four
combinations, and you can literally simulate
dozens or hundreds of combinations!!
29
The Optimal Number Of Contacts!
My clients often test four
combinations (just like in our
earlier example).
In this case, eight catalogs a
year, not twelve, yield
optimal profitability. E-mail
contacts appear to be
optimized at around 100 per
year. This strategy is different
than the 12 catalogs / 100 emails currently executed.
30
Key Takeaways
If you have a low organic percentage, you can
probably trust your matchback/attribution
algorithms.
If you have a medium or high organic
percentage, your matchback/attribution
algorithms are allocating too many orders to
your marketing activities, causing you to spend
too much $$$ on marketing.
31
Key Takeaways
Have the courage to execute both catalog
mail/holdout tests and e-mail mail/holdout
tests. Test four panels, test for a quarter or
season or year if you can.
The results are going to be breathtaking! The
results will be different than what your
matchback/attribution algorithms suggest.
32
Key Takeaways: Catalogs
Most matchbacks/allocation/attribution
models over-estimate the ability of catalogs to
drive incremental volume.
In many cases, orders that would have
happened anyway are attributed to catalogs,
causing us to spend way too much money
mailing catalogs.
33
Key Takeaways: E-Mail
Most matchbacks/allocation/attribution
models fail to properly measure what happens
when e-mail campaigns are delivered to a
customer.
E-Mail is frequently cannibalized by catalogs.
E-Mail frequently causes Search/Mobile orders
to happen.
34
Key Takeaways: Search
Search is often the outcome of catalog
marketing or e-mail marketing.
Our job is to allocate search orders driven by
catalog marketing and e-mail marketing (and
search expense, converted and non-converted
clicks) back to the catalog segment that caused
search to happen. This may cause us to
actually not mail certain segments!!!!!
35
Key Takeaways: Search
Search is often the outcome of catalog
marketing or e-mail marketing.
Our job is to allocate search orders driven by
catalog marketing and e-mail marketing (and
search expense, converted and non-converted
clicks) back to the catalog segment that caused
search to happen. This may cause us to
actually not mail certain segments!!!!!
36
Key Takeaways: Social/Mobile
In the early stages of a channel, sales are
frequently cannibalized from existing channels,
or the existing channels cause the sale to
happen in the new channel.
Over time, new channels become “organic”,
and do not require old-school channels in
order to create sales on their own.
Tests/Holdouts can validate this for you!
37
Key Conclusion
It is my opinion that most of our industry is
incorrectly using
matchbacks/allocation/attribution models, and
as a result, we spend way too much money on
marketing, attributing orders that would have
happened anyway to marketing channels.
Run a test, and learn how you, too, can save
yourself money and increase profit!
38
Questions?
Kevin Hillstrom
President, MineThatData
Website: http://minethatdata.com
Blog: http://blog.minethatdata.com
Twitter: http://twitter.com/minethatdata
E-Mail: [email protected]
Phone: 1-206-853-8278
39