Transcript Customer Analytics

Why, and How, your
Analytics Project will Fail
Peter McCallum
Director, CBI
Agenda





Introduction
Pyle’s 9 Rules for Analytics Project
Failure
Why navigating Pyle’s 9 Rules still
doesn’t guarantee success
Incorporating the analytical model into
the business process
Summary
Introduction

Who am I?



20 years experience in the IT industry
The last 12 years working exclusively
delivering Business Intelligence &
Analytical solutions
Have experienced the frustration of seeing
a data mining project fail to deliver the
quick wins promised
Agenda





Introduction
Pyle’s 9 Rules for Analytics Project
Failure
Why navigating Pyle’s 9 Rules still
doesn’t guarantee success
Incorporating the analytical model into
the business process
Summary
Pyle’s 9 Rules



Who is Dorian Pyle?
What are his rules?
Why are they still relevant?
Pyle’s Rule #1

# 1. Jump Right In




Ignore the business
Use whatever data is on hand
Use whatever tools you’re most
comfortable with
And don’t worry about how (or whether)
your results can actually be applied
Pyle’s Rule #2

# 2. Frame the problem in terms of the
data




You’ve been given data – mine it!
Don’t stop to ask whether there might be
other methods of solving the problem
Don’t think outside of the current data set –
simply ignore any environmental or
organisational factors
Restate the objective based on “whatever
the data can be persuaded to reveal”
Pyle’s Rule #3

# 3. Focus only on the most obvious
way to frame the problem



Don’t waste your time exploring the data
Concentrate on the technical merits of the
model to the exclusion of all else
Aim for the highest degree of technical
perfection
Pyle’s Rule #4

# 4. Rely on your own judgment




The data miner knows best
The data contains all the required
information – focus on revealing the
nuggets within
Input from others, especially the business,
is unnecessary & should be ignored
Remember – the miner knows best
Pyle’s Rule #5

# 5. Find the best algorithms



For any set of data one particular algorithm
will produce the best model
So focus on finding the best algorithm
It’s what data mining is all about
Pyle’s Rule #6

# 6. Rely on memory




Don’t waste your time documenting
Press on with the data investigation…. As
fast as possible
Should you ever need to duplicate the
investigation you’ll remember exactly what
you did and why
Should anyone ever dare ask you to justify
or explain your results, you will remember
Pyle’s Rule #7

# 7. Intuition is more important than
standard practice



Data mining is an art, not a science
Standards are really only intended for
“newbies”
All data sets are different, so simply rely on
your instincts
Pyle’s Rule #8

# 8. Minimize interaction between
miners and business managers



Stay away from the business
Rely exclusively on what the data tells you,
irrespective of what the business might try
to tell you
After all, mining is primarily about letting
the tools do the talking
Pyle’s Rule #9

# 9. Minimize data preparation




Creating the models themselves is the
most interesting part of data mining
Data preparation is dull, tedious & time
consuming
Let the tools look after the data preparation
for you
Do as little preparation as possible and cut
straight to the modeling
Agenda





Introduction
Pyle’s 9 Rules for Analytics Project
Failure
Why navigating Pyle’s 9 Rules still
doesn’t guarantee success
Incorporating the analytical model into
the business process
Summary
The Bigger Picture
“Data mining is part, and a very small
part, of a much larger business process.
It may be an essential part of a data
mining project, but incorporating the
results of mining with all the related
parts of the corporate project is equally,
if not more, important for ultimate
success”
Dorian Pyle
Virtuous Cycle of Data
Mining
Transform Data
Act on the
Information
Identify business
problem
Measure the results
Berry & Linoff
Realising Business Value
“The heart of data mining is
transforming data into actionable
results”
Berry & Linoff
Where’s the payback?





Large multi-national
Undertook a review of their churn
management process
Led by an international consulting firm
Executive management sponsorship
Chasing millions in potential benefits
What went right

Everything!







Fully engaged with the business
Invested time in data exploration &
preparation
Focused on the business issue rather than
the technicalities
Every step documented
Project uncovered some excellent insights
Models developed showed lift of 3X or
more
All we had to do was deploy the models
What went wrong

Deploying the models
Agenda





Introduction
Pyle’s 9 Rules for Analytics Project
Failure
Why navigating Pyle’s 9 Rules still
doesn’t guarantee success
Incorporating the analytical model into
the business process
Summary
The Starting Point
Data
Warehouse
Manual Data
Extracts
Campaign
Management
System
Mining Tool
Churn Lists
Outbound Call
Lists
Customer
Management
System
The Issues




Poor Integration
Huge degree of manual effort
Large amount of latency
Non existent feedback loop
The Impacts




Introduced a high degree of risk every
time the model was refreshed
Restricted how often the churn
propensity models could be run
Drastically reduced the value in running
the models
Made it extremely difficult to measure
the performance of retention efforts
The Goal


To overcome the issues with the
existing process
To make the churn propensity scores
more widely available
The Goal (cont’d)
Contact List
Direct Connect
Campaign
Management
System
Data
Warehouse
Mining Tool
Churn Scores
Direct Connect
Automated Update
Customer
Management
System
Outbound Call
Lists
Challenge #1
Contact List
Direct Connect
Campaign
Management
System
Data
Warehouse
Mining Tool
Churn Scores
Direct Connect
Automated Update
Customer
Management
System

Outbound Call
Lists
The Data Mining platform & licenses had
to be completely upgraded
Challenge #2
Contact List
Direct Connect
Campaign
Management
System
Data
Warehouse
Mining Tool
Churn Scores
Direct Connect
Automated Update
Customer
Management
System

Outbound Call
Lists
The Data Warehouse was re-platformed
mid project
Challenge #3
Contact List
Direct Connect
Campaign
Management
System
Data
Warehouse
Mining Tool
Churn Scores
Direct Connect
Automated Update
Customer
Management
System

Outbound Call
Lists
The Campaign Management System was
replaced mid project
Challenge #4
Contact List
Direct Connect
Campaign
Management
System
Data
Warehouse
Mining Tool
Churn Scores
Direct Connect
Automated Update
Customer
Management
System

Outbound Call
Lists
The automated process to update the
churn scores in the CRM just did not
work
Finally
Contact List
Direct Connect
Campaign
Management
System
Data
Warehouse
Mining Tool
Churn Scores
Direct Connect
Automated Update
Customer
Management
System
Outbound Call
Lists
The Long Awaited Benefits




The time required to refresh the model
was slashed by a factor of 10
Churn propensity scores could be
refreshed across the entire customer
base on a monthly basis
It became possible to accurately
measure the success of the retention
efforts
The Customer Services Representatives
could finally recognize at risk
customers during inbound calls.
Incorporating the model into
the business
“The more that the use of the analytical
solution can be embedded into the
business process being supported, the
more likely it is that benefits will be
realised”
Incorporating the model into
the business (cont’d)
“The key to successful data mining is to
incorporate the models into the
business”
Berry & Linoff
Agenda





Introduction
Pyle’s 9 Rules for Analytics Project
Failure
Why navigating Pyle’s 9 Rules still
doesn’t guarantee success
Incorporating the analytical model into
the business process
Summary
Summary


Remember Pyle’s 9 Rules
BUT more importantly…
Remember The Bigger Picture
The Bigger Picture
“Data mining is part, and a very small
part, of a much larger business process.
It may be an essential part of a data
mining project, but incorporating the
results of mining with all the related
parts of the corporate project is equally,
if not more, important for ultimate
success”
Dorian Pyle