Transcript Exploratory Data Mart

Managing Knowledge
in
Business Intelligence Systems
Dr. Jan Mrazek
Our mission is to optimize the business process
(CVM, BPM)
Relationship Mapping
Channels and Organization
Market Conditions
Customer Transacts
Modeling Behavior
Profitability Calculation Customer Segmentation Business Performance Analysis
Customer Relationship Analysis
Customer Opens an Account
Model Scoring
Cross/UP Sell
Prospects
Query Server
Retail & Commercial
Divisional Leaders
POS
Mortgages
CKDB
MIND
HR
Investment Products
MBANX Direct
Exploratory Data Mart
Customer based flat file with more than 1,000 variables
Sample of 1.5 mil. customers
Information Warehouse
Mech
IPS
Uniform BI Data Architecture
NCCS
BI Metadata Repository
Uniform BI Technical Architecture
Web Server
CVM Architecture
CVM Exploratory System (Advanced Analytics)
Model Development
Exploratory Data Mart
Sample
set of
variables
Models
(PMML)
CVM Core Analytical System
Scoring
Customer
Segmentation
Decision
about offers
Campaign Management
Treatment
Selection
Feedback CRM Front End System
data
Cust. Serv.
Profile
CRM Database
Contact Management
ODS System (“Special” transactions)
CVM
Analytical Database
(taking the role of
a customer centric
marketing database)
Transactional
ODS
(Holds only “special” transactions)
Event driven filter
of transactions right
during the load
OCIF & Householding System
Customering
OCIF
Feedback
Assessment
(Analysis)
Householding
Raw account level
data in monthly
aggregates
Primary sources (operational systems)
Variables
Value Creation
DW + Profitability System = CVM Base
Account Profitability
Customer Aggregations
Household Aggregations
DW + DMs
Raw Data
CCAPS
Detailed transactions
in a daily batch load
Treatment
Authoring
Monthly run on all customers
Daily re-run for customers with “special” transactions
Offer Selection
Legend:
Data Warehousing/Business Intelligence Environment
OCIF System
Operational Systems
Key objective
At the Bank of Montreal one of our key objectives is to excel in
our service to our customers.
To be able to achieve this key objective, we have to learn how to
anticipate our customers’ preferences in a timely manner.
Since only a timely understanding can deliver true service
excellence, we are focussed on streamlining knowledge discovery
processes along an integrated system architecture so, that the
time needed from knowledge discovery to knowledge application is
minimized.
Overview of the Knowledge Discovery Process
Identification
of
Objectives
Data
Acquisition
Data
Preparation
Model
Development
Model
Execution
(Scoring)
Scores
Deployment
Results
Analysis
Knowledge Discovery Executed in a Non-integrated Environment
DM technology A
DM technology B
data
data
DM technology C
data
DB2 UDB EEE
Data Warehouse
data
DM technology D
•Data preparation
•Model development
•?Model execution
(Scoring)
•? Scores
deployment
•? Results analysis
Disadvantages of the Non-integrated Knowledge Discovery Environment
•Data preparation responsibility of analysts/modelers
•Not optimal HW/SW for data preparation
•Data about all customers need to be moved to place of model execution
•Limited capabilities for model execution in the DW environment
•Scores not automatically stored in systems with general availability and access
•Limited ability to analyze results, quality of models
•That all results in lost of precious time to apply the discovered knowledge
Knowledge Discovery Executed in a Highly Integrated Environment
DM technology A
DM technology B
DM technology C
DM technology D
•Model development
model
(PMML)
model
(PMML)
model
(PMML)
data
data
data
•Data preparation
model
(PMML)
•Model execution
(Scoring)
IM Scoring
Exploratory
Data Mart
(Large sample
of data)
data
scores
data
•Model validation and
results analysis
•Mass scores deployment
DB2 UDB EEE
Data Warehouse
Advantages of the Integrated Knowledge Discovery Environment
•Data preparation executed by DW transformation professionals
•Robust DW HW/SW utilized for data preparation
•Modelers concentrate on actual model development
•Only samples of data moved to modelers’ environments
•Models delivered to IM Scoring in PMML format from different data mining
technologies
•IM Scoring executes models utilizing all robust DW HW/SW processing
power
•Scores immediately stored in the DW environment where they can be
accessed and used by many applications and users
•Full ability to analyze results, quality of models
•That all results in:
•Reduction of time needed for knowledge discovery and knowledge
deployment
•Optimal use of HW/SW and professional resources
•Improved process quality
Maintaining Model Version Control - DM Metadata
> Model built when, by whom
> What tool, algorithm
> Variables (links to Metadata repository)
> Variables’ transformation rule - link to ETL Metadata
> When last time re-balanced, by whom
> Since when in production
> Who is the owner, contact
> QA of PMML translation, who
> Treat as slow moving dimension
Where you can meet me
•August 15 in Anaheim, California on TDWI World
Conference Summer 2001 and Best Practices Summit
•IBM Webcast on Enhancing CRM with IBM's DB2 Intelligent Miner
Scoring http://webevents.broadcast.com/ibm/datamining/home.asp
•Adastra Prague: call +420-2-7173 3303 to arrange for a meeting
2001 Best Practices In Data Warehousing
Award (TDWI)
2000 Best Data Warehouse Award
(RealWare Awards)
2000 ADT 2000 Software Innovator
Award for Data Warehousing
(Application Development Trends)
1999 DCI Excellence in Business
Information Award