Transcript Presentation for technology demonstration using Customs Data

TECHNOLOGY DEMONSTRATION
BUSINESS INTELLIGENCE
-DATA WAREHOUSE
-OLAP
-DATA MINING /KNOWLEDGE MANAGEMENT
ANALYTICS & MODELLING DIVISION
NATIONAL INFORMATICS CENTRE
Ph: 24362530 email: [email protected]
PROTOTYPE DEVELOPMENT
INTERACTION WITH NIC-CUSTOMS DIVISION
LIMITED INPUT
CUSTOMS IMPORT DATA
ITEM LEVEL DATA
[BE, ASS VAL, DUTY, DFG, CTH, CETH, BCD, NOTFN, ….]
17 SITES ONLY
TIME RANGE: APRIL 2002 TO SEP 2004
DATA SIZE = 1.2 Cr RECORDS = 10GB
QUERIES ARE BASED ON WHAT WE COULD PERCEIVE
[NIC-CUSTOMS AND A&M DIV]
PROTOTYPE DEVELOPMENT
TO DEMONSTRATE THE POWER OF THE TECHNOLOGY
/TOOL
RDBMS TOOLS vs BI TOOLS
- TO HANDLE LARGE VOLUME OF DATA
- IN CONTRAST TO STORAGE AND RETREIVAL
WHAT HAPPENED vs FUTURE BETTER PLAN
STAGES INVOLVED
BUILD THE RDBMS TABLES [OLTP APPLICATIONS]
BUILD THE WAREHOUSE
- WHAT DATA TO BE POPULATED
- WHERE ARE THE DATA
- HOW TO BRING THE DATA
- ON WHAT PLATFORM TO BUILD THE DW
- BUILD THE DW
- HOW TO USE THE DW
TOOL USED FOR THIS EXERCISE
TO GET THE DATA AND TO BUILD DW
- COGNOS DECISION STREAM
TO BUILD AND ANALYZE CUBE “OLAP”
- COGNOS POWERPLAY
FOR GENERATING STATIC REPORTS
- COGNOS REPORTNET & VISUALIZER
FOR KNOWLEDGE EXTRACTION “DM”
- SPSS CLEMENTINE
DW PHASES
- DATA FLOW
- RDBMS
- DATA STAGING
- BUILD
- ANALYSIS BUILD
- STATIC REPORT &
MINING
=> TO SAN DATA SERVER
=> ORACLE
=> ETL
=> DW SERVER
=> OLAP SERVER
=> REPORTNET
& MINING SERVER
CUSTOMS DW ARCHITECTURE
RETRIEV
STORAGE
AREA
NETWORK
(SAN)
Extract
DATA SERVER
(ORACLE 9i/10G)
Store
ETL
(DECISION STREAM)
Load
CLIENT
To access & change the
layout of Powerplay
and
ReportNet
reports…
Request
COGNOS SERVER
 POWERPLAY
 REPORTNET
 CLEMENTINE
Respons
e
WINDOWS 2000 SERVER
(OS)
WHO ARE ALL THE USERS
DIRECTORATE OF SYSTEMS & DATA MANAGEMENT
DIRECTORATE OF REVENUE INTELLIGENCE
TAX RESEARCH UNIT
DGFT
DGCIS
DOV
DIRECTORATE OF SERVICE TAX
NCB
CENTRAL EXCISE INTELLIGENCE
DIRECTORATE OF STAT. AND INTELLIGENCE
MINISTRY OF FINANCE AND ASSOCIATED MIN.S
MIN. OF STEEL
WHY THE WAREHOUSE
-NEED OF INFORMATICS REQUIREMENT IS DYNAMIC
-ALSO NEED OF EVERY DEPTT VARIES
[SECRETARY/COMMISSIONER/JT.COMM/DY.COMM.]
-IDENTIFY THE INDIVIDUAL DEPTT/OFFICER NEEDS
-BUILD A CUBE AROUND ABOVE
-PUBLISH IT ON THE WEB
-ONLY THE AUTHORISED ONE CAN ACCESS THE CUBE
AD HOC QUERY AND REPORTS
-CAN USE THE CUBE FOR AD HOC QUERYING
-REPORTNET CAN BE USED TO GENERATE REGULAR
REPORTS
-ALL TYPE OF GRAPHICAL REPORTS POSSIBLE
[LINE, BAR, PIE, STACK…]
-ANY NO OF DIMENSIONS ONE CAN VIEW USING A CUBE
DATA MINING
DATA MINING vs DATA EXPLORATION
WHAT ARE THE BROAD MINING AREAS
- CLUSTERING
- ASSOCIATION
- PREDICTION
TECHNIQUES
- STATISTICAL METHODS
- NEURAL METHODS
- RULE INDUCTION
- WHAT IF?
REGULAR UPDATE
- NETWORKING OF NEW SITES
- DATA FLOW MECHANISM [DAILY/WEEKLY/MONTHLY]
- AVOIDING WRONG ENTRY
- GENERATION OF NEW CUBES
- INFORM THE CONCERNED OFFICIAL ABOUT THAT
- PUBLISH THE CUBES/REPORTS ON THE WEB
-
WHO ARE ALL THE USERS
CUSTOMS DEPARTMENT
DEPARTMENT OF REVENUE INTELLIGENCE
TAX RESEARCH UNIT
DGFT
DGCSI
MINISTRY OF FINANCE + ….
DATA MART TO DW OR DW TO DM
I --DM FOR INDIVIDUAL USER DEPTT.S
II --CONNECT ALL DM’S TO FORM A DW
OR
--CREATE A DW FIRST
--GET DATA FROM THE DW TO FORM DM’S AT
EACH DEPARTMENT LEVEL.
WHICH ONE TO FOLLOW?
DEPENDS UP ON THE USER NEEDS
POSSIBLE COURSE OF ACTION
FOR TOTAL PROJECT
-FORM A WORKING GROUP
-UNDERSTAND THE DATA AND ANALYTICAL NEEDS
-IDENTIFY THE OBJECTIVES
-SRS PREPARATION
-IDENTIFICATION OF HW AND SW TOOLS
-NO OF USERS [FOR DEVELOPMENT]
-WEB DEPLOYMENT
-CREATION OF CUBES FOR DIFFERENT LEVELS
OF USERS
A SINGLE WAREHOUSE FOR ALL DEPTTS/AGENCIES OF
CUSTOMS/MIN.OF FINANCE AND ASSOCIATED AGENCIES