Transcript Analytics_Framework_&_CERN_Use_Cases_-_v2

1
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Insert Information Protection Policy Classification from Slide 13
Analytics Framework & CERN Use Cases
Giuseppe Calabrese
Business Development Manager - Business Intelligence
2
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Insert Information Protection Policy Classification from Slide 13
Unstructured
Semistructured
Structured
Analytics Framework
Master &
Reference
Machine
Generated
Text, Image,
Video, Audio
ETL/ELT
Files
ChangeDC
NoSQL
Real-Time
Replication
HDFS
Hadoop
Transactions
Data
3
RDBMS
(OLTP)
Acquire
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
(MapReduce)
Organize
Insert Information Protection Policy Classification from Slide 13
ODS
DWH
Data Mart
OLAP
Cube
Analyze
In-database
Advanced
Analytics
Business
Intelligence
Alerting &
Recommendations
Data
Exploration
Decide
Unstructured
Semistructured
Structured
Oracle Technology mapped to Analytics Framework
Master &
Reference
Transactions
Oracle Data
Integrator
Machine
Generated
Files
Oracle
NoSQL
Text, Image,
Video, Audio
Oracle
Hadoop
Oracle
HDFS
MapReduce
Acquire
Organize
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Oracle 12g
Oracle
Times Ten
Oracle
Golden Gate
Data
4
Oracle 12g
Insert Information Protection Policy Classification from Slide 13
Hybrid
search /
analytical DB
Oracle
Essbase
Analyze
Oracle R
Enterprise &
Oracle Data
Mining
Oracle BI
Foundation
Oracle Real Time
Decisions
Oracle Endeca
Information
Discovery
Decide
Oracle Platform for Information Architecture
Oracle Big Data
Appliance
Oracle
Big Data
Connectors
Hadoop
Oracle R
Enterprise
Oracle NoSQL
Database
5
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Oracle Big Data
Connectors
Oracle
Exadata
In-Database
Advanced
Analytics
Oracle
Exalytics
Oracle Endeca
Information Discovery
Data
Warehouse
Oracle Data
Integrator
Insert Information Protection Policy Classification from Slide 13
Oracle
Database
Oracle Business Intelligence
Foundation
USE CASE 1 : SUSPICIOUS EVENT DETECTION
 Short Description :
– 100 GB logs per day
– Each system generates its own flat file
– Logs are scanned with algorithms implemented by CERN
– Suspect activities are stored in an oracle database
– Alerts are forwarded to the employees who have done the suspect activity
 Issues:
– Poor performance
– Flat files easy to write but hard to analyze
– Not a real-time analysis
6
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Insert Information Protection Policy Classification from Slide 13
Unstructured
Semistructured
Structured
Oracle Technology mapped to Analytics Landscape
Master &
Reference
Transactions
Oracle Data
Integrator
Machine
Generated
Files
Oracle
NoSQL
Text, Image,
Video, Audio
Oracle
Hadoop
Oracle
HDFS
MapReduce
Acquire
Organize
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Oracle 12g
Oracle
Times Ten
Oracle
Golden Gate
Data
7
Oracle 12g
Insert Information Protection Policy Classification from Slide 13
Hybrid
search /
analytical DB
Oracle
Essbase
Analyze
Oracle R
Enterprise &
Oracle Data
Mining
Oracle BI
Foundation
Oracle Real Time
Decisions
Oracle Endeca
Information
Discovery
Decide
USE CASE 2 : LHC – POSTMORTEM SYSTEM
 Short Description :
– Data is produced every second by sensors
– 50 TB / year; 15 GB / day. Oracle DB storage
– Analysis done with Labview with 10’ delta
– Analysis algorithms developed in Matlab and called from LabVIEW
– Some signals are analyzed over many years in order to compile statistics
 Issues:
– No real-time alerts on incidents
– No preventive alerts based on trend analysis
– Due to performance issues, two separate storage systems are used. One of them is
dedicated to perform the analysis (optimize for query).
8
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Insert Information Protection Policy Classification from Slide 13
Unstructured
Semistructured
Structured
Oracle Technology mapped to Analytics Landscape
Master &
Reference
Transactions
Oracle Data
Integrator
Machine
Generated
Files
Oracle
NoSQL
Oracle
HDFS
Oracle
Hadoop
MapReduce
Acquire
Organize
Text, Image,
Video, Audio
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Oracle 12g
Oracle
Times Ten
Oracle
Golden Gate
Data
9
Oracle 12g
Insert Information Protection Policy Classification from Slide 13
Hybrid
search /
analytical DB
Oracle
Essbase
Analyze
Oracle R
Enterprise &
Oracle Data
Mining
Oracle BI
Foundation
Oracle Real Time
Decisions
Oracle Endeca
Information
Discovery
Decide
USE CASE 3: LOGS ANALYSIS OF SERVERS
 Short Description :
– Daily logs analysis
 Issues:
– Find correlations on what drives to failures
– Log files stored as flat files
10
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Insert Information Protection Policy Classification from Slide 13
Unstructured
Semistructured
Structured
Oracle Technology mapped to Analytics Landscape
Master &
Reference
Transactions
Oracle Data
Integrator
Machine
Generated
Files
Oracle
NoSQL
Oracle
HDFS
Oracle
Hadoop
MapReduce
Acquire
Organize
Text, Image,
Video, Audio
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Oracle 12g
Oracle
Times Ten
Oracle
Golden Gate
Data
11
Oracle 12g
Insert Information Protection Policy Classification from Slide 13
Hybrid
search /
analytical DB
Oracle
Essbase
Analyze
Oracle R
Enterprise &
Oracle Data
Mining
Oracle BI
Foundation
Oracle Real Time
Decisions
Oracle Endeca
Information
Discovery
Decide
USE CASE 4: SYSTEMS HEALTH CHECK
 Short Description :
– Daily logs analysis
– Information stored in a relational model (Oracle DB)
 Issues:
– Find correlations on what drives to failures (Memory leaks, failures on services, etc.)
12
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Insert Information Protection Policy Classification from Slide 13
USE CASE 5: PREDICTIVE MAINTENANCE ON
CONTROL DEVICES
 Short Description :
– Signals coming from various sources (switch, electric flow, etc)
– Trigger alarms about potential problems determined by different variables depending on the
device itself: counters, number of time a valve was opened, error rates, etc.
 Issues:
– Need for preventive maintenance
– Mining rules to forecast failures
– Define maintenance and replacement policies
13
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Insert Information Protection Policy Classification from Slide 13
USE CASE 6: VIBRATION ANALYSIS
 Short Description :
– Compressors in Cryogenics / Cooling and Ventilation systems
 Issues:
– Find the cause of the vibrations
– Remove devices generating interferences
– Need for preventive maintenance
14
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Insert Information Protection Policy Classification from Slide 13
Unstructured
Semistructured
Structured
Oracle Technology mapped to Analytics Landscape
Master &
Reference
Transactions
Oracle Data
Integrator
Machine
Generated
Files
Oracle
NoSQL
Oracle
HDFS
Oracle
Hadoop
MapReduce
Acquire
Organize
Text, Image,
Video, Audio
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Oracle 12g
Oracle
Times Ten
Oracle
Golden Gate
Data
15
Oracle 12g
Insert Information Protection Policy Classification from Slide 13
Hybrid
search /
analytical DB
Oracle
Essbase
Analyze
Oracle R
Enterprise &
Oracle Data
Mining
Oracle BI
Foundation
Oracle Real Time
Decisions
Oracle Endeca
Information
Discovery
Decide
USE CASE 7: POWER CUTS ANALYSIS
 Short Description :
– Power cuts usually trigger an important number of alarms.
– In the simplest cases just looking back in time is enough to detect the source.
– But in most of the cases more detailed analysis of the data is needed
– Information provided by telemetry is stored in an Oracle DB
 Issues:
– Determine the type of analysis that will fit better with the issue
– Need to predict power cuts (preventive maintenance)
16
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Insert Information Protection Policy Classification from Slide 13
Unstructured
Semistructured
Structured
Oracle Technology mapped to Analytics Landscape
Master &
Reference
Transactions
Oracle Data
Integrator
Machine
Generated
Files
Oracle
NoSQL
Oracle
HDFS
Oracle
Hadoop
MapReduce
Acquire
Organize
Text, Image,
Video, Audio
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Oracle 12g
Oracle
Times Ten
Oracle
Golden Gate
Data
17
Oracle 12g
Insert Information Protection Policy Classification from Slide 13
Hybrid
search /
analytical DB
Oracle
Essbase
Analyze
Oracle R
Enterprise &
Oracle Data
Mining
Oracle BI
Foundation
Oracle Real Time
Decisions
Oracle Endeca
Information
Discovery
Decide
USE CASE 8: ELECTRICAL CONSUMPTION
 Short Description :
– CERN ENS (Electrical Network Supervisor) system archives values from
measurements
– Around 12’000 every 10 minutes
– The storage is managed by an external company on an instance of Oracle relational
database.
– The DB instance is installed on one dedicated server connected to CERN Technical
Network
 Issues:
– Make data aggregation (by time and by families of measurements)
– Electricity load forecast by system (families of measurements) using the historical
data
– Make the result obtained accessible to CERN users from General Purpose Network
18
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Insert Information Protection Policy Classification from Slide 13
Unstructured
Semistructured
Structured
Oracle Technology mapped to Analytics Landscape
Master &
Reference
Transactions
Oracle Data
Integrator
Machine
Generated
Files
Oracle
NoSQL
Oracle
HDFS
Oracle
Hadoop
MapReduce
Acquire
Organize
Text, Image,
Video, Audio
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Oracle 12g
Oracle
Times Ten
Oracle
Golden Gate
Data
19
Oracle 12g
Insert Information Protection Policy Classification from Slide 13
Hybrid
search /
analytical DB
Oracle
Essbase
Analyze
Oracle R
Enterprise &
Oracle Data
Mining
Oracle BI
Foundation
Oracle Real Time
Decisions
Oracle Endeca
Information
Discovery
Decide
USE CASE 9: BUDGET FOLLOW-UP
 Short Description :
– Financial forecasts and cost control
20
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Insert Information Protection Policy Classification from Slide 13
Unstructured
Semistructured
Structured
Oracle Technology mapped to Analytics Landscape
Master &
Reference
Transactions
Oracle Data
Integrator
Machine
Generated
Files
Oracle
NoSQL
Oracle
HDFS
Oracle
Hadoop
MapReduce
Acquire
Organize
Text, Image,
Video, Audio
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Oracle 12g
Oracle
Times Ten
Oracle
Golden Gate
Data
21
Oracle 12g
Insert Information Protection Policy Classification from Slide 13
Hybrid
search /
analytical DB
Oracle
Essbase
Analyze
Oracle R
Enterprise &
Oracle Data
Mining
Oracle BI
Foundation
Oracle Real Time
Decisions
Oracle Endeca
Information
Discovery
Decide
DW Global Leaders Program
22
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Insert Information Protection Policy Classification from Slide 13
DW Global Leaders Program
23
1.
a worldwide program, driven out of DW Development,
supported by our executive management
2.
for selected Oracle DW customers, by invitation only
3.
perpetual
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Insert Information Protection Policy Classification from Slide 13
DW Global Leaders Program – WHY?
1) offer a platform for our members to share experience
and best practice in Data Warehousing and interact
with us from Oracle DW Product Development
2) Support our future DW product directions through
ongoing, close and proactive relationship.
24
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Insert Information Protection Policy Classification from Slide 13
DW Global Leaders in EMEA
25
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Insert Information Protection Policy Classification from Slide 13
What are we doing?
DW Global Leaders Meetings – twice a year (Jun / Dec)
26
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Insert Information Protection Policy Classification from Slide 13
27
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Insert Information Protection Policy Classification from Slide 13