Transcript CERN Expenditure Tracking

CERN – European Organization for Nuclear Research
Administrative Support - Internet Development Services
CET and the quest for optimal
implementation and maintenance
efficiency
with Data warehousing and J2EE Components
Mikael Angberg, AS-IDS
Outline
 Introduction
to CET
 Maintenance and Implementation issues
 Quality Assurance
 Maintainability and Versatility
 Conclusions
 Questions
CERN
IDS
Challenges facing CERN Today
Build the worlds largest scientific
instrument…
Budget
Staff
With less budget..
8.6 km
fewer staff…
CERN
IDS
for more scientists…
with higher
expectations...
Challenges facing the CET Team Today
Build the worlds coolest Financial
Decision Support application…
Budget
Staff
With a small budget..
CERN
IDS
few staff…
for more financial
managers…
with higher
expectations...
CERN Expenditure Tracking
CERN
IDS
CERN Expenditure Tracking
 Critical for CERN
 1000+ “Financial” Managers
 Globally Distributed
 Wide range of user needs
CERN
IDS
High Level System Requirements
 Quality
 Maintainability
 Versatility
CERN Expenditure Tracking
CERN
IDS
The “System Idea”
User Interface
Application
(Java)
Data
Warehouse
CERN
IDS
Business Logic
Financial Data
Data Warehouse Quality
 Data and Data Warehouse Quality
o Capturing the “right” data
o Validating the data
o Error handling
o Centralised Data
o Managed complexity
o Coding Standards
o Code Inspections
CERN
IDS
Data Warehouse Quality
 Centralize data
One Place, One Format, No Redundancy
Financial
Data Sources
Stores
Purchase
Finance
CERN
IDS
Error Handling
Process
and
Validation
Extraction
Loading
Transform
Data
Warehouse
Data Warehouse Quality
Managed Complexity
SUM..GROUP BY
1999
SUM..GROUP BY
SUM..GROUP BY
2000
SUM..GROUP BY
2001
SUM..GROUP BY
Partitioned
Table
SUM..GROUP BY
 Reduced number of DB Objects
 Parameterized Extraction
CERN
IDS
 Single Entry Point
Application Quality
 Structured Development Process
o Iterative…
o Development
o Testing
o Deploy
 Coding Standards
o Based on Sun’s Java Specification
 Code Inspections
CERN
IDS
o Inspired by Fagan and NASA
Quality Framework Benefits
 Ensured Data Consistency
 Extended data quality assurance through
automated validation processes
 Greatly Reduced Database Maintenance
 Increased production code quality (less bugs)
 Higher developer productivity
 Increased transparency and knowledge transfer
within and between development teams
CERN
IDS
Maintainability and Versatility
Objectives
Satisfy a Global user community
…consisting of more than 1000 people with
different needs (and many requests)
…by providing powerful analysis
and reporting capabilities
With a development team of
maximum 5 people, minimum 1
CERN
IDS
- Maintainability
- Versatility
Facts and Dimensions
The Dimensional Model
Dimension
Data Warehouse
Time
Dimension
Location
Measures / Facts
Dimension
Payments
Commitments
.
Keys
Order
Partitioned
Fact Table
CERN
IDS
Dimension
Supplier
Maintainability and Versatility
Managed Complexity
SUM..GROUP BY
1999
SUM..GROUP BY
SUM..GROUP BY
2000
SUM..GROUP BY
2001
SUM..GROUP BY
CERN
IDS
Partitioned
Table
SUM..GROUP BY
Aggregates and Query Rewrite
Query Re-write
SELECT SUM(sales) FROM raw_data;
Dimension Tables
Aggregate Tables /
Materialized Views
1999
1999
2000
2000
2001
2001
Raw Data Table
CERN
IDS
SUM..GROUP BY
SUM..GROUP BY
SUM..GROUP BY
Scalability and maintained simplicity
Partitioning
<2000
2000
2001
2002
Partitioned Index
Partitioned Table
<2000
2000
2001
2002
Partitioned Index
Partitioned Table
CERN
IDS
Design Conclusions DWH
The data warehouse structure allows :
 Access to any aggregate level
 Access to any range of data
…transparently to the user
Maintenance and Versatility perspective :
 “Simple” design – Automation of
recurrent maintenance
 Flexible – Easy to tune and extend
 Scalable – Facilitates growing with
preserved simplicity
CERN
IDS
ART – the Java Reporting Framework
“Increase maintainability, by applying existing design/code and documentation
standards and ensure in-house knowledge of the product. “
“Use of the framework should be possible with knowledge of standard languages
only such as Java, SQL, XML.
“
User Interface
Form
Generation &
Validations
Three Main
Components
Report
Generation
HTML, Excel,
PDF…
CERN
IDS
Query Generation
Data Retrieval &
Preprecessing
ART – the Java Reporting Framework
 Web-based
 Java components
 XML Templates
Web report
Input Objects
XML
Templates
Input
Validators
Query Builder
Report Generator
CERN
IDS
Key “Success” Factors
 Development process integrating
quality assurance
 Design assures data quality
 Robust yet Flexible Data Warehouse Design
 Simplicity - Low on maintenance
 Component based (Java) reporting framework
CERN
IDS
Conclusions
Users
World-wide, highly reliable and available
Financial Decision Support
Developers
 Decrease in maintenance efforts
 Faster “time-to-market”
 Increased productivity and quality
 Development focuses on solving
business problems – Stable Infrastructure
CERN
IDS
Thank You
For More Information
Browse to:
http://ais.cern.ch/
or Email:
[email protected]
CERN
IDS
Summary of CET

Web-Based, multi-lingual
 20 GB Data Warehouse - 360 GB Raw Data
 Over 1000 active users
 ~ 1000 Reports per day
 ~ 24 / 7 Availability
 100% Automated Recurrent Maintenance
 No Ad Hoc Querying
CERN
IDS