Transcript VIEWS_EPA_Data_Summit_200802

What is VIEWS?
The Visibility Information Exchange Web System is a database system
and set of online tools originally designed to support the Regional Haze
Rule enacted by the EPA to reduce regional haze in national parks and
wilderness areas.
Some facts:
 Over 600 registered users
 Over 200 organizations represented
 Almost 100 countries represented
 300+ unique hits a day
What are some of its other goals?
 Linked to by over four dozen sites




 Over 40 million records of air data
Provide easy online access to a wide variety of air quality data.
Provide online tools for exploring and analyzing this data.
Maintain a catalog of relevant air quality-related resources.
Facilitate the research and understanding of global air quality issues.
 Dozens of monitoring networks
 CSU Research Initiative Award
 Uses the new Manifold GIS
 Monitoring site photographs
Web Address: http://vista.cira.colostate.edu/views
Sponsor: Five EPA Regional Planning Organizations (RPOs)
Guiding Body: VIEWS Steering Committee
Location: Cooperative Institute for Research in the Atmosphere (CIRA),
Colorado State University, Fort Collins, CO
Staff: Scientists, researchers, and IT professionals
Affiliations: Interagency Monitoring of Protected Visual Environments
(IMPROVE)
 Class I Area webcams
 Visibility photographs
 Visibility Grey Literature
 Periodic Newsletter
 Contour Maps
 Trends Analysis
 Air Mass Composition Analysis
Import
Validation
Source Data
Database
Rules
Storage
Transformation
Source Data
Source Data
Retrieval
Analysis
Presentation
Interpretation
Program
Logic
Front End
Back End
Import: Getting data into the system
Validation: Ensuring data accuracy
Storage: Managing data, backup, and archival
Transformation: Sorting, joining, aggregating
Retrieval: Getting the data out
Presentation: Displaying the data
Analysis: Making the data understandable
Interpretation: Making the data usable
VIEWS Architecture Overview
2
VIEWS Architecture Detail: Data Acquisition & Import
Data Acquisition System:
Metadata Import System:
• Accepts submission of data in a variety of • Facilitates the entry of new
schemas and formats
metadata
• Can automatically extract data from known • Validates new metadata
online sources
entries
• Uses database replication where possible • Detects overlap with existing
• Initially imports data and metadata “as-is”
metadata
into the source database
Data Import System:
•
•
•
•
•
•
Extracts data from the source database
Scrubs data and performs conversions
Maps source metadata to integrated metadata
Transforms the data into an integrated schema
Verifies and validates imported data
Loads data into the back-end OLTP system
VIEWS Architecture Detail (cont’d): Data Management
OLTP:
Data Warehouse Generation System:
Data Warehouse:
• Functions as the “back-end” database
• Fully relational and in 3rd normal form
• Used for data import, validation, and
management
• Technologies: Microsoft SQL Server
•
•
•
•
•
•
•
•
•
•
•
Extracts data from the OLTP
De-normalizes and transforms data
Loads data into the Data Warehouse
Builds table indexes
Archives “snapshots” of the database
Technologies: VB, stored procedures
Functions as the “front-end” database
Uses a de-normalized “star schema”
Used for querying and archiving data
Automatically generated from the OLTP
Technologies: Microsoft SQL Server
VIEWS Architecture Detail (cont’d): Data Backup, Restore, and Archival
Backup and Restore System:
Replication and Archival System:
• Automatically and periodically backs-up critical VIEWS
databases
• Restores database backups on demand
• Vertically partitions the Data Warehouse by time period
• Takes a full “snapshot” of the data warehouse at regular
intervals
• Creates a historical audit trail for verifying archive integrity
VIEWS Architecture Detail (cont’d): Data Presentation and Analysis
VIEWS Architecture Detail (cont’d): Data Presentation and Analysis
Site Browser:
For exploring detailed
monitoring site metadata,
history, photographs.
Third Party Tools:
A collection of relevant air quality
research tools provided by various
organizations and institutions.
ASCII Data File Collection:
A collection of data and
metadata in ASCII text file
format.
Class I Area Webcams:
A growing collection of links to Class I Area
webcams that provide a visual method for
assessing visibility and general air quality in
national parks and wilderness areas.
http://vista.cira.colostate.edu/views
Author: Shawn McClure
[email protected]