Transcript A New Generation of Data Services for Earth System Science

A New Generation of Data Services for
Earth System Science Education and
Research: Unidata’s Plans and
Directions
AGU Fall Meeting
San Francisco, CA
6 December 2005
Dr. Mohan Ramamurthy
Director, Unidata Program Center
UCAR Office of Programs
Boulder, CO
Science Drivers



Environmental problems like
global change & water cycle
transcend disciplinary as well as
geographic boundaries, requiring
multidisciplinary approaches and
global teams for solving them;
Rapid advances in observational
technologies, especially in
remote sensing;
Increasing use of complex,
coupled modeling systems;
Research studies on societal impact of hurricane-related
flooding involve integrating data from atmospheric
sciences, oceanography, hydrology, geology, geography,
and social sciences.
Science Drivers: Examples
OrbComm
LEO Satellite
Zero-pressure
Balloon
Gondola
(24 sonde capacity)
Hourly data at flight level
6 hours
between
drops
High-resolution vertical
profiles of Temperature, Wind,
Moisture, Pressure
Ground Station
NORTH
AMERICA
ATLANTIC OCEAN
EUROPE
End to End Information Services
GIS
Integration
Ensemble
Predictions
Emergency
Response
Coastal Environments
Need integrated services
Education Drivers


A “holistic” Earthsystem science
approach to education
Active, studentcentered learning. i.e.,
learning science by
doing science
• Observations (data)
• Tools (models,
visualization)
• Discovery
Technology Trends Enabling a
New Generation of Data Services
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Internet & the World Wide Web
Commodity microprocessors
Object-oriented programming
Open standards
Web services
Extensible Markup Language
(XML)
Global, high-bandwidth and
wireless networks
Digital libraries
Collaboratories
Grid Computing/e-Science
Data Portals and Federated,
distributed Servers
Geographic Information Systems
Knowledge environments
Ontologies and Semantic web
Data mining and knowledge
discovery
Data Services: An Evolution
An evolution from proprietary data systems towards
more open standards-based data services – i.e. web
services.
Data services should address the myriad applications
and needs of the community: research, education,
outreach, collaboration, etc.
This transition to web services poses many challenges.
Service-Oriented Science
Web Services are self-contained, selfdescribing, modular applications that can be
published, located, and invoked across the
Web.
XML based Web Services are emerging as
tools for creating next generation distributed
systems that facilitate program-to-program
interaction without the user-to-program
interaction.
Source: Ian Foster,
Science, 6 May 2005
Besides recognizing the heterogeneity as a
fundamental ingredient, these web services,
independent of platform and environment,
can be packaged and published on the
internet as they can communicate with other
systems using the common protocols.
Emerging web services standards are
enabling much easier system-to-system
integration.
Google Maps and Personal
Weather Data
How is this magic performed?
Answer: DHTML, JavaScript, CSS, XML, and XSLT
Google Earth
A Partial List of Data Services















Collection
Transport
Notification
Cataloging and metadata generation
Metadata submission
Subsetting
Aggregation
Decoders/format converter
Querying
Visualization
Collaboration
Ontology
Data mining
Weblogging or blogging
…
End–To–End Data Service
Development at Unidata
IDD
Data
LDM
Data
Storage
Locally
Generated
Data
THREDDS
Catalog
THREDDS
Data
Server
(TDS)
THREDDS
Data
Repository
(TDR)
Browse
Access
Put Data
Notify
TDS
Client
TDR
Client
E-mail
Application
(e.g. IDV)
Service
Internet Data Distribution
Satellite
Source
LDM
LDM
Radar
Model
LDM
Source
Source
LDM
LDM
LDM
Internet
LDM
LDM
LDM
File
Formats
Local/Remote
Services
Underlying
Interfaces
Primary
Interfaces
TDS: A Collection of Services
TDS
OGC WCS (Web Coverage Server)
(THREDDS Data Server Interface)
OPeNDAP
THREDDS
catalog
OpenDAP
NetCDF interface
ADDE
netCDF via HTTP
IOsp
GRID
Jgoffs
GRIB
NetCDF
NetCDF
AREA
DMSP
Station
NIDS
GINI
TDR: Another View
Data Services for Education
People
Discovery and
Publication Tools
og ols
tal To
Ca tion
ra
ne
Ge
Documents
THREDDS
Middleware
Da
ta
Se Cata
rvi
l
ce og
s
Discovery and
Publication Services
Analysis and
Visualization Tools
Data Services
Data
For effective incorporation of data into digital
libraries, we need a range of data services tailored
for education:
integrate data, models, viz. tools with learning
objects and other curricular materials
LEAD: Data Services for NWP
OU
NCSA
Unidata
Assimilation
Service
Decoder
Service
Prediction
Service
UAH
IU
Unidata
OU
Orchestration
Service
Product Generation
& Mining Service
Data Service
User Orchestrates
Web Services to
Create Regional
Forecast
User running local
analysis and
display tools
Data Services for Field Projects
GALEON IE: Data services for
GIS Integration
Concluding Remarks





At Unidata, we are in the process of building many of
these data-related services and technologies;
Despite the significant progress, much work still lies
ahead;
Other organizations and communities are engaged in
similar exercises (e.g., NOAA, IOOS, and CUAHSI) ;
We will partner with those groups where necessary
and leverage each other’s work;
We are diligently building bridges with other
communities for mutual benefit.