Transcript - wgiss

NOAA's
National Operational Model Archive and Distribution System
(NOMADS)
Access and Archives for Multi-Model Ensembles
and Climate Models
Glenn Rutledge
National Oceanic and Atmospheric Administration (NOAA)
National Climatic Data Center
Asheville, NC
CEOS WGISS-27
11-15 May, 2009
Toulouse, France
Outline
1. Developing advanced archive and access
capabilities for the next generation AOGCM models:
- NOMADS and “NOMADS-Next”.
2. The NOAA National Climate Model Portal (NCMP)
-
An OSSE capability for Climate Models
3. THORPEX Interactive Grand Global Experiment
(TIGGE)
-
Global Interactive Processing System (GIFS)
Multi-Model ensemble access prototype
NOAA National Operational Model Archive and Distribution System
NOMADS Goals
• provide distributed access to
models and associated data; and
promote model evaluation.
• foster research within the geoscience communities (ocean,
weather, and climate) to study
multiple earth systems using
collections of distributed data.
• develop institutional partnerships through open source
and proprietary technologies, standards, and tools.
NOAA National Operational Model Archive and Distribution System
New NOMADS Services
• High Availability Servers
operational in NOAA’s
National Weather Service:
- Boulder, CO
- Ft. Worth, TX
- Silver Spring, MD
• Ocean-NOMADS at the
National Ocean Data Center:
- Charleston, SC.
nomads.ncdc.noaa.gov
NCDC Architecture
NCDC
HDSS
Archive
Direct Client Access
GDS
and TDS
NCEP Dual Ingest & QC
NOAA-wide LAS
“sister-servers”
Live Access
Portals
Server
GrADS, Ferret, MatLab, IDL,
IDV, Web browsers or any
OPeNDAP enabled client
• Project ACCESS: NASA,
GMU, GMU & OPeNDAP
• Project GALEON Unidata
• NOMADS Web-Plotter,
http, ftp & binary subsetting
• GIS access (WCS, W*S)
Collaboration Focus
http / ftp access
DOE’s
Earth System Grid
“NOMADS-Next” under Development
NOMADS Data
NWP Model
•
•
•
•
•
•
•
•
•
•
•
Global Forecast System (GFS), 1 and ½ degree
NCEP Spectral Statistical Interpolation (SSI) Global Data Assimilation System (GDAS) w/ restart files
North American Mesoscale (NAM, formally Eta) 1 and 3 hourly
Rapid Update Cycle (RUC) 13km and 20km
NCEP North American Regional Reanalysis (NARR) 30 years 32km
NCEP/NCAR R1/R2 Reanalysis (Climate Data Assimilation System -CDAS)
NCEP Regional Special Model (RSM)
NCEP Global Ensembles and SREF
NCEP Ocean models (Ocean-NOMADS)
NCEP Sea Ice Models
NWS Analysis of Record Real-time Mesoscale Analysis (RTMA downscaled RUC to 5km)
In situ
•
•
•
NCDC Global Historical Climate Network (GHCN) surface temperature and precipitation anomalies
NCDC Integrated Global Radiosonde Archive (IGRA) upper air reference quality data set (formally CARDS)
NCDC Smith-Reynolds Extended Reconstructed and O/I Sea Surface Temperatures (ERSST) and climatologies )
NOAA Satellite
•
No longer serve sat and radar data to focus on Models.
Climate Models / Coupled AOGCM
•
•
GFDL CM2.0 and CM2.1 Climate Experiments
Paleoclimate Model Intercomparison Project (PMIP)
Expected availability Winter 2010!
NOAA Reanalysis
“Explaining Climate to Improve Prediction” (ECIP):
1) Climate Forecast System Reanalysis and Reforecast
(CFSRR) Project (Saha et al.) Global 1 degree
– 1978-2008 Reanalysis. ~230TB (Grib2)
2) Climate of the 20th Century Project (Compo, et al.,)
– 1850 to present. ~120TB (Grib1)
3) Climate Prediction Center Reanalysis CPCR
– 1944 to present. ~235TB (FY11)
http://nomads.ncdc.noaa.gov/NOAAReanalysis/
NOAA
Reanalysis
Community
Web Page
Informational
User Survey
Feedback
Registration
Coming Soon!
Climate Models
• GFDL, (NCAR/DOE/NASA)
– CM.x, Requirements tdb
– Models to support impacts user community
• Unified Synthesis Product (USP)
– Coupled downscaled (4 TB initially, 3 vars)
• Earth System Grid (ESG) and NOAA’s ESRL
– NOMADS collaboration with ESG (D. Williams)
– ESMF and NOAA’s ESRL (C. DeLucia)
Datasets On the Horizon
• IPCC Assessment Report 5 (AR5, 2010/11)
– Multi-model approach via ESG in process
• THORPEX Interactive Grand Global
– TIGGE International distributed Archive
– GIFS Multi Center requirements
• Other TBD Experiments (e.g., YOTC)
NCMP
National Climate Model Portal
The conceptual architecture is based on
three (3) main components; the
Portal, the Catalog Node, and the
Data Repository.
NOAA Climate Services Portal
1) The Climate Model Portal is the user’s
real-time interface to the system,
manage requests, download data,
receive user input and catalog
browsing.
2) the Catalog Node is the heart of the
system and concentrates on
connecting partners, metadata,
search and discovery and secure
peer-to-peer connectivity.
3) the Data Repository will be based on
advanced real-time access
components, and will also use the
Data Center IT infrastructure for long
term storage and access
NCDC
Archives…
NCEP
GFDL …*
Data Sources….
• Collaborators: DOE (ESG), NASA, EPA,
UCAR/NCAR, SDSC, RENCI & others
NCMP
National Climate Model Portal
= NCMP Components
NCMP
A NOMADS OSSE Capability
• The NCDC NOMADS data sets, which include conventional and non-conventional
observations, are archived and accessible through the NOMADS servers.
• for NWP OSSE develop and integrate a massive on-line data
base with users so there is controlled access to
data set selection, location, spatial extent,
Variable, quality and time period.
• Climate OSSE needs a new process
NOMADS
UW, Madison
Steps for the OSSE Capability:
1) Generate a "nature" atmosphere or ocean
2) Compute synthetic observations
3) Assimilate the synthetic observations
4) Assess the impact on the resulting analysis
forecast, prediction or assessment.
NCMP
Climate OSSE
• Climate OSSE’s may not use the same NWP
processes (decades of files for integration)
• New approaches need to be developed or
leveraged. (OLR’s etc.)
• Climate OSSE’s can establish appropriate
confidence in climate change scenarios.
• Perturbed Physics Ensembles (B. Wielicki, et
al.,) and a climateprediction.net approach
THORPEX Interactive Grand Global Experiment (TIGGE)
Objectives
• Production of internationally coordinated
advance warnings and forecasts for high
impact weather events…
– to mitigate loss of life and property,
– to contribute to the welfare of all World
Meteorological Organization (WMO) nations, with
a particular emphasis on least developed and
developing countries.
Toth, Z., J. Desmarais, G. Brunet, Houtekamer, Y. Zhu, R. Wobus, R. Hogue, R. Verret, L. Wilson, B. Cui, G. Pellerin, B. Gordon, E. O'Lenic, D. Unger,
2005: The North American Ensemble Forecast System (NAEFS). Proceedings of the 1st THORPEX International Science Symposium, December
2004, Montreal, Canada. Available at http://wwwt.emc.ncep.noaa.gov/gmb/ens/papers/ toth_naefs_thorpex_montreal.pdf
TIGGE Status
• Develop a unified ensemble operational
concept (CONOPS) to allow reliable production
and international exchange of ensemble
products:
– Identify unified ensemble operations requirements
• standard output fields in standard format and exchange
parameters
• ensemble configuration including output intervals and
forecast length, membership, approximate spatial
resolution
GEO Task co-lead & participant: Rutledge/Toth
Probability Density Functions (PDF’s)
Multi-Model Ensemble Access
20 model runs,
64 fcst projections (6hrs)
1280 queries
ONE request in ~10 seconds
OPeNDAP constraint expression
Script URL is: http://nomads.ncdc.noaa.gov/dods/enshires/archive/ens20040809/...
ensc0_00z_1x1.ascii?pratesfc[3:3][125:125][277:277]
Demonstration web page showing the URL queries of the ensembles
OPeNDAP GDS query
constrained by time and location
of Paris airport for precipation
for each ensemble component.
NOMADS Ensemble
Probabilities on the fly
(demo)
We loop the previous
queries over all forecast
times to 384-hrs and graph
the result for the
probabilities of precipitation
for Paris out 16 days.
Combinations of
probabilities (temp / precip)
can derive products: frost,
freezing rain etc.,
[email protected]
Thank you !
Historical
http://nomads.ncdc.noaa.gov
Real-Time:
http://nomads.ncep.noaa.gov/
Backup
NOMADS Capabilities
NWP
Ensembles
Reanalysis
Climate
Programmatic Servers
NAM
Lo-Res
NARR
CM2.X
SRRS
LAS
GFS
Hi-Res
Global R1
CFS
NDFD
GDS
RUC
Probability
Tool
Global R2
SST
RTMA
TDS
Most frequently used data & capabilities
LAS: Server side analysis, coordinate transformation, and user interface engines
GDS: Server side analysis and format conversions
TDS: file aggregations, I/O-SP, http, wget, OGC: WCS, WMS; GeoPortal Tooolkit
NetCDF: utilities and subsetting (in TDS)
NOMADS Wgrib: direct file level grib subsetting, / grib1-grib2 conversions /
SRRS: NOMADS Service Records Retention System the only legally mandated
dataset at NCDC
NDFD: National Digital Forecast Database- NOMADS archive sub-setting
RTMA: Real-Time Mesoscale Analysis.
OPeNDAP and http services via all 3 servers. Very popular.
Short and Long Term Goals
• Operational Forecasting• Ensemble Prediction Access: flow-dependant prediction of
weather and climate- nowcasting, medium range and seasonal.
• Atmospheric and Oceanic Research• Scalar and Vector processing and Workstation models
• Model output statistics; reanalysis; data assimilation techniques
• Global Climate Change and Advanced Analysis• Clouds, initial conditions, true coupled simulations.
• Long term climate monitoring: in-situ analysis, trends, data
homogeneity, extremes, downscaling, reducing uncertainty...
• On-demand Data Mining and Product Generation-convective process
NCMP Key Development Strategies
• Maintain and minimize impact on legacy systems
• Develop Services and Component based Architecture
(SCBA) using a top-down, bottom-up iterative
software development process
• Evolutionary development through pilot projects
• Coordinate activities through Communities of
Interest organized by structural “Data Types”
– Grids, time-series, moving-sensor multi-dimensional, profiles,
trajectories, geospatial framework, point data and metadata
• DOE Earth System Grid: initial collaborator.
NOMADS Data
Global Data Assimilation System, Spectral Forecast Model and the
Spectral Statistical Interpolation Cycling Analysis System
- NOAA-15/16 AMSU-A/B TOVS 1B Radiances (IEEE)
- Analysis Bias Corrected Information / Obs Toss List
- SFC U/A, ACRS, Aircft (BUFR)
- 6HR fcst guess from previous run (BUFR)
- ERSCAT Sat obs / HIRS 14/15, MSU TOVS (IEEE)
- Guess prep and and fcst guess output (BUFR)
- Analysis ready QC’ed Obs. (prepBUFR)
- Profiler, TOVS, Wind Obs. (BUFR)
- SFC Analysis Restart Files
- SST’s (GRIB)
- Radar VAD Winds (BUFR)
Data Assimilation: GDAS SSI
• NOMADS saves the minimum data necessary to regenerate
model output products.
• Analysis files are in the models own coordinate system.
• Input files are constructed with computer and computational
efficiency in mind, and sometimes not in standard coordinate
systems.
• Programs to convert these files are available upon request:
• spectral to gaussian
• gaussian to lat/lon
• sigma to pressure