Transcript Final_Step2 - wgiss

IDN TASK TEAM
Update
Presented by
Lola Olsen
Budapest, Hungary
CEOS WGISS-21
May 9, 2006


Notes
from the
Ukraine
meeting
MD9.5ß Software Release Demonstrated
Primary action items assigned to the IDN
Task Team
Completed tasks:
(1) Set up GEOSS portal for WGISS to review
(2) Prepared related service/tool (SERF)
descriptions for GEOSS-related services.
(3) Sent the URL to all of WGISS for the
Conference Calendar (provided on IDN
website-see “calendar” tab).
(4) Updated lyrics of ‘We WGISS’ and
captured into an audio file.
IDN Website – Current Homepage
New IDN Map
New IDN Web Site Header
New tabs that link Data Sets, Data Services & the
Conference Calendar. Action (3) from Cordoba
Status of IDN Site
• IDN Website continues to use Zope/Plone.
• Discussion List. Used the Interoperability Forum
to provide feedback on:
–
–
–
–
–
Data resolution options
DIF extension for Models
Platform/Instrument Restructure
Renaming of Keyword Fields
Geographic Location Hierarchy
Numerical Model Fields Proposed (2002/03)
•
Anne Sophie Archambeau from IPSL, France proposed an addition to
the DIF to handle numerical output data sets:
Group: Numerical_Experiment
Model_Name:
Model_Version:
Model_URL:
Model_Configuration:
Model_Resolution:
Model_Calendar:
Group: Model_Integration_Period
Start_Date:
Stop_Date:
End_Group
Simulation_Name:
Initial_Conditions:
Perturbation:
Imposed_Boundary_Conditions:
End_Group
•
First proposed in
2002
Interop discussions on the proposed new fields are summarized in the
CEOS IDN Interop Newsletter #15 for April 2003:
http://gcmd.gsfc.nasa.gov/pipermail/interop/2003-April/000011.html
Instrument and Platform
Upgrade
• Implement an option to list all instruments and platforms
alphabetically for searching.
• Expanded database table structures for Instrument and
Platform information and categories.
• Extend Supplemental Information to include expanded
Instrument and Platform information.
• Feedback Welcome!
Instrument and Platform Upgrade
• Instrument database information can include:
– Instrument Name (currently exists as short name > long
name
– Categories: types, subtypes (can be used in web interface for
browsing. For example:
• Earth Viewing vs. Space Viewing vs. in situ, etc.
• Wavelength region (microwave vs. visible, etc.)
– Channels (if appropriate)
– Spectral information; instrument information.
• Platform database information can include:
– Platform name (currently exists as short name > long name
– Categories: type of platform (e.g. Landsat Series)
– Instrument names (if appropriate): link to instrument
database table.
– Platform orbital and launch information.
Group: Instrument
Group: Instrument_Identification
*Instrument_Category:
*Instrument_Class:
*Instrument_Type:
*Instrument_Subtype:
*Short_Name:
*Long Name:
End_Group
Group: Associated_Sensors
*Short_Name (r):
End_Group
Group: Spectral/Frequency Information
*Wavelength Keyword: (r)
Number_Channels:
Spectral/Frequency
Coverage/Range:
Spectral/Frequency Resolution:
IFOV:
Radiometric_Resolution
End_Group
Group: Description
End_Group
Online_Resource: (r)
Creation_Date:
Revision_Date:
*IDN_Node: (r)
Group: Instrument_Logistics
Instrument_Start_Date:
Instrument_Owner (r):
End_Group
End_Group
Instrument
Group
(r) = repeatable field
* = controlled keywords
italics = required fields.
Group: Platform
Group: Platform_Identification
*Platform_Category:
*Platform_Series/Entity:
*Short_Name:
*Long_Name :
End_Group
Group: Alternate_Platform_Names
Short_Name (r):
End_Group
Group: Associated Instruments
*Short_Name: (r)
End_Group
Group: Orbit
Orbit_Altitude:
Orbit_Inclination:
Equator_Crossing:
Period:
Repeat_Cycle:
Perigee:
Apogee:
*Orbit_Type:
End_Group
Group: Description
<text>
End_Group
Creation_Date:
Revision_Date:
Online_Resource: (r)
*IDN_Node: (r)
Group: Platform_Logistics
Launch_Date:
Launch_Site:
Primary_Sponsor (r):
End_Group
End_Group
Platform Group
(r) = repeatable field
* = controlled keywords
italics = required fields.
Note: Instrument short_names will be valid Sensor_Names in
the database.
Note: Alternate Platform Names will refer to Platform short
names in the database
Group: Instrument
Group: Instrument_Identification
Instrument_Category: Earth Remote Sensing Instruments
Instrument_Class: Active Remote Sensing
Instrument_Type: Altimeters
Instrument_Subtype: Lidar/Laser Altimeters
Short_Name: GLAS
Long Name: Geoscience Laser Altimeter System
End_Group
Group: Associated_Sensors
*Short_Name:
End_Group
Group: Spectral/Frequency Information
Wavelength_Keyword: Radar
Number_Channels: 2
Spectral/Frequency Coverage Range: 1064 nm, 532 nm
Spectral/Frequency Resolution:
IFOV: 70 m
Radiometric_Resolution:
End_Group
Group: Description
End_Group
Online_Resource:
http://www.csr.utexas.edu/glas/Instrument_Description/
Creation_Date: 2006-03-23
Revision_Date:
IDN_Node: USA/NASA
Group: Instrument_Logistics
Instrument_Start_Date:
Instrument_Owner: NASA
End_Group
End_Group
Instrument
Example
Group: Instrument
Group: Instrument_Identification
*Instrument_Category: Earth Remote Sensing Instrument
*Instrument_Class: Passive Remote Sensing
*Instrument_Type: Photon/Optical Detectors
*Instrument_Subtype: Photometer
*Short_Name: Sun Photometer
*Long Name: Sun Photometer
End_Group
Group: Associated_Sensors
*Short_Name (r): Photometer
End_Group
Group: Spectral/Frequency Information
*Wavelength Keyword: (r) Varied
Number_Channels:
Spectral/Frequency Coverage Range:
Spectral/Frequency Resolution:
IFOV:
Radiometric_Resolution
End_Group
Group: Description
End_Group
Online_Resource: (r)
Creation_Date: 2006-04-04
Revision_Date: 2006-04-04
*IDN_Node: USA/NASA
Group: Instrument_Logistics
Instrument_Start_Date:
Instrument_Owner (r):
End_Group
End_Group
Instrument
Example
Group: Platform
Group: Platform_Identification
Platform_Category: Earth Observing Satellite
Platform_Series/Entity: EOS
Short_Name: Aqua
Long_Name: Aqua
End_Group
Group: Alternate Platform Names
Short_Name: PM-1
End_Group
Group: Associated Instruments
Instrument_Short_Name: MODIS
Instrument_Short_Name: AMSR-E
Instrument_Short_Name: AMSU-A
Instrument_Short_Name: CERES
Instrument_Short_Name: AIRS
Instrument_Short_Name: HSB
End_Group
Group: Orbit
Orbit_Altitude: 705 km
Orbit_Inclination: 98.2 degrees
Equator_Crossing: 1:30 pm, ascending node
Period: 99 minutes
Repeat_Cycle: 16 days
Perigee: 673 km *
Apogee: 686 km *
Orbit_Type: Sun Synchronous
End_Group
Group: Description
End_Group
Creation_Date: 2005-08-25
Revision_Date: 2006-03-22
Online_Resource: http://aqua.nasa.gov
IDN_Node: USA/NASA
Group: Platform_Logistics
Launch_Date: 4 May 2002
Launch_Site: Vandenberg AFB
Primary_Sponsor: NASA
End_Group
End_Group
Platform
Example
Node Reports
JCADM Report
Antarctic Master Directory (AMD)
- 3900 Total Records
- 30% Contain Direct Data
Links (estimated)
- Cover 13 Earth Science
Disciplines
- Contributions from
22 Countries
- Over 10,000 Hits in 2005
- 5,500 Retrievals in 2005
- Highly Customized
Portal Interfaces with 16
NADC Portals
http://gcmd.nasa.gov/portals/amd/
Data Access Portals
The GCMD hosts a total
of 70 highly customized
portals
16 NADC AMD portals
8 new portals:
http://gcmd.nasa.gov/portals/amd_my/
Spain, Uruguay,
Italy – Feb.
France – Jun.
Chile – Jul.
New Zealand,
Malaysia – Oct.
China – Apr. ‘06
Antarctic Master Directory - NADC Portals
The alias format is http://gcmd.nasa.gov/portals/portal_name/
Examples of alias: http://gcmd.nasa.gov/portals/amd_ar/
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Belgium Antarctic Program – Jan. 2003
Finnish Antarctic Program – Jan. 2003
Centro de Datos Antárticos, Argentina – Aug. 2003
Canadian Polar Commission/Canadian Committee for Antarctic Research – Oct. 2003
Swiss Committee on Polar Research – Nov. 2003
US Antarctic Data Centers/US Antarctic Data Coordination Center - Feb. 2004
Japan National Institute for Polar Research – Feb. 2004
Estonian Antarctic Data Center – Jul. 2004
Centro Nacional de Datos Antárticos, Spain – Feb. 2005
Instituto Antartico Uruguayo – Feb. 2005
Italian Antarctic Program – Feb. 2005
Institut Polaire Francais Paul Emile Victor, France – Jun. 2005
Centro Nacional de Datos Antarticos, Chile – Jul. 2005
Malaysian Antarctic Data Center – Oct. 2005
Antarctica New Zealand Data Center – Oct. 2005
Chinese Antarctic and Arctic Data Centre – Apr. 2006
AMD Content Growth
AMD Content Distribution by Country
Contributions from 22 Countries
AMD Search Page Hits by Country
Most Hits in 2005 from US, Italy, Argentina,
Belgium, and New Zealand
AMD Search Page Hits vs. Retrievals
10,000 Hits in 2005
5, 500 Retrievals in 2005
ARCTIC PORTAL
~ 900 Records
http://gcmd.nasa.gov/portals/arctic/
OBIS: Following the JCADM Model …
-
OBIS Taxonomic Categories
http://iobis.org/CategorySearch
- to be implemented in GCMD
Portal created in Dec. 2005
40+ metadata Records
All have a “GET DATA” link
10 Regional OBIS Portals
Customized “Templates”
Creating/Using Templates …
1.
2.
3.
4.
Select Document Type
Either select existing
template or “no template”
Either select “create new
document” or “working with
an existing document”
Enter Document Identifier
1.
2.
Enter fields that apply - select
for help documentation
Options:
-“Preview Document Display”
-“Save As Template”
-“Email Document to Yourself”
Support for foreign and mathematical
characters in DIF/SERF display
DIF
SERF
Demo Mapserver for Budapest
GCMD Prototype Mapserver
Tools
GIS Data
Layers
Map
View
OGC WMS Get Capabilities Request:
http://mapserver.gsfc.nasa.gov/wmsconnector/com.esri.wms.Esrimap/CEOS_2005?request=get
capabilities&service=WMS&version=1.1.1
Prototype Mapserver Interoperability
 Access GCMD’s Web Mapping Service (WMS) through an OGC WMS GET Capabilities Request.
 OGC WMS compatible clients can access GCMD’s map services.
GCMD Mapserver
User Friendly Desktop
Internet GIS (WMS Client)
Mapdex Indexes GCMD Mapserver
http://www.mapdex.org
New Keywords since Last
CEOS Meeting
PART C
New/Modified Earth Science
Keywords Since September 2005
NEW Variables for Numerical Models
•
•
•
•
•
•
•
•
•
Decision Support Models
Digital Elevation/Terrain Models
Earth System Component Models
Global Coupled Climate System Models
Global General Circulation Models
Phenomenological Models
Process Models
Regulatory Models
Solar-Atmosphere/Space Weather Models
URL Content Type Keywords Cleanup
Provide accurate descriptions of related information in
DIFs and SERFs.
• Provide “Direct Access” to URL for quick access to data.
1. GET DATA
e.g: http://www.mgs.md.gov/coastal/vmap/erosion/shp/Aberdeenshp.zip
2. GET WEB MAPPING SERVICE (WMS)
e.g: http://www.kgis.scar.org/cgi-bin/kgis_wms?
• Provide “Direct Access” to Services
1. GET SERVICE
e.g: http://ldas.gsfc.nasa.gov/
URL Content Type Keywords Cleanup
Previous List
New List
ASSOCIATED DATA SET(S)
BDP METADATA
CALIBRATION/VALIDATION DATA
DIAL DATA SERVER
DISC-AIRS
DISC-MODIS
DISC-OMI
DODS_DIR
DODS_FILESERVER
DODS_LAS
DODS_URL
EML METADATA
FGDC METADATA
GCMD MAPSERVER
GET DATA
GET SERVICE
NBII METADATA
NOMADS DATA SERVER
PROJECT HOME PAGE
REFERENCE MATERIALS
RELATED DATA SET DESCRIPTION
RELATED DATA SET METADATA
RELATED SERVICE METADATA
SOAP API
SOFTWARE PACKAGES
UDDI REGISTRY
WEB COVERAGE SERVICE (WCS)
WEB FEATURE SERVICE (WFS)
WEB MAPPING SERVICE (WMS)
GET DATA
VIEW EXTENDED METADATA
GET SERVICE
VIEW PROJECT HOME PAGE
VIEW RELATED INFORMATION
GET RELATED DATA SET METADATA (DIF)
GET RELATED SERVICE METADATA (SERF)
ACCESS WEB SERVICE
GET SOFTWARE PACKAGE
GET WEB COVERAGE SERVICE (WCS)
GET WEB FEATURE SERVICE (WFS)
GET WEB MAPPING SERVICE (WMS)
GET MAP SERVICE
ACCESS MAP VIEWER
New Data Center, Location,
Instrument, and Platform
Hierarchies for Improved Search
Multi-year Effort
Comments Welcome
New Location Hierarchy Search
New search box for location
Search or map date search
New Location Hierarchy Search
Select
Continent
Choose from among top level
location categories
New Location Hierarchy Search
Breadcrumbs show where you
are in the hierarchy
Select Europe
“Continent” category subdivided
into individual continents
New Location Hierarchy Search
Breadcrumbs
Select Eastern
Europe
Category “Europe” is
further subdivided into
individual countries and
regions.
New Location Hierarchy Search
Breadcrumbs
Select Hungary
Category “Eastern Europe”
is further subdivided into
individual countries and
regions.
New Location Hierarchy Search
Breadcrumbs
Results for search on
“Hungary” displayed.
DIF/SERF POPULATION
Date
Ju
l-0
5
O
ct
-0
5
Ja
n06
Ju
l-0
4
O
ct
-0
4
Ja
n05
Ap
r-0
5
12000
Ju
l-0
3
O
ct
-0
3
Ja
n04
Ap
r-0
4
Ju
l-0
2
O
ct
-0
2
Ja
n03
Ap
r-0
3
Ju
l-0
1
O
ct
-0
1
Ja
n02
Ap
r-0
2
Ju
l-0
0
O
ct
-0
0
Ja
n01
Ap
r-0
1
Ja
n00
Ap
r-0
0
# DIFs
Total DIFs January 2000 - April 2006
18000
16000
14000
WHFC children
DIFs
split of Aqua
MODIS DIFs from
Terra MODIS DIFs
10000
new Antarctic DIFs
8000
6000
Cumulative DIF Population 2003-2006
17500
16500
# DIFs
15500
14500
13500
12500
11500
Jan- Mar- May- Jul03
03
03
03
Sep- Nov- Jan- Mar- May- Jul03
03
04
04
04
04
Sep- Nov- Jan- Mar- May- Jul04
04
05
05
05
05
Month
Sep- Nov- Jan- Mar05
05
06
06
New and Revised Supplemental Descriptions
200
180
160
120
100
80
60
40
20
Ja
n06
M
ar
-0
6
-0
5
5
N
ov
p0
5
Se
Ju
l0
Ja
n05
M
ar
-0
5
M
ay
-0
5
-0
4
4
N
ov
p0
4
Se
Ju
l0
Ja
n04
M
ar
-0
4
M
ay
-0
4
-0
3
3
N
ov
p0
Se
Ju
l0
3
0
Ja
n03
M
ar
-0
3
M
ay
-0
3
# New/Revised
140
Month
Data Centers New2
Sensors New4
Data Centers Revised2
Sensors Revised4
Sources New3
Campaigns New5
Sources Revised3
Campaigns Revised5
l03
DIFs New
l04
# DIFs
DIFs Deletes
Revised
l05
5
N
ov
-0
5
Ja
n06
M
ar
-0
6
p0
Se
Ju
4
N
ov
-0
4
Ja
n05
M
ar
-0
5
M
ay
-0
5
p0
Se
Ju
3
N
ov
-0
3
Ja
n04
M
ar
-0
4
M
ay
-0
4
p0
Se
Ju
03
M
ar
-0
3
M
ay
-0
3
Ja
n-
Month
New, Revised, and Deleted DIFs 2003-2006
1800
1600
1400
1200
1000
800
600
400
200
0
DIF Population by Earth Science
Topic
SUN-EARTH
INTERACTIONS-1.6%
SOLID EARTH-11.2%
AGRICULTURE-9.7%
SPECTRAL/ENG.-10.3%
ATMOSPHERE-30.2%
PALEOCLIMATE-4.5%
OCEANS-25.0%
BIOSPHERE-29.2%
CLIMATE
INDICATORS-1.4%
LAND SURFACE-23.0%
HYDROSPHERE-14.2%
CRYOSPHERE-9.1%
HUMAN DIMENSIONS18.0%
DIF Population by Node
CLIVAR
0%
AMD
21%
G3OS
2%
NCAR
1%
UN
4%
NASA
39%
UNEP/GRID
3%
JST
1%
CSIRO
0%
RAS
0%
JAXA
1%
DLR
0%
CCRS
4%
INPE
CNES ESA/ESRIN
0%
1%
0%
NEONET
0%
USGS
5%
CONAE
0%
CIESIN
1%
USDA
4%
NOAA
12%
NASA
NOAA
USDA
CIESIN
USGS
CONAE
INPE
CCRS
ESA/ESRIN
DLR
CNES
NEONET
RAS
JAXA
JST
CSIRO
UNEP/GRID
AMD
G3OS
CLIVAR
UN
NCAR
DIF Population by Node, April 2006
UN/UNEP
8%
AMD
22%
JAXA
2%
NASA
40%
CSIRO
0%
CNES
0%
DLR
0%
ESA/ESRIN
1%
USGS
6%
CCRS
4%
INPE
0%
NASA
ESA/ESRIN
CONAE
0%
NOAA
DLR
CIESIN
1%
USDA
CNES
NOAA
12%
USDA
4%
CIESIN
JAXA
USGS
CSIRO
CONAE
AMD
INPE
UN/UNEP
CCRS
IDN SERF Population
Total SERFs In GCMD
Jan. 2001- April 2006
1600
1400
1200
SERF Totals
1000
800
600
400
200
0
Jan-01 M ay-01 Sep-01
Jan-02 M ay-02 Sep-02
Jan-03 M ay-03 Sep-03
Month
Jan-04 M ay-04 Sep-04
Jan-05 M ay-05 Sep-05
Jan-06
Distribution of SERFs
Models
13%
Metadata Handling
3%
Data Analysis &
Visualization
31%
Reference and
Information Services
10%
Hazards Management
3%
Environmental Advisory
8%
Education/Outreach
15%
Data Analysis & Visualization
Education/Outreach
Hazards Management
Metadata Handling
Data Management/
Data Handling
17%
Data Management/Data Handling
Environmental Advisory
Reference and Information Services
Models
Distribution of SERF Usage
Models
16%
Data Analysis &
Visualization
23%
Metadata Handling
10%
Data Handling
12%
Information and
Reference Services
9%
Hazards Management
8%
Environmental
Advisory
11%
Education/Outreach
11%
WEB USAGE
IDN Usage by Domain Type 2006
.us
1%
international
23%
.gov
2%
.edu
7%
.org
1%
.com
18%
.net
23%
numeric
24%
.mil
1%
.gov
.edu
.org
.com
.net
.mil
numeric
international
.us
IDN Usage by Domain 2002-2006
16000
14000
Cache opened to
Internet robots
12000
.gov
8000
Introduction
of the new
web page
.edu
.org
.com
.net
.mil
numeric
international
6000
.us
4000
2000
Month
Ja
n06
M
ar
-0
6
Ja
n05
M
ar
-0
5
M
ay
-0
5
Ju
l-0
5
Se
p05
N
ov
-0
5
Ja
n04
M
ar
-0
4
M
ay
-0
4
Ju
l-0
4
Se
p04
N
ov
-0
4
0
Ja
n03
M
ar
-0
3
M
ay
-0
3
Ju
l-0
3
Se
p03
N
ov
-0
3
Users by Domain
10000
# Web Page Hits Since January 2001
8000000
7000000
Cache opened to Internet
Search robots
6000000
4000000
Introduction
of the new
web page
3000000
2000000
1000000
0
Ja
n0
M 3
ar
M 03
ay
-0
Ju 3
l-0
Se 3
pN 03
ov
-0
Ja 3
n0
M 4
ar
M 04
ay
-0
Ju 4
l-0
Se 4
pN 04
ov
-0
Ja 4
n0
M 5
ar
M 05
ay
-0
Ju 5
l-0
Se 5
pN 05
ov
-0
Ja 5
n0
M 6
ar
-0
6
#hits
5000000
month
DIF/SERF SEARCH/RETRIEVAL
Searches by Controlled Keyword
Sun-Earth Interactions
3%
Solid Earth
7%
Spectral/Engineering
4%
Agriculture
10%
Atmosphere
14%
Paleoclimate
4%
Oceans
16%
Biosphere
7%
Land Surface
12%
Climate Indicators
6%
Hydrosphere
6%
Cryosphere
4%
Human Dimensions
7%
Agriculture
Atmosphere
Biosphere
Climate Indicators
Cryosphere
Human Dimensions
Hydrosphere
Land Surface
Oceans
Paleoclimate
Spectral/Engineering
Sun-Earth Interactions
Solid Earth
User Support Questions
Answered
50
45
Questions Answered
40
35
30
25
20
15
10
5
0
Jan
Feb
Mar
Apr
2004
May
2005
Jun
2006
Jul
M
ontAhs
NAS
P AO 2005
Aug
Sep
NAS A P AO 2006
Oct
Nov
Dec
60
Conference Calendar Entries
Number of New Entries
50
40
30
20
10
0
Jan-04 Mar-04 May-04 Jul-04 Sep-04 Nov-04 Jan-05 Mar-05 May-05 Jul-05 Sep-05 Nov-05 Jan-06 Mar-06
Month
New
Middleware Search/Retrieval Component
(Integration of spatial, freetext, and controlled queries)
Multi-Layer Search Component
delegates
1) User performs
query
7) Refines search
3) joins
Controlled Vocabulary Database Layer
Freetext (Lucene) Database Index Layer
Spatial Database Index Layer
2)
Controller
Cache
4) Set of unique IDs
Search
Results
6) Returns to user
Result
Processor
5) Title set information, brief
summary, dataset links, etc.
New Features
New Features added since
September 2005
• New Data Resolution
refinement available in
NASA DAAC portals
–
–
–
–
Earth-Sun System DAACs
Goddard Space Flight Center
Langley Research Center
National Snow and Ice Data
Center
• docBUILDER
– New preview of tool available on start
pages
– Supplemental descriptions supported
• New collaborative portals
– 16 portals added
• A New Advanced Free-text • New Keyword Subscription
Service
Search
– Map/Date search
– Uses new Google Map
– Receive Science and/or Service keyword
updates
New Portals added since September 2005
(Current total: 70)
 Tropical Cyclones
 GoMODP
 Malaysia AMD
 New Zealand AMD
 OBIS
 Australia
 Canada
 Europe
 Indian Ocean
 New Zealand
 South America
 Sub-Saharan Africa
 United States
 NASA Langley DAAC
 NASA NSIDC
 China AMD
Data Resolution “Valids”
• Reminder! DIF authors can now populate their DIFs by selecting horizontal,
vertical and temporal resolution in docBUILDER.
• Vertical Resolution refers to both Altitude and Depth resolution.
Horizontal Resolution Range:
<1m
1 m - < 30 m
30 m - < 100 m
100 m - < 250 m
250 km - < 500 m
500 m - < 1 km
1 km - < 10 km
10 km - < 50 km
50 km - < 100 km
100 km - < 250 km
250 km - < 500 km
500 km - < 1000 km
> 1000 km or > 10 degrees
Vertical Resolution Range:
<1m
1 m - < 10 m
10 m - < 30 m
30 m - < 100 m
100 m - < 1km
> 1km
Temporal Resolution Range:
< 1 second
1 second - < 1
minute
1 minute - < 1 hour
Hourly - < Daily
Daily - < Weekly
Weekly - < Monthly
Monthly - < Annual
Hourly Climatology
Daily Climatology
Pentad Climatology
Weekly
Climatology
Monthly
climatology
Annual
Annual
climatology
Decadal
Climate Normal
(30-year
climatology)
MD9.5.1 Release: Highlights
Spatial Search with
Map
Refinement by Data Resolution (for NASA DAAC portals only)
Support for refinement in SERFs (platforms, instruments, service providers, science
parameters, service parameters, etc.)
Support for foreign characters in DIF/SERF display
Support for supplemental descriptions in HTML docBUILDER
Subscription Service for Science Keywords
OAI Repository
Web Page design mockups for 12 NASA Application Areas and 7 NASA focus areas
Several bugs fixed and enhancements added (208 total)
PostgreSQL
• Database setup scripts are being customized to
work with an open source database.
– Scripts are being updated to work with
PostgreSQL 7.4.7 database.
– Currently testing MD Server setup with
PostgreSQL.
– Install instructions will be rewritten to make
installation as easy as possible for Nodes.
Highlights of New docBUILDER Tool
(Changes the users will see)
• Offers option to use personalized templates.
• Supports three-color coding for required, recommended
and optional fields.
• Provides automatic population of Data Center /Service
Provider URLs.
• Allows direct editing of the XML document.
• Offers more intuitive interface for Personnel and Data
Center fields.
• Incorporates spell-checker for all fields and full document.
= Both
= HTML
= Standalone
Highlights of New docBUILDER Tool
(Changes “behind the scenes”)
• Complies fully with XML Schema (previous version only
supported DTD).
– Provides capability of rendering custom authoring tool
according to the XML schema
– Renders customized widgets based on XML element names
association.
• Contains new business logic with a completely rewritten
business layer, resulting in much faster access to XML
documents and valids.
• Separates business logic from presentation logic, making
changes to presentation much easier.
– Refactored framework using Struts and JSP allowing for
complete portal customizations.
HTML docBUILDER
docBUILDER
HTML docBUILDER
(Opening Screen)
HTML docBUILDER
(Main Screen)
Menu with options
to save, email, etc.
Legend
Checkboxes
indicating progress
Editable text
area
HTML docBUILDER
(Data Resolution Field)
Field Name
Field Description:
short description of the
filed, including the type
of information expected.
Link to full field
description: link to
the DIF/SERF Writer's
Guide page that fully
describes the field,
including characters
allowed, definition,
format, etc.
Add/Remove field:
user can click the green
(+) to add a new field, or
the red (–) to remove that
field.
HTML docBUILDER
(Science Keywords Field)
Subset list selector
Variable level
filter: if checked,
removes all items
with Variable level
List search: filters
list as you type, keeping
only those items that
match.
Option to add
detailed variable:
allows user to add
detailed variables to
any selected items in
the list
HTML docBUILDER
(Personnel Field)
Personnel
Lookup link:
when clicked,
presents the user
with all personnel
currently in the
GCMD DB. After
the user selects a
person and clicks
the Accept button,
the information is
then populated in
the form on the
Personnel page.
HTML docBUILDER
(Data Center Field)
Data Center Lookup
link: allows the user to select
from a current list of data
centers, then populates the Data
Center form with this info,
including URL.
Keyword Subscription Service
• Receive notifications of updates, removals,
and additions to keywords for:
–
–
–
–
–
Data Centers/Service Providers
Platforms
Instruments
Locations
Projects
Search Refinements for SERF
• Refine by:
–
–
–
–
–
–
–
–
Service Providers
Instruments
Platforms
Projects
Science Keywords
Service Keywords
Parent SERFs
Child SERFs
Close-up of Refinement
Methods to
refine by
Users can also refine
by Parent and
Children SERFs
New Refinement for Data Services
2. Refine by
“Science
Parameters”
+
Free-Text
Search
1. Select
“Models”
266 Records
3. Select
Query Result #1
Metadata Record
2 Records
Refine by Temporal/Spatial Resolution
1. Refine By
“Temporal
Resolution
Range”
2. Select
“Daily - <
Weekly” &
result #1
Query Refinement by Data
Resolution
1. Query by:
 Platform AQUA
 Sensor MODIS
2. Refine by Data Resolutions
Query Refinement by Data
Resolution
3. Choose Refine by:
 Horizontal Resolution Range
 250 meters - < 1 km
4. Query Results
New Spatial Search using
• Enhanced capability for
performing spatial searches.
• Intuitive tools such as
directional arrows and zooming
assist in choosing a geographic
area on the map.
• Integration with keyword and
temporal search provides more
precise results.
Technology
Conducting a Spatial Search
Example showing a search for data sets in the
United Kingdom.
Conducting a Spatial Search Continued ……
Narrow spatial search using zoom tools
Conducting a Spatial Search Continued ……
Spatial Search Results
Map/Date Search – Google Earth
User
Performs
a Spatial,
Keyword,
& Temporal
Search
Legend
OAI-PMH Architecture
OAI-PMH
Repository
Computer running
harvester software
Why OAI-PMH?
OAI Server independent
of main database
II.
Administrator controls
metadata availability
Queries standardized
Harvesting Process
I.
Database
I.
II.
Metadata subset
transferred from a
database to a
repository
Metadata is
harvested from the
repository
Software is open source
Metadata Compatibility
•Dublin Core, FGDC,
DIF, ISO-19115
•XSLT or XML
Schema
GCMD OAI-PMH Capabilities
List Metadata Formats – Lists available metadata formats
http://gcmd.gsfc.nasa.gov/OAIPMH/oai.do?verb=ListMetadataFormats
List Sets – Lists all sets of metadata on server (Can be disabled)
http://gcmd.gsfc.nasa.gov/OAIPMH/oai.do?verb=ListSets
List Identifiers – Lists all document names in set.
http://gcmd.gsfc.nasa.gov/OAIPMH/oai.do?verb=ListIdentifiersmetadataPrefi
x=dif&set=NCAR
List Records – Lists all XML documents in set.
http://gcmd.gsfc.nasa.gov/OAIPMH/oai.do?verb=ListRecords&metadataPrefi
x=dif&set=NCAR
Identify – Lists information about the server
http://gcmd.gsfc.nasa.gov/OAIPMH/oai.do?verb=Identify
Get Record – Retrieve individual XML document
http://gcmd.gsfc.nasa.gov/OAIPMH/oai.do?verb=GetRecord&metadataPrefix
=dif&set=NCAR&identifier=GCMD_NCAR_DS123.1
GCMD OAI-PMH Capabilities
ctd.
OAI Repository Explorer (a tool for testing OAI-PMH servers)
baseURL – URL for the GCMD OAI-PMH Server
Verbs - list metadata attributes
and request metadata
Parameters – Refine by dates,
metadata formats, ID’s, metadata set
names
Validate – Authenticate metadata
against an XML Schema
Formatting – View metadata
as “parsed” or “raw” XML
“OAI Repository Explorer”
courtesy of U. of Cape Town
http://re.cs.uct.ac.za/
OAI/PMH (Open Archives Initiative-Protocol
for Metadata Harvesting) Capabilities
OAI-PMH
Repository
Identify
ListMetadataFormats (DIF,
FGDC, Dublin Core)
 ListIdentifiers
 Get Record
 ListRecords


GCMD
Metadata
Controlled
Vocabulary
Identifies
protocol;
supported
formats
Retrieves DIF
Metadata
identifiers,
records in XML
OAI-PMH
Harvester
Converting Metadata using XSLT Technology
1. Apply Mapping to an XSLT Stylesheet
Sample
ISO 19115
XML
document
Edit XSLT
Stylesheet
DIF XML
XSLT Editor
DIF HTML
2. Translate,
Format, and
Submit to the
GCMD
ISO19115 XML
Formatted DIF XML
Format
Translate
using
using
HTML
Xalan
Tidy
DIF vs. ISO 19115 Core Metadata
DIF Required Fields:
Entry ID
Entry Title
Science Keywords
*ISO Topic Category
Data Center
Summary
*Metadata Name
*Metadata Version
*Field added for ISO
19115 compliance
*Mandatory ISO fields
*Mandatory under
certain conditions or
optional ISO fields
DIF Recommended Fields:
Data Set Citation
-Dataset Title
-Dataset Release Date
Personnel (DIF Author)
Instrument
Platform
Temporal Coverage
Spatial Coverage
Location
Data Resolution
Project
Quality
Access Constraints
Use Constraints
Distribution
Related URL
DIF Revision History
DIF Optional Fields:
Keyword
Paleo-Temporal Coverage
Data Set Progress
Data Set Language
Originating Center
Multimedia Sample
Reference
Parent DIF
IDN Node
DIF Creation Date
Last DIF Revision Date
Future DIF Review Date
http://gcmd.nasa.gov/Aboutus/standards/
DIF vs. FGDC Core Metadata
DIF Required
Fields:
Entry ID
Entry Title
Science Keywords
ISO Topic Category
Data Center
Summary
Metadata Name
Metadata Version
* Mandatory FGDC
fields
• Mandatory under
certain conditions or
optional FGDC fields
DIF Recommended Fields:
Data Set Citation
Personnel (DIF Author)
Instrument
Platform
Temporal Coverage
Spatial Coverage
Location
Data Resolution
Project
Quality
Access Constraints
Use Constraints
Distribution
Related URL
DIF Revision History
DIF Optional Fields:
Keyword
Paleo-Temporal Coverage
Data Set Progress
Data Set Language
Originating Center
Multimedia Sample
Reference
Parent DIF
IDN Node
DIF Creation Date
Last DIF Revision Date
Future DIF Review Date
http://gcmd.nasa.gov/Aboutus/standards/
GCMD/IDN Achieves CMMI
Level 2
CMMI: Capability Maturity Model Integration
A set of best practices that address the development and
maintenance of products and services covering the product
life-cycle from conception through delivery and maintenance.
• CMMI certification is through the Software Engineering
Institute (SEI) of Carnegie Mellon University (CMU).
• There are 6 Capability levels. Level 2 is the “Managed” level.
• GCMD demonstrated capability in 7 Process Areas:
• Requirements Management
• Project Management
• Project Monitoring and Control
• Supplier Agreement Management
• Measurement and Analysis
• Process and Product Quality Assurance
• Configuration Management
After 5 months of intensive training and review, CMMI Level 2 was achieved
on February 24, 2006.
New DIF/SERF Changes
• Parameters: will be renamed to Science Keywords:
• Service Parameters: will be renamed to Service
Keywords:
• Keyword: will be renamed to Ancillary Keyword:
• Add field to Temporal Coverage for Relative Time:
Group: Temporal Coverage
Start_Date: [yyyy-mm-dd]
Stop_Date: [yyyy-mm-dd]
Relative_Start_Date: -x (where x= # days ago)
End_Group
Relative time permits:
– Auto-updating of temporal coverage for near-real time data.
– Specifying number of days data available.