Transcript ODM Tools

CUAHSI
HIS
Sharing hydrologic data
http://his.cuahsi.org/
Hydroinformatics Lecture:
HydroServer .NET/PHP
Daniel P. Ames and Jeffery S. Horsburgh
The CUAHSI HIS is
Supported by NSF
Grant# EAR 0622374
CUAHSI HIS
The CUAHSI Hydrologic Information System (HIS) is an internet based system
to support the sharing of hydrologic data. It is comprised of hydrologic
databases and servers connected through web services as well as software for
data publication, discovery and access.
HIS Central
Data Discovery and
Integration platform
HydroServer
Data Publication
platform
Like web
servers
Data Services
Water Data Services
Spatial Data Services
Like HTML
Like search portals
Google, Yahoo, Bing
HydroDesktop
Data Synthesis and
Research platform
Like browsers
HydroServer Goals
• A platform for organizing and publishing
space-time hydrologic datasets that:
– Provides local control of data
Put the data service as
close to the data
producer as possible.
– Makes data universally available
Share the data as
widely and openly as
possible.
– Is autonomous – e.g., functional
independent of the rest of HIS
Serve as a useful data
store regardless of
the fate of CUAHSI
HIS.
KEY HydroServer Functionality:
• Read ODM
• Share it with WebServices
Add on Visualization
Capabilities
Capabilities
Database
Configuration
Tool
WaterOneFlow
ODM
WaterOneFlow
Services
WaterOneFlow
ODM
WaterOneFlow
HydroServer
Capabilities
Database
Spatial
Services
Add on GIS Services
through ArcGIS Server
ODM
ODM Databases and Web Services
ArcGIS Server Spatial Data Services
HydroServer Key Concepts: ODM
Streamflow
Precipitation
& Climate
Water Quality
Groundwater
levels
Soil
moisture
data
Flux tower data
• A relational database at the single observation level
• Metadata for unambiguous interpretation
• Traceable heritage from raw measurements to usable
information
• Promote syntactic and semantic consistency
• Cross dimension retrieval and analysis
Horsburgh, J. S., D. G. Tarboton, D. R. Maidment, and I. Zaslavsky (2008), A relational model for environmental and water resources data,
Water Resources Research, 44, W05406, doi:10.1029/2007WR006392.
ODM 1.1.1
What are the basic attributes to be
associated with each single data value and
how can these best be organized?
DateTime
Interval (support)
“When” Time, T
Accuracy
t
Variable
Method
Quality Control Level
Sample Medium
Value Type
Data Type
Source/Organization
Units
A data value
vi (s,t)
s “Where”
Space, S
Vi
“What”
Variables, V
Censoring
Qualifying comments
Location
Feature of interest
Data Series – Metadata Description
Time
End Date Time, t2
There are C measurements
of Variable Vi at Site Sj from
time t1 to time t2
Count, C
Begin Date Time, t1
Variable, Vi
Variables
Site, Sj
Space
Series Catalog
Sj
Time
Vi
End Date Time, t2
Count, C
Begin Date Time, t1
Variable, Vi
t1
t2
C
Variables
Site, Sj
Space
ODM Controlled Vocabularies
HIS CV Website
ODM Data
Manager
ODM
Tools
Local ODM
Database
XML
Local
HydroServer
ODM Controlled
Vocabulary Moderator
ODM
Controlled
Vocabulary
Web Services
Master ODM
Controlled
Vocabulary
How do we get data out?
MOST IMPORTANT WAY:
Using WaterML and WaterOneFlow
• Set of query functions
• Returns data in WaterML
WaterML is an XML schema for communicating water data
WaterOneFlow is a set of web services based on WaterML
Let’s look at an example…
http://river.sdsc.edu/wateroneflow/NWIS/UnitValues.asmx
Slide from David Valentine
For example: to get a site, send this “SOAP” request
to the server:
Specify the site and an optional
authorization token
Get this response:
Result returns in this string.
2 Flavors of HydroServer:
HydroServer (.NET)
• HydroServer (.NET):
• http://hydroserver.codeplex.com
• Model implementation of the HydroServer CUAHSI HIS
specification
• Runs on Windows servers
• Requires Microsoft SQL Server
• Uses ArcGIS Server
• Uses Streaming Data Loader, Data Loader and SQL tools to
manage the database.
Loading Data into ODM
ODM Data Loader
• Interactive ODM Data Loader
– Loads data from spreadsheets and
comma separated tables in simple
format
• Streaming Data Loader (SDL)
ODM SDL
– Loads data from datalogger files on
a prescribed schedule
– Interactive configuration
• SQL Server Integration Services
(SSIS)
– Microsoft application accompanying
SQL Server useful for programming
complex loading or data
management functions
SSIS
Editing Data in HydroServer
using “ODM Tools”
• Query and export –
export data series
and metadata
• Visualize – plot and
summarize data
series
• Edit – delete,
modify, adjust,
interpolate, average,
etc.
HydroServer Web Data Loader
The general format for input to the
ODMWDL is a single file containing a
table with a one row header that
uses ODM field names in the
header, followed by the data in
subsequent rows.
Where do we get these tools?
ODM Tools, Streaming Data Loader, etc. can all be downloaded
from the HydroServer web site:
Web Data Loader…
ODM Tools
Data Loader
Streaming Data Loader
Publication of
Geospatial (GIS)
Datasets
• Publishing spatial
datasets using ArcGIS
Server
– Using OGC standards that
can be consumed by a
number of GIS clients
– WMS, WFS, WCS
Data Presentation Via a Map Interface
• Internet Map Server
built using ArcGIS
• Web browser client
• Combine spatial data
and observational
data
• Launch data
visualization tools
• Based on a “Region”
http://maps.usu.edu/map/
Data Preview, Visualization, and Analysis
Time Series Analyst
• Web Browser Client
• Multiple ODM Database
Support
• Variety of plot types
• Descriptive statistics
• Linked to the map
application
• Data preview and
download
Let’s try it! http://icewater.usu.edu/tsa/
HydroServer
Website
• HydroServer home
page
• Observational data
services
• GIS data services
• Online map
• Time Series Analyst
• Data Query and
Download
http://icewater.usu.edu
HydroServer Capabilities Database
Ties together Map Services and HIS Services
HydroServer Capabilities Database Configuration Tool
• Simple Windows application for creating metadata records in your
database
• Associating Regions with services and maps
HydroServer Capabilities Web Service
• Publish capabilities of each
HydroServer
– Listing of published
observational data services
– Listing of published spatial
data services
• Supports automatic cataloging
of available services at HIS
Central
• Makes HydroServers self
describing
Catalog
Server
Data
Services
Desktop
How do I create a HydroServer?
1. Get a Windows Server Machine with IIS and .Net
Framework
2. Install Microsoft SQL Server
3. Install HydroServer Software from his.cuahsi.org
–
–
–
ODM and Utilities
WaterOneFlow web services
HydroServer Website, TSA, Map, Capabilities
4. Install ArcGIS
5. Create data services and document them in the
capabilities database
6. Register your services with HIS Central and register
your variables with the HIS Ontology so your data will
be searchable
HydroServer System Requirements
• Required Hardware (Generally less than $5000)
–
–
–
–
–
Can be one or more physical or virtual machine(s)
Dual or quad-core processor at 2+ GHz
> 4 GB of RAM
> 500 GB of hard drive space
Network adapter
• Required Commercial Software
– Microsoft Windows 2003 or 2008 Server (preferred)
• IIS Web Server
• .NET Framework Version 3.5
– Microsoft SQL Server 2005 or 2008 (preferred)
• Any edition – Express (free), Standard, Enterprise
– ESRI ArcGIS Server 9.3.1
2 Flavors of HydroServer:
HydroServer PHP (HydroServer Lite)
• HydroServer Lite:
• http://hydroserverlite.codeplex.com
• Simplified implementation of the CUAHSI HIS
specification (all required services are supported).
• Runs on the free LAMP stack (Linux, Apache, MySQL,
PHP)
• Uses front end web pages to load data in the database.
What is HydroServer Lite?
• Lots of the same ideas, but different
implementation
• PHP and MySQL mean easy/fast/cheap setup
• Focus on easy extends to user interfaces.
• Developer community at
http://hydroserverlite.codeplex.com/
• Let’s try it out: http://worldwater.byu.edu/
• Browse data from any database, upload data
using the “sandbox” database.
Can Citizen Science Data be Shared and
Intermingled with “Real Data”?
(Custom Banner…)
(Map Based Interface…)
(Interactive Graphing Tool…)
(Interactive Graphing Tool…)
HydroServers
for the Developing World
• http://worldwater.byu.edu
• Custom PHP-based Catalog for Developing
Countries
Internationalization Effort
CUAHSI
HIS
Sharing hydrologic data
Build your own HydroServer Lite instance
by following the instructions at :
http://hydroserverlite.codeplex.com/
Support
EAR 0622374