Transcript Summary of Progress on CUAHSI HIS, 28 July 2004

CUAHSI Hydrologic Information System
Status Review, July 28, 2004
Agenda
• Review the work of the five project partners
–
–
–
–
–
CUAHSI (Rick Hooper, Jon Duncan)
San Diego Supercomputer Center (John Helly, ….)
University of Texas (David Maidment, …)
University of Illinois (Praveen Kumar ….)
Drexel University (Michael Piasecki…)
• Involving the collaborators: V. Lakshmi, X. Liang,
Y. Liang, U. Lall, L. Poff, K. Reckhow, D.
Tarboton, I. Zaslavsky, C. Zheng
• HIS review meetings
– SDSC (August 12-13) – technical detail
– Logan (August 23) – user needs assessment
Agenda
• Review the work of the five project
partners
– CUAHSI (Rick Hooper, Jon Duncan) –
meeting with NSF today
– San Diego Supercomputer Center (John
Helly, ….)
– University of Texas (David Maidment, …)
– University of Illinois (Praveen Kumar,...)
– Drexel University (Michael Piasecki,…)
Agenda
• Review the work of the five project
partners
– CUAHSI (Rick Hooper, Jon Duncan) – Neuse
HO report status
– San Diego Supercomputer Center (John
Helly, ….)
– University of Texas (David Maidment, …)
– University of Illinois (Praveen Kumar ….)
– Drexel University (Michael Piasecki…)
Agenda
• Review the work of the five project
partners
– CUAHSI (Rick Hooper, Jon Duncan) – Neuse
HO report status
– San Diego Supercomputer Center (John
Helly, ….)
– University of Texas (David Maidment, …)
– University of Illinois (Praveen Kumar ….)
– Drexel University (Michael Piasecki…)
UT Update
•
•
•
•
General issues
Landscape characterization for HO Design
Flux algebra for surface water systems
XML for interchange of groundwater
objects
Science Tools Corporation
• http://sciencetools.com/
• work with NASA and other
institutions integrating
databases for scientific
purposes
• Chief Scientist is Richard
Troy – he wants to explore
potential of working with
CUAHSI
• Commercial system that
operates over Oracle,
SQL/Server,…
• Company is based in
Oakland, CA
GenScn
• A tool for generation and
analysis of model
simulation scenarios for
watersheds
• Incorporated in EPA
Basins system
• Produced by AquaTerra in
Decatur, GA
• Handles lots of different
time series types
Suwannee River Watershed Data
• Contact from Wendy
Graham (former ViceChair of HIS
Committee)
• Offering data for
consideration in HIS
data model
• How to discuss this in
Logan?
UT Update
•
•
•
•
General issues
Landscape characterization for HO Design
Flux algebra for surface water systems
XML for interchange of groundwater
objects
Landscape Characterization for HO
Design
• Idea suggested by Larry Band at the end
of our call on July 14
– have a set of rules for defining subdivisions of
the landscape using orders of magnitude of
catchment size, type of land use, etc
– need to work with LIDAR as well as regular
DEM
– define points at outlets of these catchments
as potential gage sites
EDNA-Elevation Derivatives for
National Application
Pfaffstetter Basins
9 basins divided into 99 basins divided into 999 basins
Email from Larry Band
In this case the emphasisis on first retrieving all catchments of a certain size (or range of
> sizes) developed by specifying threshold areas or perhaps other
> criteria for identifying first order catchments. Likely this would
> actually be area as we are less interested in knowing precisely where
> channelized flow begins as identifying characteristics of catchments
> of specified drainage areas. In response to a set of scientific
> questions or hypotheses dealing with scaling issues, we may specify we
> need to gauge X streams for each of 5 orders of magnitude of drainage
> area that satisfy a set of selection criteria. You're correct that
> the procedure would select the set of candidate sites, from which a
> final set would need to be chosen.
>
> The rules can be quite simple, such as 1. all catchments with > 20%
> impervious cover (assuming we have an impervious surface layer), or
> more complex such as 2. all catchments with > 80% forest in a
> specified riparian buffer. It could also include topographic
> characteristics including extent and development of floodplain using
> some of the indices John Gallant has recently introduced. You're also
> correct that this can be a complex problem specifically with lidar
> data due to the size of the dataset and also the lack of elevation
> data in open water, and the potential apparent drainage disruption due
> to infrastructure. My impression is that most or at least many
> hydrologists who can carry out this type of activity with standard USGS DEM would have difficulty
> handling the lidar data. Most software packages cannot handle the
> data volumes. I raised this as a question regarding whether this
> would be an efficient use of the HIS groups time and abilities, or
> whether this is too specific an application and should be left to the
> individual HO. Their ability to handle these and similar problems may
> be a good attribute to consider in the proposals.
UT Update
•
•
•
•
General issues
Landscape characterization for HO Design
Flux algebra for surface water systems
XML for interchange of groundwater
objects
Mass Balancing
A South Florida Basin
Flow In
Rain
ET
Flow Out
Flow Out
What volume of water is stored within this basin?
Process
Select Time Series
Related to Basin
Horizontal
Inflow
Horizontal
Outflow
#1
Vertical
Inflow
Vertical
Outflow
Hydrologic Data
Model
Cumulative
Horizontal
Storage
Cumulative
Vertical
Storage
Cumulative
Total
Storage
Horizontal
Outflow
#2
Add to
Calculate a
Net Flow
Integrate
To Calculate
A Storage
Net
Horizontal
Inflow
Net
Vertical
Inflow
Net
Total
Inflow
Daily Averaged Vertical Fluxes
4.00
3.50
2.50
2.00
1.50
1.00
0.50
0.00
Date
Evapotranspiration
Rainfall
10/1/03
9/1/03
8/1/03
7/1/03
6/1/03
5/1/03
4/1/03
3/1/03
2/1/03
1/1/03
12/1/02
-0.50
11/1/02
Vertical Flux [in/day]
3.00
Daily Averaged Horizontal Flow Rates
4.00E+08
3.00E+08
1.00E+08
0.00E+00
-1.00E+08
-2.00E+08
-3.00E+08
Date
10/1/03
9/1/03
8/1/03
7/1/03
6/1/03
5/1/03
4/1/03
3/1/03
2/1/03
1/1/03
12/1/02
-4.00E+08
11/1/02
Flowrate [ft3/day]
2.00E+08
Daily Averaged Net Flow Rates
6.00E+03
4.00E+03
3.00E+03
2.00E+03
1.00E+03
0.00E+00
-1.00E+03
11
/1
/2
00
2
12
/1
/2
00
2
1/
1/
20
03
2/
1/
20
03
3/
1/
20
03
4/
1/
20
03
5/
1/
20
03
6/
1/
20
03
7/
1/
20
03
8/
1/
20
03
9/
1/
20
03
10
/1
/2
00
3
Daily Averaged Net Flowrate [cfs]
5.00E+03
Date
Net Horizontal Flow (cfs)
Net Vertical Flow (cfs)
Cumulative Storage Since Nov. 1, 2001
2.50E+09
2.00E+09
Volume (ft3)
1.50E+09
1.00E+09
5.00E+08
0.00E+00
-5.00E+08
-1.00E+09
-1.50E+09
1-Sep- 21-Oct- 10-Dec- 29-Jan- 20-Mar- 9-May- 28-Jun- 17-Aug- 6-Oct- 25-Nov02
02
02
03
03
03
03
03
03
03
Date
Horizontal Net Inflow
Vertical Net Inflow
Total Net Inflow
Complications to Process
Extracting time series
Need ability to query a large database to extract
relevant time series for one or more discrete
watersheds
Dimensions Conversions
Need spatial and temporal integration
Units Conversions
Need unit conversions
Discrete-Continuous Time
Need a spatiotemporal referencing system (TGIS)
UT Update
•
•
•
•
General issues
Landscape characterization for HO Design
Flux algebra for surface water systems
XML for interchange of groundwater
objects
Creating a 3D model of the subsurface
Stratigraphy from the North Carolina database (tabular), imported into ArcGIS
Importing borehole data to GMS
Data is imported from GIS into GMS (Groundwater Modeling System)
Solid model
Solids are generated in GMS using the Horizons method
Contacts are assigned horizons (from bottom to top) and then solids
are created by interpolating a surface for each horizon extruding
downward.
Solid model in GIS
The solid model is read back into ArcGIS through an XML file
Transfer of the solid model via XML
•
Solids can be represented as a set of vertices and triangles
•
Each vertex has a x, y, and z coordinates
•
Each triangle is constructed of three vertices
Storing solids in an XML file
Solids represented as a set of vertices and triangles
Vertices
Triangles
Full process
Stratigraphy information in a
spatial database
Back to spatial database
Interpolation in external software
(for example GMS)
Store solids in XML
Agenda
• Review the work of the five project
partners
– CUAHSI (Rick Hooper, Jon Duncan) – Neuse
HO report status
– San Diego Supercomputer Center (John
Helly, ….)
– University of Texas (David Maidment, …)
– University of Illinois (Praveen Kumar ….) –
Praveen is overseas…
– Drexel University (Michael Piasecki…)
The Modelshed Framework
Update July 28, 04
What is a Modelshed?
•
A volumetric spatial (GeoVolume?) model unit, registered in three
dimensions by a GIS, with which time-varying data, model fluxes, spatial
relationships and descriptive metadata are associated
What can the Modelshed
Framework do?
•
•
•
•
Store data for diverse spatio-temporal applications & phenomena
A generalized 4D data model for environmental science
Addresses issues of scale, heterogeneity, and resolution
Build on top of existing data models (e.g. ArcHydro) to leverage
existing data structures and tools
• Establish new relationships
• Models environmental fluxes
• Connects raster data and numerical models with object-relational
data models
Modelshed UML
Hydrography::HydroFeature
-HydroID : esriFieldTypeInteger
-HydroCode : esriFieldTypeString
ModelshedType
-ModelshedTypeID : esriFieldTypeInteger
-ModelshedClass : ModelshedClass
-Description : esriFieldTypeString
Modelshed
-ModelshedTypeID : esriFieldTypeInteger
1
*
StatisticalTS
-ModelShedTypeID : esriFieldTypeInteger
-FeatureID : esriFieldTypeInteger
-TSTypeID : esriFieldTypeInteger
-ZLayerID : esriFieldTypeInteger
-TSDateTime : esriFieldTypeDate
-TS_MEAN : esriFieldTypeDouble
-TS_MEDIAN : esriFieldTypeDouble
-TS_MAJORITY : esriFieldTypeDouble
-TS_COUNT : esriFieldTypeDouble
-TS_MIN : esriFieldTypeDouble
-TS_MAX : esriFieldTypeDouble
-TS_STD : esriFieldTypeDouble
-TS_SUM : esriFieldTypeDouble
-TS_SKEWNESS : esriFieldTypeDouble
-TS_KURTOSIS : esriFieldTypeDouble
-TS_ERROR : esriFieldTypeDouble = 0.0
1
Modelshed::ModelPoint
*
1
Modelshed::ModelLine
1
Modelshed::ModelArea
Timeseries UML
TSType
TimeSeries
-FeatureID : esriFieldTypeInteger
-TSTypeID : esriFieldTypeInteger
-TSDateTime : esriFieldTypeDate
-TSValue : esriFieldTypeDouble
*
*
TSTypeHasTimeSeries
1
1
-TSTypeID : esriFieldTypeInteger
-Variable : esriFieldTypeString
-Units : esriFieldTypeString
-IsRegular : AHBoolean
-TSInterval : TSIntervalType
-DataType : TSDataType
-Origin : TSOrigins
TSTypeHasStatisticalTS
StatisticalTS
-ModelShedTypeID : esriFieldTypeInteger
-FeatureID : esriFieldTypeInteger
-TSTypeID : esriFieldTypeInteger
-ZLayerID : esriFieldTypeInteger
-TSDateTime : esriFieldTypeDate
-TS_MEAN : esriFieldTypeDouble
-TS_MEDIAN : esriFieldTypeDouble
-TS_MAJORITY : esriFieldTypeDouble
-TS_COUNT : esriFieldTypeDouble
-TS_MIN : esriFieldTypeDouble
-TS_MAX : esriFieldTypeDouble
-TS_STD : esriFieldTypeDouble
-TS_SUM : esriFieldTypeDouble
-TS_SKEWNESS : esriFieldTypeDouble
-TS_KURTOSIS : esriFieldTypeDouble
-TS_ERROR : esriFieldTypeDouble = 0.0
ModelshedType
*
1
-ModelshedTypeID : esriFieldTypeInteger
-ModelshedClass : ModelshedClass
-Description : esriFieldTypeString
*
1
ZLayer
-ZLayerID : esriFieldTypeInteger
-AltitudeUnits : esriFieldTypeString
-AltitudeDatum : esriFieldTypeString
-Description : esriFieldTypeString
-LayerBottomAltitude : esriFieldTypeDouble
-LayerTopAltitude : esriFieldTypeDouble
-ZLayerAboveID : esriFieldTypeInteger
-ZLayerBelowID : esriFieldTypeInteger
Flux UML
FluxType
Modelshed::ModelArea
Modelshed::ModelLine
Modelshed::ModelPoint
-FluxTypeID : esriFieldTypeInteger
-Description : esriFieldTypeString
1
1
1
1
FluxRecord
-FluxLinkID : esriFieldTypeInteger
-DateTime : esriFieldTypeDate
-Value : esriFieldTypeDouble
*
*
1
FluxLink
1
ModelshedType
-ModelshedTypeID : esriFieldTypeInteger
-ModelshedClass : ModelshedClass
-Description : esriFieldTypeString
1
*
*
ZLayer
-ZLayerID : esriFieldTypeInteger
-AltitudeUnits : esriFieldTypeString
-AltitudeDatum : esriFieldTypeString
-Description : esriFieldTypeString
-LayerBottomAltitude : esriFieldTypeDouble
-LayerTopAltitude : esriFieldTypeDouble
-ZLayerAboveID : esriFieldTypeInteger
-ZLayerBelowID : esriFieldTypeInteger
1
-FluxTypeID : esriFieldTypeInteger
-FluxLinkID : esriFieldTypeInteger
-FromFeatureID : esriFieldTypeInteger
-ToFeatureID : esriFieldTypeInteger
-FromZLayerID : esriFieldTypeInteger
-ToZLayerID : esriFieldTypeInteger
-FromModelShedTypeID : esriFieldTypeInteger
-ToModelShedTypeID : esriFieldTypeInteger
-TSTypeID : esriFieldTypeInteger
ArcHydro Native
Implementation Classes
AreaLink UML
1
AreaLink
Modelshed::ModelArea
*
-Area1FeatureID : esriFieldTypeInteger
-Area2FeatureID : esriFieldTypeInteger
-ModelshedClass1 : ModelshedClass
-ModelshedClass2 : ModelshedClass
-FractionOf1In2 : esriFieldTypeDouble
*
1
ModelshedType
-ModelshedTypeID : esriFieldTypeInteger
-ModelshedClass : ModelshedClass
-Description : esriFieldTypeString
ArcHydro Native
Implementation Classes
OrthogonalLink UML
Modelshed::ModelArea
1
1
OrthogonalLink
-ModelShedTypeID : esriFieldTypeInteger
-FeatureID : esriFieldTypeInteger
-posYFeatureID : esriFieldTypeInteger
-posXFeatureID : esriFieldTypeInteger
-negYFeatureID : esriFieldTypeInteger
-negXFeatureID : esriFieldTypeInteger
-posXposYFeatureID : esriFieldTypeInteger
-posXnegYFeatureID : esriFieldTypeInteger
-negXnegYFeatureID : esriFieldTypeInteger
-negXposYFeatureID : esriFieldTypeInteger
*
1
ModelshedType
-ModelshedTypeID : esriFieldTypeInteger
-ModelshedClass : ModelshedClass
-Description : esriFieldTypeString
Applications: Helping Raster & Vector Talk
• How can continuous data in rasters
be related to database objects?
– Summarize the data using statistics,
aggregated by overlapping Modelshed
areas
– Statistics are stored as indexed data
records
– Modelsheds can be physically
meaningful, like watersheds
– This process can be automated for a
large number of rasters
Applications: Helping Raster & Vector Talk
Applications: Automating data management
with the Modelshed Tools
• The ModelShed Tools automate some database tasks:
– Adding new descriptive indexes
– Building the index of raster datasets
– Automatically processing a timeseries of raster datasets based
on areas in the database, and ingesting the statistical data into
the database
– Building AreaLink tables
• ModelShed Tools are an extension to ArcGIS 8, and use
ArcGIS Spatial Analyst geoprocessing routines
Dynamic Features
• Supports database features that move and
change in time
• The full range of Modelshed features are
still supported, including vertical indexing,
flux links, and area links.
• A parallel UML structure for static and
dynamic features
Dynamic Features in Time
t1
t2
t3
t4
Hydrography::HydroFeature
Modelshed::Modelshed
-HydroID
-HydroCode
Objects::AreaLink
Objects::DynamicFeatureIndex
-Area1FeatureID
-Area2FeatureID
-ModelshedTypeID1
-ModelshedTypeID2
-FractionOf1In2
DynamicFeature
-DynamicFeatureID
-Description
-ModelshedTypeID
-DynamicFeatureID
-ZLayerID
-TSDateTime
*
1
Objects::StatisticalTS
1
DynamicArea
{GeometryType = esriGeometryPolygon}
1
Objects::FluxLink
-FluxTypeID
-FluxLinkID
-FromFeatureID
-ToFeatureID
-FromZLayerID
-ToZLayerID
-FromModelshedTypeID
-ToModelshedTypeID
*
1
*
1
DynamicLine
{GeometryType = esriGeometryPolyline}
*
*
1
1
DynamicPoint
{GeometryType = esriGeometryPoint}
*
1
-ModelshedTypeID
-FeatureID
-TSTypeID
-ZLayerID
-TSDateTime
-TS_MEAN
-TS_MEDIAN
-TS_MAJORITY
-TS_COUNT
-TS_MIN
-TS_MAX
-TS_STD
-TS_SUM
-TS_SKEWNESS
-TS_KURTOSIS
-TS_ERROR = 0.0
Applications 2: ILRDB
• A prototype geodatabase of the Illinois River
Basin using the Modelshed geodata model
• Combining base hydrography from the NHD /
ArcHydroUSA database with supercomputergenerated regional climate data, remote sensing
data, land use data, and multi-layer soils data
• A proof of concept for study using a much more
extensive multi-disciplinary integrated database
Illinois River Basin
Database (ILRDB)
UPPER FOX
DES PLAINES
CHICAGO
LOWER FOX
KANKAKEE
UPPER ILLINOIS
LOWER ILLINOIS-SENACHWINE LAKE
VERMILION
SPOON
MACKINAW
LOWER ILLINOIS-LAKE CHAUTAUQUA
LA MOINE
SALT
UPPER SANGAMON
LOWER SANGAMON
LOWER ILLINOISSOUTH FORK SANGAMON
MACOUPIN
IROQUOIS
Studying the relationships between large-scale
phenomena and hydrology using the ILRDB
• Climate simulation precipitation and
humidity data is modeled along with NDVI
vegetation and surface hydrology
• Query-based analysis is used to analyze
the relationships between these datasets
3.5
3
ratio to series mean
2.5
2
1.5
1
0.5
0
1
2
3
4
5
6
7
8
9
10
month
monthly average streamflow in the Illinois River at Valley City, IL
climate simulation precipitation
normalized difference vegetation index
moisture flux
11
12
Agenda
• Review the work of the five project
partners
– CUAHSI (Rick Hooper, Jon Duncan) – Neuse
HO report status
– San Diego Supercomputer Center (John
Helly, ….)
– University of Texas (David Maidment, …)
– University of Illinois (Praveen Kumar ….) –
Praveen is overseas…
– Drexel University (Michael Piasecki…)
Drexel Progress
CUAHSI
July 28 2004
Controlled Vocabulary for the Neuse River Basin
ONTOLOGIES
Stream Gauges
Datums
Site Types
Counties
Agencies
Soil Types *
Municipal Wells *
Units *
http://loki.cae.drexel.edu/~how/cua
hsi/2004/07/neuse-station.owl
* In progress
MTF and MIF files based on ISO-19115
MTF Available in the Web : Version 01 based on ISO:19115
http://loki.cae.drexel.edu/~how/cuahsi/2004/07/cuahsi_v01.mtf
Example for municipal wells… end of this week. Controlled
vocabularies will be used to annotate the values
Agenda
• Review the work of the five project partners
–
–
–
–
–
CUAHSI (Rick Hooper, Jon Duncan)
San Diego Supercomputer Center (John Helly, ….)
University of Texas (David Maidment, …)
University of Illinois (Praveen Kumar ….)
Drexel University (Michael Piasecki…)
• Involving the collaborators: V. Lakshmi, X. Liang,
Y. Liang, U. Lall, L. Poff, K. Reckhow, D.
Tarboton, I. Zaslavsky, C. Zheng
• HIS review meetings
– SDSC (August 12-13) – technical detail
– Logan (August 24) – user needs assessment
Agenda
Involving the collaborators: V. Lakshmi, X. Liang, Y.
Liang, U. Lall, L. Poff, K. Reckhow, D. Tarboton, I.
Zaslavsky, C. Zheng
• Development of concept papers for particular
areas of HIS
– LeRoy Poff: an assessment of needs for an HIS to
support aquatic ecology……
– Manu Lall: a survey of methodology for statistical spacetime interpretation of data……
– We have a draft of a written scopes for Manu’s paper
and it has been reviewed by the group
Agenda
• HIS review meetings
– SDSC (Thursday, Friday, August 12-13) – technical
detail on HIS development especially on metadata
definition http://cuahsi.sdsc.edu/
– Logan (Monday, August 23) – Status report on HIS
project and assessment of user needs for HIS in the
hydrologic observatories
http://www.usu.edu/water/cuahsi
• CUAHSI will provide travel funds for HIS project
PI’s and collaborators to travel to one of these
meetings