Transcript ODM
Publication Alternatives: Hybrid Data Services Table Views Presented by Tim Whiteaker The University of Texas at Austin 4 June, 2009 ODM Water Data Service Texas Parks and Wildlife Dept Texas Water Development Board Texas Instream Flow Program Ingest Data From Different Sources Transform Data into Uniform Format with ODM Data Loader WaterML Observations Data Model (ODM) TPWD ODM Load Newly Formatted Data into ODM Tables in MS SQL/Server TWDB ODM TIFP ODM Wrap ODM with WaterML Web Services for Online Publication Hybrid Water Data Service TCOON Metadata are Transferred from XML to the ODM WaterML TCOON METADATA ODM Metadata From: ODM Database in Austin GetSites GetSiteInfo GetVariableInfo TCOON Water Data Service TCOON DataValues Get Values from: TCOON Web Site in Corpus Christi http://his.crwr.utexas.edu/tcoonts/tcoon.asmx?WSDL Web Services can both Query the ODM for Metadata and use a Web Scraper for Data Values Calling the WSDL Returns Metadata and Data Values as if from the same Database Hybrid Data Service Architecture GetSites GetSites GetSiteInfo GetSiteInfo GetVariableInfo GetVariableInfo GetValues GetValues Remote Data Source ODM Web Service Hybrid Water Data Service METADATA ODM TCOON Example Use Data Query to get data Form query, click button, get data… …after completing this brief survey Data returned as text with header Unique URL Header + Values Deciphering the URL http://lighthouse.tamucc.edu/pd?stnlist=014&serlist=pwl&when=07/23/2008-07/30/2008 &whentz=UTC0&-action=c&unit=metric&elev=stnd&interval=360 Deciphering the URL http://lighthouse.tamucc.edu/pd? stnlist=014 &serlist=pwl &when=07/23/2008-07/30/2008 &whentz=UTC0 &-action=c &unit=metric &elev=stnd &interval=360 Steps to Building the Service • Harvest Metadata into ODM • Wrap with WaterOneFlow (private) • Write Service to handle GetValues – Metadata from private service – Time series from TCOON website (via URL) TCOON METADATA ODM TCOON DataValues Programming the Web Service Web Service with ODM Metadata WaterOneFlow Helper (WaterOneFlow helper is included with the ODM web service) Metadata from Private Service GetSites in 3 lines of code!! Piggyback off your ODM service Populating the TimeSeries Object The work is in GetValues Building the URL • Use Site, Variable, and Dates to make URL • Default any other parameters Retrieving Values from the Website Build the URL Scrape the website Parsing the Website Response Process each line of text Assign to TimeSeries Object Returning the TimeSeries Object TCOON Data Service Architecture GetSites GetSites GetSiteInfo GetSiteInfo GetVariableInfo GetVariableInfo GetValues GetValues TCOON Website TCOON Metadata Web Service TCOON Hybrid Water Data Service METADATA ODM Summary – Hybrid Data Services • All methods except GetValues simply call the metadata service on ODM • GetValues reads remote website on the fly • Handy for dynamic data sources WaterML TCOON METADATA ODM TCOON Water Data Service TCOON DataValues Table Views Wear the ODM Hat ODM View WaterOneFlow Web Service Your Database Example: The ‘NotODM’ Database Field names different Table names different Creating the View Change field names Result Saving the View Use an ODM Table Name WaterOneFlow Can Read Views ODM View WaterOneFlow Web Service Not ODM No Programming Required! When to Use Publication Alternatives • Hybrid Service – You can get site locations and variable info, but do not wish to populate your database with time series values • Table View – You already have a SQL Server database that maps fairly easily to ODM, and you see no need to duplicate the data by loading into a separate ODM database