Transcript Slide 1
A Next Generation AmeriFlux Network Data Server Deb Agarwal, Catharine van Ingen, and Susan Holladay Berkeley Water Center and ORNL Enabling cross-site carbon-climate synthesis depends on: • Data and site ancillary data submitted to network archives by tower investigators • Common processes for data quality assessment and gap filling now evolving • Collaborative analyses that build on single site investigations • Enabling data availability browsing and simple analyses Leads to dramatically increased data availability – 600+ site years from 250+ sites now available Requires help from investigators: no longer bypass common processes and the archives, apply their own corrections, lose data files, etc which can make comparing results problematic Ameriflux data server • Common data repository capturing provenance information and common data processing • Ancillary and biological data being incorporated • Data browsing, mining, and plotting capabilities • Data repository that is easy to maintain, load with data, and expand capabilities Reduce cut & paste manipulation of data Dramatically increase the scale of the data that can be analyzed Large Data Archives Ameriflux/Fluxnet Data Intersection Local measurements All data and site ancillary data are held in a relational (SQL) database • Available data from ORNL web archive automatically downloaded and ingested into the database • Site ancillary data updated infrequently by manual load Selected data and site ancillary data flow into data cubes (OLAP database) Client visualization and analysis tools query the database or data cube Reports and Excel Pivot Table and Chart CSV Files Fully normalized schema for both data and ancillary/biological data Built in versioning to handle data changes for cleaning ORNL Ameriflux Web Site What’s going on at higher latitudes? (It should be getting colder) 6000 55.86306 BOREAS NSA 1981 burn site 55.879002 BOREAS NSA Old Black Spruce 55.90583 BOREAS NSA 1930 burn site 55.911671 BOREAS NSA 1963 burn site 55.916672 BOREAS NSA 1989 burn site 56.63583 BOREAS NSA 1998 burn site 69.133331 AK Happy Valley 70.281471 AK Upad Measurements per Month 5000 4000 BWC SQL Server Database FLUXNET Data Site – http://www.fluxdata.org ~920 site years from ~240 sites •Summaries of data and availability information •Data download •Pivot tables •Collaboration services •Site PI contact support Data Cube 3000 2000 1000 70.496002 AK Atqasuk 0 1 2 3 4 5 6 7 8 9 10 11 12 Data is often missing in the winter! Data Cube User Support Available offset offsetid repeat repeat value time yydoy units datumtype quality •Tutorials - Wednesday and Thursday, 3-5pm, Evergreen Room •One-on-one - help available at this table during meeting breaks sitesetid has_L3 siteid has_L4 datumid summaryEuro value lowVariabilityEuro time spikeDetectEuro exdatumid USTcorrectEuro offsetid gapborderEuro repeatid PIcontrolEuro qualityid gapEuro createTime lastAppendTime lastModifyTime appendOnlyTime parentSitesetid creatorid name description shortname units name offsetunits ancdata intoCube dailyCalc repeatCalc siteCalc dailyConvert exdatumtype exdatumid sitesetid siteid datumid dailyUnits siteset deleteTime E-mail: [email protected] data qualityid fixTime debris site siteid intoCube description latitude longitude shortname name state country intocube LaThuID dataset datasetid createTime lastAppendTime lastModifyTime appendOnlyTime howmade fixTime path deleteTime dataset_siteset creatorid name datasetid description sitesetid howmade comment http://bwc.berkeley.edu/ times