Transcript Schmidt-Data Sharing Guide-ORAFS.ppsx

Data Sharing Across Agencies:
A guide to getting on the same page
Bruce Schmidt
Pacific States Marine Fisheries Commission
Presentation to
Oregon Chapter, AFS
February, 2010
www.streamnet.org
Increasing Wide-scale Programs
Coordinated
Monitoring
ESA
Recovery
Planning
High Level
Indicators,
SOTR
Population
Assessments
(VSP)
Cross
Need
Jurisdictions
Shared
Data
BiOp
Compliance
Inter-jurisdictional
Management
Comanagement
Legal: Remand,
US v. OR, PST,
etc.
Benefits of Sharing Data
 Fosters collaboration
 Expand scope of analyses (ESU, DPS)
 Contributes to multi-agency processes (e.g.,
ESA analyses, recovery planning and
list/delist decisions; co-management; etc.)
 Could distribute cost of data collection
Talk of a Comprehensive Regional
Data Delivery System…
2002
Council Contract
with SAIC
2000 BiOp
RPA 198
So why isn’t there one?
NOAA
Guidance
CBCIS
/ NED
2006
Col. Basin
Data Center
2008 BiOp
RPA 72
Fish and Wildlife
Program
Impediments to Sharing Data
 Sampling is focused
 Do things differently
 Data often not
consolidated internally
 Data not on Web
 Hard for others to understand, use the data
 Concerns over misuse, first use, etc.
 Cost
Current Data Sharing Status:
 Individual agencies (States, Tribes, Federal)






Independent
Different mandates
Sample different
environments
Longstanding
approaches
No mandate to share data
Little support, capability to share data
Current Data Sharing Status:
 Wide scale database projects


Focused – specific data types
Not comprehensive
Current Data Sharing Status:
 Proposed regional data delivery applications



Comprehensive
Never built, or
not completed
Mechanisms to supply
data not in place
Ideal Distributed Network
Output
Tool
Regional
Data
Access
Standards
Standards
Fish
db
db
db
Standards
Habitat
db
db
db
db
Water
db
db
db
db
…
Others
db
Multiple types of data from multiple sources, standardized by data type
But,Use
Current
Reality
is more
like this:
Regional
Data
Projects
Output
Tool
Regional
Data
Access
StreamNet
Fish
db
Habitat
db
db
db
db
db
db
Pacific Northwest Water
Quality Data Exchange
Water
db
db
db
db
db
db
…
Others
dd
db
db
Multiple types of data from multiple sources, standardized by data type
Consolidated v. Individual Offices Approach
Database
Project
2 6 9
5 3 1
8 7 4
Agency
Database
2 6 9
5 3 1
8 7 4
2 6 9
5 3 1
8 7 4
Data sources: field offices, research projects, etc.
Most Expedient
Output
Tool
Regional
Data
Access
Fish
db
Habitat
db
db
db
db
db
db
Water
db
db
db
db
db
…
Others
dd
db
db
Multiple types of data from multiple sources, standardized by data type
The Essential Driver of a
Comprehensive Data Delivery System
Output
Tool
Data flow can be
automated
Fish
db
Habitat
db
db
Regional
Data
Access
db
db
db
db
Water
db
db
db
db
db
…
Others
dd
db
db
Multiple types of data from multiple sources, standardized by data type
Automation is the key!
 Efficiency, Speed
 Accuracy
 Automatic data updates
 Canned products
 Translation to regional format
 Essential for any data portal technology
The ultimate endpoint??
Water
db
db
db
Habitat
db
db
db
db
Fish
db
db
db
…
Others
db
db
Multiple types of data from multiple sources, standardized by data type
Considerations for Regional Data
Collection, Sharing and Exchange
‘The Data Sharing Guide’
A White Paper to:

Outline steps

Avoid overlooking critical steps

Describe roles entities play

Break it down: it’s not really that
difficult
ftp://ftp.streamnet.org/pub/streamnet/projman_files/Data_Sharing_Guide_2009-06-01.pdf
How Does the Data Sharing Guide Help?
 Organize discussions
 Consider ALL
components
 Provide a blueprint
 Identify needed
support
Assure that when a system is built,
the data are there to deliver!
The Data Sharing Guide addresses data flow
from field to highest reporting need
Non prescriptive
Any data type
Any agency
Non-technical
Primary focus is on infrastructure and
processes to assure data accessibility
What’s Needed?
1. Uninterrupted data flow, source to output
Steps in Data Flow
Regional data
application
GOAL
Data interoperability
Regional standards
Post data on the Web
Post metadata on the Web
Describe full data set (metadata)
Agency data QA / QC
Agency database system
Local use of data
Describe the data (metadata)
Quality Assurance / Quality Control
Input to electronic form
Field data creation (local office/project)
Steps in Data Flow
Regional data
application
GOAL
Data interoperability
Regional standards
Post data on the Web
Post metadata on the Web
Describe full data set (metadata)
Agency data QA / QC
Agency database system
Local use of data
Describe the data (metadata)
Quality Assurance / Quality Control
Input to electronic form
Field data creation (local office/project)
Steps in Data Flow
Any missed step can
prevent data flow!
Significant effort to
bridge the gaps!
Regional data
application
GOAL
Data interoperability
Regional standards
Post data on the Web
Post metadata on the Web
Describe full data set (metadata)
Agency data QA / QC
Agency database system
Local use of data
Describe the data (metadata)
Quality Assurance / Quality Control
Input to electronic form
Field data creation (local office/project)
What’s Needed?
1. Uninterrupted data flow, source to output
2. Data validated by agency
What’s Needed?
1. Uninterrupted data flow, source to output
2. Data validated by agency
3. Descriptive information (metadata)
X .
238
375
265
Y .
51
28
44
What’s Needed?
1. Uninterrupted data flow, source to output
2. Data validated by agency
3. Descriptive information (metadata)
4. Data and metadata on the Internet
What’s Needed?
1. Uninterrupted data flow, source to output
2. Data validated by agency
3. Descriptive information (metadata)
4. Data and metadata on the Internet
XML
SQL
Database
Spreadsheet
ASCII
Word
Processing
PDF
What’s Needed?
1. Uninterrupted data flow, source to output
2. Data validated by agency
3. Descriptive information (metadata)
4. Data and metadata on the Internet
5. Data able to roll together
Ideal: standard data collection, definition, codes
Minimum: Data translate to a standard
All of this is needed for seamless data
delivery to ANY regional tool
Sampling
Crews
Sampling
Agencies
Negotiate keyTechnical
issues assistance
Set
priorities
Metrics
System
development
Create
Establish
the
procedures
data
Policies
to
remove
Database
Funding
Methods Data tasks for agencies
Data
Set
standards,
Entrysharing
codes Entities
obstacles
to
data
Projects
Data disposition
Post
data
&
metadata
QA
Agency
data systems
Agency
support
Contract language
Feed
regional
data
tools
Describe
Post
datathe
/ metadata
data
Support data automation
Maintain the
Biological
& data
data mgt.
Policy
Regional
responsibilities
Roles
Makers
Data Users
The Data Sharing Guide Will Help To:
 Inform development of
agency data systems
 Inform development of
a regional data system
 Avoid skipping
essential components
 Get us started
Questions?
www.streamnet.org
Funded by:
Through:
Fish and Wildlife Program
Administered by:
Photographs courtesy of CalFish