Transcript Requirements from astronomy in the Virtual Observatory era

Requirements from astronomy
in the Virtual Observatory era
Bob Mann
Institute for Astronomy & NeSC
University of Edinburgh
Outline of talk

What is the Virtual Observatory (VObs)?
– How is it changing astronomy?

Who in astronomy needs e-Science skills?
–
–
–
–

Data centre staff
VObs middleware developers
Data analysis tool developers
Research astronomers
Summary
– What skills are needed?
– How can these requirements be met?
What is the
Virtual Observatory?

A goal:
– An interoperable federation of the world’s
astronomical data sources

A standards agency/coordination body:
– International Virtual Observatory Alliance
– Its members are VObs projects in Australia,
Canada, China, France, Germany, Hungary,
India, Italy, Japan, Korea, Russia, Spain,
UK, USA, and the EU.
Why is the VObs needed?

We observe across the whole electromagnetic spectrum
ROSAT ~keV DSS Optical 2MASS 2m IRAS 25m

IRAS 100m
GB 6cm
NVSS 20cm WENSS 92cm
Different views of a local spiral galaxy
– Need all of them to understand its physics fully
– The databases holding them are located all over the world
Schematic of the VObs
DB 1
DB 2
DB 3
DB 4
DB 5
App 1 Compute
Resource 1
Registry
Workflow
App 2
App 4
User
Portal
App 3
Compute
Resource 2
App 5
What services will the
Virtual Observatory need?

Data discovery
– Looking up metadata in a registry

Data access
– With/without authentication & authorization

Data integration
– Cross-matching entries in databases

Data manipulation
– Data mining, data analysis, etc
Data centre staff
DB 1
DB 2
DB 3
DB 4
DB 5
App 1 Compute
Resource 1
Registry
• Data Curation:
•Provenance, DBMS design and operations
• Data Access
• Security, Registry Metadata, Web Services
• Data Manipulation
• User-uploaded code, job scheduling
VObs middleware developers
DB 1
DB 2
Web/Grid services for the discovery,
transport and integration of data
Workflow & service composition
Metadata and service registration
Registry
Job management
Workflow
App 3
Security
Compute
Resource 2
Portal
Data analysis tool developers
DB 1
Data Access:
• OGSA-DAI/data access web services
Service Registration
Registry
Integration into workflows
Workflow
Job submission
App 3
Compute
Resource 2
Research astronomers
DB 1
DB 2
Initially:
• Use of portal
Soon:
Registry
Workflow
• Script and tool development
using APIs, local job control,
wrapping apps as web services
App 4
User
Portal
Compute
Resource 2
App 5
Summary of skills needed






Web/Grid services for data discovery,
access, integration and manipulation
Metadata for data & service registration
Data curation – provenance, preservation
Workflow creation and execution
Job control – local and remote
Security
How can these needs be met? (1)

Learning through experience
– Difficult: fast moving field; where to start?

Danger of learning the wrong technologies
– Inefficient: undirected training can be slow

Hiring people with appropriate skills
– Not many people with e-Science skills yet

…and expensive to employ on research grants
– Really need astronomers for some jobs

Danger of losing scientific direction to projects
How can these needs be met? (2)

Existing staff: training courses
– Can be expensive and time-consuming
– Need to budget time and money for them


Funders need to understand their necessity
Future staff: education
– Astronomers: MSc-level courses – either
as MSc or as part of PhD training
– IT specialists: inclusion of e-science
content in CS degree programmes