Transcript Science Overview Architectural Status (http://www.astrogrid.ac.uk

Science Overview
Architectural Status
(http://www.astrogrid.ac.u
k)
Multi-wavelength showing the jet in M87: from top to
bottom – Chandra X-ray, HST optical, Gemini mid-IR, VLA
radio
Nicholas Walton
IoA, Cambridge
credits: “NASA / Chandra X-ray Observatory / Herman Marshall (MIT)”,
“NASA/HST/Eric Perlman (UMBC), “Gemini Observatory/OSCIR”, “VLA/NSF/Eric
Perlman (UMBC)/Fang Zhou, Biretta (STScI)/F Owen (NRA)”
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
p1
Printed: 12/12/01
Drivers: The Sociology of
Astronomy

Continuing Collectivization
 Facility
class (common user) instruments
 Central
development of supporting s/w (e.g. Iraf)
 Calibrated
archives and access tools (e.g. IPAC)
 Information
services (e.g. ADS, NED, astro-ph)
 Consortium
projects (e.g. MACHO, SLOAN,
VISTA)

Evolving Developments
 Inter-operable
 Data
archives, joint queries (e.g. MAST)
mining (exploration and analysis tools)
 Information
discovery tools
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
p2
Printed: 12/12/01
Drivers: Technological Change I

The Growth of Data
 Significant
increase in number and size of
telescopes
In the optical:
ESO's 4x8-m VLT, Gemini's 2x8-m
 In the x-ray: XMM-Newton, Chandra
 In the mm: ALMA

 Significant
increase in size and multiplex
capabilities of associated instrumentation and
detectors, e.g.:
In the optical: VISTA will have a Gpixel array
 In the radio: e-Merlin with data rates of 320 Gbps will
generate

N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
p3
Printed: 12/12/01
Drivers: Technological Change II

The Growth of Data Archives
 Many
observatories hold multi-TB archives
 New
initiatives set-up to support new observing
capabilities (e.g. TeraPix)
 In


the optical, need to ingress all-sky survey's
Whole sky at 0.1 arcsec/pix is 100TB
Increasing Importance of Archival Data
 Time
on expensive facilities (e.g. HST) only
awarded if the archive has been searched before
hand
 This
trend is continuing, driving observatory and
user
demand
for Meeting,
more
archival
data
N A Walton: Science
Overview:
AstroGrid Collaboration
Dec accessible
13-14, 2001
p4
Printed:
12/12/01
Empowering Science Driven
Observational Astronomy

Break down the 'wavelength' barriers
 e.g.
Greater focus on science driven proposals
encouraging the use of data sets from across a
wide range of wavelengths

Increase Coordination across countries
 e.g.
The UK joins ESO: improved knowledge
transfer

Increase Access
 e.g.
East European countries may not be able to
fund a major new telescope but can contribute
to, and access 'Virtual Observatories' p5 Printed: 12/12/01
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
Astronomical Drivers: Enabling New
Science

Linking the near and distant Universe
Comparison of rest frame samples to study
evolution of galaxies
 Combination of UV, optical and IR datasets


Creating the 'Digital Sky'


Temporal data measures motions in the Galactic
centre, probes the creation of our Galaxy
The search for extra-solar planets

The planet-transit technique using federated survey
data for millions of stars
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
p6
Printed: 12/12/01
Astronomical Drivers: New Era of Surveys

SuperCOSMOS (UK: now till 2002)
 Based

on Schmidt plates - Science database ~2TB
Sloan Digital Sky Survey (US: now till 2005)
 Dedicated
CCD survey telescope: Science database
~10TB

UKIDSS (UK: from 2003)
 IR

VISTA (UK: from 2005)
 IR

camera on 4-m UKIRT: Science database ~30TB
camera on 4-m VISTA: Science database ~300TB
LSST (US: from 2007-8?)
 Dedicated
~8-m telescope: All sky/few
nights:~5000TB/yr
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
p7
Printed: 12/12/01
Astronomical Drivers: Pre-Discovery
Mining

Investigating the progenitors of sources that
show variability
 Dark
matter revealed by microlensing
events
 Planets
revealed by stellar variability
 Formation
of neutron stars revealed by
GRB's
 Death
of massive stars revealed by Type II
SN
The progenitor of SN1999gi
is <9 M: found from
mining pre-discovery HST
images.
(Smartt et al, 2001, ApJ, 556, L29)
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
p8
Printed: 12/12/01
Astronomical Drivers: Rare Objects

Huge data sets open up the possibility to
find rare objects
 Those

that are hard to find:
Brown dwarfs, have unique red colours
 Those
that are intrinsically rare
High-z quasars, stand out due to
suppression of their blue colour by the Ly-
forest
Hi-z QSO's found from SDSS multi-colour data:

the shaded area is domain of QSO's, solid line is
track for increasing z (Fan et al, 2001, ApJ, 121, 31)
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
p9
Printed: 12/12/01
Astronomical Drivers: New Objects

Large datasets open the possibility to
discover new objects
 Those
that have been missed before
because they are extremely rare or
short-lived
 Those
that have previously been
misclassified, revealed as outliers in new
parameter space correlations
DPOSS group, during searches for
high-z quasars, have discovered
peculiar objects, this one perhaps
a BAL QSO (Djorgovski et al, 2001,
PASP, 225, 52)
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
p10
Printed: 12/12/01
Astronomical Drivers: Cosmology thru SN
Ia
AstroGrid will enable rapid typing
of search fields via galaxy photo-z's
in support of UK/EU groups, e.g.
EC RTN SN network.
Issue: large scale image deconvolution
matching candidates to z
Example SCP data
from Perlmutter, 2001
snap.lbl.gov/pubdocs/
SAUL.pdf
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
p11
Printed: 12/12/01
Astronomical Drivers: The X-ray Universe
AstroGrid will enhance the ability to
classify new X-ray objects and speed
object sample creation
Issue: federating large scale multi-λ data
The ELIAS team is using Chandra,
XMM, ISO, SCUBA, optical imaging
and VLA data to study a number of
fields: see
www.roe.ac.uk/~omar/survey.html
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
p12
Printed: 12/12/01
Astronomical Drivers: WD's and Dark
Matter
AstroGrid products will enable
the federation of multi-epoch
data to increase the number of
White Dwarf's known. With IR
data cool objects will be found
impacting on the Dark Matter
problem.
Issue: TB's of multi-epoch, multicolour data, finding moving objects
WD0346+246, moving at 1.3''/year,
was discovered by Hambly et al.,
(2000, Nature, 403, 57). It is cool,
but has blue colours in the IR due to
collision absorption by H2
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
p13
Printed: 12/12/01
Astronomical Drivers: Solar Complexity
Solar-B will be launched
in 2005, aimed at studying
the interaction between
the Sun's magnetic field
and its corona
Solar-B will be supported
by complementary space/
ground data.
Science often targets
events with many multi-λ
observations required
Image from Marshall Space
Flight Centre, NASA
science.msfc.nasa.gov/ssl/pad/solar/Beyond_Solar-B.htm
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
p14
Printed: 12/12/01
AstroGrid: framework for
applications
Enable future
development and
deployment of AstroGrid
and External (e.g NVO)
tools:
Automated detection of outliers
in SDSS two colour data
(Connolly et al, 2001, AJ in press)
A NVO prototype of an automated
discovery tool for arcs developed
by A Szalay
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
p15
Printed: 12/12/01
AstroGrid: a Typical Challenge
• Problems in matching multi-λ survey data:
Differences in angular resolution, s/n ratios, backgrounds, etc
(Djorgovski et al, 2001, astro-ph/0108346)
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
p16
Printed: 12/12/01
Astronomical e-Science Initiatives
AstroGrid
www.astrogrid.ac.uk
European Grid of Solar Observations
www.mssl.ucl.ac.uk/gri/egso
Astrophysical Virtual Observatory
AstroVirtel
www.eso.org/avo
ecf.hq.eso.org/astrovirtel
National Virtual Observatory
www.us-vo.org
The Digital Sky
www.cacr.caltech.edu/SDA/digital_sky.html
The Virtual Sky Project
virtualsky.org
Sloan Digital Sky Survey: SkyServer
skyserver.fnal.gov/en
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
Skyview
p17
Printed: 12/12/01
skyview.gsfc.nasa.gov
Towards Phase-B: The Science
Case

Rapid development of science requirements
 Emphasis

Development of 'use cases'

Example: Formation of large scale structure




on user input via consultation
Construct unbiased cluster of galaxies sample over z to test various
cosmological models of galaxy formation
Need to operate on large data sets, constructing simulated surveys to
test for selection effects
Generate predictions of observed sample properties and compare with
observed samples
Development of 'White Paper' developing
AstroGrid's role for Phase-B and in the
context of larger VO picture
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
p18
Printed: 12/12/01
Developing the Science
Requirements


Recognising the specific need to support UK science
and associated key datasets was the backdrop to the
genesis of the AstroGrid program
Focus on community involvement
 Development
of a few scenarios for AstroGrid,
highlighting there capabilities and application to science
areas
 Invite

comment from community
Interaction channels will be via
 The
PS visiting astronomy groups
 Presentations,
 Development

both face to face and on-line
of a suitable questionnaire
Organise 2nd AstroGrid Workshop: Mar/Apr 2002.
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
p19
Printed: 12/12/01
Use Case Development: practical
req's

In addition to development of the science
requirements, generic use cases are being
developed, e.g.:
 Log
on remotely, operate remotely
 Donate
 Fix
a small catalogue to the grid
the WCS in a FITS image
 Find
observations via instrument footprint
 Synthesis
 Matching

colour data for a set of objects
objects by position with errors
As the science requirements are formalised, use
cases will be derived from them
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
p20
Printed: 12/12/01
A Baseline Architecture for
AstroGrid
Architecture must address Top Level Design
Issues:
Ease of use, emphasising a uniform
appearance
 Functionality
 Speed of Response
 Relevance
 Cost
 Reliability
 Scalabilty
 Outreach
 Level of data-provider buy in

N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
p21
Printed: 12/12/01
Architecture: Quality Assurance

AstroGrid federation capabilities will
characterise quality parameters, e.g.:
 Positional
accuracy →reference frames
 Photometric
accuracy →std colour systems
 Completeness
 Temporal
→weight maps
resolution
 Wavelength
resolution and coverage
 Photometric
system
 Image
distrotions
 Artifacts
 Catalogues
→criteria used to ID objects, etc.
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
p22
Printed: 12/12/01
Architecture: Data Inclusion Model
AstroGrid will potentially enable any data sets to
be accessed through AstroGrid services.
However the data must be adequately described.
Data
any
restricted
AstroGrid
any
(described)
How
any
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
restricted
restricted
p23
Printed: 12/12/01
Unified
Modelling
Language
(UML)
representatio
n of the
query
architecture
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
p24
Printed: 12/12/01
Architecture Overview
Major elements in the query form a circular flow:
> a command flow involving user initiation
> an inner 'grid' loop with bulk data flows
- Client s/w
WP-A3
<=======<
- Query Engine
WP-A2
|
|
- Resource Management WP-A2
<====<====< |
- Resource Mapping/index WP-A2
|
| |
- Data Harvesting
WP-A2
|
| |
- Data Services
WP-A4
|
| |
- Results Management
WP-A4
|
| |
- Data Mining
WP-A4
>====>====> |
- Visualisation
WP-A3
|=======^
Noted above are WP areas leading expansion of the architecture in
that area
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
p25
Printed: 12/12/01
Architecture Overview: UML view
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
p26
Printed: 12/12/01
Detailing the Architecture





The Data flow diagrams (DFD's) show a functional
decomposition
They do not say how functions are distributed
across hardware or software
Grid 'layer' diagrams are developed later when the
design has been matched to the analysis
The DFD's emphasise the flow of data (s/w
distribution), the transformations of data (s/w reuse) and the nature of data between
transformations (interoperability)
Flows described in a Data Dictionary
Transforms described in Class-ResponsibilityN A Walton:
Science Overview: AstroGrid Collaboration
Dec 13-14, 2001
p27
Printed: 12/12/01
Collaboration
(CRC)Meeting,
notes

UML representation
of
the 'query' process
Diagram to be read
with:
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
 CRC
Notes
 Data
Dictionary
p28
Printed: 12/12/01
UML representation of
the 'Harvest' process
Diagram to be read
with:
 CRC
Notes
 Data
Dictionary
p29
Printed: 12/12/01
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
Extract from the Data Dictionary

Query
 A list

of constraints on data to be extracted from the Grid
Authorisation
 Something
that indicates that a user is allowed to use a
resource up to some certain level

Table meta-data
 Describe
a table of data sufficiently that the table can be
understood by Grid s/w and transformed as necessary

Formal catalogue-extract
 Tabular
data in AstroGrid favoured form. The catalogue
extracts need to be merged and transformed. A rich
format such as XML is required.
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
p30
Printed: 12/12/01
Extract from the 'query' CRC

Query-form display
 Displays
forms to capture queries, passes contents of
form to Grid, passes user's credentials to the Grid.

Query scheduler
 Determines
whether a query can be run immediately;
whether is must be run in batch; whether it needs prior
approval for use in resources.

Resource authoriser
 Determines
whether a given user is allowed to use a
given resource at a particular level. Probably needs to be
built from scratch
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
p31
Printed: 12/12/01
The NVO Model: A Comparison
Usual 'Grid' fabric, resource
& collective layers added to
with astro-specfic user &
connectivity layers
NVO focussed on (mainly)
optical and near-IR large
scale survey data sets.
NVO layer model, figure taken from
nvo-proj.pdf at www.us-vo.org
N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
p32
Printed: 12/12/01
Concluding
Remarks



N A Walton: Science Overview: AstroGrid Collaboration Meeting, Dec 13-14, 2001
AstroGrid is a
major new UK
funded e-science
initiative
In partnership with
EU centres it will
play a lead role in
the realisation of a
European Virtual
Observatory
AstroGrid is poised
to significantly
enhance the
opportunities of the
UK astronomical
researchp33 Printed: 12/12/01