Transcript hawaii-SPIE-aug02

AstroGrid
http://www.astrogrid.org
•
•
•
•
who are we ?
what is a VO ?
what is a Grid ?
how do we get there ?
Andy Lawrence
2002
S.P.I.E. Hawaii Aug
AstroGrid
http://www.astrogrid.org
Belfast
Cambridge
Edinburgh
Jodrell
Leicester
MSSL
RAL
Andy Lawrence
2002
S.P.I.E. Hawaii Aug
AstroGrid
http://www.astrogrid.org
Optical
Infrared
X-ray
Radio
Solar
Space Plasma
Andy Lawrence
2002
S.P.I.E. Hawaii Aug
people
• PL =
• PS
• PM
Andy Lawrence
=
Nic Walton
=
Tony Linde
slogans
•
•
•
•
the archive is the sky (stolen from US-NVO)
everybody can be a power user
shift the results not the data
a supercomputer on your desk
the project
• PPARC funded project Sept 2001-2004
– approx $6M
• Grid technology development programme
– links to GridPP, MyGrid (Bio), OGSA (Globus)
• Stepping stone to Virtual Observatory
– partner in AVO
– working links with US-VO
• using Unified Process
• open collaborative web sites
status
• elaboration phase two-thirds finished
–
–
–
–
–
science problems / use cases / architecture
technology assessment reports
sw demos : portal, auth/auth, ontology; AVO science demos
collaborative web pages set up
Phase A study report to be published November
• construction phase begins ~end 2002
collectivisation and democratisation
• thirty year trend towards communal organisation
–
–
–
–
–
facility class (common-user) instruments
facility class data reduction s/w
calibrated archives with simple tools
information services (Vizier, ADS, NED)
large consortium projects (MACHO, 2dF, SDSS, UKIDSS, VISTA...)
• next steps
– inter-operable archives (joint queries)
– automated resource discovery (registry)
– facility-class exploration and analysis tools (data mining)
the Virtual Observatory concept
• Aim to make all archives
speak the same language
–
–
–
–
–
all searchable and analysable by the same tools
all data sources accessible through a uniform interface
all data held in distributed databases that appear as one
archives form the Digital Sky
eventual interface to real observatories
the archive is the sky
the Grid concept
• shared managed distributed resources
– documents + data + software + storage + cycles + expertise
•
•
•
•
•
•
network : ability to pass messages
web : transparent document system
a supercomputer
computational grid : transparent CPU on your desktop
datagrid : transparent data access and services
information grid, knowledge grid ... ? everybody can
be a power user
Virtual Organisations ?
what do we need ?
• sociology
– need agreed formats for data, metadata, provenance
– need standardised semantics ("ontology")
• internet technology
–
–
–
–
need protocols for publishing and exchanging data
need registry for publishing service availability and semantics
need method of transmitting authentication/authorisation
need methods for managing distributed resources
bottlenecks
• i/o bottleneck
– need database supercomputers
– need innovative search and analysis algorithms
• network bottleneck
– data centres must provide analysis service
– facility class analysis code needed
shift the results not the data
grid geometry needed
•
•
•
•
not a centralised superarchive
not P2P (like Napster)
not a hierarchy (like LHC grid)
service providers + users (like most commerce)
– some unplanned open use
– some registered use
– variety of access rights
two rivers
• academic / Globus
– remote log on
– identity/authentication/authorisation
– resource management
• commercial / W3C
– exchange of data (B2B)
– service description and publication
– "Web services" = XML + SOAP + WSDL (cf GLU)
Globus problems
• only half works
• data transfer primitive
– flat files only
• not a services-user structure
Web service problems
•
•
•
•
•
one-to-one
bulky
no general auth/auth solution
no accepted service registry solution
no ontology solution
Grid Services ?
• web services
+ auth/auth
+ lifetime management
• to be developed by OGSA project
– OGSA-DAI integrates database access
– AstroGrid and MyGrid "early adopters"
who does what
• the VO projects will :
– evolve agreed data standards
– build infrastructural toolkit for data exchange
– build registry of available data services
• Data Centres will :
– maintain data in whatever format they like
– use toolkit to write standard data services
– publish services to registry
• AstroGrid also will :
– provide first example data services for key UK databases
– provide example "point of entry" user interface
– provide some resource : warehouse, CPU, "MySpace".
who does what part(II)
• observatories / instrument builders need to :
– output data to new standards
– link observing request to VO research
– decide whether to provide data services
or farm out to data centres
FIN
Web DB access today
DB
engine
SQL
data
front
end
CGI
request
html
browser
web
page
user
Web service
native
data
web
service
XML
data
application
DB
engine
SQL
XML
request
anything
user