Transcript Goals and plans for Grid Data Warehouse

Goals and plans for
Grid Data Warehouse
Guy Rixon
AstroGrid Consortium meeting
3-4 November 2003
Consortium meeting, November 2003: Grid Data Warehouse
AstroGrid goals
Data warehouse:



Concentrate catalogues in one DB
Therefore solve distributed-join problem
Allow users to add tables to DW.
Exploit UK e-Science Grid.
OGSI at arm’s length:




Exploit OGSI.
Don’t rely on OGSI in critical places
Use OGSI/OGSA in “leaf node” services.
See separate grid-tech talk at this meeting.
2
Consortium meeting, November 2003: Grid Data Warehouse
GDW goals
Use “pure grid” sites and resources

Don’t require astro software at grid sites.
Use OGSA-DAI as remote interface to DB.
Use job-manager services?
Translate VObs  grid in A/G web-service.
Support basic DW operations:



Query preloaded catalogues
Add user catalogues
Allow query results to stay in DB for later queries
3
Consortium meeting, November 2003: Grid Data Warehouse
Design
See wiki web, keyword GdwDesign.
See
http://astrogrid.ast.cam.ac.uk/javadoc/
…but following detailed material hadn’t
made it to the web at the time of
presentation: model inside IDE. (Will
probably be on the web when this
presentation is archived.)
4
Consortium meeting, November 2003: Grid Data Warehouse
Iteration 4 planning (1)
Pre-loaded data-sets:





2MASS catalogues
XMM1
USNOB1
FIRST catalogue
INT-WFC catalogues
FIRST is first because it’s small.
5
Consortium meeting, November 2003: Grid Data Warehouse
Iteration 4 planning (2)
1. Do use cases SynchronousQuery and
AsynchronousQuery for one catalogue
by middle I4.
2. Do above for all data-sets by end I4.
3. Try to do either LoadEndUserDataSet
or AsynchronousQueryIntoTable by
end I4.
6
Consortium meeting, November 2003: Grid Data Warehouse
Iteration 4 planning (3)
Develop at Cambridge.
Deploy copy to Leicester.
If possible, deploy copy of back-end to
UK e-Science grid, possibly at
NeSC/EPCC.
7