Transcript apac04ausvo - The Australian Virtual Observatory

Australian e-Astronomy
 Aus-VO
 APAC Astronomy Grid
 Collaborations
US-Australia Workshop on High-Performance
Grids and Applications
8-9 June 2004
David Barnes
Research Fellow, School of Physics
The University of Melbourne
Aus-VO Structure
• Rachel Webster (Melb) - Lead Investigator
• David Barnes (Melb) - Project Scientist
– (sometimes Project Manager!)
• Funded year-by-year by the Australian
Research Council Linkage Infrastructure Equipment and Facilities grant program
• Institutes are broadly:
– University astronomy research groups
– National telescope facilities (AAO & ATNF)
– University and partnership computing groups
Aus-VO Partner Institutes
•
•
•
•
•
•
•
•
•
The University of Melbourne
2003 LIEF
The University of Sydney
CSIRO Australia Telescope National Facility
AUD 260K
Anglo-Australian Observatory
Swinburne University of Technology
The University of Queensland
2004 LIEF
Monash University
AUD 306K
The University of New South Wales
Victorian Partnership for Advanced
Computing
• ANU Mount Stromlo Observatory &
Supercomputing Facility
• CSIRO Mathematical and Information
Sciences
collaborators
• Australian Partnership for Advanced
Computing
APAC Astronomy Grid Structure
• Jon Smillie (ANU) – Project Leader
• + Sarah Maddison (Swin) & David Barnes (Melb)
• David Barnes (Melb) – Research Leader
• Steering Committee includes Aus-VO, Astronomy and
Grav Wave representatives
• Funded 2004-2006 by the APAC Grid
Application Support grant program
• Active members & partners are:
– Melbourne and Swinburne (via VPAC)
– UNSW and Sydney (via ac3)
– ANU (via ANU)
• Research Community is Aus-VO,
astronomers (faculty, postdocs, students)
Thanks to …
• ARC for supporting Aus-VO through the
LIEF grant scheme, year-by-year
• APAC and partners for supporting our grid
activites through the application support
program, 3 yrs
• together these allow forward planning
• and a nice division of grid infrastructure
and science infrastructure effort …
(This is not the last slide)
• Aus-VO
– science content
•
•
•
•
data
codes
metadata
registry content
– science services
• source finding
• coordinate
transformations
• catalogue matching
• visualisation
• interfaces to legacy
environments
• APAC Astro Grid
– data grids
• data store
• data access protocols
• authentication &
authorisation
• network facilities
– compute grids
• workflow
• job configuration,
submission &
monitoring
• code-to-data
paradigms
Our take on virtual observatories
• bring legacy astronomy archives on-line and ensure
future project compliance
• provide access to archived realisations of
simulations and to resources for computing against
new parameter sets
• describe all data fully, and support a small and
well-chosen set of interoperability protocols
• develop tools and interfaces to find, acquire,
process and visualise data
• build national and international grids to host the
data, tools and interfaces
APAC Astronomy Grid Project
Aus-VO & APAC-AG projects 2004
• Common format on-line archive projects:
– HIPASS catalog: HI Parkes All Sky Survey: neutral
Hydrogen spectral line survey, 4,300 sources with ~30
parameters and 1024-channel radio spectra
– SUMSS catalog: Sydney University Molonglo Sky Survey:
radio continuum survey at 843 MHz, 107,000 sources with
~15 parameters
– 2dFGRS QSO catalog: 2-degree Field Galaxy Redshift
Survey: optical spectra of >20,000 southern quasi-stellar
objects
– ATCA archive: Australia Telescope Compact Array archive:
all observations since 1988, circa 1.5 TB of more than
1,000 separate observing projects! Substantial exercise in
describing data with metadata.
– MACHO, RAVE, Pulsar timing archives, Gemini archive
and more in 2004!
Quick Time™a nd a
TIFF ( Unco mpre ssed ) dec ompr esso r
ar e nee ded to see this pictur e.
QuickTi me™ and a
TIFF ( Uncompressed) decompressor
are needed to see thi s pi ctur e.
Quick Time™a nd a
TIFF ( Unco mpre ssed ) dec ompr esso r
ar e nee ded to see this pictur e.
APAC Astronomy Grid Project
Aus-VO & APAC-AG projects 2004
• Server- and client-based visualisation tools:
– client canvas for legacy software package
AIPS++ to draw upon from a remote server.
– 3d visualisation of catalogues
• Data reduction pipelines for:
– Molonglo Observatory Synthesis Telescope
– AT Compact Array archive
– Gemini, for Australian-used instruments
• Portals for theoretical astrophysics jobs:
– configuration and execution
– monitoring
– result retrieval and initial analysis and display
APAC Astronomy Grid Project
Themes and potential collaborations
• Data mining
• catalogue matching
• searches for new empirical relationships +/ new rare
objects
• Legacy software integration
• using legacy code in VO services
• using VO services in legacy software environments
• transparent access to Web and Grid Services from the
desktop
• Visualisation
• remote visualisation service (legacy s/w drawing to
remote “canvases” via Corba, SOAP, …)
• VOTable to VRML service
• volume rendering
General future challenges &
opportunities
• Data grid: replica catalogs done properly,
bandwidth (esp. regional), certificate
authority, virtual data grids?, …
• Services for uploading user codes: is a
sandbox needed?, compiler and library
versions, is a cluster needed?
• Grid management - synchronising Globus,
Tomcat, … versions, legacy software,
account names, geographical location,
firewalls, … - use MDS?
APAC Astronomy Grid Project
Grid a HUGE social challenge!
• don’t want to turn astronomers into Grid
programmers!
• don’t want to depend (entirely) on external
third-party services for day-to-day work!
• don’t want to divert research staff!
• any demo efforts must be useful and longlasting, not rushed to meet a deadline!
• don’t want to make things harder!
Suggested Aust-US “demo”
• ePhysics portal [Aus-VO, APAC-AG]
– prototyped at Melbourne, using GridSphere,
collab. effort between Astro & Particle Px.
• grid broker(s) [GridBus, Nimrod/G, …]
• numerical codes [Aus-VO & APAC-AG,
FLASH from NCSA?]
• compute grid nodes (computational)
[APAC, …, 32 Tflop from NCSA?!, UCSD?]
• remote visualisation nodes and software
[“ship pixels”] (simple plots, slicing, …)
[QPSF?], could utilise 10Gbit Pacific link!
• data grid nodes (archiving, interim) […]