Transcript Document
The German Astrophysical Virtual Observatory (GAVO) Knowledge Networking for Astronomy in Germany and abroad Gerard Lemson1,2, Wolfgang Voges1, Joachim Wambsganss2 GAVO team 1 Max-Planck-Institut 2 Astronomisches GES 2007, 3.5.2007 für extraterrestrische Physik, Garching Rechen Institut/Zentrum für Astronomy der Universität Heidelberg Overview • Knowledge networking for astronomy: the Virtual Observatory • Standardisation: the International Virtual Observatory Alliance (IVOA) • The Virtual Observatory in Germany: GAVO • Theory in the VObs GES 2007, 3.5.2007 The status quo • Astronomy produce a huge range of valuable data products, – Of sometimes astronomical sizes (soon Petabytes/yr). – From ground-based observatories and satellites – From all-sky surveys, and from large variety of individual targeted observations. – In all wavelengths. – Don’t forget computer simulations • Requires large variety of disciplines/specialisations to produce and analyse. • Data products are relatively quickly made public (~1 yr). – Many already available as online archives, more or less standardised, homogeneous. • In combination promises interesting new science. GES 2007, 3.5.2007 Some astronomical data products Images Spectra Catalogues Simulations GES 2007, 3.5.2007 • John Hibbard http://www.cv.nrao.edu/~jhibbard/n4038/n4038.html • NASA/CXC/SAO/G. Fabbiano et al. • Di Matteo, Springel and Hernquist, 2005 GES 2007, 3.5.2007 But • • • • • • Where is the data? What do these archives contain? How can they be accessed? How do we analyse these (very) large datasets? How can we combine them? ... • The Virtual Observatory (VObs) is the proposed answer to these questions. GES 2007, 3.5.2007 The VObs ... Promises • improved communication • reuse of results • validation • comparison • combination • federation • collaboration • new science Requires • archiving • curation • online availability • description • access tools • filtering tools • analysis tools • standardisation → O(16) national VObs projects, organized in the IVOA GES 2007, 3.5.2007 IVOA GES 2007, 3.5.2007 Mission statement Facilitate the international coordination and collaboration necessary for the development and deployment of the tools, systems and organizational structures necessary to enable the international utilization of astronomical archives as an integrated and interoperating virtual observatory. GES 2007, 3.5.2007 IVOA organisation • Since 2002 • Activities divided in working - and interest groups, “tiger teams” • Meet twice a year in interoperability meetings (two weeks from now, Beijing) • Active mailing lists, wiki pages • Standardised standardisation process GES 2007, 3.5.2007 IVOA activities • Working groups create standards for – – – – – – – publication and discovery (Resource Registry) (meta-)data description (DM, Semantics) selection and remote filtering (DAL, VOQL) formats for transmitted data (VOTable) (web) services, distributed workflows (GWS) application interoperability (Applications) event notification (VOEvent) • interest groups represent special interests – Data curation – Grid – Theory GES 2007, 3.5.2007 Some results • Resource registry data models + various implementations • VOTable format (XML schema) + many support tools • Data models for – space-time coordinate systems – characterisation of observations – spectra • Simple data access protocols for – source catalogues – 2D images – 1D spectra • In development: – – – – Astronomical data query language (ADQL) Astronomical web service standards (UWS) Application messaging Simulation data models + access protocols GES 2007, 3.5.2007 The German Astrophysical Virtual Observatory http://www.g-vo.org GES 2007, 3.5.2007 GAVO I: 2002-2005 • BMBF funded • Partners – Astrophysikalisches Institut Potsdam (AIP) – Astronomisches Rechen Institut/Zentrum für Astronomy der Universität Heidelberg (ARI) – Max-Planck-Institut für extraterrestrische Physik (MPE), Garching – Hamburger Sternwarte – Asociated partner: Max-Planck-Institut fur Astrophysik (MPA), Garching • Activities – R&D – Prototyping – Special attention: • • • • Archive publication: ROSAT, RAVE Data mining: cross-matching, classification Grid computing: simulations, distributed cluster finder Theory: virtual telescopes, archiving simulations, IVOA GES 2007, 3.5.2007 GAVO II: 2006-2008 • BMBF funded • Partners – AIP, MPE, MPA, – Technische Universität Munchen-Informatik – Universität Tübingen • Focus: move to scientifically useful services • Projects • • • • • Millennium database (see later) IVOA representation (theory, VOQL) Standard services (SIA, SSA,SCS) Custom services VObs expertise center at ARI ... GES 2007, 3.5.2007 VObs expertise center @ ARI • IVOA compatible metadata repository for community • Implementation data access query protocols • Storage (smaller) data sets, especially science ready data • Tools • Outreach/PR • Help-desk GES 2007, 3.5.2007 Theory in the VObs: some observations • Simulations not as simple as observations – – – – less homogeneous complex observables no standardisation on data formats archiving ad hoc, for local use • Current IVOA standards somewhat irrelevant – no common sky – no common objects – requires data models for content, physics, code • Moore’s law for N-body) simulations – Very large simulations possible – NB: also makes useful lifetime relatively short GES 2007, 3.5.2007 “Moore’s law” for N-body simulations Courtesy Simon White GES 2007, 3.5.2007 Virgo collaboration’s Millennium database • Largest cosmological simulation to date – 10 billion particles evolving under gravity – 500 Mpc (~2Gly) box – 64 snapshots – 350000 CPU hours – O(30Tb) raw data • Derived data – – – – density fields clusters, merger trees galaxies, merger trees realistic, “observed” galaxy catalogues GES 2007, 3.5.2007 Courtesy Volker Springel Time evolution: merger trees GES 2007, 3.5.2007 Real and Mock catalogues Courtesy Volker Springel GES 2007, 3.5.2007 Database + web server • Derived data products only • SQLServer database • Apache web server – portal: http://www.mpa-garching.mpg.de/millennium/ – public DB access: http://www.g-vo.org/Millennium – private access: http://www.g-vo.org/MyMillennium • Access methods – browser producing various formats, plotting capabilities – stream based wget + IDL, R, etc allows – finite query time (30sec-7min) • Features – efficient tree storage+access – spatial indexing – MyDB GES 2007, 3.5.2007 GES 2007, 3.5.2007 GES 2007, 3.5.2007 Usage statistics • Up since Aug 2006 • Community notified via preprint server http://xxx.lanl.gov/abs/astro-ph/0608019 • 130 registered scientific users • >1.4 million individual SQL queries • > 4 billion returned rows (since March 8 2007) GES 2007, 3.5.2007 Summary • VObs is natural extension of astronomy’s history of data archiving, standardisation, online and open access. • IVOA active, questions are complex technically, but politics are sometimes even harder. • GAVO relatively small, but has found some niches, particularly theory. • To be successful requires use by non-VO scientists, requires proper PR. GES 2007, 3.5.2007 Thank you. Further thanks to: Volker Springel, Simon White, Gabriella DeLucia, Jeremy Blaizot, Manfred Kitzbichler (MPA, Garching), Carlos Frenk, John Helly , Richard Bower (ICC, Durham, UK), Alex Szalay (JHU, Baltimore) Opening picture courtesy of NASA Goddard Space Flight Center. GES 2007, 3.5.2007