Transcript presentation

Simulation Databases
for the Virtual Observatory
Work in progress and application
to the StarFormat Project
Patrick Hennebelle
Franck Le Petit
Benjamin Ooghe
VO Theory – Simulation DataModel
• SNAP (Simple Numerical Access Protocol)


DataBase : SimDB to store
Data Access Protocol to access : SimDAP to access
• SimDB / Volute Project :
UML model
SimDB pipeline
(XMI)





DataModel UML
DMTransformer
DB managing (JPA)
DB scripts
(DDL)
Java interface
WebServices (SimDAP)
intermediate model
Doc
Schemas
(HTML)
(XSD)
Java code
Browser
(web)
http://code.google.com/p/volute/
StarFormat
Simulation Databases for the Virtual Observatory
Fri, February 13th, 2009
2
StarFormat
Simulation Databases for the Virtual Observatory
Fri, February 13th, 2009
3
SimDB Infrastructure
StarFormat
Simulation Databases for the Virtual Observatory
Fri, February 13th, 2009
4
5
DATA MODEL
6
DATA MODEL
7
DATA MODEL
8
DATA MODEL
9
10
StarFormat - Integration in the DM
• Different sorts of computings to store :

MHD Simulations :

Code (RAMSES, ZEUS, …)

Target (molecular clouds, …)

Physical processes involved (MHD, heating, gravitation, …)

Parameters (initial conditions, limits, AMR description, refinement, …)

Description of results :



Multiple time steps (snapshots)
Statistics (min/mean/max/rms velocity, density, mass, …)
PDF, image files

Clumps identification (Cluster Detection)

…
StarFormat
Simulation Databases for the Virtual Observatory
Fri, February 13th, 2009
11
StarFormat - Integration in the DM
THY3D_grav_mhd
(ref. to FORM_MC_RAMSES2_01)
(ref. to MHD)
StarFormat
(ref. to Gravitation)
FORM_MC_RAMSES2_01
MHD Cooling
Simulation Databases for the Virtual Observatory
Gravitation
Fri, February 13th, 2009
12
StarFormat - Integration in the DM
FORM_MC_RAMSES2_01
FORM_MC
THY3D_grav_mag_bcl
(ref. to FORM_MC_
RAMSES2_01)
Time step #12
AMR Cells
Number of Cells
StarFormat
Simulation Databases for the Virtual Observatory
Fri, February 13th, 2009
13
P. Hennebelle
FORM_MC
FORM_MC_RAMSES2_01
THY3D_grav_mag_bcl
(ref. to FORM_MC_
RAMSES2_01)
Initial Velocity
17.8 km/s
(ref. to Initial Velocity)
AMR Cells
Molecular Cloud (MC)
(ref. to AMR Cells)
20 (ref. to
Number
of Cells)
Number
of Cells
TIME STEP #12
(ref. to MC)
5.4 cm-3
(ref. to
Mean Density)
Mean
Density
14
StarFormat - Issue with the DM
• Problem : no way to store table of values
• Examples :

Number of AMR Cells per level

Statistics dependant on a factor (density threshold)

Probability density functions
• Different solutions imagined with respect to the DM :

Statistics as a PostProcessing : Child Experiments

Multiple Experiments

Multiple Target Objects

Property Groups
StarFormat
Simulation Databases for the Virtual Observatory
Fri, February 13th, 2009
15
16
StarFormat - Bidimensional data issue
DM extension : UML proposition
StarFormat
Simulation Databases for the Virtual Observatory
Fri, February 13th, 2009
17
StarFormat - Bidimensional data issue
• Proposition submitted to the group Theory

Waiting for approval

Developing anyway in the mean time :

If approved : good, just need to adapt the code to new SimDB specs

If rejected : external development => lack of interoperability
Acceptable considering interoperable projects?
Complete and double the results :



Local version with modified DM
External version using original DM and least bad solution
Theory not official itself...
StarFormat
Simulation Databases for the Virtual Observatory
Fri, February 13th, 2009
18
StarFormat – Pipeline
• Experimentator :

runs simulations with code and collects results

calculates extra results (statistics, plots, ...)

generates metadata file (VOTable format) per snapshot
• Software :

collects and reads VOTable files with SAVOT (CDS)

stores snapshot data in an intermediate object model

use SimDB API to store in PostgreSQL database
• Admin Interface :

metadata files feeding (directory, webform, ...)

database administration (updating, removal, browsing, ...)
StarFormat
Simulation Databases for the Virtual Observatory
Fri, February 13th, 2009
19
StarFormat – Pipeline : VOTable
• Normalized format of XML

Resource / Table / {Param – Field – Data}
StarFormat
Simulation Databases for the Virtual Observatory
Fri, February 13th, 2009
20
StarFormat – Pipeline : VOTable
• Local VOTable architecture constraints :

Resource Project


Resource Experiment


Tables Project, Protocol, Code, Contact, Representation
Tables Experiment, Physics, Parameters, Target
Resource Snapshot

Tables Snapshot, Statistics, Statistics_threshold, Files
StarFormat
Simulation Databases for the Virtual Observatory
Fri, February 13th, 2009
21
StarFormat – Pipeline : VOTable
StarFormat
Simulation Databases for the Virtual Observatory
Fri, February 13th, 2009
22
StarFormat
Simulation Databases for the Virtual Observatory
Fri, February 13th, 2009
23
SimDAP - WebServices
• Browser, queries, downloads, connect to other data, …
• SimDB Browser (WebApp) :

Browse resources

Run SQL queries

Validate XML instances
with SimDB's Schema
• Prepared parametered Queries
• Work in development (SIMDAP)
StarFormat
Simulation Databases for the Virtual Observatory
Fri, February 13th, 2009
24
Example of WebService : SIMDB Browser
StarFormat
Simulation Databases for the Virtual Observatory
Fri, February 13th, 2009
25