Transcript Greene - STScI

The COMPASS (Catalogs of Objects and Measure
Parameters for All Sky Surveys) Database Overview
Gretchen Greene, Brian McLean, David Wolfe, and Charles Loomis
Space Telescope Science Institute
Antonio Volpicelli, Roberto Morbidelli and Allesandro Spagna
Osservatorio Astronomico di Torino
October 22, 2001
Introduction: Database Functionality
 Key Component in the GSC-2 catalog production
– Online access to processed plate data
Hierarchical Triangulated Mesh
Level 3
– Identification of unique sources
– Cross-indexing of multiple observations
• Overlapping photographic plates
• Multiple epochs
– Global recalibration capability
– GSC 2 is “exported” from COMPASS
 Successful demonstation of Object-Oriented technology and the
selection of commercial software, Objectivity/DB
 One of the first large-scale astronomical archives
–
Estimated 2 billion objects, 10 billion sources/observations
–
4 TB catalog derived from 8 TB image archive!
 Collaborative project with OATo and SDSS JHU Science Archive
– OATo assisted with schema development, QA (CIQA), platematching and much more!
– SDSS/STScI joint development of the HTM Indexing
Leafnode S213
Infrastructure: How is the Database designed?
 Object model or design is based on hierarchical systems for efficiency
– Federation of Databases and Containers
 Plate data information stored in 2 forms for access
– MetaData and Observations within each HTM region
HTM Region Database
for each of 32768
regions
RegionInfoContainer
Plate Database
PlateInfoContainer
GSH
RegionCatalogContainers
Plate Source Objects
for each plate
section within
region
Page = Federated Database
SourceIndex
Section of reference
catalog within
region
GSPC2
= Database
= Container
= Class/Basic Object
Tycho2
[ COMPASS Federated Database ]
for each plate
Highlights: What happened in 2001
• Completion of Loading and Cross-Matching
Observations: All-Sky, 2 bands
• Reference catalogs: Successfully loaded GSPC2
and Tycho2 into COMPASS, utilize real-time
object-matching based on GSC1
• Fully Online RAID System: Has been upgraded to
4TB using NT client/server configuration.
• Export of GSC 2.2 Public Release: The export of
~1 billion objects from the database with best
observation position selection, magnitude and
classification filtering for accuracy.
• Steady State Production Operations: Catalog
production pipeline, recalibration, and
administration of multi-Terabyte data volume
• HST Bright Object Protection (BOP): GSC-2.2
incorporation into Observation Planning Tools and
target acquisition development (APT) – special
release
• ShowSky JAVA Visualization Tool Public Release
•Multiple Catalog viewing, including SDSS,
2MASS, Tycho, DSS, GSC….
•Parameter Filtering
•HTM grid field view with level
GSC2.2 Export Catalog Fields: what will change with 2.3?
Field/Column Number
Parameter
1
GSC ID2
2
Right Ascension
3
Declination
4
Mean Epoch
5
RA Error
6
Dec Error
7
RA Proper Motion
8
Dec Proper Motion
9
RA Proper Motion Error
10
Dec Proper Motion Error
11
F Magnitude
12
F Magnitude Error
13
J Magnitude
14
J Magnitude Error
15
V Magnitude
16
V Magnitude Error
17
N Magnitude
18
N Magnitude Error
19
Classification
20
Semi Major Axis
21
Eccentricity
22
Position Angle
23
Status Flag
• Define new requirements and
evaluate current parameters
early enough to avoid
schedule slips
•need time to develop
modifications to export
task
•Coordination of DB
administation with science
goals
• How do we want to query the
catalog?
•move toward mining.
Export Catalog: Current configuration and moving
toward GSC 2.3
 Source identification
– names from GSC2.2 release should be preserved now that the catalog has been incorporated
into telescope operations, is this always possible?
 Catalog column parameters, new ones? Changes to existing?
– Position currently is based on observation closest to plate center for the same bandpass
• Astrometric residuals were verified to be a factor of 2 improved.
– Reclassification: further definition of non-star into galaxy, blend, etc.
 Plate identification
– epoch is non-unique distinction of plate source
 Multi-plate calibrated parameters, proper motion:
– API development will benefit from experience of previous catalog pipeline tasks, e.g. Plate
loading and object/source cross matching.
– Minimize the large scale transaction time, perform dynamically if possible.
• Will this parameter be changing
Multi-Archive at STScI: CASB coordination with MAST
• MAST develops and
maintains the public data
release WWW user interface
for SDSS and the STScI
WWW pages for GSC-2
– individual development
sites provide more project
specific information
-CASB
-FNAL
-JHU
• Unified view of STScI
archives
–Growth in the large scale
catalog archives
–Cross correlation of mission
data now possible,
development of large scale
correlations between ground
and space missions in progress.
Moving toward Multi-catalog archive: SDSS and
GSC-2 joint operation at STScI
• Data Archive Server is
mirrored parallel to GSC-2
on hybrid LINUX/NT dual
server.
• New Develop of
MAST/GALEX archive
based on GSC-2/SDSS
archive interoperation
• SDSS population into
COMPASS for high speed
cross-matching utilizing
HTM
-no spatial remapping
• New Database Engine??
There is a possibility that
SDSS will migrate from
Objectivity to another
commerical system, e.g. SQL
server.
Future Developments: GSC-2 enhancements to
support access for science and operations
 2MASS population into COMPASS
– GSC-2 + 2-MASS will be input catalog of NGST guidance system
– 2MASS also utilizes HTM for their file system
 SDSS population into COMPASS
 Direct public access to COMPASS observations extended for higher speed access
– HTMIndex container using sorted level10 source indexing as in export catalog
similar to the export catalog infrastructure
 Correct Internal database page file limitations
 GSC2Query Enhancements
 GSC-2, SDSS, GALEX, 2MASS =>>>> NVO (National Virtual Observatory)
– Still very early in conceptual stage, but nevertheless it is real, 10M NSF grant is
currently issued for collaboration between STScI, Caltech, JHU + others.