Transcript Document

OO Software and Data Handling in AMS
Vitali Choutko, Alexei Klimentov
MIT, ETHZ
Computing in High Energy and Nuclear Physics
Beijing , September 3-7 , 2001
Outline

AMS – particle physics experiment on the
international space station :







AMS Detector
STS91 precursor flight
AMS ISS mission
Data flow and AMS ground centers
Software development
Conditions and Tag Database
Data Processing
A.Klimentov
AMS software and data handling
CHEP01
2
AMS : a particle physics
experiment in space
PHYSICS GOALS :
Accurate, high statistics measurements of charged,
cosmic ray spectra in space > 0.1GV
• The study of dark matter (90% ?)
+
Nuclei and e- spectra measurement
• Determination of the existence or absence of antimatter in the Universe
Look for negative nuclei
• The study of the origin and composition of cosmic rays
Measure isotopes D, He, Li, Be…
A.Klimentov
AMS software and data handling
CHEP01
3
A.Klimentov
AMS software and data handling
CHEP01
4
Precursor flight :
8
10 events recorded
Trigger rates 0.1-1kHz
DAQ lifetime 90%
Results :
_
Anti-matter search :
He / He = 1.1x 10 -6
Charged Cosmic Ray spectra
Pr, D, e-+ , He, N
Geomagnetic effects on CR
under/over geomagnetic
cutoff components
Magnet : Nd2Fe14B
TOF : trigger, velocity and Z
Si Tracker : charge sign, rigidity, Z
Aerogel Threshold Cerenkov : velocity
Anticounters : reject multi particle events
A.Klimentov
AMS software and data handling
CHEP01
5
A.Klimentov
AMS software and data handling
CHEP01
6
AMS on ISS , 3 years in space
_
+ from p,p
Separate eup to 300 GeV
3
4
He, He, B, C…
+
e-,
up to 1000 GeV
A.Klimentov
AMS software and data handling
CHEP01
7
AMS 02
In Cargo Bay

separate e  from p,p up
to 300 GeV
Radiators
TRD
USS
Tracker
8 Layers
He Vessel
(S1, S2)
Magnet
TOF
Veto Counter
TOF (S3, S4)
RICH
Electronics
He3 ,He4 ,B,C,...
ECAL
e  ,to 1000 GeV
cke r
R . Be 15, 2001
st
u
g
u
A
A.Klimentov
AMS
on ISSand
fordata
3 years
AMS
software
handling
CHEP01
8
ISS to Remote AMS Centers Data Flow
AMS
White Sand, NM
facility
Payload
Operations
Control
Center
Real-time & Real-time, “Dump”, & White Sand’s
“Dump” data
LOR playback
Real-time
Data
H&S
Payload Data
Service system
ACOP
Stored data
Short
Term
Long
Term
External Communications
MSFC, Al
Monitoring
& science
data
Real-time
& “dump”
H&S
Monitoring
Science
Flight ancillary
data
GSE
NearReal-time
File transfer
High Rate
Frame MUX
ISS
playback
NASA
Telescience
Ground
centers
Infrastructure
Science
Operations
Center
Remote
AMS Sites
9
POIC@MSFC AL
POCC
POCC
External Communications
HOSC Web Server and xterm
XTerm
commands
Monitoring, H&S data
Flight Ancillary data
AMS science data (selected)
cmds archive
TReK WS
“voice”loop
TReK WS
Video distribution
GSE
PC Farm
Science
Operations
Center
GSE
Buffer data
Retransmit
To SOC
GSE
D
A
T
A
AMS Remote
center
MC production
Data
mirror
archiving
RT data
Commanding
Monitoring
NRT Analysis
S
e
r
v
e
r
Production
Farm
MC
production
NRT Data
Processing
Primary storage
Archiving
Distribution
Science Analysis
Analysis
Facilities
Data
Server
Analysis
Facilities
AMS
Station
AMS
Station
A.Klimentov
AMS
Station
CHEP01
AMS Ground Centers
10
AMS SW development
Been started mid 1996 :
basic decisions :




new code C++ only (though we had a large part of
legacy SW written on Fortran)
Existing libraries (CERNLIB, Geant, etc) incorporated via
C/Fortran interface (R.Burow)
transient and persistent classes are separated with
implementing of copy member functions
Decide to use Root and HBOOK for histogramming and
data visualization
A.Klimentov
AMS software and data handling
CHEP01
11
AMS SW development (cont’d)

Use different persistency solutions for various
type of data :
Flat files for the raw data
Ntuples and Root files for ESD
Relational Database (Oracle) tables for file catalogues
Relational Database (Oracle) [Objectivity up to Sep 1998]
o Event Tags
o Calibration data
o Slow control data
o NASA ancillary data
o Various catalogues (processing history, etc)
A.Klimentov
AMS software and data handling
CHEP01
12
Tag Storage with Oracle RDBMS

Tag

Model :
is an unsigned 32 bit integer containing 16, 1 to 5 bit long parameters such
as charge, momentum sign, ß,…
• Flat files – 2400 files, one file per DAQ run, tags are stored as an array of unsigned int.
• RootN - 10 files, each file with ~240 trees, one tree per DAQ run with single branch (tag) per tree
• RootS - 10 files, each file with ~240 trees, one tree per DAQ run , having 16 branches,
every parameter stored in a dedicated branch
• OracleN - table with 10 partitions and 1 column, mapping tag to a column
• OracleI - table with 10 partitions and 1 column with 16 bitmap indices, mapping tag to a column
• OracleS – table with 10 partitions and 16 columns, every parameter mapping to a column

Query :

Data stored on Raid array connected to AS4100
retrieve tags with 3 parameters satisfied to the given limits (query
taken from the “real analysis chain”)
(quad-CPU rated at
600MHz, 2GB RAM)
A.Klimentov
AMS software and data handling
CHEP01
13
Oracle RDBMS to store AMS tags
Method
Size
Gbyte
Query
Time
(sec)
Total write
time (sec)
Record
write time
(sec)
Flat Files
1.4
600
-
-
RootN
0.9
700
2168
22
RootS
1.2
112
2200
66
OracleN
3.4
1420
6600
66
OracleS
6.6
600
6600
66
OracleI
3.4
3.9
6600+500
1)
66
1) 500 sec to build indices for 100M tags
A.Klimentov
AMS software and data handling
CHEP01
14
Design of the Conditions Database

Collection of Time Dependent Values (TDVs)
•Name, id
•Validity begin, validity end time
•Insert time
•Array of unsigned integers (size 100 byte – 8 Mbyte)




Primary access keys : name, id, validity interval
Secondary key : insert time
Major Components : table of names and ids, default TDVs, TDVs
Applications :
Loading data into database
Fetching conditions during event reconstruction
Management utilities (TDV browser)
A.Klimentov
AMS software and data handling
CHEP01
15
AMS Conditions Database


Initially Objectivity, then flat files, now Oracle
Performance test for
TOF temperature (many short records)
Tracker pedestals (small amount of large records)

TDV
Total
Records
Flat file size
(Mbyte)
Oracle table
size (Mbyte)
Record write
time (msec)
TOF Temperature
9835
1.9
2.8
17
Tracker pedestals (a)
330
36.3
45.2
75
Tracker pedestals (b)
330
36.3
44.9
103
(a) BLOB array is stored inside the table, (b) - outside
A.Klimentov
AMS software and data handling
CHEP01
16
Oracle RDBMS to store Tags and TDVs




Currently 8 Gbyte is stored in the Conditions DB (115
different TDV types)
100 million event tags are stored in Tag DB
Oracle RDBMS performance and functionality satisfy
AMS requirements. Using of bitmap indices for tags
improves query time dramatically.
The current implementation works with distributed
CORBA technology. It allowes to reduce the number
of database clients and machine loading.
A.Klimentov
AMS software and data handling
CHEP01
17
Nominal Tables
Hosts, Interfaces
Producers, Servers…
{II}
{I}
{VI}
• {I} submit 1st server
server
server
{III}
Active Tables :
Hosts, Interfaces,
ESD
{IV}
Producers, Servers
Conditions DB
Tag DB
{V}
server
server
ESD
Raw data
server
{VI}
server
server
server
server
•{II} “cold” start
•{III} read “active”
tables (available hosts,
number of servers,
producers, jobs/host)
•{IV} submit servers
•{V} get “run”info (runs
to be processed, ESD
output path)
• {VI} submit producers
(LILO, LIRO,RIRO…)
•Notify servers
ESD
Catalogues
ESD
Oracle RDBMS
Raw data
A.Klimentov
AMS software and data handling
CHEP01
ESD
AMS Production
18
AMS Production Highlights






Stable running for more than 1 month
Average efficiency 95% (98% without Oracle)
Processes communication and control via Corba
LSF for process submission
Run Oracle server on AS4100 Alpha and Oracle clients on Linux.
Oracle RDBMS
 Tag DB with 100M entries
 Conditions DB with 100K entries
 Bookkeeping
 Production status
 Runs history
 File catalogues
A.Klimentov
AMS software and data handling
CHEP01
19