Transcript How can the HERA-B database systems help you

The HERA-B database services
detector configuration, calibration,
alignment, slow control, data classification
A. Amorim
A. Amorim, Vasco Amaral, Umberto Marconi, Tome Pessegueiro, Stefan
Steinbeck, Antonio Tome, Vicenzo Vagnoni and Helmut Wolters
• The HERA-B detector
• The database problem
• The Architecture
• The Berkeley-DB DBMS
• The client/server integration
• The domains and solutions
• Conclusions and Outlook
1
B0/B0
J/y KS
HERA-B
Experiment
Vertex Detector
Si strip
12 mm resolution
A. Amorim
B/B-tagging
RICH (p/K)
TRD (e/h)
ECAL(g+e/h)
MUON (m/h)
multianode PMT
straw tubes
+thin fibers
W/Pb scintillator
shashlik
tube,pad and gas
pixel chambers
C4F10
HiPt trigger
Magnet: 2 Tm
pad/gas pixel
Tracking:
- ITR(<20cm): MSGC-GEM
- OTR(>20cm): 5+10mm drift cells
2
The HERA-B detector
A. Amorim
SVD
(not visible)
RICH
ECAL
Muon 1,3,4
Magnet
TRD
OTR
ITR- OTR chambers
3
The main challenge: Selecting
A. Amorim
4
How do we select them?
A. Amorim
Time
scale
10ms
Pretrigger: ECAL, m
System, p T pads
L1: e/m “Tracking” in
4 SL, p T cut, mass
cut
Input
rate
10MHz
5ms
200ms
4s
L3: full track & vertex
fit, +SVD tracks, p. id
L2: + drift times,
magnet
traversal,
vertexing
50 kHz
500 Hz
TAPE
L4: + full
reconstruction,
physics
selection
50 Hz
20 Hz
5
HERA-B DAQ
A. Amorim
Detector Front End Electronics
FCS
1000 SHARC (DSP)
Event
Control
SLT/TLT
DSP SWITCH
Trigger
PC
DSP SWITCH
Trigger
PC
Trigger
PC
INTERNET SWITCH
4LT
4LT
PC
4LT
PC
Logger
PC
L2-farm:
240 PC’s L4-farm 200 PC’s
6
The HERA-B database problem
A. Amorim
To provide persistence services (including online-offline replication) to:
••Detector
DetectorConfiguration
configuration
• Common accepted schema
Calibrationand
andAlignment
alignment
••Calibration
• Distributing information to the reconstruction and trigger farms
• Associate each event with the corresponding database information
••Slow
Slow Control
control
• Manage updates without data redundancy
••Data
Data set
Set and
and event
Eventclassification
Classification
••Online
Online Bookkeeping
7
Characterizing the context
A. Amorim
8
Querying on time intervals
A. Amorim
one can select objects by the values of their attributes
Most of our requests want to query on time or version
exception: Event Tag Database: select on (Part Type, E, Pt, etc).
Query on time => Object(Time) or Object(t1,t2)
Our simple database layer on top of Berkeley DB provides that
otherwise one has to specialize the DBMS
( example the conditions database of BaBar and R&D45)
t
9
Keys, objects and client/server
A. Amorim
/PM/ Descrip. field1 ; field 2; ...
Db: /RICH/HV/
.2
.5
-.1
5
6
8
9
2
...
versions
client/server at the SDB level +
RPM -> an UDP based
communication package.
Datastore Server
C- formatter/client
C- formatter/local
MIZZI
Key= name+ version
SDB/client thin layer
SDB/local thin layer
MIZZI
Machine independent
blub of DATA
Berkeley DB / Sleepycat embedded database manager
Open Source
HERA-B client server infrastructure (DAQ software)
10
The Berkeley DB
A. Amorim
See http://www.sleepycat.com/
• Embedded transactional store with:
logging, locking, commit and roll back, disaster recovery.
•Intended for:
high-concurrency read-write workloads, transactions and recoverability.
• Cursors to speed access from many clients..
• Open Source policy
The license is free for non-commercial purposes - rather nice support
•No client/server support is provided
11
Slow Control Interface
A. Amorim
Metadata Object
Pmt1000
Pmt1003
Pmt2000
Data Object
Update
1.2 ...
1 ...
1.5 ...
2 ...
1.6 ...
2.3 ...
time
Optimized Queries
Slow Control interface
Client/server
Update
SlcTable
Datastore Server
nused : int = 0
nmax : int
carray : char **
tabledes : char *
tabledat : char *
tableupd : char *
maxnupd : int
slctem p : slctem pdat *
nchardef : int
SlcDatabase
x : Tables
dbname : char *
slctemp
C- formatter/client
C- formatter/local
MIZZI
SDB/client thin layer
SDB/local thin layer
MIZZI
nfields : int
columsize : int
fieldtype : int*
xbuf : void *
ids : int *
columdata : void *
SlcBookDatabase( )
db
Berkeley DB / Sleepycat embedded database manager
Open Source
HERA-B client server infrastructure (DAQ software)
SlcBookTable( )
SlcUnbookTable( )
MIZZItable
SlcGetDatabase( )
SlcOpenTables( )
tdat SlcCloseTables( )
SlcAdd2Table( )
SlcFindInTable( )
SlcGetDescTable( )
tupd
.dat
.upd
SlcPutDescTable( )
SlcStoreUpdTable( )
SlcStoreDataTable( )
SlcMem Table( )
SlcTableValues( )
(trough the tab ledes nam e )
SlcTableGetpt( )
SlcTableGetSpt( )
1..1
LocCopy2Table( )
LocCopyTable( )
LocMergeSlcTable( )
.sch
12
Associations to Events
A. Amorim
...
...
Index Obj.
...
Index Obj.
Index Obj.
Revision 0 - online
Calibrating
Index Obj.
Active server
interface
Index Objects
(referenced by events)
Slow Control interface
1 - offline
Client/server
Client/server
C- formatter/client
C- formatter/local
MIZZI
SDB/client thin layer
SDB/local thin layer
MIZZI
Dynamic
Associations
Datastore Server
Index Obj.
Created in active
Servers
Berkeley DB / Sleepycat embedded database manager
Open Source
HERA-B client server infrastructure (DAQ software)
13
The index API design
key
n_servers : int
max_servers : int
n_tables : int
max_tables : int
n_transtabs : int
max_transtabs : int
add2servers( )
add2tables( )
add2transtabs( )
marknotused( )
cleannotused( )
A. Amorim
Index objects can associate
also to transaction objects
which are not data but are
associated to sets of data
objects that must be
considered together.
Tools were also developed
transtabs
tables
servers
nstart
key_server
name : char *
vnew : int
key_table
name : char *
nend major : int
minor : int
vnew : int
key_transtab
x : Tables
keyt : int
transtab major : int
minor : int
vnew : int
line : int
dbname : char *
tablenam e : char *
transtab == line
14
The Parameter Distribution (cont.)
A. Amorim
Calibration/Alignm ent Process
Calibration/Alignm ent Process
5: StoreTable(Table)
Db_Server
These are the
first servers to
startup
Db_Server
Db_Server
2: SUBSCRIBE(Table)
1: SUBSCRIBE(Table)
3: SUBSCRIBE(Table)
6: UPDATE(Table)
7: UPDATE(KeyTable)
KeyDb_server
startup in
second place
C&A Manager
4: SUBSCRIBE(KeyTable)
startup in
third place
15
A. Amorim
16
Basic n-n associations (LEDA)
A. Amorim
•Associations are navigated with iterators
•Using hash tables.
•Keys as OID’s with the scope of classes.
•Explicitly loaded or saved (as containers)
Key
LEDA - Object Manager
(hash table implemented associations)
Active server
interface
Key objects
(referenced by events)
Slow Control interface
key’
Key key’
Client/server
Client/server
Datastore Server
C- formatter/client
C- formatter/local
MIZZI
SDB/client thin layer
SDB/local thin layer
MIZZI
Berkeley DB / Sleepycat embedded database manager
Open Source
HERA-B client server infrastructure (DAQ software)
17
GUI for editing and drawing
A. Amorim
From R&D: JAVA, TCL/TK, gtk
Reusing and extending widget.
Data hidden from TCL/TK
Socket:
Client/ Server
ROOT
database
Binding
18
General Architecture
A. Amorim
109 Evt./y
LEDA - Object Manager
(hash table implemented associations)
Active server
interface
Key objects
(referenced by events)
Slow Control interface
Replication
Client/server
Client/server
Tag and D/Q replication
Datastore Server
Database GUI
Spreadsheet
C- formatter/client
Socket Db GUI
Plotting
C- formatter/local
MIZZI
Tag and Data Quality
Root Object
Persistency
SDB/client thin layer
SDB/local thin layer
MIZZI
Berkeley DB / Sleepycat embedded database manager
Open Source
Tcl/Tk
Open Source
ROOT
Open Source
MySql, RDBMS
Open Source
HERA-B client server infrastructure (DAQ software)
19
The Cache Server
A. Amorim
WAN TCP/IP gateway
User
client
User
client
Cache
Cache
Db- server
User
client
Db- server
Cache
Memory
20
The Replication Mechanism
A. Amorim
firewall
ONLINE
OFFLINE
Offline
DB server
DB server
imported
Incremental
dump files
DB server
Send to
tape
Offline
DB server
21
DAQ configuration
A. Amorim
Fast Control
System
Slow Control
Hardware
Software
Components
Physical General
Hardware
Logical
Hardware
FLT
Subdetector FED
22
Software Com ponent
SMC Configuration
Software Com ponents
Software Com ponent:
Run Configuration
DAQ_SMCCfg
(from DAQ_CONF)
name : char [64]
sst : unsigned int
gsdt : char [256]
sdt : char [256]
Software Com ponent
Process Templates
DAQ_R_UITem p
DAQ_RunCfg
(from DAQ_CONF)
name : char [64]
RunCfg
SMCCfg
DAQ_R_RunProc
IUup
DAQ_R_SMCCfg
IUdown
DAQ_ProcTemp
(from DAQ_CONF)
type : const char[64]
name : char[64]
path : char[64]
exec : char[64]
env : char[256]
args : char[256]
restart : unsigned int
ProcCfg
ProcCfg
ProcTemp
DAQ_R_ProcTemp
ProcCfg
ProcTemp
DAQ_R_TempNgrp
ProcCfg
DAQ_R_ProcNgrp
NodeGrp
A. Amorim
DAQ_ProcCfg
(from DAQ_CONF)
grname : char[64]
domain : char[64]
name : char[64]
range : unsigned int [2]
env : char[256]
args : char[256]
init : char[64]
init_args : char[256]
external : unsigned int
critical : unsigned int
restart : unsigned int
synch : unsigned int
fini : char[64]
fini_args : char[256]
DAQ
(Software
Components)
up
NodeGrp
DAQ_NodeGrp
(from DAQ_CONF)
name : char[32]
type : char[32]
down
DAQ_R_ProcCfg
Software Com ponent
Process Configuration Trees
23
VDS databases
A. Amorim
24
Maintaining the system
A. Amorim
• A slow control process is permanently checking the state
of the database servers.
• It issues alarms for the detector shift crew
• Tools to start and stop the dynamic configuration of
database servers are to be used by a set of experts.
• The configuration and startup of the distributed database
server system is performed using a special configuration
database for this system.
25
Conclusions
ONLINE:
• Large number of Clients => Gigabytes per Update
A. Amorim
•broadcast simultaneously to SLT
• tree of cache database servers to the 4LT
• Correlates (dynamically) each event with the databases objects
• 600 k SLC parameters using data and update objects
•parameter history is re-clustered on the database servers
• The online database system has been successfully commissioned
OFFLINE:
• Replication mechanism decouples online from offline
•also provides incremental backup of the data
• TCP/IP gateways and proxies
• “data warehousing” for data-set classification -> MySQL
• Relation to event tag under evaluation
• Also providing persistency to ROOT objects
Using Open Source external packages has been extremely useful.
26
Future directions ...
A. Amorim
One must have a plan even if it is a wrong one ...
Berkeley DB
Client
Client Client
MySQL
Persistent State Service Persistent State Service
PSS
PSS
CORBA
Client
Client
(ORBacus)
Open Communications Interface (OCI)
IIOP
WAN
UDP/ based
Farms
27