Transcript Configuration Database

Configuration Database
Review
David Forrest
University of Glasgow
CM24 @ RAL :: 1st June 2009
1
Configuration Database
 Overview of the database
 Status
 Future Work
2
Overview :: Scope
 The configuration database deals with
cabling, calibration, geometry and set
value information ONLY
 Its use cases are useful for offline
analysis, reconstructing the
configuration of the experiment
 Eg “Give me the configuration of the
experiment at 19:15 on 31st May
2009”
3
Overview :: Temporal Data
 Imagine uploading some calibration valid for a given
month.
 Store the period of validity and label this the valid time.
 Then later we find a better calibration for that same valid
time. We want to keep both, with the same valid time.
 We also store the time the information was given to the
database, which is the transaction time.
 Now we can get the most up to date calibration by
default, but also recall the other calibrations by meta
data.
 Our database is bi-temporal
 Example holds for geometry (eg misalignment) and
cabling also. Not currently implemented for set values.
4
Overview :: API
 There are many applications which will read from and
write to the database
 The database should not make any impositions upon
their design and implementation
 When something is updated it would be easier to update
code in one place rather than in many places, versions,
etc
 So we have an API, which stands for Application
Programming Interface
 The API is also independent of the reading and writing
applications and the database management software.
 David Forrest will write the API and Database. Malcolm
Ellis will write the code for G4MICE to interact with the
API. Other people may be needed to write other client
applications eg James Leaver for the EPICs interface.
5
API-2
Database
Developer
Control
Room
Apps
DB API
DB
G4MICE
RAL Firewall
Note: Following discussions from
Software session, DB will be hosted
on a new machine in the control
room which must be bought (see
Paul K or Malcolm for details)
Grid Users
6
Overview :: Implementation
 We use the Postgres 8.3 Database
management software. Open source.
 The API is a server with code written in
Java using JDBC. C++ code, etc, can of
course connect to it!
 All the client has to do is interpret the
feedback which will be of a guaranteed
agreed format
 Full database has been built up
incrementally, testing and prototyping. It is
based at RAL, and can be accessed
remotely
7
XML
 It has been agreed that input/output from API will be
in XML
 This means we can always be backwards
compatible. XML is extendable. You can put more
stuff in if you have to.
 API can check XML is well formed, and call the right
functions itself, so potentially all the user need do is
pass an XML file (eg with a geometry) so you barely
need to know even what functions the API has
 A well defined interface between the database and the
client application means that the chance of distributed
bugs over many client applications is reduced.
8
Consistency
 A series of constraints will be included in
the final product to maintain consistency eg
Step VI cannot have three trackers. These
constraints have not been completely
catalogued yet.
 Failure on one constraint results in rolling
back the database to a state just before the
constraint was violated.
 It is envisioned that there will be a log, of
every single operation and query on the
database
9
Backup
 Envisaged to make daily backups of
the database contents to a box on the
mice network for which the disk
contents are regularly backed up
 This will piggy-back on the backup
strategy for other systems
10
Status: Calibrations
 It has been challenging to tie down all
detector groups to a calibration format
 So we won’t. The idea now is to have a
flexible calibration format which does not
require us to know the format of a
calibration in advance
 Calibrations can now be a vector of values
of elements of different types – double
floating point number, string, boolean,
integer
 Meta data included + Temporal overlap
11
Calibrations - 2
12
Status: Cablings
 Just getting to grips with this recently
 Take detectors as roots of cabling - unless theres a
good use case for the configuration DB storing
detector independent cabling
 Previously had a fairly loose approach
 We now have detector specific cabling schemes – are
these complete (so far as is related to a real use case
of the config DB)?
 Every component of cabling should have a unique ID,
often the serial number. This is unique not only over
all components of same type, but over all
components. Eg so a splitter can be assigned an ID
number for each channel without knowing in advance
the type of device it is connecting to.
13
Cablings-2
Prelim – will likely have a couple of
changes to TOF, KL, and arising from
implementation issues.
14
Geometry
15
Status: Set Values
 The following are known and recorded
as being needed:
 Magnet currents (quads, dipoles);
RFCC currents; diffuser thickness;
decay solenoid current(s);
spectrometer solenoid currents; focus
coil currents; target depth & delay
16
Set Values
 We have two types of configurations with regard to set
values. Run config and saved config.
 Run config: This is an XML file sent from EPICS which is
saved by the database. It is not parsed and read into the
database structure, it is only tagged in terms of time,
version and run number. It is used once for a run with a
time validity.
 Reference config: You can tag a configuration and use
it many times. This is read into the database structure.
 If some aspect of a run configuration is implemented in
EPICs but not yet in the configuration database
structure, this does not introduce delay – its stored as a
flat file.
17
Status: API
 The API exists now, and sits at RAL
 It really should be moved to a box which is visible to
everyone. Any offers?
 It currently takes a geometry encoded in XML and
translates that into SQL and uploads it to the
database
 XML is standardised and extendable, and highly
recommended as a means of communicating with the
API, which has good XML parsing functionality already
 In an ideal world, for writing data, just send an xml
file and the API knows what to do eg if its geometry,
calibration, cabling etc.
18
Early Testing
 Geometry almost fully implemented, with valid time (but
not transaction time yet) and full module hierarchy
 60+ geometries were uploaded and typical questions
asked about the geometry at a given time. Valid time
introduces no significant slow down to Postgres.
Transaction time will increase complexity only slightly
 This still has to be translated from SQL response to XML,
and G4MICE reconstructing the hierarchy may or may
not place further obligations on the database, but
current results are promising, unlikely that there will be
any grievous slowdown.
19
Current State + Future Work







Geometry prototyping is going well
G4MICE will very soon be able to, as well as send a geometry
to the db (already done), also reconstruct a geometry from
database into memory and compare that with what it sent.
Would like to work on calibrations and, mainly, cablings, next
Once cablings are fully prototyped, much of the hardest work
will be done
We may still need people to write code which sends requests to
the API and digests response (no db knowledge required)
Progress at this collaboration meeting in defining seams with
other systems, eg G4MICE, control room apps, EPICS, DAQ by
meeting up with JSG, Malcolm, Pierrick, and others
It is likely that the db will be placed in a box in the control
room – probably a new box unless a suitable one is available.
20
Project Plan
 Human resource: David Forrest 50%
 Long term maintenance is an open issue
 Scheduled milestones
 Reconstruction of Geometry: end of June
 Prototyping and Testing of Cabling and
Calibration : end of July
 Save and recall run configurations: end
of August
21
Temporal but not temperamental
22