Transcript Configuration - Indico

CMS Global Trigger
Control and Monitoring
Christian Hartl
CERN Austrian Doctoral Student
June 2009
Institut für Hochenergiephysik, ÖAW
Alexander Winkler
Anton Taurok
Barbara Neuherz
Bernhard Arnold
Christian Hartl
Claudia-Elisabeth Wulz
Herbert Bergauer
Herbert Rohringer
Ivan Mikulec
Josef Strauss
Kurt Kastner
Manfred Jeitler
Markus Eichberger
Michael Padrta
Philipp Glaser
Philipp Wagner
Thomas Schreiner
Thomas Themel
Vasile Mihai Ghete
Compact Muon Solenoid
2
Outline
1.
2.
3.
4.
5.
6.
Global Trigger (GT) in the Context of CMS
GT Software Environment & Architecture
GT Database
GT Configuration
GT Monitoring
GT Online Operation
3
Outline : Next:
1.
2.
3.
4.
5.
6.
Global Trigger (GT) in the Context of CMS
GT Software Environment & Architecture
GT Database
GT Configuration
GT Monitoring
GT Online Operation
CMS @ LHC:
High Luminosity : Need Event Filtering
recapitulation
• CMS designed for high luminosity 1034 cm-1s-1
– proton bunches collide every 25 ns
– bunches contain ~1015 protons
– collision energy √s = 2 x 7 TeV
• need to discern
– huge background
– small fraction of
physics of interest
4
CMS @ LHC:
Two Step Filter: L1 and HLT
• Level 1 Trigger
– hardware (FPGAs / ASICs)
– pipelined, 40 MHz
– reduce event rate
from ~1GHz to 100 kHz
• Higher Level Trigger
– software (computer farm)
– reduce event rate
from 100 kHz to 100 Hz
recapitulation
5
CMS Global Trigger:
Final Level 1 Filter Logic
• menu of trigger algorithms
recapitulation
GTL
Global Trigger Logic
– parallel calculation of >100 algorithms
• complex combinations of simple conditions
– based on CMS trigger objects
• muons (momentum + coord.)
• electrons and jets (energy + coord.)
• energy sums and missing energy
• only selected algorithms contribute
– trigger candidate = logic OR
FDL
Final Decision Logic
6
CMS Global Trigger:
Trigger Decision Distribution
recapitulation
• Does every trigger candidate pass? No…
TCS
• Trigger Control System decides:
Trigger Control System
– every beam crossing (40 MHz)
• candidate triggers too close?
• any CMS system not ready?
DEADTIME
• After a Level 1 Accept?
– all systems send data to DAQ
(data acquisition)
• Global Trigger also sends:
– muon & calorimeter data; decision info
GTFE
Global Trigger Front End
7
8
Outline: Next:
1.
2.
3.
4.
5.
6.
Global Trigger (GT) in the Context of CMS
GT Software Environment & Architecture
GT Database
GT Configuration
GT Monitoring
GT Online Operation
Global Trigger Online Software:
Main Tasks: Configuration & Monitoring
architecture
9
• Why "online"?
– it's for data taking
• hardware configuration
?
– put hardware chips into well defined state
• e.g. trigger menu parameters
– configuration data from CMS database
OK
• hardware monitoring
– system parameters (e.g. detector input status)
– physics parameters (trigger rates & deadtime)
Global Trigger Online Software:
GT Cell (Trigger Supervisor Application)
architecture
•
GT Cell is a web application
– C++ web service (XDAQ = CMS standard)
– node in L1 "Trigger Supervisor" application tree
•
GT Cell is running as a Linux service 24/7
…
–
–
–
–
on a machine in "service cavern" (CMS network)
PCI link to hardware (CAEN VME interface)
configuration & monitoring service
netw. communication with other L1 applications
netw. access to CMS database
10
Global Trigger Online Software:
GT Cell in Distributed Control System
architecture
11
Detector Applications
Tracker, HCAL, ECAL, DT, RPC, CSC...
CMS
Run Control
Data Acquisition, Filtering, Monitoring Applications
DAQ, DQM, HLT…
Trigger Hardware
Clock & Signal Distribution (TTC)
Central Trigger Cell
L1 Trigger Function Manager
Level 1 Trigger Applications
Detector
Global Trigger
Global Muon Trigger
Cathode Strip Chamber Track Finder
Drift Tube Track Finder
Resistive Plate Chamber Trigger
Global Calorimeter Trigger
Regional Calorimeter Trigger
Global Trigger
Global Muon Trigger
…
Global Trigger Online Software:
GT/GMT Software Architecture
Ph. Wagner
Trigger
Menu Cell
Th. Themel
B. Arnold
GT/GMT
Test Cell
Ph. Glaser
A. Winkler
Ch. Hartl
B. Arnold
architecture
T. Nöbauer
I. Mikulec
M. Magrans
Ph. Wagner
G. Kasieczka
Ch. Hartl
Th. Themel
B. Arnold
GT Cell GMT Cell
Trigger Supervisor Framework
commands, operations, control panels
database access, cell communication
CMS
Database
H. Sakulin
B. Neuherz
J. Strauss
B. Arnold
M. Jeitler
GT/GMT Test
Applications
GT Libraries
access to
GT module functions
XDAQ Framework
lightweight C++ web applications
generic access to hardware and database
inter-application network messaging
Scientific Linux CERN
operating system used by CMS
CMS Private Computer Network
12
13
Outline: Next:
1.
2.
3.
4.
5.
6.
Global Trigger (GT) in the Context of CMS
GT Software Environment & Architecture
GT Database
GT Configuration
GT Monitoring
GT Online Operation
Global Trigger Database:
Bookkeeping for Hardware Setups
database
complete hardware setups stored in database
this allows for:
– deterministic hardware configuration:
• full Level 1 trigger configuration defined in advance
– bookkeeping:
• which setup used in which CMS run?
– synchronisation of online and offline software:
• emulation software setup must match hardware setup
– debugging:
• reproduce hardware conditions to debug problems
14
Global Trigger Database:
Hardware Setup: What must be stored?
database
15
• Trigger Logic parameters
– selection of trigger algorithms from menu
• which algorithms allowed to cause trigger?
– physics parameters of trigger conditions
• e.g. energy or momentum thresholds
– down-scaling of trigger algorithms
• consider, say, every 100th trigger candidate only
• Hardware Timing
– alignment of trigger inputs
– readout logic delays
This is a glimpse.
The complete
hardware setup
must be defined
in the database.
Global Trigger Database:
Hardware Setup Data Organization
database
• structured information for all GT modules
• decoded items (human-interpretable names)
TSC
GT key
TSP
L1T Menu Key
16
Global Trigger Database:
Frequently Changing: "Run Settings"
database
• fine-tuned setup: modifiable ad hoc on shift
– "GT Run Settings" Database
17
Global Trigger Database:
"GT Run Settings" Database Update
database
18
• GT Run Settings GUI:
– update "current Run Settings" in the database
• i.e. those which are to be used in the next run
1
2a
trigger sources
mix
2b
selection of
algorithms from
Level 1 Menu
2c
3
down-scaling of
algorithm predecisions
Global Trigger Database:
"GT Run Settings": Selection of Algos
database
19
Global Trigger Database:
Level 1 Trigger Menu Versions
database
20
• trigger menu implemented in firmware
– Global Trigger Logic condition chips:
• main characteristics: firmware version
• threshold definition: register content (set by software)
• Level 1 trigger menu versions bookkeeping
– crucial to analysis of CMS data
• strict versioning enforced in database
• difference between two versions?
– only implementation differs? e.g. trigger objects rescaling
– versions physically different? new firmware? threshold change?
Global Trigger Database:
Level 1 Trigger Menu Data Organization
database
L1T Menu DB
populated by
Trigger Menu Editor
21
22
Outline: Next:
1.
2.
3.
4.
5.
6.
Global Trigger (GT) in the Context of CMS
GT Software Environment & Architecture
GT Database
GT Configuration
GT Monitoring
GT Online Operation
Global Trigger Configuration:
Configure and Start the Run…
23
configuration
data taking with cosmic muons (Global Runs)…
CMS systems coordinated by Run Control:
configure
•
stop
1. GT: trigger sources OFF
2. GT: stop signal to CMS systems
enabled
suspended
stop'
suspend
1. GT: start signal to CMS systems
2. GT: trigger sources ON
(physics, random, calibration)
configured
stop
enable
halted
enable
•
configure
1. GT: hardware setup
2. GT: periodic orbit signal (BC0)
to CMS systems
enable'
•
Global Trigger Configuration:
What Data to Send after Level 1 Accept?
configuration
• Level 1 Accept to the detector:
– GT sends data to Data Acquisition:
• input data from Global Muon and Calorimeter Trigger
• decision word: which algorithms caused the L1A?
– crucial for Level 1 "seeding" of Higher Level Trigger
• Global Trigger readout configuration
– what data is sent, what not?
– send how big a window around the time of L1A?
• 3·25 ns or 5·25 ns
24
Global Trigger Configuration:
No Changes While Running
configuration
25
physics data require stable trigger setup
– no trigger configuration changes while running
– exception 1: pre-scaling of trigger algorithms:
• luminosity drops while running  trigger rates decrease
decrease pre-scaling (in steps) to compensate
– using pre-defined pre-scaling sets (set index 0, 1, 2, …)
– no information lost: event data contain set index
– exception 2: random trigger rate (if "randoms on")
– no information lost: event data show dedicated ID
Global Trigger Configuration:
Operation Transitions in More Detail
configuration
26
additional transitions:
(launched by Run Control)
– define which detector partitions
(there are 32) should be serviced by
the Trigger Control system
(enabled-enabled)
– to compensate for decreasing
luminosity
enabled
suspend
• change prescaling
change prescaling
stop'
(configured-configured)
configured
enable
• repartition
repartition
stop
– reload firmware into chips
from PROMs (necessary when LHC
clock changes)
halted
enable'
(halted-halted)
configure
• cold-reset
cold-reset
suspended
Global Trigger Configuration:
Service Eight Detector Partition Groups
configuration
27
Up to eight runs can be serviced by GT in parallel/independently.
cold-reset
…
cold-reset
7
halted
halted
suspended
stop'
change prescaling
enabled
suspend
suspend
enable'
enabled
e.g.:
muon systems
(DT, DTTF,
CSC, CSCTF)
stop
change prescaling
repartition
configured
stop'
enable
stop
configured
enable
e.g.:
calorimeter systems
(ECAL, HCAL,
RCT, GCT)
configure
configure
repartition
enable'
0
suspended
Global Trigger Configuration:
From Run Control to Global Trigger Crate
configuration
28
• Communication Flow:
Level1 Trigger Online Software
CMS Run
Control
Level 1 Trigger
Function Manager
Run Control Software
Framework (Java)
Central
Cell
Global Trigger
Cell
Global Trigger
Hardware
Trigger Supervisor / XDAQ
Framework (C++)
• Run Control defines:
which detector parts are read out?
– Global Trigger reacts in repartition transition
• assign detector partition to due GT partition (0…7)
• enable detector partition to receive triggers and signals
• listen and react to detector partition's status information
Global Trigger Configuration:
HardwareFunction C++ Class
configuration
29
• translation database – hardware:
database field:
TCS.p0.PHYSICS_TRIGGER_EN
(true / false)
Trigger Control System, partition 0:
physics triggers enabled?
hardware register bits:
C++ class
HardwareFunction
TCS.PTC0_CMD_REG, bit 12
(1 or 0)
register bit for disabling
of physics triggers for partition 0.
• class HardwareFunction defines mapping:
– in central place (documentation!)
– "automatic" configuration
• database field names are HardwareFunction names
• configure by call to HardwareFunction(itemName)::set(value)
Global Trigger Configuration:
Database & Hardware Configuration Editor
configuration
• To browse through a given Global Trigger
setup in the CMS database
– what's in the GT key?
• To compare with the current hardware
configuration
– change hardware settings ad hoc for testing
– online documentation of database items
(and correspondence to hardware)
• To create a new Global Trigger setup
– make new GT key (modify existing one)
30
Global Trigger Configuration:
Database & Hardware Configuration Editor
configuration
• Step 1: Select GT setup from database
– or key from any other Level 1 Trigger subsystem
works for all Level 1
Trigger subsystems
list of available
GT setup keys
click to enter navigator
1
2
3
31
Global Trigger Configuration:
Database & Hardware Configuration Editor
configuration
• Step 2: Navigate through GT setup in DB
1
– modify configuration (in editor, not yet in database)
4
make editable,
enter name
click to store modifications in
database (enter preview)
2
tree navigation
online documention of
database items:
sub-node link
navigation
"what hardware register bits
does it correspond to?"
3
modify item
orange:
editor item modified
red:
hardware value differs
32
Global Trigger Configuration:
Database & Hardware Configuration Editor
configuration
33
• Step 3: Preview & insert new setup ("key") in
database
2
1
double-check: is that what you
want to add in the database?
add setup in database
34
Outline: Next:
1.
2.
3.
4.
5.
6.
Global Trigger (GT) in the Context of CMS
GT Software Environment & Architecture
GT Database
GT Configuration
GT Monitoring
GT Online Operation
Global Trigger Monitoring:
Hardware, Trigger, Configuration, Inputs
monitoring
35
• what's the trigger setup?
– current trigger sources and selected algorithms
• what's the configuration status?
– configured? which key? running?
• how is the trigger performance?
– trigger rates? which algorithms contribute most?
– deadtime? what are the reasons?
• what's the detector input and GT's status?
– status of GT boards and readout? detector inputs?
Global Trigger Monitoring:
Monitoring Data Flow
CMS Level 1 Trigger
GT Cell Page
L1 Page
monitoring
WBM Page
L1 Page server
Linux File System
Global Trigger
GT Cell
service
Monitoring
Collector
Web Based Monitoring
(web server)
Trigger Function
Manager
CMS Database
"SCAL" Application
CMS private network
36
Global Trigger Monitoring:
Output Trigger Rate History
monitoring
• GT output trigger rate (from event number)
– rate = Δ(number of L1As) / Δt
– live update every 5 s (L1 Page)
high rate test: 90 kHz
(mainly artificial
"random" triggers)
Cosmics Run @ Zero Tesla (CRUZET)
37
Global Trigger Monitoring:
Trigger Rates and Deadtime
GT Cell
Trigger Monitor
Panel
run
number
time since start of
run (in luminosity
segments, ~93 s)
instantaneous
trigger (L1A) rate
monitoring
38
generated
triggers (L1As)
rates of individual
trigger candidates
rate history of generated trigger candidates
rate history of lost
trigger candidates
rate
rate
rate
rate
history of
history of
history of
history of
physics
L1As
random L1As
test
L1As
calibration L1As
history of generated deadtime (in %)
- due to trigger rules
- due to calibration cycles
- due to detector/DAQ backpressure
luminosity segment (x 93 s):
164 163 162 …
Global Trigger Monitoring:
Detector Input Status and GT Status
monitoring
GT Cell
Detector & GT
Status Monitor
Panel
39
Monitoring Panel to spot quickly problems in GT or detector
while taking data.
2 CSC partitions:
WARNING
ECAL partitions: BUSY
1 HCAL partition: READY
2 HCAL partitions:
OUT OF SYNCH
2 TRK partitions: ERROR
RPC partition:
not in readout (not yellow)
status is ignored
GT boards and readout: READY
Output Status of TCS
partition 0 = IDLE
(partition 0 not running)
Combined Input Status to TCS = ERROR
(here: because of ERROR in CSC system)
Global Trigger Monitoring:
Trigger Sources, Key, Enabled Algos
monitoring
GT Cell
Configuration
Panel
GT (TCS partition 0) is
configured
physics triggers enabled (no random or calibration triggers)
"algorithms" L1_SingleJet6 and L1_SingleJet10 are enabled
(will forward trigger candidates to the Trigger Control System)
40
41
Outline: Next:
1.
2.
3.
4.
5.
6.
Global Trigger (GT) in the Context of CMS
GT Software Environment & Architecture
GT Database
GT Configuration
GT Monitoring
GT Online Operation
Global Trigger Online Operation:
Trigger Shifter Tasks
operation
• before each CMS run:
– GT basic configuration defined by GT key
• as set in Level 1 Trigger Key GUI
– GT ad hoc settings defined by GT Run Settings
• as set in GT Run Settings GUI
• while CMS is taking data:
– monitoring of rates, deadtime, detector status
• Resync or HardReset in case of detector problem
– change of random trigger rate (high rate tests)
– change of algorithm decision prescaling
42
CMS Global Trigger
Control and Monitoring
Thank you!
more:
Global Trigger official web page:
http://wwwhephy.oeaw.ac.at/p3w/electronic1/GlobalTrigger/GlobalTriggerCrate.htm
collection of wiki articles:
https://twiki.cern.ch/twiki/bin/view/CMS/GlobalTrigger
43