Transcript Discovery-aimed: p-value and significance

Nikhef Topical lectures on Statistics (3rd day)
14-16 December 2011
All information (slides, code, exercises) can be found at
 http://www.nikhef.nl/~ivov/TopicalLectures
The Higgs boson
09:00-12:30
Faster-than-light neutrino’s
14:00-17:00
Aart Heijboer and Ivo van Vulpen
Simple likelihood plot
Can everybody do this ?
If you want to reproduce this plot, but cannot please let us know,
now or step by later
TMath::Poisson( Nevt_bin, alpha )
http://www.nikhef.nl/~ivov/TopicalLectures/SimpleFit/
Di-photon mass
Excluded cross-sections
4-lepton mass
Significance
s
Counting events in a mass window
Standard Model
SM
10
Higgs
5
Data
12
Ok, now what ?
Distribution of expected number of
events from a large number of LHC’s
SM
SM+Higgs
discovery
exclusion
data
Discovery-aimed: p-value and significance
incompatibiliy with the SM-only hypothesis
SM
10
Higgs
5
Data
12
Expected significance
p-value = 8.35%
1.38 sigma
1) What is the expected significance ?
Observed significance
p-value = 30.3%
0.5 sigma
2) What is the observed significance ?
Discovery-aimed: p-value and significance
SM
Higgs
10
5
Expected significance
p-value = 8.35%
1.38 sigma
3) At what Lumi do you expect to
be able to claim a discovery ?
3 times more
LUMINOSITY
Expected significance
SM
30
Higgs
15
Discovery if p-value < 2.87x10-7
p-value =
0.19%
2.9
sigma
exected p-value
3.6 sigma
observed p-value
Exclusion: when/how do you exclude a signal
incompatibiliy with the SM+Higgs hypothesis.
Excluded if CL < 5%
SM
10
Higgs
5
Data
12
Observed CLs+b
SM
SM+Higgs
CLs+b = 18.5%
1) Observed confidence level ?
Can we exclude the SM+Higgs
hypothesis ?
>5%  NO
Expected CLs+b
2) Expected confidence level ?
Did we expect to be able to exclude
the SM+Higgs hypothesis ?
>5%  NO
CLs+b = 7.0%
Exclusion: What about 1.5 x the Higgs cross-sec ?
Expected CLs+b
SM
10
Higgs
7.5
Data
12
h /hSM  1.50
CLs+b = 2.0%

Observed CLs+b
h /hSM  1.50
CLs+b = 6.8%

3) What σh/σhSM can we exclude ?
Observed excluded cross-section
σh/σhSM , = 1.64
Observed h /hSM to be excluded

Excluded cross-sections
Expected h /hSM to be excluded

We will try to reproduce a few of these numbers
Test statistic
An (event) variable that tries to separate the 2 hypotheses:
1)The number of events
L(   1)
LR 
ˆ)
L(   
2)Likelihood ratio:
L(   1)
LR 
L(   0)
used today
A) Event counting
L(μ) =
Used in ATLAS/CMS
Π Poisson(n i | SM   Higgs)
Loop over all bins
Probability to observe ni events
When you expect λSM+ μλHiggs
B) NN output, or complex
 multi-dimensional fit, …
Exercises
ATLAS-CONF-2011-161
http://cdsweb.cern.ch/record/1406356/files/ATLASCONF2011161.pdf
Mock hγγ analysis and try to
reproduce 4 characteristic ATLAS
numbers for a single mass (126
GeV), indicated by
Quick intro into some of the usefull
functions in the code  higgs.C
In root:
.L higgs.C++
higgs()
Some crucial functions: init_pdfs
ATLAS
our imitation
Global histograms (TH1F*): pdf_bgr and pdf_sig
Some crucial functions: generate_dataset
Some crucial functions: compute_test_statistic
Some crucial functions: get_test_statistic_distribution
See also: plot_with bands (TH1F* h) from utils.h
Some additional helper functions that you need (in utils.h)
double median (TH1F*)
double get_fraction_below(TH1F* h, double x_value)
double get_fraction_above(TH1F* h, double x_value)
TH1F* read_dataset(Tstring filename)