Transcript Slide 1 - The Eclecticon of Dr French

Conditional Probability and Bayes’ Theorem
Consider two events A and B. Each can either occur or not occur. Event A not occurring is denoted by A’ and B not occurring is denoted by B’.
We can construct TWO tree diagrams to map out the possible permutations of outcomes.
The probability of A occurring given B has occurred is P(A|B). Events A and B occurring is P(A,B). Note the order does not matter for the latter.
P  B A
P  A B
B
B
A
P  A
P  A '
A
P  B ' A
B’
P  B A '
B
P  B
P  B '
P  A' B
P  A B '
A’
A
B’
A’
P  B ' A '
B’
P  A ' B '
A’
Now P(A,B) = P(B,A), hence
P( B) P( A | B)
P ( A)
P( B) P( A | B)
P ( B | A) 
P ( B ) P ( A | B )  P ( B ') P ( A | B ')
1
P ( B | A) 
P ( B ') P ( A | B ')
1
P( B) P( A | B)
P ( B | A) 
P( A) P( B | A)  P( B) P( A | B)
This is called Bayes’ Theorem, and allows P(B|A) to be computed from P(A|B), P(A) and P(B)
Mathematics topic handout: Conditional probability & Bayes Theorem Dr Andrew French. www.eclecticon.info PAGE 1
Conditional probability example
P(A|B) = 1/3, P(A) = 1/4 and P(B) = 1/5 : Find all the other probabilities in both tree diagrams corresponding to events A and B .
P  B A
B
3
4
A
2
3
A’
B
A
1
4
1
3
P  B ' A
P  B A '
B’
B
A’
P  B ' A '
1
 1  1 
P ( B | A)     
4
 5  3 
4
 P ( B | A) 
15
11
 P ( B ' | A) 
15
1
5
2
5
P  A B '
B’
P  A ' B '
B’
3
 1  2 
P ( B | A ')     
4
 5  3 
8
 P ( B | A ') 
45
37
 P ( B ' | A ') 
45
A
A’
2
1
 1   11 
P ( A | B ')    P ( B ' | A)     
5
4
 4   15 
55
 P ( A | B ') 
120
65
 P ( B ' | A) 
120
Mathematics topic handout: Conditional probability & Bayes Theorem Dr Andrew French. www.eclecticon.info PAGE 2
Consider a medical test (Pass, Y or Fail, N) for a disease. The probability of passing (or indeed failing) is conditional upon whether the patient
actually has the disease (true, T) or not (false, F). Unfortunately the latter is what we want to infer from the test, not the other way round.
Tests are never perfect, and there will be four possible outcomes:
Thomas Bayes
Possible outcome
Conditional probability
1701-1761
Pass test given person actually has the disease
P(Y|T) = t
Fail test given person actually has the disease (“false negative”)
P(N|T) = 1-t
Pass test given person doesn’t actually have the disease (“false positive”)
P(Y|F) = q
Fail test given person doesn’t actually have the disease
P(N|F) = 1-q
The middle two options are obviously undesirable, and often ignored by medical practitioners.
Probabilities we are really interested in
Reality (or model thereof). We could work this out
from historical statistics
Obviously these two views are
equivalent, hence
PT Y P(Y )  PY T PT 
This is Bayes’ Theorem
Now note:
P(Y )  PY T PT   PY F PF 
PT Y  
PY T PT 
P(Y )
Bayes’ Theorem

PY T PT 
PY T PT   PY F PF 

1
PY F PF 
1
PY T PT 
 PT Y  
1
q(1  p)
1
tp
Mathematics topic handout: Conditional probability & Bayes Theorem Dr Andrew French. www.eclecticon.info PAGE 3
Example: Testing a member of the public at random for a disease.
The chance of having disease is p = 1/1000
The disease test is 95% accurate. i.e. t = 1 - q = 0.95, and symmetric!
(i.e. false positive is equally unlikely as a false negative)
Probability of having the disease
given a positive test result is:
PT Y  
1
q(1  p)
1
tp
1

0.05  0.999
1
0.95  0.001
 1.87%
Note if q = 1 - t
k
1
1 p
t 
(1  t )(1  p)
p
1
1
tp
k
i.e. because of the low probability of actually having the disease in the first place
the overall probability of someone having the disease given a positive test result
is actually very low! To give a better than 95% accurate result, our test accuracy t
must be t = 0.999/(0.001/0.95 + 1) = 99.8% accurate.
Example from The Signal and the Noise by Nate Silver p247
PRIOR PROBABILITY
Initial estimate of how likely it is that terrorists would crash
planes into Manhattan skyscrapers
p
0.005%
A NEW EVENT OCCURS: FIRST PLANE HITS WORLD TRADE CENTER
Probability of plane hitting if terrorists are attacking
Manhattan skyscrapers
t
100%
Probability of plane hitting if terrorists are not attacking
Manhattan skyscrapers (i.e. an accident)
q
0.008%
POSTERIOR PROBABILITY
Revised estimate of probability of terror attack, given first
plane hitting World Trade Center
1
q (1  p )
1
tp
38%
But then probability of terror attack, given second
plane hitting the World Trade Center is 99.99%
since we re-do the analysis but set p = 38%
Mathematics topic handout: Conditional probability & Bayes Theorem Dr Andrew French. www.eclecticon.info PAGE 4
Cluedo example
PRIOR PROBABILITY
Probability of Colonel Mustard being of murderous intent
p
5%
Probability of Professor Plum meeting his doom given
Colonel Mustard is a potential murderer
t
50%
Probability of Professor Plum dying given Colonel Mustard is
not feeling particularly murderous. i.e. he dies via natural
causes, or someone else kills him....
q
1%
P
?
CONDITIONAL PROBABILITIES: LIKELIHOODS
POSTERIOR PROBABILITY
Probability of Colonel Mustard being the murderer of
Professor Plum, given Professor Plum is observed to be dead
P
P
PY T   t
1
1

 72%
q(1  p)
1 95
1
1
tp
50  5
PT   p
PF  1  p
Plum dies (Y)
C. Mustard
is a murderer
PN T  1  t
Plum lives (N)
PY F   q
Plum dies (Y)
C. Mustard
not a
killer
PN F  1  q Plum lives (N)
p
Mathematics topic handout: Conditional probability & Bayes Theorem Dr Andrew French. www.eclecticon.info PAGE 5
PT   p
PF  1  p
PY T   t
Pass test (Y)
PN T  1  t
Fail test (N)
PY F   q
Pass test (Y)
Hypothesis
true (T)
Hypothesis
not true (F)
PN F  1  q Fail test (N)
Mathematics topic handout: Conditional probability & Bayes Theorem Dr Andrew French. www.eclecticon.info PAGE 6