Transcript Lec8Probability

Lecture 8
5.2 Discrete Probability
5.2 Recap
• Sample space: space of all possible outcomes.
• Event: subset of of S.
• p(s) : probability of element s of S:
 p( s)  1
s 0  p ( s )  1
sS
•
p( E )   p(s)
sE
• Probability of complement p( E )  1  p( E )
n
• Disjoint events:
p(
i 1
n
Ei )   p ( Ei )
• Overlapping events: p( E1
i 1
E2 )  p( E1 )  p( E2 )  p( E1
• Three doors problem (see ex. 39 p. 362 + matlab demo)
E2 )
5.2
We flip a coin 3 times with Heads and Tails equal probability.
There are 8 possibilities: |S| = 8
S
HHH
E
HTT
THT
HHT
THH
TTH
TTT
HTH
p( E ) 
1
2
P( F ) 
F
1
2
P( E
F) 
E: event that an odd number of tails occurs
F: first flip comes up tails
3
4
P( E
F) 
1
4
5.2
S
E
HHT
THT
THH
HHH
p( E ) 
THH E|F
TTH
F
TTT
HTH
THT
HTT
TTT
TTH
1
2
P( F ) 
1
2
P( E
F) 
3
4
P( E
F) 
1
4
F
What is the probability of odd number of tails (i.e. of event E) if we know
that the first flip was tails (i.e. F happened) ?
 If we know that F happened the total number of possible outcomes shrinks to |F|
E = THH, THT, TTH, TTT.
Of those there are two that make event E happen: THH, TTT.
P(E|F) = 1/2
5.2
Theorem: If E and F are two events and p(F)>0, then the
conditional probability of E given F is given by:
p( E F )
P( E | F ) 
P( F )
Example:
What is the probability that a family with 2 kids has two boys, given that they
have at least one boy? (all possibilities are equally likely).
S: all possibilities: {BB, GB, BG, GG}.
E: family has two boys: {BB}.
F: family has at least one boy: {BB, GB, BG}.
E
F = {BB}
p(E|F) = (1/4) / (3/4) = 1/3
GG
F
E
BB
BG
GB
5.2
S
GG
S
F
E
BB
HTT
E
HHT
BG
GB
HTH
THT
THH
TTT
TTH
F
HHH
p(E) = ¼
P(E|F) = 1/3
By knowing F has happened,
I have changed the probability
that E has happened: they are
dependent
P(E)=1/2
P(E|F) = 1/2
By knowing F has happened,
I have NOT changed the probability
that E has happened: they are
independent
5.2
Equivalent statement: The probability of the intersection od 2 events
is the product of the probabilities of the separate events.
P( E | F )  P( E ) 
P( E F )
 P( E )  P( E
P( F )
F )  P( E ) P( F )
Example:
A family has 2 children (|S|=4).
Is the event E that the family has children of both sexes independent from
the event F that they have at most one boy?
E: {BG, GB}
F: {GG, BG, GB}
E
F: {BG, GB}
P( E F )  1/ 2
P( E ) P( F )  1/ 2  3 / 4  3 / 8
dependent
5.2
What about a family with 3 kids ?
|S| = 8
|E| = 8 – 2 = 6 (only BBB and GGG violate E).
|F| =| {GGG, GBB, BGB, BBG} | = 4
| E1
E2 | =| {GBB, BGB, BBG}| = 3
P( E F )  3 / 8
P( E ) P( F )  3 / 4 1/ 2  3 / 8
independent !
(again: it’s hard to get any intuition for this).
5.2
Problem:
We toss a coin 7 times, but the coin biased with probability 2/3 heads.
What is the probability that we find 4 heads in those 7 tosses?
There are C(7,4) ways to generate a sequence with 4 heads and 3 tails.
(thinks of a bit-string with 0’s and 1’s).
Each one of them has probability (2/3)^4 x (1/3)^3.
Total: probability: C(7,4) x (2/3)^4 x (1/3)^3
More general:
What is the probability of finding k 1’s in a bit-string of length n, when the
probability of finding a 1 is p?

P(k , n, p) 
n!
p k (1  p) n k
k !(n  k )!
Binomial distribution.
(matlab demo)
5.2
Does it sum to 1?
Recall:
n!
( p  q) 
p k q nk
k !(n  k )!
n
Use q = 1-p to prove the result.
5.2 Random Variables
Many problems deal with real numbers rather than set memberships.
E.g. What is the sum of the outcomes of 2 dice?
How many times do we expect to 7 1 in bit-strings of length 12?
We were able to answer those questions before, but we now introduce
some formal machinery:
Definition: A random variable X is a function (not a variable!) from sample space
to the real numbers. It assigns a real number to each possible outcome.
(and is not random!).
Example:
S={apple, pear, banana}  X(apple) = 1, X(pear) = 2, X(banana) = 3.
X: the number of times heads comes up when we toss a coin 2 times:
X(TT) = 0;
X(HT) = 1; X(TH) = 1;
X(HH) = 2;
5.2
Definition: Let P(X=r) be the probability that X takes a value r.
A distribution of a random variable X on a sample space S is the set of all
pairs (r,P(X=r)) for all r in X(S). Thus, we specify the distribution by providing all
possible P(X=r).
Example:
X is the number of heads of 2 coin tosses:
P(X=0) = ¼
P(X=1) = ½
P(X=2) = ¼
5.3
Definition: The expected value of a random variable X(s) on the sample space S
is given by:
E ( X )   X ( s ) p( s)
sS
Sometimes, it’s more efficient to compute expectations by clumping together
all elements of S that result in the same value for X(s).
p( X  r ) 

p( s)
generally true
sS , X ( s )  r
 K (r ) p ( s )
only true when all p(s) s.t. X(s)=r have
equal probability.
number of elements in S
such that X(s)=r
5.3
Example:
X is number of heads in 2 coin tosses. Expected value?
E(X) = 0 x P(X=0) + 1 x P(X=1) + 2 x P(X=2) =
0
+½
+ 2x ¼ = 1.
We expect on average that we see 1 head.
Example:
X is the value of the number that comes up on a die. Expected value?
E(X) = 1 x 1/6 + 2 x 1/6 + 3 x 1/6 + 4 x 1/6 + 5 x 1/6 + 6 x 1/6 = 21/6 = 7/2.
(matlab demo2)