Transcript Probability Review

Paul Munro
-Introduction to Information Science
-IS2000
Probability Review
•
•
•
•
•
Definitions/Identities
Random Variables
Expected Value
Joint Distributions
Conditional Probabilities
Paul Munro
-Introduction to Information Science
-IS2000
Probability Defined
• an event (experiment) has a set of
possible outcomes, each with a
probability, that measures their
relative (anticipated) frequencies of
occurrence normalized to 1.
Paul Munro
-Introduction to Information Science
-IS2000
Probability Identities
E  {e1,e2, ,en }
Probability of each outcome: pi  Pr(ei )
Probability distribution: (p1, p2, , pn )
Events and outcomes:
0  pi  1 for all i

n
 pi  1

i1
Paul Munro
-Introduction to Information Science
-IS2000
Joint Distributions
• Two (or more) events
• Each event has an outcome
• Joint distribution stipulates the
probability of every combination of
outcomes
IS2000
--
Introduction to Information Science
--
Paul Munro
Two Events
E  {e1, ,en }
E F 
F  { f1, , f m }
{(e1, f1 ), ,(e1, f m ),
(e2 ,
f1 ), ,(e2 , f m ),
(en , f1 ), ,(en , f m )}
IS2000
--
Introduction to Information Science
--
Paul Munro
Random Variables
Random variables
A rando m va riable X takes on a value from a given set. Thus, it is an event where the
outcome s x1, x2, ..., x N have numerical values.
N
The expe cted value of X is
 pi x i .
i 1
Example:
Find the expected value of X if
Pr(X = 2) = 0.15
Pr(X = 6) = 0.20
Pr(X = 5) = 0.45
Pr(X = 8) = 0.20
Answer:
N
 pi x i
i 1
 (0.15)(2)  (0.45)(5)  (0.20)(6)  (0.20)(8)  0.3  2.25  1.20  1.60  5.35
Paul Munro
-Introduction to Information Science
-IS2000
Multiple Random Variables
More than one random variable
Two random variables X and Y , with X {x1, x2, ..., x N} and Y {y1, y2, ..., y M}
Let X and Y be two simultaneous events with outcomes xi and yj. This joint event
has a probabili ty p(xi , yj ). These probabilit ies can be written in matrix form. Note
that the row s sum to the total probabili ty of the correspondin g xi , and the column s
sum to the total probabili ty of the correspondin g yj.
p(x1 , y1 )
p(x 1 ,y 2 )
...
p(x 1 , y M )

p(x1 )
p(x 2 , y1 )
p(x 2 ,y 2 )
...
p(x 2 , y M )

p(x 2 )
...
...
...
...
p(x N ,y 1 )
p(x N , y 2 )
...
p(x N , y M )


p(y1 )
p(y 2 )
...


...
p(y M )
p(x N )


1
IS2000
--
Introduction to Information Science
--
Paul Munro
Joint probability matrix
p(x1 , y1 )
p(x 1 ,y 2 )
...
p(x 1 , y M )

p(x1 )
p(x 2 , y1 )
p(x 2 ,y 2 )
...
p(x 2 , y M )

p(x 2 )
...
...
...
...
p(x N ,y 1 )
p(x N , y 2 )
...
p(x N , y M )


p(y1 )
p(y 2 )
...


...
p(x N )


p(y M )
1
The sums of the columns and rows are ma thematically expressed as follows :
Rows:
p(x i ) 
M
N
 p(x i ,y j )
Columns:
p(y j )   p(x i , y j )
j1
i1
The sum of all the joint probabili ties is 1:
N M
N
M N
M
i 1 j 1
i1
j1i 1
j 1
  p(x i , y j )  p(x i )    p(x i , y j )   p(y j )  1 .
Paul Munro
-Introduction to Information Science
-IS2000
Conditional Probability
• Random variables are often NOT
independent
• P(rain in Pittsburgh), P(rain in Monroeville),
P(rain in New York), P(rain in Hong Kong)
• P(Heads up), P(Tails down)
• P(D1=5), P(D2=6)
• P(D1=1), P(D1 + D2=2)
IS2000
--
Introduction to Information Science
--
Paul Munro
Dice Example
IS2000
--
Introduction to Information Science
--
Paul Munro
Conditional Probability
A
p(x i | y j ) 
AB
p(x i , y j )
p(y j )
OR
B
P(A|B) =
P(AB)
P(B)
p(x i ,y j )  p(x i | y j )p(y j )  p(y j | x i ) p(x i )
p(x 1 , y1 )  0.1
p(x 1 , y 2 )  0.0
p(x1 ,y 3 )  0.2
p(x 2 ,y 1 )  0.1
p(x 2 , y 2 )  0.1
p(x 2 ,y 3 )  0.5
p(y1) = 0.2
p(y2) = 0.1
p(y3) = 0.7
p(x 1 | y1 )  0.5
p(x1 |y 2 )  0.0
p(x1 | y 3 ) 
p(x 2 | y1 )  0.5
p(x 2 | y 2 )  1.0
p(x 2 | y 3 ) 
IS2000
--
Introduction to Information Science
--
Paul Munro
Example
2
7
5
7
Paul Munro
-Introduction to Information Science
-IS2000
Markov Processes
• State transition probabilities
• Matrix or Diagram
• Matrix Multiplication predicts
multiple transition probabilities
• Mk Converges to steady state