Transcript No Slide Title

Lecture Note 5
Discrete Random Variables
and Probability
Distributions
©
Random Variables
A random variable is a variable that
takes on numerical values determined
by the outcome of a random experiment.
Discrete Random Variables
A random variable is discrete if it
can take on no more than a
countable number of values.
Discrete Random Variables
(Examples)
1. The outcome of a roll of a die.
2. The number of defective items in a
sample of twenty items taken from a
large shipment.
Continuous Random Variables
A random variable is continuous if
it can take any value in an interval.
Continuous Random Variables
(Examples)
1.
2.
3.
4.
Heights of students in a class
Average temperature in November
The income in a year for a family.
The amount of oil imported into the
U.S. in a particular month.
Discrete Random Variables
We will study the fundamental concepts
of discrete random variables, namely (1)
Probability Distribution Function and (2)
Cumulative Probability Function.
First, we will learn the notation of the
Probability Distribution Function
Understanding the notation for
the probability distribution
function
Suppose you are about to roll a die. Then,
the number you would get after the roll is
a discrete random variable. Let X denote
this random variable.
Then this random variable X can take 6
possible outcomes from S={1, 2, 3, 4, 5, 6}.
Understanding the notation for the
probability distribution function, contd
Then, the probability that the random variable X
takes the value 1 (i.e., the probability that you
get 1 after rolling the die) is 1/6. This can be
conveniently expressed using the following
notation:
P(1)=P(X＝1)=1/6
Similarly, we have
P(2)=P(X=2)=1/6
P(3)=P(X=3)=1/6
.
.
P(6)=P(X=6)=1/6
Understanding the notation for the
probability distribution function, contd
 We can generalize the notation from the
previous slides to other discrete random
variables. Let X be any discrete random
variable which takes k possible values
from S={x1, x2, …, xk}
Conventionally, we use capital letter X
to denotes the random variable, and
small letter x to denote the possible
values that X can take.
Understanding the notation for the
probability distribution function, contd
Now, let x be any values from S={x1, x2, …, xk}
Then, the probability that the random variable X
takes the value x can be written as:
P(x)=P(X=x)
This notation is what it is called “Probability
Distribution Function”. The next slide re-iterates
this definition.
Probability Distribution Function
The probability distribution function, P(x),
of a discrete random variable expresses the
probability that X takes the value x, as a
function of x. That is
P( x)  P( X  x), for all valuesof x.
Probability Distribution Function
-Exercise 1Ex 1-1: Suppose you roll a die. Let X be
the random variable for this experiment.
Find the probability distribution function
P(x) by completing the table in the Excel
Sheet “Probability Distribution Function
Exercise”.
Ex 1-2: Plot the probability distribution
function.
Probability Distribution Function
-Exercise 2Consider the following game. You toss a coin. If
you get the heads, you receive \100. If you gets
the tails, you receive nothing.
The payoff for this game is a discrete random
variable. Let X denote this random variable.
Find the probability distribution of this random
variable by completing the table in “Probability
Distribution Function Exercise”.
Required Properties of Probability
Distribution Functions of Discrete
Random Variables
i.
ii.
Let X be a discrete random variable with
probability distribution function, P(x). Then
P(x)  0 for any value of x
The individual probabilities sum to 1; that is
 P( x )  1
x
Where the notation indicates summation over
all possible values x.
Cumulative Probability Function
The cumulative probability function, F(x0),
of a random variable X expresses the
probability that X does not exceed the value
x0, as a function of x0. That is
F ( x0 )  P( X  x0 )
Where the function is evaluated at all values x0
Cumulative Probability Function
-Example 33-1: Let X be the random variable for a roll
of a die. Find F(1), F(2), F(3), to F(6). Use
the Excel sheet, `Probability distribution
function exercise’.
3-2: Plot F(x) and x.
Derived Relationship Between Probability
Function and Cumulative Probability
Function
Let X be a random variable with probability function
P(x) and cumulative probability function F(x0). Then
it can be shown that
F ( x0 )   P( x)
x x0
Where the notation implies that summation is over all
possible values x that are less than or equal to x0.
Expected Value
Expected value is a similar concept as the
average. More specifically, it is the weighted
average with the weight given by the
probability of each outcome.
For example, consider the following game. You
toss a coin. If you get the heads, you receive
\100. If you get the tails, you receive nothing.
Then the expected payoff for this game is
\100×(probability of getting \100)
+ \0×(probability of getting \0) = \50
More formally, expected value is defined as
follows:
See next slide.
Expected Value
The expected value, E(X), of a discrete random
variable X is defined
E ( X )   xP( x)
x
Where the notation indicates that summation
extends over all possible values x.
The expected value of a random variable is
called its mean and is denoted x.
Understanding the notation of
the expected value
Let X be the random variable which takes
a value in S={x1, x2,..,xk}.
Then the expected value of X is computed
as follows. See next slide
Random
variable X
P(x)
xP(x)
x1
P(x1)
x1P(x1)
x2
P(x2)
x2P(x2)
:
:
:
xk
P(xk)
xkP(xk)
μX=
k
 x P( x )
i 1
k
k
Expected Value Exercise
-Exercise 4 Let X be the random variable that
shows the number of house purchase
contracts that a real estate agent can
achieve in a month. The probability
distribution of X is given by the
following.
Compute the expected number of
house purchase contracts that the real
estate agent can achieve in a month.
x
0
1
2
3
4
5
6
P(x)
0.05
0.07
0.12
0.2
0.3
0.2
0.06
Expected Value: Functions of
Random Variables
Let X be a discrete random variable with
probability function P(x) and let g(X) be
some function of X. Then the expected
value, E[g(X)], of that function is defined as
E[ g ( X )]   g ( x) P( x)
x
Expected Value: Functions of
Random Variables
-Exercise 5Continue using the real estate agent example.
Suppose that the monthly salary for the real
estate agent is given by:
g(x) =$1000+ $1500*x
Ex 5-1: Find the expected monthly salary of the
agent. Use the “Probability Distribution
Function Exercise” Excel Sheet.
Variance and Standard Deviation
Let X be a discrete random variable. The
expectation of the squared deviation about the
mean, (X - x)2, is called the variance, denoted
2x and is given by
  E( X   x )   ( x   x ) P( x)
2
x
2
2
x
The standard deviation, x , is the
positive square root of the variance.
Often we use the Var(X) to denote the variance
and SD(X) to denote the standard deviation
Understanding the procedure to
compute the variance
The table in the next slide summarizes the
procedure to compute variance.
Variable P(x)
X
xP(x)
(x- μX)2P(x)
x1
P(x1)
x1P(x1)
(x1-μX)2P(x1)
x2
P(x2)
x2P(x2)
(x2-μX)2P(x2)
:
:
:
:
xk
P(xk)
xkP(xk)
(xk-μX)2P(xk)
Sum=μX Sum=σ2X
Variance and Standard Deviation
-Exercise 6Continue using the real estate
agent example. Using the table
in the “Probability Distribution
Function Exercises” Excel sheet,
answer the following questions:
Ex 6-1 Compute the variance
Ex 6-2 Compute the standard
deviation
x
0
1
2
3
4
5
6
P(x)
0.05
0.07
0.12
0.2
0.3
0.2
0.06
Summary of Properties for Linear
Function of a Random Variable
Let X be a random variable with mean x , and
variance 2x ; and let a and b be any constant fixed
numbers. Define the random variable Y = a + bX.
Then, the mean and variance of Y are
and
Y  E(a  bX )  a  b X

2
Y
 Var(a  bX )  b 
2
so that the standard deviation of Y is
Y  b X
2
X
Linear Function of a Random
Variable
-Exercise 7 Continue using the example of the real estate agent from
exercise 4, 5, and 6.
 Suppose that the monthly salary for this agent, Y, is
determined in the following method:
Y=$1000+ $1500*X
Ex 7-1: Compute E(Y) using the formula in the previous
slide.
Ex 7-2: Compute the variance and standard deviation of
Y
Standardization of a Random
Variable
Let X be a random variable with mean X and
standard deviation X. Define a new random
variable Z as
Z
Then
E(Z)=0
Var(Z)=1
X  X
X
Exercise 8
Continue using the real estate agent
example. Let X be the random variable for
the possible number of contracts the agent
can achieve in a month.
Standardize X using the formula in the
previous slide, and verify that it has mean
zero and variance 1.
Reviews of Combinatorics and
Common discrete distributions
 Review of Combinatorics
1. Number of orderings
2. Number of combinations
3. Number of sequences of x Successes in n Trials
 Common discrete distributions
1. Bernoulli Distribution
2. Binomial Distribution
Stock Price Movement Example
To motivate the study of combinatorics
and the discrete distributions, let us
consider a simple example of a stock price
movement. The example will utilize
combinatorics and distributions.
Stock Price Movement Example
The price of Stock A today is $10. At the end of
each month the price of the stock A either goes
up by the factor of 1.2 with probability 0.4, or
goes down by the factor of 0.9 with probability
0.6.
This means that, if the price of the stock goes up
this month, then the price of the stock in the
second month will be $10×1.2 =$12. If the price
of the stock goes up this month, and goes down
in the second month, the price of the stock in the
third month will be $10×1.2×0.9 = $10.8.
Stock Price Movement Example, Contd
Suppose that you make the following contract
with a stock broker:
“You have the right to purchase a share of stock
A at the price equal to $37.61 at the beginning of
the 13th month.
Then, if the actual market price of stock A is
higher than $37.61, you purchase the stock and
immediately sell it to make a positive profit
(ignoring fees associated with the contract and
transaction fee). If the actual price is lower than
37.61, you do not have to exercise the right.
Then consider the following question:
See next slide
Stock Price Movement Example,
Contd
Question
What is the probability that you make some
strictly positive profit from this contract
(ignoring all the fees associated with the
contract and buying and selling of the share)?
Answer to this question requires tools such as
combinatorics and Binomial distribution.
In the following slides, we will review
combinatorics and some common distributions.
After studying these concepts, we will come
back to the question.
Review of Combinatorics
-Number of orderingsYou have n cards numbered from 1 to n.
The number of ways you can order this
card is given by
n!=n×(n-1) ×(n-2) ×…×3×2×1
n! reads n factorial
0! is defined to be 1.
Number of ordering example
There are 5 cards numbered from 1 to 5.
What is the number of ways you can
order this card?
Review of Combinatorics
-CombinationsSuppose there are n cards numbered from 1 to
n. If you take x cards out of n cards, the number
of possible combinations of the cards is given by
C
n
x
n!

x!( n  x )!
Where n!=n×(n-1) ×(n-2) ×…×3×2×1
Cnx reads n choose x.
Combinations
-ExampleA personnel officer has eight candidate to
fill four similar positions. What is the total
number of possible combinations of four
candidates chosen from eight?
Review of combinatorics
- sequences of x Successes in n
TraialsConsider a random trial where there are
only two outcomes, “Success” or
“Failure”.
Then, if you make n independent trials,
the number of sequences that contains
exactly x successes is equal to Cnx.
Sequences of Successes
-ExerciseConsider tossing a coin 10 times. Find the
number of ways the head appears exactly
4 times.
Number of sequences
-ExerciseSuppose that the figure below is a road
map of a certain area. If you start from
point A and walk to point B, how many
possible routes can you take? (Suppose
you do not walk back.)
B
A
Bernouli Trial
Bernouli Random Trial with success
probability .
This is a random experiment with two
possible outcomes, “success” or “failure”,
where the probabilities are given by
P(Success) = 
P(Failure) = 1- 
Bernouli Random Variable
Consider a Bernouli Trial with P(Success)= .
Define a random variable X in the following
way.
X=1 if the Bernouli random trial turn out to be success
X=0 if the Bernouli random trial turns out to be a
failure.
Then, X is called a Bernouli Random Variable with
success probability .
Bernouli Distribution
Consider a Bernouli random variable X
with success probability . Then the
probability distribution function for X is
called Bernouli Distribution with success
probability . This is given by
P(0)=(1- ) and P(1)= 
Bernouli Distribution
-ExampleConsider the following game. You toss a
coin. If you get the heads, you receive $1.
If you get the tails, you receive nothing.
Let X be the random variable for the
payoff of this game.
X has the Bernouli distribution with
success probability 0.5.
Mean and Variance of a
Bernoulli Random Variable
The mean is:
 X  E( X )   xP( x)  (0)(1   )  (1)  
X
And the variance is:
  E[( X   X ) ]   ( x   X ) P( x)
2
X
2
2
X
 (0   ) 2 (1   )  (1   ) 2    (1   )
Binomial Distribution
-ExampleI use an example to illustrate Binominal
Distribution.
You inspect a production line by randomly
checking the items. It is known that 5% of the
products from this production line are defect
items.
Suppose you randomly choose 4 items from the
production line. Then, the number of defect
items you would find is a discrete random
variable.
Binomial Probability
-Examples, ContdLet X be the number of defect items in the
4 randomly picked items. What is the
probability that exactly 2 of them are
defect items?
To answer to this question, first, consider
the possible sequences that 2 defect items
are picked.
The number of
possible sequences
you pick 2 defect
items is given in the
table. D denotes the
defect item, and G
denotes the good
item.
Note that the number
of sequences is given
by C42=6.
The possible sequences
you pick 2 defect items
Sequence 1 DDGG
Sequence 2 DGDG
Sequence 3 GDDG
Sequence 4 GGDD
Sequence 5 GDGD
Sequence 6 DGGD
Now, what is the probability of
getting Sequence 1? That is, what is
the probability that you first get D,
second you get D, third you get G
and fourth you get G?
The answer to this question: since the
probability of getting D is 0.05, and
the probability of getting G is (10.05)=0.95, the probability of getting
this sequence is 0.05*0.05*0.95*0.95
=0.002256.
In fact, the probability of getting any
of the 6 sequences is the same; since
each sequence contains exactly 2
defective items and 2 good items, the
probability of getting each sequence
is 0.0520.952=0.002256.
Since there are 6 sequences, the
probability that you get exactly 2
defect items is given by
(0.002256)×6=0.013536.
This is equal to C42(0.05)2(1-0.05)2
We can generalize this problem. Consider a
production line. The probability that an item
from this production line is a defect item is .
Suppose that you pick n products from this
production line. Then, the probability that the
number of defect items is exactly x is given by
P( x)  C  (1   )
n
x
x
n x
•This probability distribution function is called
Binomial Distribution with “success” probability .
Next slide summarizes the binomial distribution.
Binomial Distribution
Suppose that a random experiment can result in two possible
mutually exclusive and collectively exhaustive outcomes, “success”
and “failure,” and that  is the probability of a success resulting in a
single trial. If n independent trials are carried out, the distribution
of the resulting number of successes “x” is called the binomial
distribution. Its probability distribution function for the binomial
random variable X = x is:
P(x successes in n independent trials)=
n!
x
( n x )
P( x) 
 (1   )
x!(n  x)!
for x = 0, 1, 2 . . . , n
Mean and Variance of a Binomial
Probability Distribution
Let X be the number of successes in n independent
trials, each with probability of success . The x follows
a binomial distribution with mean,
 X  E( X )  n
and variance,
  E[( X   ) ]  n (1   )
2
X
2
Binomial Probabilities
- An Example
A sales person randomly visits houses to sell a certain
product. He believes that for each visit, the probability of
making a sale is 0.40.
If the sales person visits 5 houses, what is
the probability that he makes at least 3
sales?
Answer
Let X be the random number for the number of sales. Then,
P(At leaset 3 sale) = P(X ≥ 3) = P(X =3) + P(X = 4)+P(X=5)
5!
P (X  3)  P (3)
(0.4) 3 (0.6) ( 53)  0.2304
3!(5 - 3)!
5!
P (X  4)  P (4)
(0.4) 4 (0.6) ( 5 4 )  0.0768
4!(5 - 4)!
5!
P (X  5)  P (5)
(0.4) 5 (0.6) ( 55)  0.01024
5!(5 - 5)!
P(makes at least 3 sales)=P(3)+P(4)+P(5)
=0.2304+0.0768+0.01024=0.31744
Computing Binomial Distribution
using Excel
Let n be the total number of trials. Let x be the number
of success, and  be the success probability. The Excel
function to compute binomial probabilities is
P(X=x)= BINOMDIST(x, n, , FALSE)
P(X≤x)=BINOMDIST(x,n, , TRUE)
Note that if you put FALSE at the end, it computes the
binomial probability distribution. If you put TRUE at
the end, it computes the cumulative binomial
distribution.
Binomial Distribution
-ExerciseOpen “Binomial Distribution Exercise”
Find the Binomial Distribution Function
for n=50 and =0.3. Then, graph the
Binomial Probability Distribution.
Binomial Distribution Function
with n=50 and =0.3.
Binimial Distribution With n=50 and Success prob=0.3
0.14
P(x): Probability Distribution
0.12
0.1
0.08
0.06
0.04
0.02
0
x: Number of successes
Stock Price Movement Example
Now, we come back to the Stock Price
Movement example that we saw at the
beginning.
First, take a look at how the stock price
may move in the first three months. See
the next slide
Stock price movement for the first three months
Third month
x1 = $14.4
This
month
$10
Second
month
$12
x2 = $10.8
$9
x3 = $8.1
As it can be seen, there are three possible values for
the stock price in the third month. Now answer the
questions in the next slide.
Q1: In the 13th month, how many possible values
for the stock price are there?
Q2. Show that the 4th highest price is $37.61
Q3. What is the probability that the stock price at
the beginning of 13th month is equal to the 4th
highest price, $37.61?
Q4. Remember the contract. The contract is “You
have the right to purchase a share of the stock A
in the 13th month at the price equal to $37.61”.
Then, what is the probability that you get some
positive profit from this contract (ignoring all
the fees associated with buying and selling the
stock).
Joint Probability Functions
Consider two stocks, Stock A and Stock B. Let X
denote the random variable for the return of
stock A. Let Y denote the random variable for
the return of Stock B.
Suppose that X takes 4 possible values 0, 0.05,
0.1, and 0.15.
Further, suppose that Y takes 4 possible values,
0, 0.05, 0.1 , and 0.15.
The joint probabilities of X and Y are given in
the table in the next slide.
Joint Probability Function
-ExampleY return
X return
0
0.05
0.1
0.15
0
0.02
0.1
0.05
0.01
0.05
0.03
0.1
0.1
0.05
0.1
0.1
0.12
0.01
0.01
0.15
0.15
0.13
0.01
0.01
Joint Probability Functions
Let X and Y be a pair of discrete random variables.
Their joint probability function expresses the
probability that X takes the specific value x and
simultaneously Y takes the value y, as a function of
x and y. The notation used is P(x, y) so,
P( x, y)  P( X  x  Y  y)
Marginal Probability Functions
Let X and Y be a pair of jointly distributed random
variables. In this context the probability function of the
random variable X is called its marginal probability
function and is obtained by summing the joint
probabilities over all possible values; that is,
P( x)   P( x, y)
y
Similarly, the marginal probability function of the
random variable Y is
P( y)   P( x, y)
x
Exercise
Open “Joint Probability Exercise”. This
sheet contains the joint probability
example of stock A return (X) and Stock B
return (Y) in the previous example.
Find the Marginal distribution.
Conditional Probability
Functions
Let X and Y be a pair of jointly distributed discrete random
variables. The conditional probability function of the random
variable Y, given that the random variable X takes the value x,
expresses the probability that Y takes the value y, as a function of
y, when the value x is specified for X. This is denoted P(y|x), and
so by the definition of conditional probability:
P ( x, y )
P( y | x) 
P( x)
Similarly, the conditional probability function of X, given Y = y
is:
P ( x, y )
P( x | y ) 
P( y )
Exercise
Continue using the same example.
Compute the following conditional
probabilities.
P(X=0|Y=0.1)
P(Y=0.15|X=0.1)
Independence of Jointly
Distributed Random Variables
The jointly distributed random variables X and Y are
said to be independent if and only if their joint
probability function is the product of their marginal
probability functions, that is, if and only if
P( x, y)  P( x) P( y) for all possible pairsof values x and y.
And k random variables are independent if and only if
P( x1, x2 ,, xk )  P( x1 ) P( x2 )P( xk )
Exercise
Using the example in the “Joint
Distribution Exercise” check to see if X
and Y are statistically independent.
Definition
Let X and Y be jointly distributed discrete
random variables with joint probability
distribution function P(x,y). Then, the
expectation of a function of X and Y,
g(X,Y) is defined as
E[ g ( X , Y )]  
x
 g( x, y)P(x, y)
y
Covariance
Let X be a random variable with mean X , and let Y be a
random variable with mean, Y . The expected value of (X
- X )(Y - Y ) is called the covariance between X and Y,
denoted Cov(X, Y).
For discrete random variables
Cov( X , Y )  E[( X   X )(Y  Y )]   ( x   x )( y   y ) P( x, y)
x
y
An equivalent expression is
Cov( X , Y )  E( XY )   x  y   xyP( x, y)   x  y
x
y
Correlation
Let X and Y be jointly distributed random variables.
The correlation between X and Y is:
  Corr( X , Y ) 
Cov( X , Y )
 XY
Covariance and Statistical
Independence
If two random variables are statistically
independent, the covariance between them is 0.
However, the converse is not necessarily true.
Exercise
Consider the following joint distribution of X
and Y
Value of Y
3
Value 1 0.1
of X
2 0.3
4
0.2
0.4
Exercise: Cov(X,Y), Var(X), Var(Y) and
CORR(X,Y).
Portfolio Analysis
The random variable X is the price for stock A and the
random variable Y is the price for stock B. The market
value, W, of the portfolio is given by the linear function,
W  aX  bY
Where, a, is the number of shares of stock A and, b, is the
number of shares of stock B.
Portfolio Analysis
The mean value for W is,
W  E[W ]  E[aX  bY ]
 a X  bY
The variance for W is,
  a   b   2abCov( X , Y )
2
W
2
2
X
2
2
Y
or using the correlation,
  a   b   2abCorr( X , Y ) X  Y
2
W
2
2
X
2
2
Y
Exercise
Consider Stock A and Stock B. Let X and
Y denote the market price of stock A and
B, respectively. It is known that E(X)=10,
E(Y)=20, Var(X)=2, Var(Y)=4, and
Cov(X,Y)=‒1. Now consider the following
portfolio.
W=5X +4Y.
Find E(W), Var(W).