Transcript X - Mysmu .edu mysmu.edu

Chapter 2: Random Variable
and Probability Distributions
Yang Zhenlin
[email protected]
http://www.mysmu.edu/faculty/zlyang/
Chapter Contents
Chapter 2
Random Variable (r.v.)
– Discrete r.v.
– Continuous r.v.
Distribution of a Random Variable
– Probability mass function (pmf)
– Probability density function (pdf)
Expectation of a Random Variable
Variance and Standard Deviation
Moment Generating Function
STAT151, Term II 2015-16
2
© Zhenlin Yang, SMU
Random Variable
Chapter 2
In real-world problems, we are often faced with one or more
quantities that do not have fixed values.
The values of such quantities depend on random actions and
they change from one experiment to another,
Number of babies born in a certain hospital
Number of traffic accidents occurred on a road per month
The amount of rainfall in Singapore per year
The starting price of a stock each day, etc.
In probability, quantities introduced in these diverse examples
are called Random Variables.
Study the properties of random variables allows us to have a
better understanding on the real-world phenomena, and to
control or to predict their behavior, … .
STAT151, Term II 2015-16
3
© Zhenlin Yang, SMU
Chapter 2
Random Variable
Example 2.1 If in the experiment of rolling two fair dice, X is
the sum, then X is a variable. Since the value of X changes from
one experiment to another, X is also a "random variable". The
possible values of X and the corresponding probabilities are
summarized as follows:
X
pi
2
3
4
5
6
7
8
9
10
11
12
1
2
3
4
5
6
5
4
3
2
1
36
36
36
36
36
36
36
36
36
36
36
where, for example,
p2 = P(X=2) = P{(1,1)} = 1/36,
p3 = P(X=3) = P{(1,2), (2,1)} = 2/36,
p7 = P(X=7) = P{(1,6), (6,1), (2,5), (5,2), (3,4), (4,3)} = 6/36
STAT151, Term II 2015-16
4
© Zhenlin Yang, SMU
Random Variable
Chapter 2
In Example 2.1, numerical values of X depend on the
outcomes of the experiment, e.g.,
• if the outcome e = (2,3), then X = 2+3 = 5,
• if e = (5,6), then X = 5+6 = 11, etc.
Thus, X is a real-valued function.
On the other hand, if, for example, X = 3, then it must be that
e = (1,2) or (2,1). Therefore, the inverse image of the function
X is an event!
Definition 2.1. Let S be the sample space of an experiment. A
real-valued function X defined on S and taking values in R (the
set of real numbers) is called a random variable (r.v.) if for
each interval I  R, the inverse image {e: X(e)  I } is an event.
STAT151, Term II 2015-16
5
© Zhenlin Yang, SMU
Chapter 2
Random Variable
Example 2.2 Suppose that we toss a coin having a probability p
of coming up heads, until the first head appears. Let N denote the
number of flips required, then assuming the outcome of
successive flips are independent, N is a random variable taking
values 1, 2, 3, …, with respective probabilities
P{N =1} = P(H) = p,
P{N =2} = P(TH) = (1p)p,
P{N =3} = P(TTH) = (1p)2p,

n 1
P{N  n}  P(TT

T
H
)

(
1

p
)
p

n 1
All probabilities add up to 1: pn 1 (1  p)

STAT151, Term II 2015-16
6
n 1
p

1
1  (1  p)
© Zhenlin Yang, SMU
Distribution of a Random Variable
Chapter 2
Definition 2.2 (Cumulative Distribution Function) The
cumulative distribution function (CDF) of a random variable X
is defined by,
F(x) = P{X ≤ x},
for any real x.
F(x) “cumulates” all of the probabilities of the values of X up to
and including x.
A function F(x) is a CDF if and only if it satisfies:
(a) limx  F(x) = 0 and limx  F(x) = 1
(b) limh0 F(x+h)= F(x), h > 0 (right continuous)
(c) a < b implies F(a) ≤ F(b) (nondecreasing).
STAT151, Term II 2015-16
7
© Zhenlin Yang, SMU
Chapter 2
Distribution of a Random Variable
Example 2.3. A four-sided die has a different number, 1, 2, 3, or
4, affixed to each side. On each roll, each of the four numbers is
equally likely to occur. A game consists of rolling the die twice.
If X represents the maximum of the two rolls, then X is a
random variable. Find and sketch the CDF of X
Solution: Possible values for X are 1, 2, 3, and 4, and the corresponding
probabilities are:
P(X = 1) = P{(1,1)} = 1/16
P(X = 2) = P{(1,2), (2,1), (2,2)} = 3/16
P(X = 3) = P{(1,3), (2,3), (3, 3), (3,2), (3,1)} = 5/16
P(X = 4) = P{(1,4), (2,4), (3,4), (4,4), (4,3), (4,2), (4,1)} = 7/16.
F(x)
1
Thus, the CDF of X is,
x
F(x)
1
2
3
4
1/16
4/16
9/16
1
9/16
1/4
1/16
0
STAT151, Term II 2015-16
8
1
2
3
4
x
© Zhenlin Yang, SMU
Distribution of a Random Variable
Chapter 2
Example 2.4. A function is specified as follows.
0
x 4

F(x) = 1 2
 x 12  1 2
1
x0
0  x 1
1 x  2
2 x3
x  3.
(a) Verify that it is a CDF.
(b) Plot F(x) and calculate: P(X < 2),
P(X = 2), and
P(1 ≤ X < 3).
Solution: (a) … ,
F(x)
1
(b) : P(X < 2) = F(2–) = 1/2.
P(X = 2) = F(2) – F(2–)
= (2/12+1/2)–1/2 = 1/6.
3/4
2/3
1/2
P(1 ≤ X < 3) = P(X < 3) – P(X < 1)
= F(3–) – F(1–)
= (3/12+1/2) – 1/4 = 1/2.
STAT151, Term II 2015-16
1/4
0
9
1
2
3
x
© Zhenlin Yang, SMU
Distribution of a Random Variable
Chapter 2
Note. Random variables can be classified as discrete, continuous,
or mixed. The function given in Example 2.4 is a mixed CDF
which has “jumps” at certain points. We will concentrate on pure
discrete and pure continuous types of random variables.
Definition 2.3 (Discrete R.V. and Probability Mass Function)
If the set of all possible values of a random variable, X, is a
countable set, x1, x2, . . . , xn, or x1, x2, . . ., then X is called a
discrete random variable. The function
p(x) = P(X = x),
x = x 1 , x2 , . . .
that assigns the probability to each possible value of X will be
called probability mass function (pmf).
So, pmf tells how the probabilities are distributed across the values of r.v. X.
STAT151, Term II 2015-16
10
© Zhenlin Yang, SMU
Distribution of a Random Variable
Chapter 2
A function p(x) is a pmf if and only if
(i) p(x) ≥ 0 for all x, and (ii)  p ( x )  1.
all x
Example 2.5. Is each of the following functions a pmf? If it is,
find the constant k.
(a) p(x) = k(1/2)x , x = 0, 1, 2, . . .
(b) p(x) = k [(1/2)x – 1/2], x = –2, –1, 0, 1, 2.
Solution: (a) p(x) > 0 for all x = 0, 1, 2, . . . ; and
2
p
(
x
)

k
(
1

1
/
2

(
1
/
2
)
 )  2k ,  k  1 / 2

all x
(b) Since p(0) = k/2, and p(2) = –k/4, the two values of the function have
opposite signs no matter what value k takes. Therefore, it is impossible for
p(x) to be always nonnegative, and hence p(x) cannot be a pmf.
STAT151, Term II 2015-16
11
© Zhenlin Yang, SMU
Distribution of a Random Variable
Chapter 2
The r.v's introduced above take countable number of values.
However, many social phenomena can only be described by
variables with uncountable values, e.g., arrival time of a train,
lifetime of a transistor, etc.
Definition 2.4 (Continuous R.V. and Probability Density
Function) A r.v. X is said to be a continuous random variable if
there exists a nonnegative function f, defined for all real x  (– ,
), such that for any set B of real numbers
P{X  B} =
B f (x)dx.
The function f is called the probability density function (pdf) of
the r.v. X.
So, pdf f(x) tells how likely values of r.v. X occurs around x.
STAT151, Term II 2015-16
12
© Zhenlin Yang, SMU
Distribution of a Random Variable
Chapter 2
Some important properties can be deduced immediately from Definition 2.4:
i)
ii)
t
F(t) =   f (x)dx (let B = (–  , t] );
b
P(a ≤ X ≤ b) = a f (x)dx ( let B = [a, b]);
a
iii) P(X = a) = a f (x)dx = 0 (let a = b in ii) )
iv) The pdf can be obtained from CDF by differentiation:
d
'
f(x) = F (x) =
F(x), for all x where F'(x) exists.
dx
Note that it is possible that F'(x) does not exist at finite number of points,
where f(x) can be defined as either ‘left’ or ‘right’ derivatives.
STAT151, Term II 2015-16
13
© Zhenlin Yang, SMU
Distribution of a Random Variable
Chapter 2
Example 2.6 A CDF has the following expression
0,
 x,
F(x) = 0.25  0.5 x,

1,
x0
0  x  0.5
0.5  x  1.5
1.5  x
Sketch the graph of F(x) and find the pdf f(x).
Solution:
The graph of F(x) is shown below.
Clearly, F(x) is not differentiable at x
= 0, 0.5 and 1.5. Take the ‘right’
derivatives, i.e., f(0) = 1, f(0.5) = 0.5,
and f(1.5) = 0, we then have
F(x)
1
0,
1,
f ( x)  
0.5,

0,
.5
0
.5
STAT151, Term II 2015-16
1
1.5
x
14
x0
0  x  0.5
0.5  x  1.5
1.5  x
© Zhenlin Yang, SMU
Distribution of a Random Variable
Chapter 2
A function f(x) is a pdf if and only if it satisfies the properties
(i) f(x) ≥ 0 for all x
and
(ii)



f ( x)dx  1
Example 2.7 A machine produces copper wire, and occasionally
there is a flaw at some point along the wire. The length of the
wire (in meters) produced between successive flaws is a
continuous r.v. X with pdf of the form:
f(x) = c(1+x)3, x > 0,
where c is a constant. Find c, and CDF.
1
Solution: 1 =   f (x)dx = 0 c(1 x) dx = c   ,  c =2.
2
x
Now, P(X ≤ x) = 0 2(1  t) 3 dt = 1 – (1+x)–2, hence
x0
0,
F ( x)  
2
1  (1  x) , x  0

STAT151, Term II 2015-16

3
15
© Zhenlin Yang, SMU
Expectation of a Random Variable
Chapter 2
Definition 2.5. (Expectation) The expected value of a discrete r.v.
X with pmf p(x) is defined by
 = E(X) =
 x p( x),
allx
i.e., a weighted average of all possible values of X. It is also called
mean of the population represented by X.
Expected value of a discrete r.v. is just the weighted average of the
possible values of X, weighted by their chances to occur.
The expected value of a continuous r.v. with pdf f(x) is defined by
 = E(X) =



x f ( x)dx,
if the integral is absolutely convergent. Otherwise we say that
E(X) does not exist.
STAT151, Term II 2015-16
16
© Zhenlin Yang, SMU
Chapter 2
Expectation of a Random Variable
In Example 2.1, X is the sum of the two numbers shown on the
two dice. The expected sum is:
1
2
3
1
E(X) =  x p( x)  (2)  (3)  (4)    (12)  7
36
36
36
36
all x
In Example 2.7, we have
x
3
E(X) = 0 2x(1  x) dx = –
(1  x)2



+
0

0

1
1
dx = –
(1  x) 0
(1  x)2
= 1.
Expectation is an extremely important concept in summarizing
characteristics of distributions. It gives a measure of central
tendency. It has following simple but important property:
If a and b are constants, then
E[aX + b] = a E[X] + b
STAT151, Term II 2015-16
17
© Zhenlin Yang, SMU
Chapter 2
Function of a Random Variable
The following results are important:
If X is a r.v., then a function of it, u(X), is also a r.v. ,
E[u(X) ] =  u( x) p( x) , if X is a discrete r.v. with pmf p(x),
all x

E[u(X) ] =  u( x) f ( x)dx , if X is a continuous r.v. with pdf f(x).
Example 2.8 The r.v. X has pmf p(x) = 1/3, for x = 1, 0, 1. Find
E(X) and E(X2).
Solution:
E( X )   xp( x)  (1)13   (0)13   (1)13   0
all x
E( X )   x p( x)  (1)
2
2
all x
   (0)    (1) 
2 1
3
2 1
3
2 1
3
2

3
The above result says that the expectation of the r.v., u(X), can
simply be found through the distribution of the original r.v. X !
STAT151, Term II 2015-16
18
© Zhenlin Yang, SMU
Chapter 2
Variance of a Random Variable
Another important quantity of interest is the variance of a r.v. X,
denoted by Var(X), which is defined by
Var(X) = E[(X –)2],
where  = E(X). A related quantity is the standard deviation,
(X) = {Var(X)}1/2.
Variance and standard deviation (sd) measure how much the
values of X vary around its mean .
As (X –)2 is a function of X, i.e., u(X) = (X –)2, from the results
given in the last slide, we have
2
Var(X) =  ( x   ) p( x) , if X is discrete with pmf p(x),
all x
Var(X) =
STAT151, Term II 2015-16



( x   ) 2 f ( x)dx , if X is continuous with pdf f(x).
19
© Zhenlin Yang, SMU
Chapter 2
Variance of a Random Variable
Example 2.9. The time elapsed, in minutes, between the placement of an order of
pizza and its delivery is a random variable with pdf
1 15 if 25  x  40
f(x) = 

0
otherwise
(a) Determine the mean and standard deviation of the time it takes for the pizza
shop to deliver pizza.
(b) Suppose that it takes 12 minutes to bake pizza. Determine the mean and
standard deviation of the time it takes for the delivery person to deliver pizza.
Solution: Let X be the time between order and delivery.
(i) E(X) =

40
25
x (1 / 15)dx = 32.5,
40
2 = Var(X) =  ( x  32.5) 2 (1 / 15)dx = 18.75, and  = 4.33.
25
(ii) The time it takes for delivery person to deliver pizza is Y = X –12. Therefore,
E(Y) = E(X – 12) =

25
Var(Y) = E[(X – 12 –
STAT151, Term II 2015-16
40
( x  12) (1 / 15)dx = 32.5 – 12 = 20.5,
20.5)2]
=

40
25
( x  12  20.5) 2 (1 / 15)dx = 18.75.
20
© Zhenlin Yang, SMU
Variance of a Random Variable
Chapter 2
We note in Example 2.9 Var(X) and Var(Y) are the same. Why?
If X is a r.v. with mean , and a and b are arbitrary constants, then
E(aX + b) = aE(X) + b
Var(X) = E(X2) – 2
To show third result,
Var(aX + b)
= E[(aX + b – E(aX + b))2]
= E[(aX + b – aE(X) – b)2]
= E[(aX –aE(X))2]
= E[a 2(X –E(X))2]
= a 2E[ (X –E(X))2]
= a2Var(X)
Var(aX + b) = a2Var(X)
(aX + b) = |a|(X).
To show the second result,
Var(X) = E[(X –)2]
= E(X2 –2X+ 2)
= E(X2) –2 E(X)+ 2
= E(X2) – 2
STAT151, Term II 2015-16
21
© Zhenlin Yang, SMU
Variance of a Random Variable
Chapter 2
Example 2.10. The monthly sales at a computer store have a
mean of $25,000 and a standard deviation of $4,000. Profits are
30% of the sales less fixed costs of $6,000. Find the mean and
standard deviation of the monthly profit.
Solution: Let X = Sales. E(X) = 25,000, and Var(X) = 4,0002
Let Y = Profit. Then, Y = .30(X) – 6,000.
E(Profit) = E(Y)
= .30 E(X) – 6,000
= (.30)(25,000) – 6,000 = 1,500.
Var(Profit) = Var(Y)
= Var[(.30)(X) – 6,000]
= (.30)2Var(X) = 1,440,000
(Profit) = (Y) = 0.3(X) = 1,200.
STAT151, Term II 2015-16
22
© Zhenlin Yang, SMU
Moment Generating Function*
Chapter 2
The mean, variance and standard deviation are important
characteristics of a distribution.
For some distributions, it is rather difficult to compute these
quantities directly.
A special function defined below can help. More importantly,
the uniqueness property of this function often help to find the
distribution of some useful functions of r.v.s.
Definition 2.12 (Moment-Generating Function) Let X be a
random variable, discrete or continuous. If there exists a positive
number h such that
M(t) = E(etX)
exists and is finite for –h < t < h, then M(t) is called the MomentGenerating Function (MGF) of X.
STAT151, Term II 2015-16
23
© Zhenlin Yang, SMU
Chapter 2
Moment Generating Function
Why is it called the Moment Generating Function?
 because it generates moments.
What are the moments?
• the term ‘moment’ is from mechanics, representing the
product of a distance and its weight.
• So, xi p(xi) is a moment, and xi p(xi) or E(X) is the moment
of the ‘system’.
• In statistics, E(X), E(X2), E(X3), …, are the 1st, 2nd , 3rd, …,
moments about the origin;
• E(X), E[(X)2], E[(X)3], etc, are the 1st, 2nd , 3rd
central moments, or moments about the mean.
• See p69 of text for details.
STAT151, Term II 2015-16
24
© Zhenlin Yang, SMU
Chapter 2
Moment Generating Function
Property of MGF:
M.G.F., if it exists, completely determines the distribution
function. In other words, if two random variables, assuming
the same set of vaules, have the same M.G.F., they must have
the same distribution function.
E(Xr) = M(r)(0), the rth derivative of M(t) evaluated at 0.
We use a discrete r.v.s to demonstrate these properties: if X and Y
have possible values {v1, v2, …}, and pmfs p(x) and q(x), then,
M X (t )  E (etX )  et v1 p(v1 )  et v2 p(v2 )  ,
M Y (t )  E (etY )  et v1 q(v1 )  et v2 q(v2 )  .
Thus, if MX(t) = MY(t), it must be that
p(vi) = q(vi), i = 1, 2, …
STAT151, Term II 2015-16
25
© Zhenlin Yang, SMU
Moment Generating Function
Chapter 2
To understand the property E(Xr) = M(r)(0), we have
M (t ) 
d
dt
e
t x1

p( x1 )  et x 2 p ( x2 )  et x3 p( x3 )  
 et x1 x1 p( x1 )  et x 2 x2 p( x2 )  et x3 x3 p( x3 )  
Thus, M (0)  x1 p( x1 )  x2 p( x2 )  x3 p( x3 )    E( X )
M (t ) 
d2
dt 2
e
t x1

p( x1 )  et x2 p( x2 )  et x3 p( x3 )  
 et x1 x12 p( x1 )  et x2 x22 p( x2 )  et x3 x32 p( x3 )  
2
2
2
2


M
(
0
)

x
p
(
x
)

x
p
(
x
)

x
p
(
x
)



E
(
X
)
Thus,
1
1
2
2
3
3
In general,
M (t )  i 1 e x p( xi ),
(r )
STAT151, Term II 2015-16

t xi
r
i
M
26
(r )
(0)  i 1 xir p( xi )  E( X r )

© Zhenlin Yang, SMU
Moment Generating Function
Chapter 2
Example 2.11. If X has the MGF,
 3  2 t  2  3t  1 
M (t )  e    e    e  
6
6
6
t
then, as the coefficients of the e terms are probabilities, the
probabilities must be 3/6, 2/6, and 1/6; the values beside t are the
values of X, which are 1, 2, and 3. The pmf of X is
x
1
2
3
p(x)
3/6
2/6
1/6
Example 2.12. Suppose the MGF of X is
et 2
M (t ) 
, t  ln 2.
t
1 e 2
Find the distribution of X.
STAT151, Term II 2015-16
27
© Zhenlin Yang, SMU
Moment Generating Function
Chapter 2
Solution: Until we expand M(t), we cannot detect the
tx
coefficients of e i .
1
Recall:
 1  z  z 2  z 3  ,
1 z
1  z  1
we have,
t
e 2
M (t ) 
1  et 2
et

2
 e t e t 2 e 3t

1   2  3   


2
2
2


2
3
t 1
2t  1 
3t  1 
 e    e    e    
2
2
2
STAT151, Term II 2015-16
28
That is, P(X = x) = (1/2)x,
for positive integer x; the
pmf of X is thus,
x
1
p( x)    , x  1,2,3,.
2
© Zhenlin Yang, SMU