Transcript app_bPROBABILITY - Memorial University of Newfoundland

ECON 6002
Econometrics
Memorial University of Newfoundland
Review of Probability Concepts
Adapted from Vera Tabakova’s notes

B.1 Random Variables

B.2 Probability Distributions

B.3 Joint, Marginal and Conditional Probability
Distributions

B.4 Properties of Probability Distributions

B.5 Some Important Probability Distributions
Principles of Econometrics, 3rd Edition
Slide B-2

A random variable is a variable whose value is unknown until it is
observed.

A discrete random variable can take only a limited, or countable,
number of values.

A continuous random variable can take any value on an interval.
Principles of Econometrics, 3rd Edition
Slide B-3

The probability of an event is its “limiting relative frequency,” or the
proportion of time it occurs in the long-run.

The probability density function (pdf) for a discrete random
variable indicates the probability of each possible value occurring.
f ( x)  P  X  x 
f ( x1 )  f ( x2 ) 
Principles of Econometrics, 3rd Edition
 f ( xn )  1
Slide B-4
Principles of Econometrics, 3rd Edition
Slide B-5
Figure B.1 College Employment Probabilities
Principles of Econometrics, 3rd Edition
Slide B-6

The cumulative distribution function (cdf) is an alternative way to
represent probabilities. The cdf of the random variable X, denoted
F(x), gives the probability that X is less than or equal to a specific
value x
F  x  P  X  x
Principles of Econometrics, 3rd Edition
Slide B-7
Principles of Econometrics, 3rd Edition
Slide B-8

For example, a binomial random variable X is the number of
successes in n independent trials of identical experiments with
probability of success p.
n x
P  X  x   f  x     p (1  p ) n x
 x
 n
n!
where n!  n  (n  1 n  2 
 
 x  x! n  x !
Principles of Econometrics, 3rd Edition
(B.1)
 21
Slide B-9

For example, if we know that the MUN basketball team has a chance
of winning of 70% (p=0.7) and we want to know how likely they are
to win at least 2 games in the next 3 weeks
Principles of Econometrics, 3rd Edition
Slide B-10

For example, if we know that the MUN basketball team has a chance
of winning of 70% (p=0.7) and we want to know how likely they are
to win at least 2 games in the next 3 weeks

First: for winning only once in three weeks, likelihood is 0.189, see?
n!
x!
nx
!


3!
1!
31
!
3
Times
px 
1 pnx 0. 7 1 
1 0. 72 0. 063
Principles of Econometrics, 3rd Edition
Slide B-11

For example, if we know that the MUN basketball team has a chance
of winning of 70% (p=0.7) and we want to know how likely they are
to win at least 2 games in the next 3 weeks…

The likelihood of winning exactly 2 games, no more or less:
n!
3!

 3!  321 3
211
x!
nx
!
2!
32
!
2!
1
!
px 
1 pnx 0. 7 2 
1 0. 71 0. 147
Principles of Econometrics, 3rd Edition
Slide B-12

For example, if we know that the MUN basketball team has a chance
of winning of 70% (p=0.7) and we want to know how likely they are
to win at least 2 games in the next 3 weeks

So 3 times 0.147 = 0.441 is the likelihood of winning exactly 2 games
Principles of Econometrics, 3rd Edition
Slide B-13

For example, if we know that the MUN basketball team has a chance
of winning of 70% (p=0.7) and we want to know how likely they are
to win at least 2 games in the next 3 weeks

And 0.343 is the likelihood of winning exactly 3 games
n!
x!
nx
!

3!
3!
33
!

3!
3!1

3!
3!1
1
px 
1 pnx 0. 7 3 
1 0. 70 0. 343
Principles of Econometrics, 3rd Edition
Slide B-14

For example, if we know that the MUN basketball team has a chance
of winning of 70% (p=0.7) and we want to know how likely they are
to win at least 2 games in the next 3 weeks

For winning only once in three weeks: likelihood is 0.189

0.441 is the likelihood of winning exactly 2 games

0.343 is the likelihood of winning exactly 3 games

So 0.784 is how likely they are to win at least 2 games in the next 3
weeks

In STATA
di Binomial(3,2,0.7) di Binomial(n,k,p)
Principles of Econometrics, 3rd Edition
Slide B-15

For example, if we know that the MUN basketball team has a chance
of winning of 70% (p=0.7) and we want to know how likely they are
to win at least 2 games in the next 3 weeks

So 0.784 is how likely they are to win at least 2 games in the next 3
weeks

In STATA
di binomial(3,2,0.7) di Binomial(n,k,p) is the
likelihood of winning 1 or less

So we were looking for 1- binomial(3,2,0.7)
Principles of Econometrics, 3rd Edition
Slide B-16
Figure B.2 PDF of a continuous random variable
Principles of Econometrics, 3rd Edition
Slide B-17
P  20  X  40  
40
 f  x  dx .355
20
P  X  x 
x
 f  t  dt  F  x 

P  20  X  40   F (40)  F (20)  .649  .294  .355
Principles of Econometrics, 3rd Edition
Slide B-18
1
2

X 
3
4
high school diploma or less
some college
four year college degree
advanced degree
0 if had no money earnings in 2002
Y 
1 if had positive money earnings in 2002
Principles of Econometrics, 3rd Edition
Slide B-19
 f  x, y   1
x
Principles of Econometrics, 3rd Edition
y
Slide B-20
f X ( x )   f ( x, y )
for each value X can take
y
fY ( y )   f ( x, y )
(B.2)
for each value Y can take
x
4
fY  y    f  x, y 
y  0,1
x 1
fY 1  .19  .06  .04  .02  .31
Principles of Econometrics, 3rd Edition
Slide B-21
Principles of Econometrics, 3rd Edition
Slide B-22
P(Y  y, X  x) f ( x, y )
f ( y | x)  P(Y  y | X  x) 

P  X  x
f X ( x)
Principles of Econometrics, 3rd Edition
y
f  y | X  3
0
.04/.18=.22
1
.14/.18=.78
(B.3)
Slide B-23

Two random variables are statistically independent if the conditional
probability that Y = y given that X = x, is the same as the
unconditional probability that Y = y.
P  Y  y | X  x   P Y  y 
(B.4)
f ( x, y)
f ( y | x) 
 fY ( y )
f X ( x)
(B.5)
f ( x, y )  f X ( x ) f Y ( y )
(B.6)
Principles of Econometrics, 3rd Edition
Slide B-24
Y = 1 if shaded Y = 0 if clear
X = numerical value (1, 2, 3, or 4)
Principles of Econometrics, 3rd Edition
Slide B-25
Principles of Econometrics, 3rd Edition
Slide B-26
Principles of Econometrics, 3rd Edition
Slide B-27
Principles of Econometrics, 3rd Edition
Slide B-28

B.4.1
Mean, median and mode
E[ X ]  x1P  X  x1   x2 P  X  x2  

 xn P  X  xn 
(B.7)
For a discrete random variable the expected value is:
  E[ X ]  x1 f ( x1 )  x2 f ( x2 ) 
 xn f ( xn )
n
  xi f ( xi )   xf ( x)
i 1
(B.8)
x
Where f is the discrete PDF of x
Principles of Econometrics, 3rd Edition
Slide B-29

For a continuous random variable the expected value is:
  EX  

 xf  x dx

The mean has a flaw as a measure of the center of a probability
distribution in that it can be pulled by extreme values.
Principles of Econometrics, 3rd Edition
Slide B-30

For a continuous distribution the median of X is the value m such that
P  X  m   P( X  m)  .5

In symmetric distributions, like the familiar “bell-shaped curve” of
the normal distribution, the mean and median are equal.

The mode is the value of X at which the pdf is highest.
Principles of Econometrics, 3rd Edition
Slide B-31
E[ g ( X )]   g ( x) f ( x)
(B.9)
x
Where g is any function of x, in particular;
E  aX   aE  X 
(B.10)
E  g  X    g  x  f  x   axf  x   a xf  x   aE  X 
Principles of Econometrics, 3rd Edition
Slide B-32
E  aX  b  aE  X   b
E  g1  X   g2  X   E  g1  X   E  g2  X 
Principles of Econometrics, 3rd Edition
(B.11)
(B.12)
Slide B-33
The variance

The variance of a discrete or continuous random variable X is the
expected value of
g  X    X  E  X  
Principles of Econometrics, 3rd Edition
2
Slide B-34

The variance of a random variable is important in characterizing the
scale of measurement, and the spread of the probability distribution.

Algebraically, letting E(X) = μ,
var( X )    E  X    E[ X 2 ]  2
2
Principles of Econometrics, 3rd Edition
2
(B.13)
Slide B-35

The variance of a constant is?
Principles of Econometrics, 3rd Edition
Slide B-36
Figure B.3 Distributions with different variances
Principles of Econometrics, 3rd Edition
Slide B-37
var(aX  b)  a2 var( X )
(B.14)
var(aX  b)  E  aX  b  E  aX  b    E  aX  b  a  b 
2
2
 E  a  X      a E  X     a 2 var  X 
2
Principles of Econometrics, 3rd Edition
2
2
Slide B-38
3

E  X   

skewness   3

4

E  X   

kurtosis  
4
Principles of Econometrics, 3rd Edition
Slide B-39
E[ g ( X , Y )]   g ( x, y ) f ( x, y )
x
(B.15)
y
E  X  Y   E ( X )  E (Y )
(B.16)
E  X  Y     x  y  f  x, y   xf  x, y    yf  x, y 
x
y
x
y
x
y
  x  f  x, y    y  f  x, y    xf  x    yf  y 
x
y
y
x
x
y
 E  X   E Y 
Principles of Econometrics, 3rd Edition
Slide B-40
E (aX  bY  c)  aE ( X )  bE (Y )  c
(B.17)
E  XY   E  g  X , Y     xyf  x, y   xyf  x  f  y 
x
y
x
y
  xf  x   yf  y   E  X  E Y  if X and Y are independent.
x
y
g ( X , Y )  ( X   X )(Y  Y )
Principles of Econometrics, 3rd Edition
(B.18)
Slide B-41
Figure B.4 Correlated data
Principles of Econometrics, 3rd Edition
Slide B-42
Covariance and correlation coefficient
cov( X , Y )   XY  E  X   X Y  Y   E  XY    X Y
cov  X , Y 
 XY


var( X ) var(Y )  X Y

(B.19)
(B.20)
If X and Y are independent random variables then the covariance and
correlation between them are zero. The converse of this relationship is
not true.
Principles of Econometrics, 3rd Edition
Slide B-43
Covariance and correlation coefficient
cov  X , Y 
 XY


var( X ) var(Y )  X Y


(B.20)
The correlation coefficient is a measure of linear correlation between
the variables
Its values range from -1 (perfect negative correlation) and 1 (perfect
positive correlation)
Principles of Econometrics, 3rd Edition
Slide B-44

If a and b are constants then:
var  aX  bY   a 2 var( X )  b2 var(Y )  2ab cov( X , Y )
(B.21)
var  X  Y   var( X )  var(Y )  2cov( X , Y )
(B.22)
var  X  Y   var( X )  var(Y )  2cov( X , Y )
(B.23)
Principles of Econometrics, 3rd Edition
Slide B-45

If a and b are constants then:
var  X  Y   var( X )  var(Y )  2cov( X , Y )
(B.22)
So:
var
X Yvar
Xvar
Y2
x y
Why is that? (and of course the same happens for the case
of var(X-Y))
Principles of Econometrics, 3rd Edition
Slide B-46

If X and Y are independent then:
var  aX  bY   a 2 var( X )  b2 var(Y )
(B.24)
var  X  Y   var( X )  var(Y )
(B.25)
var  X  Y  Z   var  X   var Y   var  Z 
Principles of Econometrics, 3rd Edition
Slide B-47

If X and Y are independent then:
var  X  Y  Z   var  X   var Y   var  Z 
Otherwise this expression would have to include all the doubling of each
of the (non-zero) pairwise covariances between variables
as summands as well
Principles of Econometrics, 3rd Edition
Slide B-48
4
E  X    xf  x   1 .1   2  .2    3  .3   4  .4   3   X
x 1
  E  X  X 
2
X
2
2
2
2
2







 1  3  .1   2  3  .2   3  3  .3   4  3  .4 

 
 
 

  4  .1  1 .2    0  .3  1 .4 
1
Principles of Econometrics, 3rd Edition
Slide B-49

B.5.1
The Normal Distribution

If X is a normally distributed random variable with mean μ and
variance σ2, it can be symbolized as X ~ N  , 2  .
 ( x  )2 
f ( x) 
exp 
,

2
22
 2

1
Principles of Econometrics, 3rd Edition
  x  
(B.26)
Slide B-50
Figure B.5a Normal Probability Density Functions with Means μ and Variance 1
Principles of Econometrics, 3rd Edition
Slide B-51
Figure B.5b Normal Probability Density Functions with Mean 0 and Variance σ2
Principles of Econometrics, 3rd Edition
Slide B-52

A standard normal random variable is one that has a normal
probability density function with mean 0 and variance 1.
X 
Z
~ N (0,1)


(B.27)
The cdf for the standardized normal variable Z is
( z )  P  Z  z  .
Principles of Econometrics, 3rd Edition
Slide B-53
a 
 X  a 

 a  
P[ X  a]  P 


P
Z






 
 
 
  
(B.28)
a 
 X  a 

 a  
P[ X  a]  P 


P
Z


1





 
 
 
  
(B.29)
b 
a 
 b 
 a  
P[a  X  b]  P 
Z 
 
  


 
 
  
  
(B.30)
Principles of Econometrics, 3rd Edition
Slide B-54

A weighted sum of normal random variables has a normal
distribution.
X 1 ~ N  1 , 12 
X 2 ~ N   2 , 22 
Y  a1 X1  a2 X 2 ~ N  Y  a11  a22 , Y2  a1212  a2222  2a1a212 
Principles of Econometrics, 3rd Edition
(B.27)
Slide B-55
V  Z12  Z22 
 Zm2 ~ (2m)
(B32)
E[V ]  E (2m )   m
2

var[V ]  var ( m )   2m
Principles of Econometrics, 3rd Edition
(B.33)
Slide B-56
Figure B.6 The chi-square distribution
Principles of Econometrics, 3rd Edition
Slide B-57

A “t” random variable (no upper case) is formed by dividing a
standard normal random variable Z ~ N  0,1 by the square root of an
independent chi-square random variable, V ~ (2m) , that has been
divided by its degrees of freedom m.
Z
t
V
Principles of Econometrics, 3rd Edition
~ t( m )
(B.34)
m
Slide B-58
Figure B.7 The standard normal and t(3) probability density functions
Principles of Econometrics, 3rd Edition
Slide B-59

An F random variable is formed by the ratio of two independent chisquare random variables that have been divided by their degrees of
freedom.
V1 m1
F
~ F( m1 ,m2 )
V2 m2
Principles of Econometrics, 3rd Edition
(B.35)
Slide B-60
Figure B.8 The probability density function of an F random variable
Principles of Econometrics, 3rd Edition
Slide B-61















binary variable
binomial random variable
cdf
chi-square distribution
conditional pdf
conditional probability
continuous random variable
correlation
covariance
cumulative distribution function
degrees of freedom
discrete random variable
expected value
experiment
F-distribution
Principles of Econometrics, 3rd Edition














joint probability density function
marginal distribution
mean
median
mode
normal distribution
pdf
probability
probability density function
random variable
standard deviation
standard normal distribution
statistical independence
variance
Slide B-62