Transcript Document
Random Vectors and Matrices
• A random vector is a vector whose elements are random variables.
• The collective behavior of a p x 1 random vector is described by a
joint probability density function f(x1,x2,…,xp) = f(x).
• If the joint density of a p x 1 random vector can be factored as
f(x1,x2,…,xp) = f1(x1) f2(x2)∙∙∙ fp(xp) then the p continuous random
variables X1,X2,…Xp are mutually independent.
STA347 - week 9
1
Mean and Variance of Random Vector
• The expected value of a random vector is a vector of the expected values of
each of its elements. That is, the population mean vector is
• The population variance-covariance matrix of a px1 random vector x is a
p x p symmetric matrix
CovX E x x ' ij
where σij = Cov(Xi, Xj) = E(Xi – μi)(Xj – μj).
• The population correlation matrix of a px1 random vector x is a p x p
symmetric matrix ρ = (ρij)
where ij
ij
ii jj
.
STA347 - week 9
2
Properties of Mean Vector and Covariance Matrix
STA347 - week 9
3
Functions of Random variables
• In some case we would like to find the distribution of Y = h(X) when the
distribution of X is known.
• Discrete case
P X x
pY y PY y Ph X y P X h 1 y
xh 1 y
• Examples
1. Let Y = aX + b , a ≠ 0
1
PY y PaX b y P X y b
a
2. Let Y
X2
P X y P X y
PY y P X 2 y P X 0
0
STA347 - week 9
if y 0
if y 0
if y 0
4
Continuous case – Examples
1. Suppose X ~ Uniform(0, 1). Let Y X 2 , then the cdf of Y can be found as
follows
FY y PY y P X 2 y P X y FX y
The density of Y is then given by
2. Let X have the exponential distribution with parameter λ. Find the
1
density for Y
X 1
3. Suppose X is a random variable with density
x 1
f X x 2
0
, 1 x 1
, elsewhere
Check if this is a valid density and find the density of Y X 2 .
STA347 - week 9
5
Theorem
• If X is a continuous random variable with density fX(x) and h is strictly
increasing and differentiable function form R R then Y = h(X) has density
fY y f X
d 1
h y
h y
dy
1
for y R .
• Proof:
STA347 - week 9
6
Theorem
• If X is a continuous random variable with density fX(x) and h is strictly
decreasing and differentiable function form R R then Y = h(X) has density
dyd h y
f Y y f X h 1 y
1
for y R .
• Proof:
STA347 - week 9
7
Summary
• If Y = h(X) and h is monotone then
fY y f X
d 1
h y
h y
dy
1
• Example
X has a density
x3
f X x 4
0
for 0 x 2
otherwise
Let Y X 6. Compute the density of Y.
STA347 - week 9
8
Change-of-Variable for Joint Distributions
• Theorem
Let X and Y be jointly continuous random variables with joint density
function fX,Y(x,y) and let DXY = {(x,y): fX,Y(x,y) >0}. If the mapping T given
by T(x,y) = (u(x,y),v(x,y)) maps DXY onto DUV. Then U, V are jointly
continuous random variable with joint density function given by
f xu , v , y u , v J u , v
fU ,V u , v X ,Y
0
if u , v DU ,V
otherwise
where J(u,v) is the Jacobian of T-1 given by
x
J u, v u
y
u
x
v
y
v
assuming derivatives exists and are continuous at all points in DUV .
STA347 - week 9
9
Example
• Let X, Y have joint density function given by
e x y
f X ,Y x, y
0
Find the density function of U
if x, y 0
otherwise
X
.
X Y
STA347 - week 9
10
Example
• Show that the integral over the Standard Normal distribution is 1.
STA347 - week 9
11
Example
• A device containing two key components fails when and only when both
components fail. The lifetime, T1 and T2, of these components are
independent with a common density function given by
e t
fT t
0
t 0
otherwise
• The cost, X, of operating the device until failure is 2T1 + T2. Find the
density function of X.
STA347 - week 9
12
Convolution
• Suppose X, Y jointly distributed random variables. We want to find the
probability / density function of Z=X+Y.
• Discrete case
X, Y have joint probability function pX,Y(x,y). Z = z whenever X = x and
Y = z – x. So the probability that Z = z is the sum over all x of these joint
probabilities. That is
pZ z p X ,Y x, z x .
x
• If X, Y independent then
pZ z p X x pY z x .
x
This is known as the convolution of pX(x) and pY(y).
STA347 - week 9
13
Example
• Suppose X~ Poisson(λ1) independent of Y~ Poisson(λ2). Find the
distribution of X+Y.
STA347 - week 9
14
Convolution - Continuous case
• Suppose X, Y random variables with joint density function fX,Y(x,y). We want to
find the density function of Z=X+Y.
Can find distribution function of Z and differentiate. How?
The Cdf of Z can be found as follows:
FZ z P X Y z
zx
f x, y dydx
X ,Y
x y
z
f x, v x dvdx
X ,Y
x v
z
f x, v x dxdv.
X ,Y
v x
If
f x, v xdx
XY
is continuous at z then the density function of Z is given by
x
• If X, Y independent then
f Z z
f Z z
f x, z xdx
XY
x
f x f z x dx
X
Y
x
This is known as the convolution of fX(x) and fY(y).
15
Example
• X, Y independent each having Exponential distribution with mean 1/λ. Find
the density for W=X+Y.
STA347 - week 9
16
Order Statistics
• The order statistics of a set of random variables X1, X2,…, Xn are the same
random variables arranged in increasing order.
• Denote by
X(1) = smallest of X1, X2,…, Xn
X(2) = 2nd smallest of X1, X2,…, Xn
X(n) = largest of X1, X2,…, Xn
• Note, even if Xi’s are independent, X(i)’s can not be independent since
X(1) ≤ X(2) ≤ … ≤ X(n)
• Distribution of Xi’s and X(i)’s are NOT the same.
STA347 - week 9
17
Distribution of the Largest order statistic X(n)
• Suppose X1, X2,…, Xn are i.i.d random variables with common distribution
function FX(x) and common density function fX(x).
• The CDF of the largest order statistic, X(n), is given by
FX n x PX n x
• The density function of X(n) is then
f X n x
d
FX n x
dx
STA347 - week 9
18
Example
• Suppose X1, X2,…, Xn are i.i.d Uniform(0,1) random variables. Find the
density function of X(n).
STA347 - week 9
19
Distribution of the Smallest order statistic X(1)
• Suppose X1, X2,…, Xn are i.i.d random variables with common distribution
function FX(x) and common density function fX(x).
• The CDF of the smallest order statistic X(1) is given by
FX 1 x PX 1 x 1 PX 1 x
• The density function of X(1) is then
f X 1 x
d
FX x
dx 1
STA347 - week 9
20
Example
• Suppose X1, X2,…, Xn are i.i.d Uniform(0,1) random variables. Find the
density function of X(1).
STA347 - week 9
21
Distribution of the kth order statistic X(k)
• Suppose X1, X2,…, Xn are i.i.d random variables with common distribution
function FX(x) and common density function fX(x).
• The density function of X(k) is
f X k x
n!
FX x k 1 1 FX x nk f X x
k 1!n k !
STA347 - week 9
22
Example
• Suppose X1, X2,…, Xn are i.i.d Uniform(0,1) random variables. Find the
density function of X(k).
STA347 - week 9
23
Computer Simulations - Introduction
• Modern high-speed computers can be used to perform simulation
studies.
• Computer simulation methods are commonly used in statistical
applications; sometimes they replace theory, e.g., bootstrap methods.
• Computer simulations are becoming more and more common in
many applications such as quality control, marketing, scientific
research etc.
STA347 - week 9
24
Applications of Computer Simulations
• Our main focus is on probabilistic simulations. Examples of
applications of such simulations include:
Simulate probabilities and random variables numerically.
Approximate quantities that are too difficult to compute
mathematically.
Random selection of a sample from a very large data sets.
Encrypt data or generate passwords.
Generate potential solutions for difficult problems.
STA347 - week 9
25
Steps in Probabilistic Simulations
• In most applications, the first step is to specify a certain probability
distribution.
• Once such distribution is specified, it will be desired to generate one
or more random variables having that distribution.
• The build-in computer device that generates random numbers is
called pseudorandom number generator.
• It is a device for generating a sequence U1, U2, … of random values
that are approximately independent and have approximately uniform
distribution of the unit interval [0,1].
STA347 - week 9
26
Simulating Discrete Distributions - Example
• Suppose we wish to generate X ~ Bernoulli(p), where 0 < p < 1.
• We start by generating U ~ Uniform[0, 1] and then set:
1
X
0
Up
Up
• Then clearly X takes two values, 0 and 1. Further,
P X 1 PU p p
• Therefore, we have that X ~ Bernoulli(p).
• This can be generalized to generate Y ~ Binomial(n, p) by generating
U1, U2, … Un. Setting Xi as above and let Y = X1 + ∙∙∙ + Xn.
STA347 - week 9
27
Simulating Discrete Distributions
• In general, suppose we wish to generate a random variable with
probability mass function p.
• Let, x1 < x2 < x3 < ∙∙∙ be all the values for which p(xi) > 0.
• Let U ~ Uniform[0, 1].
• Define Y by:
j
Y min x j : pxk U
k 1
• Theorem 1: Y is a discrete random variable, having probability
mass function p.
• Proof:
STA347 - week 9
28
Simulating Continuous Distributions - Example
• Suppose we wish to generate X ~ Uniform[a, b].
• We start by generating U ~ Uniform[0, 1] and then set:
X b aU a
• Using one-dimensional change of variable theorem we can easily
show that X ~ Uniform[a, b].
STA347 - week 9
29
Simulating Continuous Distributions
• In general, simulating continuous distribution is not an easy task.
• However, for certain continuous distributions it is not difficult.
• The general method for simulating continuous distribution makes
use of the inverse cumulative distribution function.
• The inverse cdf of a random variable X with cumulative distribution
function F is defined by:
F 1 t min x : F x t
for 0 < t < 1.
STA347 - week 9
30
Inversion Method for Generating RV
• Let F be any cumulative distribution function, and let
U ~ Uniform[0, 1].
• Define a random variable Y by: Y F 1 U
• Theorem 2: Y has cumulative distribution function given by F. That is,
PY y F y
• Proof:
STA347 - week 9
31
Important Notes
• The theorem above is valid for any cumulative distribution function
whether it corresponds to a continuous distribution, a discrete
distribution or a mixture of the two.
• The inversion method for generating random variables described
above can be used whenever the distribution function is not too
complicated and has a close form.
• For distributions that are too complicated to sample using the
inversion method and for which there is no simple trick , it may
still be possible to generate samples using Markov chain methods.
STA347 - week 9
32
Example – Exponential Distribution
• Suppose X ~ Exponential(λ). The probability density function of X is:
e x
f X x
0
• The cdf of X is:
x0
otherwise
x
F x e t dt 1 e x
0
x
• Setting U F x 1 e
and solving for x we get…
• Therefore, by theorem 2 above, X 1 ln 1 U where
U ~ Uniform[0, 1], has an Exponential(λ) distribution.
STA347 - week 9
33
Example – Standard Normal Distribution
• Suppose X ~ Normal(0,1). The cdf of X is denoted by Ф(x). It is
given by:
t2
x
1 2
x
e dt
2
• Then, if U ~ Uniform[0, 1], by theorem 2 above
Y 1 U min x : x U
has a N(0,1) distribution.
• However, since both Ф and Ф-1 don’t have a close form, i.e., it is
difficult to compute them, the inversion method for generating RV is
not practical.
STA347 - week 9
34