Transcript Lecture 12 - The Department of Mathematics & Statistics

Combining Random Variables
Quite often we have two or more random variables
X, Y, Z etc
We combine these random variables using a
mathematical expression.
Important question
What is the distribution of the new random variable?
An Example
Suppose that a student will take three tests in the next
three days
1. Mathematics (X is the score he will receive on this
test.)
2. English Literature (Y is the score he will receive on
this test.)
3. Social Studies (Z is the score he will receive on this
test.)
Assume that
1. X (Mathematics) has a Normal distribution with
mean m = 90 and standard deviation s = 3.
2. Y (English Literature) has a Normal distribution
with mean m = 60 and standard deviation s = 10.
3. Z (Social Studies) has a Normal distribution with
mean m = 70 and standard deviation s = 7.
Graphs
0.14
X (Mathematics)
m = 90, s = 3.
0.12
0.1
0.08
Z (Social Studies)
m = 70 , s = 7.
0.06
0.04
Y (English Literature)
m = 60, s = 10.
0.02
0
0
20
40
60
80
100
Suppose that after the tests have been written an overall
score, S, will be computed as follows:
S (Overall score) = 0.50 X (Mathematics) + 0.30 Y
(English Literature) + 0.20 Z (Social Studies) +
10 (Bonus marks)
What is the distribution of the overall score, S?
Sums, Differences, Linear Combinations of R.V.’s
A linear combination of random variables, X, Y, . . . is
a combination of the form:
L = aX + bY + … + c (a constant)
where a, b, etc. are numbers – positive or negative.
Most common:
Sum = X + Y
Difference = X – Y
Others
Averages = 1/3 X + 1/3 Y + 1/3 Z
Weighted averages = 0.40 X + 0.25 Y + 0.35 Z
Means of Linear Combinations
If
L = aX + bY + … + c
The mean of L is:
Mean(L) = a Mean(X) + b Mean(Y) + … + c
mL = a mX + b mY + … + c
Most common:
Mean( X + Y) = Mean(X) + Mean(Y)
Mean(X – Y) = Mean(X) – Mean(Y)
Variances of Linear Combinations
If X, Y, . . . are independent random variables and
L = aX + bY + … + c then
Variance(L) = a2 Variance(X) + b2 Variance(Y) + …
s L2  a 2s X2  b 2s Y2 
Most common:
Variance( X + Y) = Variance(X) + Variance(Y)
Variance(X – Y) = Variance(X) + Variance(Y)
The constant c has no effect on the variance
Combining Independent Normal Random Variables
If X, Y, . . . are independent normal random variables,
then L = aX + bY + … is normally distributed.
In particular:
X + Y is normal with mean m X  mY
standard deviation
s X2  s Y2
X – Y is normal with mean m X  mY
standard deviation
s X2  s Y2
Example: Suppose that one performs two
independent tasks (A and B):
X = time to perform task A (normal with mean 25
minutes and standard deviation of 3 minutes.)
Y = time to perform task B (normal with mean 15
minutes and std dev 2 minutes.)
X and Y independent so T = X + Y = total time is normal
mean
m  25  15  40
with
standard deviation s  32  2 2  3.6
What is the probability that the two tasks take more than 45
minutes to perform?
45  40 

PT  45  P Z 
  PZ  1.39  .0823
3.6 

Example 2:
A student will take three tests in the next three days
1. X (Mathematics) has a Normal distribution with
mean m = 90 and standard deviation s = 3.
2. Y (English Literature) has a Normal distribution
with mean m = 60 and standard deviation s = 10.
3. Z (Social Studies) has a Normal distribution with
mean m = 70 and standard deviation s = 7.
Overall score, S = 0.50 X (Mathematics) + 0.30 Y
(English Literature) + 0.20 Z (Social Studies) +
10 (Bonus marks)
Graphs
0.14
X (Mathematics)
m = 90, s = 3.
0.12
0.1
0.08
Z (Social Studies)
m = 70 , s = 7.
0.06
0.04
Y (English Literature)
m = 60, s = 10.
0.02
0
0
20
40
60
80
100
Determine the distribution of
S = 0.50 X + 0.30 Y + 0.20 Z + 10
S has a normal distribution with
Mean mS = 0.50 mX + 0.30 mY + 0.20 mZ + 10
= 0.50(90) + 0.30(60) + 0.20(70) + 10
= 45 + 18 + 14 +10 = 87
ss 
 0.5 s
2
X
 0.5
2
2

2
  0.3 s   0.2  s
2
2
Y
2
3   0.3 10   0.2  7
2
2
2
2
Z
2
 2.25  9  1.96  13.21  3.635
Graph
0.12
0.1
distribution of
S = 0.50 X + 0.30 Y + 0.20 Z + 10
0.08
0.06
0.04
0.02
0
0
20
40
60
80
100
The distribution of averages (the mean)
• Let x1, x2, … , xn denote n independent random
variables each coming from the same Normal
distribution with mean m and standard deviation s.
n
• Let
x
x
i 1
n
i
1
1
   x1    x2 
n
n
What is the distribution of x ?
1
   xn
n
The distribution of averages (the mean)
Because the mean is a “linear combination”
1
1
m x    m x1    m x2 
n
n
1
1
  m  m 
n
n
1
   m xn
n
1
1
   m  n  m  m
n
n
and
2
2
2
1 2 1 2
1 2
s    s x1    s x2     s xn
n
n
n
2
2
2
s2 s2
1 2 1 2
1 2
   s   s    s  n 2 
n
n
n
n
n
2
x
Thus if x1, x2, … , xn denote n independent random
variables each coming from the same Normal
distribution with mean m and standard deviation s.
Then
n
x
x
i
i 1
n
1
1
   x1    x2 
n
n
1
   xn
n
has Normal distribution with
mean m x  m and
variance s x2 
s2
n
standard deviation s x 
s
n
Example
• Suppose we are measuring the cholesterol level of
men age 60-65
• This measurement has a Normal distribution with
mean m = 220 and standard deviation s = 17.
• A sample of n = 10 males age 60-65 are selected and
the cholesterol level is measured for those 10 males.
• x1, x2, x3, x4, x5, x6, x7, x8, x9, x10, are those 10
measurements
Find the probability distribution of x ?
Compute the probability that x is between 215 and 225
Example
• Suppose we are measuring the cholesterol level of
men age 60-65
• This measurement has a Normal distribution with
mean m = 220 and standard deviation s = 17.
• A sample of n = 10 males age 60-65 are selected and
the cholesterol level is measured for those 10 males.
• x1, x2, x3, x4, x5, x6, x7, x8, x9, x10, are those 10
measurements
Find the probability distribution of x ?
Compute the probability that x is between 215 and 225
Solution
Find the probability distribution of x
Normal with m x  m  220
s
17
and s x 

 5.376
n
10
P  215  x  225
 215  220 x  220 225  220 
 P



5.376
5.376
5.376


 P  0.930  z  0.930  0.648
Graphs
0.08
The probability
distribution of
the mean
0.06
0.04
The probability
distribution of
individual
observations
0.02
0
150
170
190
210
230
250
270
290
310
An Excel file illustrating the sampling
distribution of x
mean.xls
Normal approximation to the Binomial
distribution
Using the Normal distribution to calculate
Binomial probabilities
Binomial distribution n = 20, p = 0.70
0.2500
Approximating
Normal distribution
0.2000
m  np  14
s  npq  2.049
0.1500
Binomial distribution
0.1000
0.0500
-0
-0.5
2
4
6
8
10
12
14
16
18
20
Normal Approximation to the Binomial
distribution
PX  a  Pa  12  Y  a  12 
• X has a Binomial distribution with
parameters n and p
• Y has a Normal distribution
m  np
s  npq
1
2
 continuity correction
0.2500
Approximating
Normal distribution
0.2000
P[X = a]
0.1500
Binomial distribution
0.1000
0.0500
-0
-0.5
2
4
6
8
10
a  12
12
a
14
a
16
1
2
18
20
0.2500
0.2000
Pa  12  Y  a  12 
0.1500
0.1000
0.0500
--
-0.5
a
0.2500
0.2000
P[X = a]
0.1500
0.1000
0.0500
--
-0.5
a
Example
• X has a Binomial distribution with
parameters n = 20 and p = 0.70
We want PX  13
The exact valu e PX  13
 20 
13
7
  0.70 0.30  0.1643
 13 
Using the Normal approximation to the
Binomial distribution
PX  13  P12 12  Y  13 12 
Where Y has a Normal distribution with:
m  np  20(0.70)  14
s  npq  20.70.30  2.049
Hence
P12.5  Y  13.5
12.5  14 Y  14 13.5  14 
 P



2
.
049
2
.
049
2
.
049


 P 0.73  Z  0.24
= 0.4052 - 0.2327 = 0.1725
Compare with 0.1643
Normal Approximation to the Binomial
distribution
Pa  X  b  p(a)  p(a  1)   p(b)
1
1

 P a  2  Y  b  2
• X has a Binomial distribution with
parameters n and p
• Y has a Normal distribution
m  np
s  npq
1
2  continuity correction
0.2500
Pa  X  b
0.2000
0.1500
0.1000
0.0500
--
-0.5
a  12
a
b
b  12
0.2500
Pa  12  Y  b  12 
0.2000
0.1500
0.1000
0.0500
--
-0.5
a  12
a
b
b  12
Example
• X has a Binomial distribution with
parameters n = 20 and p = 0.70
We want P11  X  14
The exact valu e P11  X  14
 p(11)  p(12)  p(13)  p(14)
 20 
 20 
11
9
14
6




  0.70 0.30     0.70 0.30
 11 
 14 
 0.0654  0.1144  0.1643  0.1916  0.5357
Using the Normal approximation to the
Binomial distribution
P11  X  14  P10 12  Y  14 12 
Where Y has a Normal distribution with:
m  np  20(0.70)  14
s  npq  20.70.30  2.049
Hence
P10.5  Y  14.5
10.5  14 Y  14 14.5  14 
 P



2
.
049
2
.
049
2
.
049


 P1.71  Z  0.24
= 0.5948 - 0.0436 = 0.5512
Compare with 0.5357
Comment:
• The accuracy of the normal
appoximation to the binomial
increases with increasing values of n
Example
• The success rate for an Eye operation is 85%
• The operation is performed n = 2000 times
Find the probability that:
1. The number of successful operations is
between 1650 and 1750.
2. The number of successful operations is at
most 1800.
Solution
• X has a Binomial distribution with
parameters n = 2000 and p = 0.85
We want P1680  X  1720
 P1679.5  Y  1720.5
where Y has a Normal distribution with:
m  np  2000(0.85)  1700
s  npq  200.85.15  15.969
Hence P1680  X  1720
 P1679.5  Y  1720.5
1679.5  1700 Y  1700 1720.5  1700 
 P



15
.
969
15
.
969
15
.
969


 P1.28  Z  1.28
= 0.9004 - 0.0436 = 0.8008
Solution – part 2.
We want PX  1800
 PY  1800.5
 Y  1700 1800.5  1700 
 P


15
.
969
15
.
969


 PZ  6.29
= 1.000
Sampling Theory
sampling distributions
Note:It is important to recognize the dissimilarity
(variability) we should expect to see in various
samples from the same population.
• It is important that we model this and use it
to assess accuracy of decisions made from
samples.
• A sample is a subset of the population.
• In many instances it is to costly to collect
data from the entire population.
Statistics and Parameters
A statistic is a numerical value computed from a
sample. Its value may differ for different samples.
e.g. sample mean x , sample standard deviation s, and
sample proportion p̂.
A parameter is a numerical value associated with a
population. Considered fixed and unchanging. e.g.
population mean m, population standard deviation s,
and population proportion p.
Observations on a measurement X
x1, x2, x3, … , xn
taken on individuals (cases) selected at random from a
population are random variables prior to their
observation.
The observations are numerical quantities whose
values are determined by the outcome of a random
experiment (the choosing of a random sample from
the population).
The probability distribution of the observations
x1,
x2, x3, … , xn
is sometimes called the population.
This distribution is the smooth histogram of the the
variable X for the entire population
0.07
0.06
0.05
0.04
0.03
0.02
0.01
0
0
10
20
30
40
50
60
the population is unobserved (unless all observations
in the population have been observed)
0.07
0.06
0.05
0.04
0.03
0.02
0.01
0
0
10
20
30
40
50
60
A histogram computed from the observations
x2, x3, … , xn
Gives an estimate of the population.
0.07
0.06
0.05
0.04
0.03
0.02
0.01
0
0
10
20
30
40
50
60
x1,
A statistic computed from the observations
x1, x2, x3, … , xn
Is also a random variable prior to observation of the
sample.
A statistic is also a numerical quantity whose value is
determined by the outcome of a random experiment
(the choosing of a random sample from the
population).
The probability distribution of statistic computed
from the observations
x1, x2, x3, … , xn
is sometimes called its sampling distribution.
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01
0
0
10
20
30
40
50
60
It is important to determine the sampling distribution
of a statistic.
It will describe its sampling behaviour.
The sampling distribution will be used the asses the
accuracy of the statistic when used for the purpose of
estimation.
Sampling theory is the area of Mathematical Statistics
that is interested in determining the sampling
distribution of various statistics
Many statistics have a normal distribution.
This quite often is true if the population is Normal
It is also sometimes true if the sample size is
reasonably large. (reason – the Central limit theorem,
to be mentioned later)
Two important statistics that have a normal distribution
The sample mean
x

x
i
n
The sample proportion:
pˆ  X
n
• X is the number of successes in a Binomial
experiment
The sampling distribution of the sample mean
Let x1, x2, … , xn denote n independent random
variables each coming from the same Normal
distribution with mean m and standard deviation s.
n
Let
x
x
i 1
n
i
1
1
   x1    x2 
n
n
What is the distribution of
x?
1
   xn
n
The distribution of averages (the mean)
Because the mean is a “linear combination”
1
1
m x    m x1    m x2 
n
n
1
1
  m  m 
n
n
1
   m xn
n
1
1
   m  n  m  m
n
n
and
2
2
2
1 2 1 2
1 2
s    s x1    s x2     s xn
n
n
n
2
2
2
s2 s2
1 2 1 2
1 2
   s   s    s  n 2 
n
n
n
n
n
2
x
Thus if x1, x2, … , xn denote n independent random
variables each coming from the same Normal
distribution with mean m and standard deviation s.
Then
n
x
x
i
i 1
n
1
1
   x1    x2 
n
n
1
   xn
n
has Normal distribution with
mean m x  m and
variance s x2 
s2
n
standard deviation s x 
s
n
Example
• Suppose we are measuring the cholesterol level of
men age 60-65
• This measurement has a Normal distribution with
mean m = 220 and standard deviation s = 17.
• A sample of n = 10 males age 60-65 are selected
and the cholesterol level is measured for those 10
males.
• x1, x2, x3, x4, x5, x6, x7, x8, x9, x10, are those 10
measurements
Find the probability distribution of x ?
Compute the probability that x is between 215 and
225
Solution
Find the probability distribution of x
Normal with m x  m  220
s
17
and s x 

 5.376
n
10
P  215  x  225
 215  220 x  220 225  220 
 P



5.376
5.376
5.376


 P  0.930  z  0.930  0.648
Graphs
0.08
The sampling
distribution of
the mean
0.06
0.04
The probability
distribution of
individual
observations
0.02
0
150
170
190
210
230
250
270
290
310
Standard Error of the Mean
In practice, the population standard deviation s is rarely
known, so we cannot compute the standard deviation of x ,
s
s.d.( x ) =
.
n
In practice, we only take one random sample, so we only have
the sample mean x and the sample standard deviation s.
Replacing s with s in the standard deviation expression gives
us an estimate that is called the standard error of x .
s
s.e.( x ) =
.
n
For a sample of n = 25 weight losses,
the standard deviation is s = 4.74 pounds.
So the standard error of the mean is 0.948 pounds.
The Central Limit Theorem
The Central Limit Theorem (C.L.T.) states that if n
is sufficiently large, the sample means of random
samples from a population with mean m and finite
standard deviation s are approximately normally
distributed with mean m and standard deviation
.
s
Technical Note:
n
The mean and standard deviation given in the CLT
hold for any sample size; it is only the “approximately
normal” shape that requires n to be sufficiently large.
Graphical Illustration of the Central Limit Theorem
Distribution of x:
n=2
Original Population
10
20
30
x
10
20
Distribution of x:
n = 30
Distribution of x:
n = 10
10
x
x
30
10
20
x
Applications of the sampling distribution of the sample
mean is and the Central Limit Theorem
• When the sampling distribution of the sample mean is
(exactly) normally distributed, or approximately
normally distributed (by the CLT), we can answer
probability questions related to the sample mean.
Example
Example:
15.
Consider a normal population with m = 50 and s =
Suppose a sample of size 9 is selected at random. Find:
1) P ( 45  x  60)
2) P ( x  47.5)
Solutions: Since the original population is normal, the distribution of the
sample mean is also (exactly) normal
1) m x  m  50
2) s x  s
n  15
9  15 3  5
Example
45
 1.00
z=
x-m
s
n
;
50
0
60
2.00
x
z
 45  50
60  50
 z 
P (45  x  60)  P

 5
5 
 P( 1.00  z  2.00)
 0.8413  0.0228  0.8185
Example
0.3085
47.5 50
-0.50
z=
x-m
s
n
;
0
x
z
 x  50 47.5  50

P( x  47.5)  P

 5
5 
 P( z  .5)
 0.5000  01915
 0.3085
.
Example
Example: A recent report stated that the day-care cost per week in Boston is
$109. Suppose this figure is taken as the mean cost per week and that the
standard deviation is known to be $20.
1) Find the probability that a sample of 50 day-care centers would show a mean cost of
$105 or less per week.
2) Suppose the actual sample mean cost for the sample of 50 day-care centers is $120.
Is there any evidence to refute the claim of $109 presented in the report?
Solutions:
•
The shape of the original distribution is unknown, but the sample size, n, is large. The
CLT applies.
•
The distribution of x is approximately normal
m x  m  109
sx  s
n  20
50  2.83
Example
1)
0.0793
105
141
.
z=
x-m
s
n
;
109
0

105  109 
P( x  105)  Pz 


2.83 
 P ( z  141
. )
 0.0793
x
z
Example
2)
• To investigate the claim, we need to examine how likely an observation
is the sample mean of $120
• Consider how far out in the tail of the distribution of the sample mean
is $120
z=
x-m
s
n
;



P ( x  120)  P  z  120 109 

2.83 
 P ( z  3.89 )
 1.0000 - 0.9999 = 0.0001
• Since the probability is so small, this suggests the observation of $120 is very
rare (if the mean cost is really $109)
• There is evidence (the sample) to suggest the claim of m = $109 is likely wrong
Summary
• The mean of the sampling distribution of x is equal to the mean of the original
population:
m m
x
• The standard deviation of the sampling distribution of x (also called the
standard error of the mean) is equal to the standard deviation of the original
population divided by the square root of the sample size:
sx  s n
Notes:
– The distribution of x becomes more compact as n increases. (Why?)
2
2
– The variance of x : s x  s n
• The distribution of x is (exactly) normal when the original population is
normal
• The CLT says: the distribution of x is approximately normal regardless of the
shape of the original distribution, when the sample size is large enough!
Sampling Distribution
for Any Statistic
Every statistic has a sampling distribution,
but the appropriate distribution may not always
be normal, or even approximately bell-shaped.
Construct an approximate sampling distribution
for a statistic by actually taking repeated samples
of the same size from a population and constructing
a relative frequency histogram for the values of the
statistic over the many samples.
Sampling Distribution for Sample Proportions
Let p = population proportion of interest
or binomial probability of success.
Let
X
no. of succeses
pˆ 
n

no. of bimomial trials
= sample proportion or proportion of successes.
Then the sampling distributi on of p̂
is a normal distribution with
mean m pˆ  p
s pˆ 
p (1  p )
n
Example
Sample Proportion Favoring a Candidate
Suppose 20% all voters favor Candidate A. Pollsters take a
sample of n = 600 voters. Then the sample proportion who favor
A will have approximately a normal distribution with
mean m pˆ  p  0.20
s pˆ 
p (1  p )
n

0.20 (0.80 )
600
 0.1633
Sampling distributi on of p̂
30
25
20
15
c
10
5
0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Using the Sampling distribution:
Suppose 20% all voters favor Candidate A. Pollsters take a
sample of n = 600 voters.
Determine the probability that the sample proportion will be
between 0.18 and 0.22
i.e. the probabilit y, P0.18  p  0.22
Solution:
Recall m pˆ  p  0.20
s pˆ 
p (1  p )
n

0.20 (0.80 )
600
 0.1633
 0.18  0.20 p  0.20 0.22  0.20 
P0.18  p  0.22  P 


0.1633
0.1633 
 0.1633
 P1.225  z  1.225  0.8897  0.1103  0.7794
Sample proportion within 5 percentage points
Determine the probability that the sample proportion will be
between 0.15 and 0.25
i.e. the probabilit y, P0.18  p  0.22
Solution:
Again m pˆ  p  0.20
s pˆ 
p (1  p )
n

0.20 (0.80 )
600
 0.1633
 0.15  0.20 p  0.20 0.25  0.20 
P0.15  p  0.25  P 


0.1633
0.1633 
 0.1633
 P 3.062  z  3.062  0.9989  0.00110  0.9978