Transcript Inference for a Single Population Proportion (p)

Inference for a Single Population
Proportion (p)
Sampling Distribution of the
Sample Proportion
Sample Proportion
p^ = estimate of population proportion p
= proportion of sample having a
specified attribute
x
= n
where
n = sample size and
x = the observed number of “successes” in
the sample
pˆ is random and varies from sample to sample
Sampling Distribution of the Sample Proportion
The histograms below show the estimated sampling distribution of
the sample proportion based upon 1000 samples of the drawn for
the given sample size (n).
For larger samples,
the sampling
distribution of the
sample proportion
is approximately
normal.
Sampling Distribution of the
Sample Proportion
Sampling Distribution of pˆ
1. Provided n is “sufficiently large”, the sampling
distribution is normal, generally
np  10
*
and n (1  p )  10
*
*
5 is also used.
2. The mean of the sampling distribution is
p = true population proportion
3. Standard deviation of the sampling distribution or the
standard error of sample proportion is given by:
SE ( pˆ ) 
p (1  p )
n
Notation
:

pˆ ~ N  p ,


p (1  p ) 


n

Implications for Inference
(n “large”)
CI for Population Proportion (p)
pˆ  ( z - value)
pˆ (1 - pˆ )
n
z-values from standard normal
90% = 1.645
95% = 1.960
99% = 2.578
Effect Size
Test Statistic (Ho: p = po)
z
pˆ  p o
p o (1  p o )
n
~ standard
normal
d 
| pˆ  p o |
p o (1  p o )
Hypothesis Testing for a Population
Proportion (p)
Null Hypothesis
Ho: p = po
Alternative Hypotheses
(upper-tailed)
p-value
P-value
0 z
P-value
(lower-tailed)
z
0
Hypothesis Testing for a Population
Proportion (p)
Null Hypothesis
Ho: p = po
Alternative Hypotheses
(two-tailed)
p-value
-z
0
z
For two-tailed tests in general it is preferable to simply
construct a confidence interval for the parameter of
interest and see if the hypothesized value under the null
hypothesis is contained in the CI. If it is we fail to reject
Ho and if it is not then we reject Ho.
Example: Treatment of Kidney Cancer
• Historically, one in five kidney cancer patients
survive 5 years past diagnosis, i.e. 20%.
• An oncologist using an experimental therapy
treats n = 40 kidney cancer patients and 16 of
them survive at least 5 years.
• Is there evidence that patients receiving the
experimental therapy have a higher 5-year
survival rate?
Step 1: Formulate Hypotheses
p = the proportion of kidney cancer patients receiving
the experimental therapy that survive at least 5 years.
H o : p  . 20 (5 - yr. survival
rate is not better)
H A : p  . 20 (5 - yr. survival
rate is better)
Step 2: Determine test criteria
Choose a  .05 (may want to consider smaller?)
Use large sample test ? Definitely questionable as we have…
np = (40)(.20) = 8 > 5 and n(1-p) = (40)(.80) = 32 > 5
Step 3: Collect data and compute
test statistic
pˆ 
16
 . 40 or a 40% 5 - yr. survival
rate
40
z
pˆ  p o
p o (1  p o )
n

. 40  . 20
. 20 (1  . 20 )
 3 . 16
40
The observed 5-yr. survival rate for kidney cancer
patients undergoing the experimental therapy is 3.16
standard errors above the historical rate!
Step 4: Compute p-value
P-value = .0008
0
z = 3.16
It is highly unlikely we would
obtain a 40% 5-yr. survival rate in
our sample, if in fact the 5-yr.
survival rate for the population
of patients treated with the
experimental therapy was truly
20%.
Step 5: Make Decision and Interpret
We have very strong evidence to suggest the 5-year
survival rate for kidney cancer patients undergoing
the experimental therapy is greater than the current
5-yr. survival rate of 20% (p = .0008).
Step 6: Quantify significant results
Confidence Interval
pˆ  1 . 96
pˆ (1  pˆ )
n
 . 40  1 . 96
(. 40 )(. 60 )
40
Effect Size
d 
| . 40  . 20 |
 1 . 25
(. 20 )(. 80 )
A very large effect whi
ch is not surprising
 . 40  (1 . 96 )(. 077 )
the success rate using the experiment
 . 40  . 15
is twice as large.
 (. 25 ,. 55 ) or (25% , 55%)
given
al therapy
Power Calculation
For power use software
Baseline proportion is the
proportion under the null
hypothesis (.20 or 20% here)
Difference to detect is the
absolute difference between p
under alternative and p under
null (i.e., .40 - .20 = .20)
Power is calculated once sample
size and difference information is
entered (here, Power = .935).
Power Curve for n = 40 and po = .20
For a difference of .20
Power = .935 as seen on
previous slide
Sample Size and CI’s for p
• Suppose we wish to estimate p using a 95% CI and
have a margin of error of 3%. What sample size do
we need to use?
• Recall the CI for p is given by:
pˆ  ( z - value)
pˆ (1 - pˆ )
n
MARGIN OF ERROR (E)
Sample Size and CI’s for p
• Here for a 95% CI we want E = .03 or 3%
E  1 . 96
pˆ (1 - pˆ )
 . 03
n
• After some wonderful algebraic manipulation
1 . 96 pˆ (1  pˆ )
2
n
E
2
Oh, oh! We don’t know p-hat !!
1. “Guesstimate”
2. Use p-hat from pilot or prior
study.
3. Largest n we would ever need
comes when p-hat = .50.
Sample Size and CI’s for p
1. Informed approach
n
2
1 . 96 pˆ (1  pˆ )
E
2
pˆ  from prior knowledge
2. Conservative approach (i.e. worst case scenario)
n
1 . 96
4E
2
2
uses pˆ  .50
Standard normal values
90% = 1.645
95% = 1.960
99% = 2.578
Sample Size and CI’s for p
•
•
•
Original Question: Suppose we wish to estimate p
using a 95% CI and have a margin of error of 3%.
What sample size do we need to use?
Assume that we estimate the 5 yr. survival rate for a
new kidney cancer therapy, and we know historical
that it this survival rate is around 20%.
Using informed approach
n
2
1 . 96 pˆ (1  pˆ )
E
2
2

1 . 96 (. 20 )(. 80 )
. 03
2
 682 . 95  n  683 subjects
Sample Size and CI’s for p
•
•
•
Original Question: Suppose we wish to estimate p
using a 95% CI and have a margin of error of 3%.
What sample size do we need to use?
Assume that we estimate the 5 yr. survival rate for a
new kidney cancer therapy, and we know historical
that it this survival rate is around 20%.
Using conservative approach
n
1 . 96
2
2

1 . 96
2
2
 1067 . 1  n  1068 subjects
4E
4 (. 03 )
This is why in media polls you they usually report a sampling
error of + 3% and that the poll was based on a sample of
n = 1000 individuals.
Small Sample Inference for p:
Binomial Exact Test
• When the sample size is “small” the sampling
distribution cannot be approximated by a standard
normal distribution.
• However, regardless of sample size the EXACT
sampling distribution of the number of “successes” in n
independent trials ALWAYS has a Binomial
Distribution.
• Thus if we knew more about the binomial distribution
we could use it to find p-values when conducting a
hypothesis test and also when constructing confidence
intervals for the pop. proportion p.
Binomial Probability Distribution
A binomial random variable X is defined to the number
of “successes” in n independent trials where the
P(“success”) = p is constant.
Notation: X ~ BIN(n,p)
In the definition above notice the following conditions
need to be satisfied for a binomial experiment:
1. There is a fixed number of n trials carried out.
2. The outcome of a given trial is either a “success”
or “failure”.
3. The probability of success (p) remains constant
from trial to trial.
4. The trials are independent, the outcome of a trial is
not affected by the outcome of any other trial.
Binomial Distribution
• If X ~ BIN(n, p), then
 n x
n!
n x
x
n x
P( X  x)    p (1  p)

p (1  p)
x!(n  x)!
 x
x  0,1,..., n.
• where
n!  n  ( n  1)  ( n  2 )  ...  1, also 0!  1 and 1!  1
n
   " n choose x"  the number
x
x " successes"
P (" success " )  p
in n trials.
of ways to obtain
Binomial Distribution
• If X ~ BIN(n, p), then
 n x
n!
n x
x
n x
P( X  x)    p (1  p)

p (1  p)
x!(n  x)!
 x
x  0,1,..., n.
• E.g. when n = 3 and p = .50 there are 8 possible
equally likely outcomes (e.g. flipping a coin)
SSS SSF SFS FSS SFF FSF FFS FFF
X=3 X=2 X=2 X=2 X=1 X=1 X=1 X=0
P(X=3)=1/8, P(X=2)=3/8, P(X=1)=3/8, P(X=0)=1/8
• Now let’s use binomial probability formula instead…
Binomial Distribution
• If X ~ BIN(n, p), then
 n x
n!
n x
x
n x
P( X  x)    p (1  p)

p (1  p)
x!(n  x)!
 x
x  0,1,..., n.
• E.g. when n = 3, p = .50 find P(X = 2)
3
3!
3!
3  2 1
  


 3 ways
 2  2! ( 3  2 )! 2! 1! ( 2  1)  1
SSF
SFS
FSS
3 2
3 2
2
1
P ( X  2 )   . 5 (. 5 )
 3 (. 5 )(. 5 )  . 375 or 3
8
2
Example: Treatment of Kidney Cancer
• In our example we had n = 40 patients and if
we assume the experimental therapy is no
better than current treatments then probability
of 5-year survival is p = .20.
• Thus the number of patients in our study
surviving at least 5 years has a binomial
distribution, i.e. X ~ BIN(40,.20).
Example: Treatment of Kidney Cancer
• X ~ BIN(40,.20), find the probability that exactly 16
patients survive at least 5 years.
 40  16
24


P
(
X

16
)

.
20
.
80
 . 001945
•
 16 
 
• This requires some calculator gymnastics and some
scratchwork! Also, keep in mind for a p-value we
need to find the probability of having 16 or more
patients surviving at least 5 yrs.
• Remember p-value is defined as evidence as extreme
or more extreme.
Example: Treatment of Kidney Cancer
• So we actually need to find:
p-value = P(X = 16) + P(X = 17) + … + P(X = 40)
 40  16
24
P ( X  16 )   . 20 . 80  . 001945
 16 
+
…
+
 40  17
23
P ( X  17 )   . 20 . 80  . 000686
 17 
 40 
40
0
P ( X  40 )   . 20 . 80  0
 40 
EXACT p-value = .002936
YIPES!
Example: Treatment of Kidney Cancer
• X ~ BIN(40,.20), find the probability that 16 or more
patients survive at least 5 years.
• USE COMPUTER!
• Binomial Exact Test p-value calculator in JMP
p-values are computed
automatically for all three
possible alternatives
Enter
n = sample size
x = observed # of “successes”
po = proportion under Ho
Example: Treatment of Kidney Cancer
• X ~ BIN(40,.20), find the probability that 16 or more
patients survive at least 5 years.
• USE COMPUTER!
• Binomial Exact Test p-value calculator in JMP
Exact p-value = .0029362
Contrasting this EXACT p-value (p = .0029) to the one calculated
earlier using the normal approximation (p = .0008) we see a fairly
substantial difference! MORAL: USE EXACT WHEN n is SMALL !!!
Exact CI for p using the
binomial distribution
• Find LCL and UCL for p by finding
probabilities that meet the following
requirements:
P(X > x|p = LCL) = a/2
and
e.g. for 95% confidence
P(X < x|p = UCL) = a/2 a  .05  a/2  .025
• Use computer to find these probabilities.
Exact CI for p using the
binomial distribution
• Find a 95% CI for p for the kidney cancer study
For the lower confidence limit we find LCL = .248 or .249
Exact CI for p using the
binomial distribution
• Find a 95% CI for p for the kidney cancer study
For the upper confidence limit we find UCL = .566 or .567
Therefore based on an EXACT 95% confidence interval we
estimate that the success rate of the experimental therapy
is between 24.8% and 56.7%.
Summary of Inference for a Single
Population Proportion (p)
• When n is “large” use large sample methods
based on the sampling distribution being
approximately normal. (Easy)
• When n is “small” use exact methods based on
the binomial distribution, which requires specific
software or tables. (Hard)
• Exact methods can always be used!
• In general, precise estimates of a population
proportion requires a large samples size,
e.g. media polls which typically use n = 1,000.