Transcript p value

P Values - part 2
Samples &
Populations
Robin Beaumont
2011
With much help from
Professor Chris Wilds material
University of Auckland
Aspects of the P value
Resume
A P value is a conditional probability considering a range of outcomes
Sample value
P value = P(observed summary value + those more extreme |population value = x)
Hypothesised
population value
Populations and samples
Ever constant
at least for your study!
= Parameter
estimate = statistic
One sample
Size matters – single samples
Size matters – multiple samples
We only have a rippled mirror
Standard deviation - individual level
Area! Wait and
see
But does not take into account
sample size
= t distribution
'Standard Normal distribution'
Area:
95%
68%
Total Area = 1
SD value
=
0
1
2
Between + and - three- standard deviations from the mean = 99.7% of area
Therefore only 0.3% of area(scores) are more than 3 standard deviations ('units') away.
Defined by sample size aspect
~ df
= measure of variability
Sampling level -‘accuracy’ of estimate
Talking about means here
= 5/√5 = 2.236
SEM = 5/√25 = 1
SEM =
𝑠𝑞𝑢𝑎𝑟𝑒 𝑟𝑜𝑜𝑡 𝑜𝑓 𝑛𝑢𝑚𝑏𝑒𝑟 𝑖𝑛 𝑠𝑎𝑚𝑝𝑙𝑒
𝑠𝑡𝑎𝑛𝑑𝑎𝑟𝑑 𝑑𝑒𝑣𝑖𝑎𝑡𝑖𝑜𝑛 𝑜𝑓 𝑠𝑎𝑚𝑝𝑙𝑒
We can predict the
accuracy of your
estimate (mean)
by just using the
SEM formula.
From a single
sample
From: http://onlinestatbook.com/stat_sim/sampling_dist/index.html
Example - Bradford Hill, (Bradford Hill, 1950 p.92)
• mean systolic blood pressure for 566 males around Glasgow
= 128.8 mm. Standard deviation =13.05
• Determine the ‘precision’ of this mean.
All possible values of
POPULATION mean
• “We may conclude that our observed mean may differ from
the true mean by as much as ± 2.194 (.5485 x 4) but not
more than that in around 95% of samples. page 93. [edited]
Sampling summary
• The SEM formula allows us to:
• predict the accuracy of your estimate (
i.e. the mean value of our sample)
• From a single sample
• Assumes Random sample
Variation what have we ignored!
Onto Probability now
Probabilities are rel. frequencies
Frequency - No. of
Frequency Distribution (Histogram) of exam results
12
11
10
9
8
7
6
5
4
3
2
1
0
40-44
45-49
50-54
55-59
60-64
65-69
70-74
75-79
80-84
85-90
Scores
Relative
frequency =
Probability
0.25
Probability Distribution
total 48 scores
0.2
The total area = 1
0.15
All outcomes at
any one time = 1
0.1
0.05
0
Scores
40-44
45-49
50-54
55-59
60-64
65-69
70-74
75-79
80-84
85-90
Multiple outcomes at any one time
Probability Density Function
11
10
Probability
Density
The total area = 1
total 48 scores
9
8
7
6
5
4
3
2
1
0
B
A
33
37
43
p(score<45) = area A
47
53
57
63
Scores
67
73
77
83
p(score > 50) = area B
P(score<45 and score >50)
=
Just add up the individual
outcomes
87
What happens in the past affects the present
= Conditional Probability
Disease X
P(disease x |male)
Male
P(male)
Disease X AND Male
P(disease AND male) = P(male) x P(disease x | male)
No Disease X
female
Disease X
P(disease AND male) /P(male) = P(disease x | male)
Multiple each branch of the tree
to get end value
No Disease X
Screening Example
0.1% of the population carry a particular faulty gene.
A test exists for detecting whether an individual is a carrier of the gene.
In people who actually carry the gene, the test provides a positive result with
probability 0.9.
In people who don’t carry the gene, the test provides a positive result with
probability
0.01. result when
If someone
gets a positive
P( P | G)
tested,
the probability
they P = test is positive
Let Gfind
= person
carriesthat
gene
for gene N = test is negative for gene
actually are a carrier of the gene.
We want to find
P(G | P) =
P(G and P)
P(P)
0.0009
P(P) = P(GP(G
and| P)
+ P(G'
andP)
= 0.0009 +
P) =
0.0826
0.01089
0.00999 = 0.01089
P(P | G) ≠ P (G | p)
ORDER MATTERS
Errors
Survival analysis
• Each years
survival depends
on previous ones
or does it?
Probability summary
•
•
•
•
•
All outcomes at any one time add up to 1
Probability histogram = area under curve =1
-> specific areas = set of outcomes
-> specific areas = ‘equal to or more extreme’
Conditional probability – present dependent
on past – ORDER MATTERS
Putting it all together
Statistics
• Summary measure – SEM, Average etc
• T statistic – different types, simplest:
Tstatistic  observed
difference in estimated mean and population value
sampling variability in means
Tstatistic  observed
difference in estimated mean and population value
SEM

observed difference in estimated mean and population value
expected variability in means due to random samping
Signal

Noise
So when t = 0 means 0/anything = estimated and
hypothesised population mean are equal
So when t = 1 observed different same as SEM
So when t = 10 observed different much greater than
SEM
T statistic example
Serum amylase values from a random sample of 15
apparently healthy subjects. The mean = 96
SD= 35 units/100 ml.
How likely would such a sample be obtained from a
GIVEN the population value
population of serum amylase determinations
= the null hypothesis
with a mean
of
120.
(taken
from
Daniel 1991
96  120
24
T


 2.656
35
p.202
adapted)
9.037
statistic
15
This looks like a rare occurrence?
But for what
t density:
Original units:
What does the shaded
area mean!
Shaded
area
=0.0188
s x = 9.037 n =15
120
96
Given that the sample was obtained
from a population with a mean of 120
a sample with a T(n=15) statistic of 2.656 or 2.656 or one more extreme
will occur 1.8% of the time = just
under two samples per hundred on
average.
....
Given that the sample was obtained
from a population with a mean of 120
2.656
0
-2.656
t
Serum amylase values from a random sample of 15
a sample of 15 producing a mean of 96
apparently healthy subjects. mean =96 SD= 35 units/100
ml.
(120-x where x=24) or 144 (120+x
How likely would such a sample be obtained from a
where x=24) or
extreme will
population of serum amylase determinations with a
Butone
it thismore
not a P value
mean of 120. (taken from Daniel 1991 p.202 adapted)
occur 1.8% of the time, that is just
P value = 2 · P(t < t| H is true) = 2 · [area to the left of t under a t distribution with df = n − 1]
under two samples per hundred on
0
(n−1)
o
P value and probability for t statistic
p value
= 2 x P(t(n-1) values more extreme than t(n-1) | Ho is true)
= 2 · [area to the left of t under a t distribution
A p value is a special type of
with n − 1 shape]
probability with:
Multiple outcomes + conditional
upon the specified parameter value
Putting it all together
Do we need it!
Rules
Set a level of acceptability =
critical value (CV)!
t density:
Original units:
s x = 9.037 n =15
120
96
Shaded
area
=0.0188
0
t
-2.656
0
2.656
Say one in twenty 1/20 =
Or 1/100
Or 1/1000
or . . . .
If our result has a P value of less than
our level of acceptability.
Reject the parameter value. Say 1 in 20
(i.e.CV=0.5)
Given that the sample was obtained
from a population with a mean
(parameter value) of 120 a sample
with a T(n=15) statistic of -2.656 or 2.656
or one more extreme with occur 1.8%
of the time, This is less than one in
do we replace
twenty therefore What
we dismiss
theit with?
Fisher – only know and only consider
the model we have i.e. The parameter
we have used in our model –
when we reject it we accept that any
value but that one can replace it.
Neyman and Pearson + Gossling
Must have an alternative specified
value for the parameter
If there is an alternative - what is it – another distribution!
•Power – sample size
•Affect size
•– indication of clinical
importance:
Serum amylase values from a random sample of 15
apparently healthy subjects. mean =96 SD= 35 units/100
ml.
How likely would such a sample be obtained from a
population of serum amylase determinations with a
mean of 120. (taken from Daniel 1991 p.202 adapted)
= 96
= 120
α = the
reject
region
Correct
decisions
incorrect
decisions
Insufficient
power – never
get a significant
result even when
effect size large
Too much power
get significant
result with trivial
effect size
Life after P values
•
•
•
•
Confidence intervals
Effect size
Description / analysis
Bayesian statistics - qualitative approach by the back door!
• Planning to do statistics for your dissertation?
see: My medical statistics courses:
Course 1:
www.robin-beaumont.co.uk/virtualclassroom/stats/course1.html
YouTube videos to accompany course 1:
http://www.youtube.com/playlist?list=PL9F0EBD42C0AB37D0
Course 2:
www.robin-beaumont.co.uk/virtualclassroom/stats/course2.html
YouTube videos to accompany course 2:
Your attitude to your data
Where do they fit in!
Students bloomers
• The p value did not indicate much statistic
significance
• Given that the population comes from one
population
• The p value is 0.003 thus rejecting the null
hypothesis and there is a statistical significance
• Correlation = 0.25 (p<0.001) indicating that
assuming that the data come from a bivariate
normal distribution with a correlation of zero you
would obtain a correlation of <0.000. There is