Transcript p value
P Values - part 2
Samples &
Populations
Robin Beaumont
2011
With much help from
Professor Chris Wilds material
University of Auckland
Aspects of the P value
Resume
A P value is a conditional probability considering a range of outcomes
Sample value
P value = P(observed summary value + those more extreme |population value = x)
Hypothesised
population value
Populations and samples
Ever constant
at least for your study!
= Parameter
estimate = statistic
One sample
Size matters – single samples
Size matters – multiple samples
We only have a rippled mirror
Standard deviation - individual level
Area! Wait and
see
But does not take into account
sample size
= t distribution
'Standard Normal distribution'
Area:
95%
68%
Total Area = 1
SD value
=
0
1
2
Between + and - three- standard deviations from the mean = 99.7% of area
Therefore only 0.3% of area(scores) are more than 3 standard deviations ('units') away.
Defined by sample size aspect
~ df
= measure of variability
Sampling level -‘accuracy’ of estimate
Talking about means here
= 5/√5 = 2.236
SEM = 5/√25 = 1
SEM =
𝑠𝑞𝑢𝑎𝑟𝑒 𝑟𝑜𝑜𝑡 𝑜𝑓 𝑛𝑢𝑚𝑏𝑒𝑟 𝑖𝑛 𝑠𝑎𝑚𝑝𝑙𝑒
𝑠𝑡𝑎𝑛𝑑𝑎𝑟𝑑 𝑑𝑒𝑣𝑖𝑎𝑡𝑖𝑜𝑛 𝑜𝑓 𝑠𝑎𝑚𝑝𝑙𝑒
We can predict the
accuracy of your
estimate (mean)
by just using the
SEM formula.
From a single
sample
From: http://onlinestatbook.com/stat_sim/sampling_dist/index.html
Example - Bradford Hill, (Bradford Hill, 1950 p.92)
• mean systolic blood pressure for 566 males around Glasgow
= 128.8 mm. Standard deviation =13.05
• Determine the ‘precision’ of this mean.
All possible values of
POPULATION mean
• “We may conclude that our observed mean may differ from
the true mean by as much as ± 2.194 (.5485 x 4) but not
more than that in around 95% of samples. page 93. [edited]
Sampling summary
• The SEM formula allows us to:
• predict the accuracy of your estimate (
i.e. the mean value of our sample)
• From a single sample
• Assumes Random sample
Variation what have we ignored!
Onto Probability now
Probabilities are rel. frequencies
Frequency - No. of
Frequency Distribution (Histogram) of exam results
12
11
10
9
8
7
6
5
4
3
2
1
0
40-44
45-49
50-54
55-59
60-64
65-69
70-74
75-79
80-84
85-90
Scores
Relative
frequency =
Probability
0.25
Probability Distribution
total 48 scores
0.2
The total area = 1
0.15
All outcomes at
any one time = 1
0.1
0.05
0
Scores
40-44
45-49
50-54
55-59
60-64
65-69
70-74
75-79
80-84
85-90
Multiple outcomes at any one time
Probability Density Function
11
10
Probability
Density
The total area = 1
total 48 scores
9
8
7
6
5
4
3
2
1
0
B
A
33
37
43
p(score<45) = area A
47
53
57
63
Scores
67
73
77
83
p(score > 50) = area B
P(score<45 and score >50)
=
Just add up the individual
outcomes
87
What happens in the past affects the present
= Conditional Probability
Disease X
P(disease x |male)
Male
P(male)
Disease X AND Male
P(disease AND male) = P(male) x P(disease x | male)
No Disease X
female
Disease X
P(disease AND male) /P(male) = P(disease x | male)
Multiple each branch of the tree
to get end value
No Disease X
Screening Example
0.1% of the population carry a particular faulty gene.
A test exists for detecting whether an individual is a carrier of the gene.
In people who actually carry the gene, the test provides a positive result with
probability 0.9.
In people who don’t carry the gene, the test provides a positive result with
probability
0.01. result when
If someone
gets a positive
P( P | G)
tested,
the probability
they P = test is positive
Let Gfind
= person
carriesthat
gene
for gene N = test is negative for gene
actually are a carrier of the gene.
We want to find
P(G | P) =
P(G and P)
P(P)
0.0009
P(P) = P(GP(G
and| P)
+ P(G'
andP)
= 0.0009 +
P) =
0.0826
0.01089
0.00999 = 0.01089
P(P | G) ≠ P (G | p)
ORDER MATTERS
Errors
Survival analysis
• Each years
survival depends
on previous ones
or does it?
Probability summary
•
•
•
•
•
All outcomes at any one time add up to 1
Probability histogram = area under curve =1
-> specific areas = set of outcomes
-> specific areas = ‘equal to or more extreme’
Conditional probability – present dependent
on past – ORDER MATTERS
Putting it all together
Statistics
• Summary measure – SEM, Average etc
• T statistic – different types, simplest:
Tstatistic observed
difference in estimated mean and population value
sampling variability in means
Tstatistic observed
difference in estimated mean and population value
SEM
observed difference in estimated mean and population value
expected variability in means due to random samping
Signal
Noise
So when t = 0 means 0/anything = estimated and
hypothesised population mean are equal
So when t = 1 observed different same as SEM
So when t = 10 observed different much greater than
SEM
T statistic example
Serum amylase values from a random sample of 15
apparently healthy subjects. The mean = 96
SD= 35 units/100 ml.
How likely would such a sample be obtained from a
GIVEN the population value
population of serum amylase determinations
= the null hypothesis
with a mean
of
120.
(taken
from
Daniel 1991
96 120
24
T
2.656
35
p.202
adapted)
9.037
statistic
15
This looks like a rare occurrence?
But for what
t density:
Original units:
What does the shaded
area mean!
Shaded
area
=0.0188
s x = 9.037 n =15
120
96
Given that the sample was obtained
from a population with a mean of 120
a sample with a T(n=15) statistic of 2.656 or 2.656 or one more extreme
will occur 1.8% of the time = just
under two samples per hundred on
average.
....
Given that the sample was obtained
from a population with a mean of 120
2.656
0
-2.656
t
Serum amylase values from a random sample of 15
a sample of 15 producing a mean of 96
apparently healthy subjects. mean =96 SD= 35 units/100
ml.
(120-x where x=24) or 144 (120+x
How likely would such a sample be obtained from a
where x=24) or
extreme will
population of serum amylase determinations with a
Butone
it thismore
not a P value
mean of 120. (taken from Daniel 1991 p.202 adapted)
occur 1.8% of the time, that is just
P value = 2 · P(t < t| H is true) = 2 · [area to the left of t under a t distribution with df = n − 1]
under two samples per hundred on
0
(n−1)
o
P value and probability for t statistic
p value
= 2 x P(t(n-1) values more extreme than t(n-1) | Ho is true)
= 2 · [area to the left of t under a t distribution
A p value is a special type of
with n − 1 shape]
probability with:
Multiple outcomes + conditional
upon the specified parameter value
Putting it all together
Do we need it!
Rules
Set a level of acceptability =
critical value (CV)!
t density:
Original units:
s x = 9.037 n =15
120
96
Shaded
area
=0.0188
0
t
-2.656
0
2.656
Say one in twenty 1/20 =
Or 1/100
Or 1/1000
or . . . .
If our result has a P value of less than
our level of acceptability.
Reject the parameter value. Say 1 in 20
(i.e.CV=0.5)
Given that the sample was obtained
from a population with a mean
(parameter value) of 120 a sample
with a T(n=15) statistic of -2.656 or 2.656
or one more extreme with occur 1.8%
of the time, This is less than one in
do we replace
twenty therefore What
we dismiss
theit with?
Fisher – only know and only consider
the model we have i.e. The parameter
we have used in our model –
when we reject it we accept that any
value but that one can replace it.
Neyman and Pearson + Gossling
Must have an alternative specified
value for the parameter
If there is an alternative - what is it – another distribution!
•Power – sample size
•Affect size
•– indication of clinical
importance:
Serum amylase values from a random sample of 15
apparently healthy subjects. mean =96 SD= 35 units/100
ml.
How likely would such a sample be obtained from a
population of serum amylase determinations with a
mean of 120. (taken from Daniel 1991 p.202 adapted)
= 96
= 120
α = the
reject
region
Correct
decisions
incorrect
decisions
Insufficient
power – never
get a significant
result even when
effect size large
Too much power
get significant
result with trivial
effect size
Life after P values
•
•
•
•
Confidence intervals
Effect size
Description / analysis
Bayesian statistics - qualitative approach by the back door!
• Planning to do statistics for your dissertation?
see: My medical statistics courses:
Course 1:
www.robin-beaumont.co.uk/virtualclassroom/stats/course1.html
YouTube videos to accompany course 1:
http://www.youtube.com/playlist?list=PL9F0EBD42C0AB37D0
Course 2:
www.robin-beaumont.co.uk/virtualclassroom/stats/course2.html
YouTube videos to accompany course 2:
Your attitude to your data
Where do they fit in!
Students bloomers
• The p value did not indicate much statistic
significance
• Given that the population comes from one
population
• The p value is 0.003 thus rejecting the null
hypothesis and there is a statistical significance
• Correlation = 0.25 (p<0.001) indicating that
assuming that the data come from a bivariate
normal distribution with a correlation of zero you
would obtain a correlation of <0.000. There is