Transcript p value
P Values - part 3 The P value as a ‘statistic’ Robin Beaumont 1/03/2012 With much help from Professor Geoff Cumming P values - Putting it all together Summary Reviewso far • A P value is a conditional probability which considers a range of outcomes – shown as a ‘area’ in a graph. • The SEM formula allows us to: predict the accuracy of your estimate ( i.e. the mean value of our sample) across a infinite number of samples! Summary far What is a so statistic? • A statistic is just a summary measure, technically we have reduced a set of data to one or two values: • Range (smallest – largest) • Mean, median etc. • Inter-quartile range, SD Variance • Z score, T value, chi square value, F value etc • P value T value • T statistic – different types, simplest 1 sample: Tstatistic observed difference in estimated mean and population value sampling variability in means Tstatistic observed difference in estimated mean and population value SEM observed difference in estimated mean and population value expected variability in means due to random samping Signal Noise So when t = 0 means 0/anything = estimated and hypothesised population mean are equal So when t = 1 observed different same as SEM So when t = 10 observed different much greater than SEM T statistic example Serum amylase values from a random sample of 15 apparently healthy subjects. The mean = 96 SD= 35 units/100 ml. How likely would such a ‘unusual’ sample be obtained from a population of serum amylase determinations with a mean of 120. (taken from Daniel 1991 p.202 adapted) The population value = the null hypothesis Tstatistic 96 120 24 35 9.037 15 2.656 This looks like a rare occurrence? t density: s x = 9.037 n =15 120 96 Given that the sample was obtained from a population with a mean of 120 a sample with a T(n=15) statistic of 2.656 or 2.656 or one more extreme will occur 1.8% of the time = just under two samples per hundred on Shaded area average. =0.0188 .... What does the shaded Given that the sample was obtained area mean! from a population with a mean of 120 0 a sample of 15 producing a mean of 96 2.656 0 -2.656 t (120-x where x=24) or 144 (120+x Serum amylase values from a random sample of 15 apparently healthy subjects. mean =96 SD= 35 units/100 ml. where x=24) or one more extreme will How likely would such a unusual sample be obtained from a population of serum amylase determinations with a mean of 120. occur 1.8% of the time, that is just (taken from Daniel 1991 p.202 adapted) under two samples per hundred on =P value average. P value = 2 · P(t < t| H is true) = 2 · [area to the left of t under a t distribution with df = n − 1] Original units: (n−1) o P value and probability for the one sample t statistic p value = 2 x P(t(n-1) values more extreme than obtained t(n-1) | Ho is true) = 2 X [area to the left of t under a t distribution with n − 1 shape] Statistic -> sampling distribution -> PDF -> p value No sampling distribution! Create a virtual one P Value Variability Taking another random sample the P value be different How different? – Does not follow a normal distribution Depends upon the probability of the null hypothesis being true! Remember we have assumed so far that the null hypothesis is true. Dance of the p values – Geoff Cummings Simplified dance of the p values when the null hypothesis is true Example from Geoff Cummings dance of the p values The take home message is that we can obtain very small p values even when the null hypothesis is true. Why no CI for the P Value if it varies across trials • P value -> statistic but • Not all statistics represent values that are reflected in a population value • Other ways of getting an idea of variability across trials: • Reproducibility Probability Value (RP) Goodman 1992 and also 2001 journal articles Hung, O’Neill, Bauer & Kohne 1997 Biometrics journal Shao & Chow 2002 – Statistics in Medicine journal Boos & Stefanki 2011 – Journal of the American statistical association Cummings 2008 + and book Cumming’s Reproducibility (replication) Probability Value Given Pobtained = 0.05 What is the interval in which we are likely to see 80% of subsequent P values? Answer: We have 80% of seeing subsequent p values fall within the zero to 0.22 boundary 0, 0.22 [One sided] This means that we have a 20% of them being subsequently > 0.22 What about when the null hypothesis is not true?