Transcript Slide 1
Prediction statistics • Prediction generally • True and false, positives and negatives • Quality of a prediction • Usefulness of a prediction • Prediction goes Bayesian • Use of prediction statistics Prediction • Three major reasons for doing research – To understand the processes of nature – To be able to predict future event – To be able to control future events • Three major activities in medical care – To make a diagnosis – To define the prognosis – To intervene and alter the progress of events • Clinical prediction – Signs, symptoms, tests results – Diagnosis, prognosis Types of prediction • Using one observation (test) to predict another observation (outcome) – Both test and outcome are continuous measurements • e.g. Using age to predict height • Regression analysis, curve fitting – Test is a criteria and outcome a measurement • e.g. using sex to predict weight • Confidence intervals between two means – Both test and outcome are criteria • e.g. using the presence of calcification in a mammogram to predict the presence of a carcinoma • Prediction statistics – Test is a continuous measurement and outcome a criteria • E.g. using body temperature to predict the presence of an infection • Receiver Operator Characteristics Prediction statistics • Prediction generally • True and false, positives and negatives • Quality of a prediction • Usefulness of a prediction • Prediction goes Bayesian • Use of prediction statistics Prediction statistics • To evaluate how one criteria can predict another from data collected • The number of cases with different combinations of test and outcome results collated Test + Test Total Outcome + True + (TP) False + (FN) TP + FN Outcome False + (FP) True - (TN) FP + TN Total TP + FP FN + TN Prediction statistics • Prediction generally • True and false, positives and negatives • Quality of a prediction • Usefulness of a prediction • Prediction goes Bayesian • Use of prediction statistics Quality of test Sensitivity : the proportion of test positives amongst those outcome positive Specificity : the proportion of test negatives amongst those outcome negative Outcome + Outcome Total Test + True + (TP) False + (FP) TP + FP Test - False + (FN) True - (TN) FN + TN Total TP + FN FP + TN Prediction statistics • Prediction generally • True and false, positives and negatives • Quality of a prediction • Usefulness of a prediction • Prediction goes Bayesian • Use of prediction statistics Usefulness of a test Positive Predictive Value : the proportion of outcome positives amongst those test positive Negative Predictive Value : the proportion of outcome negatives amongst those test negative Outcome + Outcome Total Test + True + (TP) False + (FP) TP + FP Test - False - (FN) True - (TN) FN + TN Total TP + FN FP + TN Usefulness of a test • What the user want to know – How likely will outcome be + if test is + – How likely will outcome be - if test is – • Predictive values are prevalence dependent – Positive predictive value is 0 (100%) if prevalence is 1 (100%) and 0 (0%) if prevalence is 0 (0%) – Negative predictive value is 0 (0%) if prevalence is 1 100% and 1 (100%) if prevalence is 0 (0%) – The shape of these curves depend on a complex relationship between Sensitivity, Specificity, and prevalence ( Sen)(Prev) PPV ( Sen)(Prev) (1 Pr ev)(1 Spec) Spec(1 Pr ev) NPV Spec(1 Pr ev) Pr ev(1 Sen) Sensitivity = 0.8 Specificity = 0.8 Positive Predictive Value 1 0.8 0.6 0.4 Negative Predictive Value 0.2 0 0 0.2 0.4 0.6 Prevalence 0.8 1 Prediction statistics • Prediction generally • True and false, positives and negatives • Quality of a prediction • Usefulness of a prediction • Prediction goes Bayesian • Use of prediction statistics Prediction as probabilities • Pre-test probability = Prevalence • Post-test probability (test +) = Positive Predictive Value • Post-test probability (test -) = 1 - Negative Predictive Value Bayes Theorem • Perceptions of probability is altered by experience or observations – Final probability = Preconceived probability x Bayes factor • Prediction can be presented as an example of Bayes model – Post-test probability = function (pre-test proabability, Bayes factor) Quality of test as Bayes factor Likelihood Ratio • Likelihood Ratio – The ratio of probabilities of getting a positive test, between outcome positives and outcome negatives – The more LR>1, the more likely outcome + – The more LR<1, the less likely outcome - • Likelihood Ratio + Test – Likelihood Ratio for those test + – LR+ = Sensitivity / (1 – Specificity) • Likelihood Ratio – Test – Likelihood Ratio for those test – – LR - = (1-Sensitivity) / Specificity Using Likelihood Ratio Converting pre-test probability to Post-test probability Oddspre test P r obabilitypre test /(1 P r obabilitypre test ) Oddspost test Oddspre test Likelihood_ Ratio P r obabilitypost test Oddspost test /(1 Oddspost test ) Prediction statistics • Prediction generally • True and false, positives and negatives • Quality of a prediction • Usefulness of a prediction • Prediction goes Bayesian • Use of prediction statistics The use of prediction statistics • Development of diagnostic tools and tests – Tested clinically to evaluate quality • This is usually under controlled conditions • Often in a high prevalence population to get sufficient sample size – Quality of the test published as Sensitivity and Specificity, or increasingly commonly as Likelihood Ratios • Interpretation of statistics – Using local prevalence (pre-test probability) to convert quality measurements into post-test probabilities