Transcript APPENDIX B. SOME BASIC TESTS IN STATISTICS

Slides for Introduction to Stochastic Search
and Optimization (ISSO) by J. C. Spall
APPENDIX B
SOME BASIC TESTS IN STATISTICS
•Organization of appendix in ISSO
–Standard one-sample test
•P-values
•Confidence intervals
–Basic two-sample tests
•Matched pairs t-test
•Unmatched pairs t-test with identical variances
•Unmatched pairs t-test with nonidentical variances
–Other approaches to testing
•One- and two-sample tests important in stochastic
search, optimization, and Monte Carlo simulation
The Standard One-Sample Test
• One set of data {Xi } for testing on   E(Xi)
• Famous test statistics
z  X μ or t  X μ
σ n
s n
• z and t have a N(0, 1) and t-distribution, respectively
• t-statistic useful in small samples; both z and t often
used with non-normal samples
• Large values of |z| or |t | indicate rejection of null
hypothesis that  is some chosen value (commonly
 = 0)
B-2
P-Values
• P-value: Probability that future experiment
would have value of test statistic at least as
extreme as that observed in the current
experiment
• Provides info. beyond binary accept/reject null
hypothesis
– Useful as indicator of strength of rejection
• Example: If z = 2.15, P-value is 0.016 based on
null hypothesis that   0
– Fairly strong evidence that  > 0
B-3
Two-Sample Tests
• Two sets of data {Xi } and {Yi } for testing X = Y
– E.g., Xi and Yi represent simulation outputs under two
scenarios
• Generic test statistic form
t  X Y
()
where () denotes appropriate variance estimate
• Three basic categories of tests affecting ()
– matched pairs
2)
– unmatched pairs; identical variances ( 2X  Y
2)
– unmatched pairs; non-identical variances ( 2X  Y
• Large values of |t | indicate rejection of null
hypothesis that X = Y
B-4