Transcript Calculate Sn, Sp, LRs

PTP 560
• Research Methods
Week 9
Thomas Ruediger, PT
Calculate Sn, Sp, LRs
Confidence Intervals
with small samples
 What is small sample size?
 Less than 30 (one of those special numbers in stats)
 Sampling distribution tends to spread out
 Standard normal curve not adequate
 Use the t-distributions





Theoretical sampling distributions
Flatter peak, wider at the tails
Approaches normal curve as sample size increases
Use values of t instead of z
Described by degrees of freedom (n-1 for confidence
intervals)
Hypothesis testing
• Are the differences
– Representative of “real” effects?
– Just by chance?
• Null hypothesis ( HO)
– Means are not different
– Stated in terms of population parameter
– μA=μB
• Alternative hypothesis (H1)
– Difference too large to be by chance
– μA≠μB
– May be directional or non-directional
Truth
Decision
Ho is True Ho is False
Reject Ho
Do Not Reject Ho
Type I error
α
Correct
Correct
Type II error
β
Type I Error
• Significance Level (Alpha level , α level)
• Your choice of how much risk you are willing to
take of saying there is a difference when there
really is no difference
• Set this before the study
– Conventionally is 0.05
• This is arbitrary, but almost always what we choose
– Choose the level based on the Type I error concern
Type I Error
• Probability Values (Evaluated after the study)
– Probability of finding this big a difference by chance
• p = .07 of this big a difference by chance
– You are not stating the probability of the inverse
• p = .93 that it is real difference is not appropriate
– Compare p-value to alpha level if greater
– Compare your p-value (calculated after the study) with
your α level (set before the study)
– If the p-value is less than α, reject the null
– If the p-value is greater than α, fail to reject the null
Type II Error
Statistical Power
• Beta (β)
– Probability of failing to reject a false HO(null hypotheses)
– Β of 0.20 is 20% chance we will make a Type II error
• Statistical Power
– Complement of β (not compliment)
– In this example 0.80 (1.00 – 0.20 = 0.80)
– 80% probability of correctly rejecting the null
• Before: a priori - power used to determine sample
size
• After: post hoc – power reported if HO not rejected
“If there was a difference, could we have found it?”
Determinants of
Statistical Power
• Significance Criterion
– As α increases, power increases
(As α increases from 0.05 to 0.10, power increases)
• Variance
– As variance decreases, power increases
• Sample size
– As sample size increases, power increases
• Effect size (difference b/w the group means)
– As effect size increases, power increases
z• z - score represents the distance between:
– A sample score and
– Sample mean
– Divided by the standard deviation
You will see this in osteoporosis scores
(+2 for z- score is 2 SD away from a healthily woman mean)
• z - ratio represents the distance between:
– A sample mean and
– Population mean
– Divided by the standard error of the mean
Critical Region
 That portion of the curve above and below z
 If calculated z > critical z, reject HO
 One or two tailed test?
 Non directional hypothesis– two tailed
 z of 2.00 encompasses 95.44 % (non-critical)
 4.66% is the critical area
 z of 1.96 encompass 95%,
 Critical region is 5%, 2.5% in each tail of a non-directional test
 Directional hypothesis– one tailed
 z of 1.645 encompasses 95%
 Critical region is 5%, all in one tail of a directional test, while NON-direction will be 2.5%
 Practically, you are disregarding everything in the other tail
 Table A.1 back of P & W
Figure A: Intervention 1 is different than Intervention 2
Figure B: Intervention 1 is less effective than Intervention 2
Parametric Statistics
• Used to estimate population parameters
• Based on assumptions
– Randomly drawn from a normally distributed population
– Variances in the samples equal (at least roughly)
– Interval or ratio scale
• Classically, if assumptions violated, use nonparametric tests
• Many view parametric stats as Robust enough to
withstand even major violation
t-test
 Examines two means
 Two groups
 Two conditions/two performances
 Statistical significance based on
 Difference in the means
 Between the groups
 The effect size
 Variance
 Within the groups
 How variable are the scores
Fig 19.1
t-test
 Based on a ratio
 Difference between group means/Variability within groups
 Difference between means
 Treatment effect and error variance
 One mean- second mean and variability between the groups. In both the numerator and
denominator of t-test ratio, so holds it to zero.
 Variability within groups
 Error variance alone
 Equal and unequal variances affect t-ratio
 SPSS and most other packages automatically test for this. Where?

Ratio can be written: Treatment effect and error variance/Error variance
NOTE:
Error variance
 Not mistakes
 Is anything that is not due to the independent variable
t-test
 If the null is true
 Ratio reduces to: Error/Error
 The bigger the difference - The bigger the ratio
How does the ratio get bigger?
 This ratio is compared to the critical
 Determines significance
 Is the ratio significant?
 Based on critical value (but now t instead of z)
 Entering arguments (Table A.2)
 Alpha level (almost always 0.05
 Degrees of freedom ( one or a few less than n )
 CI can be constructed for where the true mean difference lies
t-test
 Independent t-test




Usually random assignment
Can be convenience assignment
No inherent relationship between the groups
Degrees of freedom = total sample size – 2
 Paired t-test
 An inherent relationship between the groups
 Self (repeated measure test)
 Twins
 Difference scores for each pair compared
 Degrees of freedom = number of paired scores – 1
 Find this in P & W or in an SPSS output table – don’t calculate for this class!
ANOVA
 Examines three (or more) means
 Three (or more) groups
 Three (or more) conditions/ three (or more) performances
 Statistical significance based on
 Difference in the means
 Between the groups
 The effect size
 Variance
 Within the groups
 How variable are the scores
 This should sound familiar
ANOVA
 Based on ratio
Treatment effect and error variance/Error variance

 For ANOVA it is the F-ratio
 Derived from the Sum of Squares (SS)
 Larger the SS the larger the ______________?
 Calculate SS
(each score minus sample mean, square each result, sum them)
 Then determine the Mean Square (MS)
 MSb = SSb/dfb (dfb = one less than the number of groups)
 MSe = SSe/df e (df e = total N – number of groups)
 F statistic is the ratio = MSb/Mse
 Ratio of the between groups variance to error variance

Find this in P & W or in an SPSS output table – don’t calculate for this class!