Transcript Education 793 Class Notes

Education 793 Class Notes
Decisions, Error and Power
22 October 2003
Today’s agenda
• Class and lab announcements
• Questions?
• Decisions, Error, Power
2
Review: Chain of reasoning
Population with
parameters
Random
selection
Inference
Probability
Sample with
statistics
Underlying
distributions
of the statistic
Random selection can take several forms (such as simple,
systematic, cluster, or stratified random sampling), and is
intended to generate a sample that represents the
population.
3
Four Steps
1. State the hypothesis
H0 vs. HAlternate
2. Identify your criterion for rejecting H0
Directional
One-tailed
or
or
non-directional test
two-tailed
Set alpha level (Prob. incorrectly rejecting H0)
3. Compute test statistic
General form: Test = statistic – expected parameter
standard error of statistic
4. Decide about H0
4
Million Dollar Question
• To Reject or Not To Reject?
5
Review: Inferential error
• Type I: Alpha
– Rejecting the null hypothesis with the null
hypothesis is really true
• Type II: Beta
– Failing to reject the null hypothesis when the null
hypothesis is in fact false
6
Possible outcomes
Decision
Reject H0
(H0 false)
Fail to
reject H0
(H0 true)
In the population, In the population,
H0 is true
H0 is false
Type I error
a
Correct decision
Correct decision
Type II error
b
7
Considerations
• Basic considerations
– What is the statistic of interest?
– Is a one-tailed versus two-tailed test appropriate?
– What is the nature of the sample?
• Dependent versus independent
• How much power is available?
– Power is 1 – b
– Factors influencing power:
• Increasing the number of observations
• Reducing the error in the data
8
Statistical Power
• Power is the probability of correctly rejecting
a false null hypothesis. As such, power is
defined as:
1 - ß where ß is the Type II error probability
• Two kinds of power analyses:
– A priori: Used to identify what the sample size
needs to be to identify a specified effect
– Post hoc: Used when the null hypothesis has not
been rejected, and when you want to know the
probability that you have committed a Type II error
9
What Power is all about
• Power analysis is about Type II errors, “missed
effects” – failing to reject H0: when there really is
an effect in the population
• “Power” is the antithesis of “risk of Type II error”
– Risk of Type II error = 1 - power
– Power = 1 - Risk of Type II error
10
Trade off Between a and Power
• Conundrum: As we decrease a (Type I
Error), b (Type II Error) increases. The
two errors have an inverse relationship.
In order to deal with the problem,
researchers follow an established set of
guidelines. a=.05 or .01
• Which error is the most dangerous?
11
Power of a Test
• Example: Ho: m=72, H1: m<72 a=.05, n=36
Review: Given the sample data, a decision is
made to reject or not reject the null
hypothesis.
zcrit= -1.65
This distribution is
assumed to be true
under the Null
a=Type I Error
m=72
12
Power of a Test
zcrit= -1.65
a=Type I Error
m=72
• If the null hypothesis is false, then the above
distribution is NOT the one we sampled from. We
must have sampled from one of the possible
alternative distributions, of which there are an infinite
number. Recall H1: m<72
13
Power of a Test
• The steps in computing power require
the researcher to set a predetermined
difference from the two means (D1) that
is meaningful (Effect Size).
• With three pieces of information, a, N,
D, we can estimate the power of a test.
• We will let computer programs do this
for us.
14
Factors Affecting Power
• Size of the difference between population
means, what the book refers to as D1.
– The greater the effect size (D1), the greater the
power will be. Common sense, as the difference
gets larger, it will be easier to detect.
• Significance level
– As the power of a statistical test increases so does
a. This is the trade off between Type I and Type II
Errors.
15
Factors Affecting Power
• Variance
– All other factors being equal (a, N, D), the
smaller the standard deviation in the
population, the greater the power.
• Sample size
– By increasing N, we decrease the standard
deviation in the sampling distribution.
Hence the power is increased.
16
Power as a function of…
the true
difference in m
significance
level
sample
size
17
Next Week
• Chapter 12 p. 333-367 and
Available through JSTOR
18