Transcript Chapter 8

Psy B07
POWER
Chapter 8
Slide 1
Psy B07
Chapter 4 flashback
H0 true
H0 false
Reject H0
Type I error
Correct
Fail to
reject H0
Correct
Type II error
 Type I error is the probability of rejecting the
null hypothesis when it is really true.
 The probability of making a type I error is
denoted as .
Chapter 8
Slide 2
Psy B07
Chapter 4 flashback
 Type II error is the probability of failing
to reject a null hypothesis that is really
false
 The probability of making a type II
error is denoted as .
 In this chapter, you’ll often see these
outcomes represented with
distributions
Chapter 8
Slide 3
Psy B07
Distributions
 To make these representations clear, let’s first
consider the situation where H0 is, in fact,
true:
correct
failure to
reject
Alpha
Type I
Error
 Now assume that H0 is false (i.e., that some
“treatment” has an effect on our dependent variable,
shifting the mean to the right).
Chapter 8
Slide 4
Psy B07
Distributions
Distribution
Under H0
Correct
Rejection
Distribution
Under H1
Type II
error
Chapter 8
Power
Alpha
Slide 5
Psy B07
Definition of Power
 Thus, power can be defined as follows:
 Assuming some manipulation effects the
dependent variable, power is the probability
that the sample mean will be sufficiently
different from the mean under H0 to allow
us to reject H0.
 As such, the power of an experiment depends
on three (or four) factors:
Chapter 8
Slide 6
Psy B07
Factors affecting power
Alpha
 As alpha is moved to the left (for example, if
one used an alpha of 0.10 instead of 0.05),
beta would decrease, power would increase ...
but, the probability of making a type I error
would increase.
 1 - 2 :
 The further that H1 is shifted away from H0,
the more power (and lower beta) an
experiment will have.
Chapter 8
Slide 7
Psy B07
Factors affecting power
Standard error of the mean
 The smaller the standard error of the
mean (i.e., the less the two distributions
overlap), the greater the power. As
suggested by the CLT, the standard
error of the mean is a function of the
population variance and N. Thus, of all the
factors mentioned, the only one we can
really control is N.
Chapter 8
Slide 8
Psy B07
Effect size
 Most power calculations use a term called effect size
which is actually a measure of the degree to which the
H0 and H1 distributions overlap.
 As such, effect size is sensitive to both the difference
between the means under H0 and H1, and the standard
deviation of the parent populations.
Specifically:
1   2
d

Chapter 8
Slide 9
Psy B07
Effect size
 In English then, d is the number of
standard deviations separating the
mean of H0 and the mean of H1.
 Note: N has not been incorporated in
the above formula. You’ll see why
shortly
Chapter 8
Slide 10
Psy B07
Estimating effect size
 As d forms the basis of all calculations of power, the first
step in these calculations is to estimate d.
 Since we do not typically know how big the effect will
be a priori, we must make an educated guess on the
basis of:
1) Prior research.
2) An assessment of the size of the effect that would
be important.
3) General Rule (small effect d=0.2, medium
effect d=0.5, large effect d = 0.8)
Chapter 8
Slide 11
Psy B07
Estimating effect size
 The calculation of d took into account 1)
the difference between the means of H0
and H1 and 2) the standard deviation of
the population.
 However, it did not take into account
the third variable the effects the overlap
of the two distributions; N.
Chapter 8
Slide 12
Psy B07
Estimating effect size
 This was done purposefully so that we have
one term that represents the relevant
variables we, as experimenters, can do nothing
about (d) and another representing the
variable we can do something about; N.
 The statistic we use to recombine these factors
is called delta and is computed as follows:
  d[ ƒ( N)]
 where the specific ƒ(N) differs depending on the
type of t-test you are computing the power for.
Chapter 8
Slide 13
Psy B07
Power calcs for one-sample t
 In the context of a one sample t-test, the ƒ(N)
alluded to above is simply: N
 Thus, when calculating the power associated
with a one sample t, you must go through the
following steps:
1) Estimate d, or calculate it using:
1   2
d

Chapter 8
Slide 14
Psy B07
Power calcs for one-sample t
 Calculate δ using:
d N
 3) Go to the power table, and find the power
associated with the calculated δ given the level
of α you plan to use (or used) for the t-test
Chapter 8
Slide 15
Psy B07
Power calcs for one-sample t
Example:
Say I find a new stats textbook and after looking at it, I
think it will raise the average mark of the class by about
8 points. From previous classes, I am able to estimate
the population standard deviation as 15. If I now test
out the new text by using it with 20 new students, what
is my power to reject the null hypothesis (that the new
students marks are the same as the old students
marks).
How many new students would I have to test to bring
my power up to .90?
Note: Don’t worry about the bit on “noncentrality
parameters” in the book.
Chapter 8
Slide 16