Transcript Null Hypothesis Testing

Psych 5500/6500
Null Hypothesis Testing: General Concepts
and Introduction to
The t Test for a Single Group Mean
Fall, 2008
1
Introduction
Time to move on to the heart of experimental
science, formulating hypotheses to test
theories, gathering data, and then deciding
which hypothesis is supported by the
data.The general approach is called 'null
hypothesis testing'.
2
t Test for a Single Group Mean
We will introduce null hypothesis testing by seeing
how it is carried out in what is called the t test for
a single group mean.
This test is rarely used in statistics, but it makes a
very nice lead into null hypothesis testing from our
earlier work on the sampling distribution of the
mean.
As the semester progresses we will return to null
hypothesis testing again and again, each time will
simply be a variation on the same theme.
3
t Test for a Single Group Mean
The t test for a single group mean involves
making hypotheses about the true value of
the mean of the population from which we
are sampling. Its limitation is that you need
to have some theoretical reason for
hypothesizing specific values about the
population mean...
4
Example
We have an IQ test that has a μ=100 for the
general population of the United States. We
want to test a theory which predicts that the
population of people in the country of Elbonia
should have a different μ than that of the U.S.
The theory simply predicts that the mean IQ of
Elbonians is different, it doesn’t predict
specifically whether it should be greater or less
than that of the U.S. (we will cover that type of
hypothesis testing later).
5
General Approach
We will draw a sample of Elbonians and find
the mean IQ of that sample. From that we
will decide whether or not the mean of the
population of Elbonians is different than the
mean of the population of the U.S. (i.e.
whether μElbonians100)
6
The Challenge
The challenge we face is this, let’s say that the
mean of the sample of Elbonians is 106, does
that prove the Elbonians have higher IQ’s (i.e.
that μElbonians  μAmericans )? Or, might it simply be
that Elbonians are the same as people in the
USA (i.e. μElbonians = μAmericans = 100) but that we
just happened to get a high sample mean due to
chance (i.e. your sample has random bias). This
is exactly the sort of thing that null hypothesis
testing was set up to handle.
7
Two Hypotheses
The first step in null hypothesis testing is to
divide reality into two mutually exclusive and
exhaustive hypotheses (i.e. one, and only one,
of these will be true):
1. null hypothesis (H0): the hypothesis that you
set up in hopes of rejecting. Often, but not
always, it is the hypothesis of 'no difference'.
2. alternative hypothesis (HA): the hypothesis
you hope to prove. Often, but not always, it is
the hypothesis of 'difference'.
8
The exact form of the null and alternative
hypotheses depend upon the statistical
procedure you are using. In our example
(t test for a single group mean):
H0: μElbonia = 100
HA: μElbonia  100
Note:
1) Hypotheses are always about populations,
2) Together H0 and HA cover all possibilities.
9
Two Possible Results
The results are always stated in terms of the null
hypothesis (this is ‘null hypothesis testing’ after
all). There are two and only two possible results
of an experiment (in this class always give one of
the following two as the ‘result of the
experiment’):
1. ‘Do not reject H0’
2. ‘Reject H0’
Later we will take a look at why it is more accurate to say ‘do
not reject H0’ than ‘accept H0’’.
10
Implications Regarding HA
Although our decision is stated in terms of H0,
once we decide about H0 that implies
something about HA (as the two hypotheses
are mutually exclusive and exhaustive)
‘reject H0’ implies ‘accept HA’.
‘do not reject H0’ implies ‘do not accept HA’.
11
Four Possibilities
Reality
1) Reject H0
1) Ho true
2) Ho false
Type 1 error
Correct
decision
p = ‘alpha’
p = ‘power’
Decision
2) Do not
reject H0
Correct
decision
Type 2 error
p = ‘beta’
12
Type 1 Error and alpha
Type 1 error: rejecting H0 when in reality H0 was
true.
alpha (α): the probability that you will make a type
1 error. This is a conditional probability:
Correct interpretation of alpha:
α=p(you will decide to reject H0 | H0 is actually true).
Incorrect interpretations of alpha:
1) p(H0 is true | you decide to reject H0)
2) p(making a type one error | you decide to reject H0)
3) p(making a type one error | you are doing null hypothesis testing)
13
Type 2 Error and beta
Type 2 error: not rejecting H0 when in reality
H0 was false.
beta (β): the probability you will make a type
2 error. This is also a conditional
probability:
β = p(deciding to not reject H0 | H0 is actually false)
14
power
power = p(deciding to reject H0 | H0 is actually false)
Power involves making a correct decision, to reject
H0 when in reality H0 was false.
We rarely know the actual values of power and beta,
but we do know the following:
power + beta = 1.00 (so we know that when power
goes up beta goes down and vice versa). We will
devote a whole lecture just on power later in the
semester.
15
Basic Approach in Null
Hypothesis Testing
1. Determine the probability of getting the data you
did if H0 were true. In this case, what is the
probability that we would have gotten a sample
mean (106) so far away from what H0 predicted
(100) if H0 were actually true? i.e. p(sample
mean of 6 or more away from 100 | H0 is
actually true)
2. If that probability is very small, then reject H0.
16
Coin Example
H0: the coin is fair, i.e. p(Heads)=0.5
HA: the coins is biased, i.e. p(Heads)0.5
1st coin flip = Heads, p(one head|H0)=0.5
2nd coin flip = Heads, p(two heads|H0)=.25
3rd coin flip = Heads, p(three heads|H0)=.125
4th coin flip = Heads, p(four heads|H0)=.0625
5th coin flip = Heads, p(five heads|H0)=.03125
6th coin flip = Heads, p(six heads|H0)=.01563
7th coin flip = Heads, p(seven heads|H0)=.0078
17
Significance Level
At what point do we say that the data are so
unlikely given H0 that we want to reject H0 in favor
of HA?
Where we draw the line is called our ‘significance
level’, it is how improbable the results have to be
to reject H0. In psychology we usually use p=.05,
or if we wish to be conservative and not reject H0
without more proof, we use p=.01
Note: H0 could still be true, we can’t rule that out,
we can only say the results are improbable if H0
were true, so we will go with HA instead.
18
Significance Level and alpha
Once you set your significance level, you have also
set the value of alpha (‘α’).
alpha = your significance level.
Thus the value of alpha--the probability of making a
type one error given H0 is true--is under your
direct control based upon the significance level
you choose (an a priori decision).
19
In Our Example
1) Our hypotheses were:
H0: μElbonia = 100
HA: μElbonia  100
2) We set our significance level at .05.
3) The mean of our sample of Elbonians was 106.
4) If the probability of getting a sample mean at least six away
from 100 if H0 were true is, say, p=.04, then we would
decide to ‘reject H0’, as the probability of getting such a
sample mean if H0 were true is less than .05 (our
significance level). Our sample mean was improbable if H0
were true.
5) If, however, the probability of getting a sample mean at
least six away from 100 if H0 were true is, say, p=.17, then
we would decide to ‘not reject H0’, as the sample mean is
not all that unlikely if H0 were true (i.e. p > significance
level of .05).
20
beta and power
Usually you can not directly set the value of
beta (‘β’) or power (but at least you do
know that beta+power=1.00)
21