Transcript Chapter 6

Psy B07
CATEGORICAL DATA & χ2
Chapter 6
Slide 1
Psy B07
A Quick Look Back
 Reminder about hypothesis testing:
1) Assume what you believe (H1) is wrong.
 Construct H0 and accept it as a default.
2) Show that some event is of sufficiently low
probability given H0***.
3) Reject H0.
*** In order to do this, we need to know the distribution
associated with H0, because we use that distribution as the
basis for our probability calculation.
Chapter 6
Slide 2
Psy B07
z-score
 Use when we have acquired some data set,
then want to ask questions concerning the
probability of certain specific data values (e.g.,
do certain values seem extreme?).
 In this case, the distribution associated with
H0 is described by X and S2 because the data
points reflect a continuous variable that is
normally distributed.
Chapter 6
Slide 3
Psy B07
Chi-square (χ2) test
 The Chi-square test is a general purpose
test for use with discrete variables.
 It has a number of uses, including the
detection of bizarre outcomes given
some a priori probability for binomial
situation, and for multinomial
situations.
Chapter 6
Slide 4
Psy B07
Chi-square (χ2) test
 In addition, it allows us to go beyond
questions of bizarreness, and move into the
question of whether pairs of variables are
related. For example:
Female
Male
Legalize
Do not Legalize
9
9
23
7
 It does so by mapping the discreet variables
unto a continuous distribution assuming H0,
the chi-square distribution.
Chapter 6
Slide 5
Psy B07
The chi-square distribution
 Let’s reconsider a simple binomial problem.
Say, we have a batter who hits .300 [i.e.,
P(Hit)=0.30], and we want to know whether it
is abnormal for him to go 6 for 10 (i.e., 6 hits
in 10 at bats).
 We could do this using the binomial stuff that I
did not cover in Chapter 5 (and for which you
are not responsible)
 But we can also do it with a chi-square test
Chapter 6
Slide 6
Psy B07
The way of the chi2
 We can put our values into a contingency table
as follows:
Observed
Expected?
Hits
Outs
6
4
 Then consider the distribution of the following
formula given H0:
(Observed  Expected) 2

Expected
Chapter 6
Slide 7
Psy B07
The way of the chi2
Attempt Expected (E)
1
2
3
4
5
6
7
8
9
10
Chapter 6
Observed (O)
3/7
3/7
3/7
3/7
3/7
3/7
3/7
3/7
3/7
3/7
Slide 8
Psy B07
The way of the chi2
Attempt Expected (E)
1
2
3
4
5
6
7
8
9
10
Chapter 6
3/7
3/7
3/7
3/7
3/7
3/7
3/7
3/7
3/7
3/7
Observed (O)
hit
out
out
hit
hit
hit
out
hit
out
hit
Slide 9
Psy B07
The way of the chi2
In-Class Example:
 Note that while the observed values are
discreet, the derived score is continuous.
 If we calculated enough of these derived
scores, we could plot a frequency distribution
which would be a chi-square distribution with
1 degree of freedom or 2(1).
 Given this distribution and appropriate tables,
we can then find the probability associated
with any particular 2 value.
Chapter 6
Slide 10
Psy B07
The way of the chi2
Continuing the Baseball Example:
Observed
Expected
2
(
O

E
)
2  
E
(6  3) 2 (4  7) 2


3
7
9 9
   4.29
3 7
Chapter 6
Hits
Outs
6
3
4
7
So if the probability of
obtaining a 2 of 4.29 or
greater is less than , then the
observed outcome can be
considered bizarre (i.e., the
result of something other than
a .300 hitter getting lucky).
Slide 11
Psy B07
The way of the chi2
 Just like the t-test, chi2 distribution is
based on degrees of freedom
 Thus, since our obtained 2 value of 4.29
is greater than 3.84, we can reject H0
and assume that hitting 6 of 10 reflects
more than just chance performance.
Chapter 6
Slide 12
Psy B07
The way of the chi2
Going a Step Further:
 Suppose we complicate the previous
example by taking walks and hit by
pitches into account. That is, suppose
the average batter gets a hit with a
probability of 0.28, gets walked with a
probability of .08, gets hit by a pitch
(HBP) with a probability of .02, and gets
out the rest of the time.
Chapter 6
Slide 13
Psy B07
The way of the chi2
 Now we ask, can you reject H0 (that this batter
is typical of the average batter) given the
following outcomes from 50 at bats?
Observed
Expected
Hit
12
Walk HBP Out
3
8
27
1) Calculate expected values (Np).
2) Calculate 2 obtained.
3) Figure out the appropriate df (C-1).
4) Find 2critical and compare 2 obtained to it.
Chapter 6
Slide 14
Psy B07
The way of the chi2
Observed
Expected
Hit
12
14
Walk HBP Out
3
8
27
4
1
31
2
(
O

E
)
2  
E
(12  14) 2 (3  4) 2 (8  1) 2 (27  31) 2




14
4
1
31
 50.51
Chapter 6
Slide 15
Psy B07
Two types of chi2 tests
 So far, all the tests have been to assess
whether some observation or set of
observations seems out-of-line with some
expected distribution. This is also known as
the goodness-of-fit chi-square test
 However, the logic of the chi-square test can
be extended to examine the issue of whether
two variables are independent (i.e., not
systematically related) or dependent (i.e.,
systematically related).
Chapter 6
Slide 16
Psy B07
χ2 test for independence
 Consider the following data set again:
Female
Male
Legalize
Do not Legalize
9
9
23
7
 Are the variables of gender and opinion
concerning the legalization of marijuana
independent?
Chapter 6
Slide 17
Psy B07
χ2 test for independence
Legalize
Do Not Legalize Total
Female
9
23
32
Male
9
7
16
Total
18
30
48
From the marginal totals we can calculate:
P(Female)
= 32/48 = 0.667
P(Male)
= 16/48 = 0.333
P(Legalize)
= 18/48 = 0.375
P(Do Not Legalize) = 30/48 = 0.625
Chapter 6
Slide 18
Psy B07
χ2 test for independence
 If these two variables are independent, then
by the multiplicative law, we expect that:
P(Female,Legalize) = P(Female) x P(Legalize)
= .667 x .375
= .250125
EV(Female, Legalize) = Np
= 48 x .250125 = 12
Chapter 6
Slide 19
Psy B07
χ2 test for independence
 If we do this for all four cells, we get:
Legalize
Female
Male
Total
Chapter 6
Do Not Legalize Total
9
23
Expect: 12
Expect: 20
9
7
Expect: 6
Expect: 10
18
30
32
16
48
Slide 20
Psy B07
χ2 test for independence
 Are the observed values different enough from
the expected values to reject the notion that
the differences are due to chance variation?
(O  E )
 
E
2
2
2
2
(9  12) (23  20) (9  6) (7  10)




12
20
6
10
 3.6
2
2
Chapter 6
Slide 21
Psy B07
χ2 test for independence
 The df associated with 2 variable contingency
tables can be calculated using the formula:
df = (C-1)(R-1)
 where C is the number of columns and R is the
number of rows.
Chapter 6
Slide 22
Psy B07
χ2 test for independence
 Thus, to finish our previous example,
the 2 critical with alpha equal .05 and 1 df
equals 3.84. Since our 2 is not bigger
than that (i.e., 3.6) we cannot reject H0.
Chapter 6
Slide 23
Psy B07
Assumptions of χ2
Independence of observations:
 Chi-square analyses are only valid when
the actual observations within the cells
are independent.
 This independence of observations is
different from the issue of whether the
variables are independent, that is what
the chi-square is testing.
Chapter 6
Slide 24
Psy B07
Assumptions of χ2
Independence of observations:
 You know your observations are not
independent when the grand total is larger
than the number of subjects.
 Example: The activity level of 5 rats was
tested over 4 days, producing these values:
Chapter 6
Low
Activity
Medium
High
3
7
10
Slide 25
Psy B07
Assumptions of χ2
Normality:
 Use of the chi-square distribution for finding
critical values assumes that the expected
values (i.e., Np) are normally distributed.
 This assumption breaks down when the
expected values are small (specifically, the
distribution of Np becomes more and more
positively skewed as Np gets small).
Chapter 6
Slide 26
Psy B07
Assumptions of χ2
Normality:
 Thus, one should be cautious using the
chi-square test when the expected
values are small.
 How small? This is debatable but if
expected values are as low as 5, you
should be worried.
Chapter 6
Slide 27
Psy B07
Assumptions of χ2
Inclusion of Non-Occurrences:
 The chi-square test assumes that all
outcomes (occurrences and nonoccurrences) are considered in the
contingency table.
 As an example of a failure to include a
non-occurrence, see page 160 of the text.
Chapter 6
Slide 28
Psy B07
A tale of tails
 We only reject H0 when values of 2 are
larger than 2 obtained.
 This suggests that the 2 test is always
one-tailed and, in terms of the rejection
region, it is.
 In a different sense, however, the test
is actually multiple tailed.
Chapter 6
Slide 29
Psy B07
A tale of tails
 Reconsider the following “marking scheme” example:
Option 1
Option 2
Option 3
38
57
5
 If we do not specify how we expect the results to fall
out then any outcome with a high enough 2 obtained can
be used to reject H0.
 However, if we specify our outcome, we are allowed to
increase our alpha - in the example we can increase
alpha to 0.30 if we specified the exact ordering (in
advance) that was observed.
Chapter 6
Slide 30
Psy B07
Measures of Association
 The chi-square test only tells us whether two
variables are independent, it does not say
anything about the magnitude of the
dependency if one is found to exist.
 Stealing from the book, consider the following
two cases, both of which produce a significant
2 obtained, but which imply different strengths
of relation:
Chapter 6
Slide 31
Psy B07
Measures of Association
Smoking Behaviour
Male
Female
Nonsmoker
400
350
Smoker
100
150
  13.333
2
Primary Food Shopper
Male
Female
Chapter 6
Yes
400
100
No
100
400
 2  12.737
Slide 32
Psy B07
Measures of Association
 There are a number of ways to quantify
the strength of a relation (see sections
in the text on the contingency
coefficient, Phi, & Odds Ratios), but the
two most relevant to psychologists are
Cramer’s Phi and Cohen’s Kappa.
Chapter 6
Slide 33
Psy B07
Measures of Association
 Cramer’s Phi (φc) can be used with any
contingency table and is calculated as:
2
c 
N(k  1)
 Values of range from 0 to 1. The values the
tables on the previous page are 0.12 and 0.60
respectively, indicating a much stronger
relation in the second example.
Chapter 6
Slide 34
Psy B07
Measures of Association
 Often, in psychology, we will ask some “judge”
to categorize things into specific categories.
 For example, imagine a beer brewing
competition where we asked a judge to
categorize beers as Yucky, OK, or Yummy.
 Obviously, we are eventually interested in
knowing something about the beers after they
are categorized.
Chapter 6
Slide 35
Psy B07
Measures of Association
 However, one issue that arises is the
judges abilities to tell the difference
between the beers.
 One way around this is to get two
judges and show that a given beer is
reliably rated across the judges (i.e.,
that both judges tend to categorize
things in a similar way).
Chapter 6
Slide 36
Psy B07
Measures of Association
 Such a finding would suggest that the judges
are sensitive to some underlying quality of the
beers as opposed to just guessing.
Judge 2
Yuck!
OK
Yummy
Chapter 6
Yuck!
3
1
0
Judge 1
OK
2
15
1
Yummy
3
2
3
Slide 37
Psy B07
Measures of Association
 Note that if you just looked at the
proportion of decisions that me and
Judge 2 agreed on, it looks like we are
doing OK:
P(Agree)=21/30 = 0.70 or 70%
Chapter 6
Slide 38
Psy B07
Measures of Association
 There is a problem here, however, because
both judges are biased to judge a beer as OK
such that even if they were guessing, the
agreement would seem high because both
would guess OK on a lot of trials and would
therefore agree a lot.
Solution:
 fO –  fE
k=
=?
N –  fE
Chapter 6
Slide 39
Psy B07
Measures of Association
 Such a finding would suggest that the judges
are sensitive to some underlying quality of the
beers as opposed to just guessing.
Judge 2
Yuck!
OK
Yummy
Chapter 6
Yuck!
3 (1.07)
1
0
Judge 1
OK
2
15(10.8)
1
Yummy
3
2
3(1.07)
Slide 40
Psy B07
O  E

N  E
21  12.94

30  12.94
 .47
Chapter 6
Slide 41