Transcript Unit 2 Slides: Basic Statistics

Unit 1B: Everything you
wanted to know
about basic statistics …
PSYC 4310
COGS 6310
MGMT 6969
Department of
Cognitive Science
© 2015, Michael Kalsher
Michael J. Kalsher
Learning Outcomes
After completing this section, students will be able to:
• Demonstrate knowledge concerning what a statistical
model is and why we use them (e.g., the mean)
• Demonstrate knowledge of what the ‘fit’ of a model is
and why it is important (e.g., the standard deviation)
• Distinguish models for samples and populations.
• Describe and discuss problems with NHST and
modern approaches (e.g., reporting confidence intervals
and effect sizes).
Slide 2
The Research Process
Slide 3
which$was$unnecessary$because$the$real$bridge$was$perfectly$safe—the$model$was$a$bad$
representation$ of$ reality).$ We$ can$ have$ some$ confidence,$ but$ not$ complete$ confidence,$ in$
predictions$from$this$model.$The$final$model$is$completely$different$to$the$realJworld$situation;$it$
bears$no$structural$similarities$to$the$real$bridge$and$is$a$poor$fit.$As$such,$any$predictions$based$on$
this$model$are$likely$to$be$completely$inaccurate.$Extending$this$analogy$to$science,$it$is$important$
when$we$fit$a$statistical$model$to$a$set$of$data$that$it$fits$the$data$well.$If$our$model$is$a$poor$fit$of$
the$observed$data$then$the$predictions$we$make$from$it$will$be$equally$poor.$
Building Statistical Models
The Real World
Good Fit
Moderate Fit
Poor Fit
$
$
Figure'2.2:'Fitting'models'to'real5world'data'(see'text'for'details)'
Jane'Superbrain'Box'2.1'Types'of'statistical'models'(1)'
Slide 4
Populations and Samples
• Population
– The collection of units (be they people,
plankton, plants, cities, suicidal authors,
etc.) to which we want to generalize a set
of findings or a statistical model.
• Sample
– A smaller (but hopefully representative)
collection of units from a population used
to determine truths about that population
Slide 5
The Only Equation You Will Ever
Need … only partly kidding!
Slide 6
A Simple Statistical Model
• In statistics we fit models to our data (i.e. we
use a statistical model to represent what is
happening in the real world).
• The mean is a hypothetical value (i.e. it doesn’t
have to be a value that actually exists in the data
set).
• As such, the mean is simple statistical model.
Slide 7
The Mean
The mean (or average) is the value from which
the (squared) scores deviate least (it has the
least error).
n
Mean ( X ) 
 xi
i 1
n
Slide 8
The Mean as a Model
Slide 9
Measuring the ‘Fit’ of the Model
• The mean is a model of what happens
in the real world: the typical score.
• It is not a perfect representation of the
data.
• How can we assess how well the
mean represents reality?
Slide 10
A Perfect Fit
Rating (out of 5)
6
5
4
3
2
1
0
0
1
2
3
4
5
6
Rater
Slide 11
Calculating ‘Error’
• A simple deviation is the difference between
the mean and an actual data point.
• Deviations can be calculated by taking each
score and subtracting the mean from it:
Slide 12
Slide 13
Use the Total Error?
We could just take the error between the mean
and each data point and add them up.
But, this is what
we would get!
( X  X )  0
Score
Mean
Deviation
1
2.6
-1.6
2
2.6
-0.6
3
2.6
0.4
3
2.6
0.4
4
2.6
1.4
Total =
0
Slide 14
Sum of Squared Errors
• Merely summing the deviations to find out
the total error is problematic.
• This is because deviations cancel out
because some are positive and others
negative.
• Therefore, we square each deviation.
• If we add these squared deviations we get
the Sum of Squared Errors (SS).
Slide 15
SS 

Score
Mean
Deviation
Squared
Deviation
1
2.6
-1.6
2.56
2
2.6
-0.6
0.36
3
2.6
0.4
0.16
3
2.6
0.4
0.16
4
2.6
1.4
1.96
Total
5.20
( X  X )  5.20
2
Sum of Squared
Errors (SS)
Slide 16
Mean Squared Error (MSe)
Although the SS is a good measure of the accuracy
of our model, it depends on the amount of data
collected. To overcome this problem, we use the
average, or mean, squared error, but to compute
the average we use the degrees of freedom (df) as
the denominator.
Slide 17
Degrees of Freedom
The number of degrees of freedom generally refers to the number of independent observations in
a sample minus the number of population parameters that must be estimated from sample data
(please refer to Jane Superbrain 2.2 on p. 49 of the Field textbook).
Sample
Population
7
11
8
12
9
X  10
15
8
?
  10
Slide 18
The Standard Error
• The standard deviation (SD) tells us how well
the mean represents the sample data.
• If we wish to estimate this parameter in the
population (μ is the symbol used to connote the population
mean) then we need to know how well the
sample mean represents the values in the
population.
• The standard error of the mean (SE or
standard error) tells us the extent to which our
sample mean is representative of the population
parameter.
Slide 19
The SD and the Shape of a Distribution
Slide 20
Samples vs. Populations
• Sample
– Here, the Mean and SD describe only the sample
from which they were calculated.
• Population
– Here, the Mean and SD are intended to describe the
entire population (very rare in practice).
• Sample to Population:
– Mean and SD are obtained from a sample, but are
used to estimate the mean and SD of the population
(very common in practice).
Slide 21
Sampling Variation
X  30
X  25
X  33
X  30
X  29
Slide 22
 = 10
M = 10
M=9
M = 11
M = 10
M=9
M=8
M = 12
M = 11
M = 10
s
X 
N
Mean = 10
SD = 1.22
4
3
Frequency
The SE can be
estimated by dividing
the sample standard
deviation by the square
root of the sample size.
2
1
0
6
7
8
9
10
11
Sample Mean
12
13
14
Slide 23
Confidence Intervals
• In statistical inference, we attempt to estimate population
parameters using observed sample data.
• A confidence interval gives an estimated range of values
which is likely to include an unknown population parameter,
the estimated range being calculated from a given set of
sample data.
• Confidence intervals are constructed at a confidence level,
such as 95 %, selected by the user.
• What does this mean?
• It means that if the same population is sampled on numerous
occasions and interval estimates are made on each
occasion, the resulting intervals would bracket the true
population parameter in approximately 95 % of the cases.
Slide 24
Confidence Intervals
(see discussion starting on p. 54 of Field textbook)
• Domjan et al. (1998)
– ‘Conditioned’ sperm release in Japanese Quail.
• True Mean
– 15 Million sperm
• Sample Mean
– 17 Million sperm
• Interval estimate
– 12 to 22 million (contains true value)
– 16 to 18 million (misses true value)
– CIs constructed such that 95% contain the true
value.
Slide 25
Slide 26
Showing Confidence Intervals
Visually
Slide 27
Types of Hypotheses
• Null hypothesis, H0
– There is no effect.
– Example: Big Brother contestants and members
of the public will not differ in their scores on
personality disorder questionnaires
• The alternative hypothesis, H1
– AKA the experimental hypothesis
– Example: Big Brother contestants will score
higher on personality disorder questionnaires
than members of the public
Slide 28
Test Statistics
• A Statistic for which the frequency of
particular values is known.
• Observed values can be used to test
hypotheses.
Slide 29
What does Null Hypothesis
Significance Testing (NHST) Tell Us?
• The importance of an effect?
– No, significance depends on the sample size.
• That the null hypothesis is false?
– No, it is always false.
• That the null hypothesis is true?
– No, it is never true.
• One problem with NHST is that it encourages
all or nothing thinking.
Slide 30
One- and Two-Tailed Tests
Slide 31
Type I and Type II Errors
• Type I error
– occurs when we believe that there is a
genuine effect in our population, when in fact
there isn’t.
– The probability is the α-level (usually .05)
• Type II error
– occurs when we believe that there is no effect
in the population when, in reality, there is.
– The probability is the β-level (often .2)
Slide 32
Confidence Intervals & Statistical
Significance
Slide 33
Effect Sizes
An effect size is a standardized measure of
the size of an effect:
– Standardized = so comparable across studies
– Not (as) reliant on the sample size
– Allows people to objectively evaluate the size
of observed effect.
Slide 34
Effect Size Measures
• There are several effect size measures that
can be used:
–
–
–
–
–
Cohen’s d
Pearson’s r
Glass’ Δ
Hedges’ g
Odds Ratio/Risk rates
• Pearson’s r is a good intuitive measure
– Oh, apart from when group sizes are different …
Slide 35
Effect Size Measures
• r = .1, d = .2 (small effect):
– the effect explains 1% of the total variance.
• r = .3, d = .5 (medium effect):
– the effect accounts for 9% of the total variance.
• r = .5, d = .8 (large effect):
– the effect accounts for 25% of the variance.
• Beware of these ‘canned’ effect sizes though:
– The size of effect should be placed within the
research context.
Slide 36
Important Concepts
Alternative hypothesis
Standard error of the mean (SE)
Confidence interval
Sum of squared errors (SS)
Degrees of freedom
Test statistic
Deviation
Unexplained variance
Effect Size (measures)
Explained variance
Frequency distribution
Mean
Mean Squared Error (MSe)
Model
Null hypothesis
Null Hypothesis Significance
Testing (NHST)
Population
Sample
Standard deviation
Slide 37