Transcript Lecture7 - University of Idaho

PSYC512: Research Methods
Lecture 7
Brian P. Dyre
University of Idaho
PSYC512: Research Methods
Lecture 7 Outline





Questions about material covered in Lecture 6
 Measures: scales and sensitivity
More on Measurement
 Reliability, Precision, and Validity
Hypothesis testing and Variables
Variables and Research Design
Defining Variables
PSYC512: Research Methods
Features of Measures:
Reliability


The ability of a measure to produce consistent results
when repeated measurements are taken under
identical conditions
Types:
 precision: physical measurement (1/noise)
 margin of error: sampling in surveys
 interrater reliability: observers viewing the same
behavior
 Test-retest, parallel forms and split-half
reliabilities: psychological tests
PSYC512: Research Methods
Other Features of Measures



Accuracy
 does a measure produce results that agree with a
known standard?
 Accuracy vs. Precision
Validity
 Measurement validity: the extent to which your
measure indeed measures what it is intended to
measure
 Types: Face validity, Content validity, Criterionrelated validity (concurrent vs. predictive),
Construct validity
Relationship between reliability and validity
PSYC512: Research Methods
Hypothesis Testing: Variables


Hypothesis testing is the process by which hypothetical relationships
between variables (something that varies in quantity or quality) are
assessed (the relationships are deduced from one or more theories)
Types of variables
 Dependent variable  measure
 Independent variable  manipulation
 Extraneous variable  not pertinent to hypotheses
 Confounding variable  extraneous variable that covaries with
your manipulated variable (typically we try to control these to
eliminate the covariance)
 Intervening variable  theoretical construct of interest that is
not directly observable (e.g., group cohesiveness, mental
workload)
PSYC512: Research Methods
Variables and Research
Designs

Relationships can be hypothesized between
 Multiple dependent measures  correlational
research design: presence or absence of a relation
between the variables can be tested, but not causality
 Manipulated (independent) variables and some
measure  experimental design, with proper control
of confounding variables (e.g., random assignment to
experimental treatment groups) causality may be
inferred
PSYC512: Research Methods
Defining Variables:
Operationism

Operationism: psychological concepts are
equivalent to the operations (manipulations or
measures) used to define those concepts

Hunger: the state produced by food
deprivation


Only observable operations are included in
theoretical or hypothetical statements
You cannot separate the concept from its
operations—cannot generalize, concept has
no external validity
PSYC512: Research Methods
Defining Variables: Converging
Operations or Network Specification


Multiple operations or a set of operations can be used
to define a concept, not just one
Operations can converge to scientifically isolate
intervening variables through a process of converging
operations (Garner, Hake, & Eriksen, 1956)

selective influence – experimental manipulations
affect particular intervening variables but not others

convergence – different operations can be used to
manipulate or measure a common intervening
variable or psychological construct
PSYC512: Research Methods
Converging Operations




Example: The phenomenon of “Perceptual” Defense
(Garner, Hake, & Eriksen, 1956)
Two Possibilities

perceptual discrimination of vulgar words takes
longer

responding with a vulgar word takes longer
Operationist: perception is the discrimination response,
therefore, we can’t tell which
Converging operations: add a second, orthogonal
operation—exchange the vulgar and neutral response
mappings
PSYC512: Research Methods
Network Specification of
Meaning


Psychological Concepts are defined by their relations
with other concepts rather than a unitary operational
definition
Introduction and Discussion sections of papers describe
the relationships of our variables to all other relevant
variables and concepts—what G, H, & E call assumed
operations

Method and results sections describe the specific
converging operations we use
PSYC512: Research Methods
Construct Validity


The soundness of our operations, do they manipulate or
measure the intervening variable that they are intended
to manipulate or measure?
Types (Campbell & Fiske, 1959)
 Discriminant validation: operation should not affect or
correlate with operations on other intervening
variables
 Convergent validation: operation should affect or
correlate with other operations on the same
intervening variable
PSYC512: Research Methods
Testing Hypotheses




Hypothesis testing is the process by which hypothetical
relationships between intervening variables are assessed
Hypotheses are always tested relative to one-another or
to a “null” hypothesis
Examples
 Comparing Groups
 Assessing Performance Interventions
 Assessing Relationships between variables
Problem: Measurement Noise
PSYC512: Research Methods
Hypothesis Testing:
Probability and Statistics


Why are probability and statistics important?
 Used to assess variability in a measure
 Effect (treatment) Variance
 Variability due to relationship between
variables or effect of different levels of
independent variable (treatments)
 “Good” variance that we want to maximize
 Error Variance
 Variability in measure due to factors other than
the treatment
 “Bad” variance that we want to minimize
Probability and Statistics are simply tools used to
assess (descriptive statistics) and compare
(inferential statistics) these sources of variability
PSYC512: Research Methods
Visualizing Variability: Distributions
of Frequency and the Histogram
Histograms: used to represent
frequencies of data in different
classes or categories
Bin
6
Frequency

4
2
0
0
1
2
3
4
5
6
7
8
9 10
Grade
PSYC512: Research Methods
Frequency
0
0
1
0
2
0
3
0
4
3
5
1
6
6
7
4
8
2
9
6
10
1
Displaying Histograms:
Stem and Leaf Plots
Stem and Leaf plots are used to display histograms
graphically (on their side) using only typed characters
Stem
Leaf
(hypothetical histogram for IQ)

6
7
8
9
10
11
12
13
78
35668
012234445555667777889
00011233333334445566667889999
01112233334444445566677777888899
0001122233444566777899
0012569
02
PSYC512: Research Methods
Distributions of Probability Density

Similar to frequency
histogram except y-axis
now represents
probability density
(mass) rather than
frequency
Probability density =
Frequency/N
0.5
Probability Density

0.4
0.3
0.2
0.1
0.0
0
1
2
3
4
5
6
Grade
PSYC512: Research Methods
7
8
9 10
Some Types of Distributions
Gamma
0.14
0.18
0.12
0.16
Probability Density
Probability Density
Normal
0.10
0.08
0.06
0.04
0.14
0.12
0.10
0.08
0.06
0.04
0.02
0.02
0.00
0.00
0
5
10
15
20
25
Data
0
5
10
Data
PSYC512: Research Methods
15
20
25
Measures of the Center of a
Distribution




Measures of center represent the general
magnitude of scores in a distribution
Mode: most frequent score
Median: the middle score of an ordered
distribution
Mean (average):
where X is the data and
N is the total number of
observations
X


N
PSYC512: Research Methods
Measures of the Spread of a
Distribution




Measures of spread are used to assess the
consistency of scores in a distribution
Range = max score – min score
Interquartile range = score(Q3) – score(Q1)
Variance (s2) and standard deviation (s)
where X is the data,
2
m is the mean of the data,


X


and N is the total number
s2  
N
of observations
PSYC512: Research Methods
More on Variance
Standard Deviation (s) = sqrt(variance)
where X is the data,
2
X 
m is the mean of the data,
s
and N is the total number
N
of observations
Why N instead of N-1? Populations vs. Samples
 Remembering how to compute variance
“the mean of the squares – square of the means”



s
2
X


N
2
X
 
 N




2
PSYC512: Research Methods
Describing Distributions
Parametrically: Statistical Moments





Any distribution based on interval or ratio data can be
summarized by its statistical moments
First Moment: Mean—location of distribution on x-axis
Second Moment: Variance—dispersion of distribution
Third Moment: Skewness—symmetry of distribution
Fourth Moment: Kurtosis—degree of “peakedness”
PSYC512: Research Methods
Estimators


Sample statistics estimate population parameters
 Mean: M or X vs. M
 Variance: s2
vs. s2
Properties of Estimators
 Sufficiency: uses all information in sample (mean and variance
are sufficient, mode and range are not)
 Unbiasedness: expected value approaches real value with
increased sampling
 Efficiency: tightness of cluster of sample statistics relative to
the population parameter
 Resistance: influence of outliers on sample statistic
PSYC512: Research Methods
Next Time…


Topic: Research Designs and Inferential
Statistics
Be sure to:
 Read the assigned readings (Howell chapters
6-7)
 Continue searching and reading the scientific
literature for your proposal
PSYC512: Research Methods