Transcript Research Methods Notes - Reading Community Schools

Research Methods & Statistics:
“Those who trust in their own wits
are fools”.
-Proverbs 28:26
Guiding Principles
• The goal is research literacy.
– Most of us will not be career researchers.
– BUT….we are all consumers of research.
– How do you tell the good from the bad?
• No research is perfect. Everything is open to
critique—a great platform for critical thinking.
Methods Is Like Whac-A-Mole…
a fun game for which there is no perfect solution
Whac-A-Mole
• If I’m conducting my own research, the goal is
to minimize the moles.
• If I’m considering the research of others, the
goal is to identify as many moles as I can.
98% Certainty
• Answer the questions on the next slide by
writing a small number and a large number
such that you are at least 98% certain that the
correct answer is in between.
98% Certainty
1. The area of the US in square miles?
2. The population of Australia 2007?
3. American battle deaths in Spanish-American
War?
4. Female psychiatrists in the US in 2005?
5. Operating nuclear plants worldwide in 2007?
98% Certainty
1.
2.
3.
4.
Area of US:
Australian pop.:
Battle deaths:
Female
psychiatrists:
5. Nuclear plants:
3.6 million sq. miles
20.4 million
385
13,079
435
How’d that work out?
Topic 1: Moles
• Hindsight bias (Myers Activity)
• https://www.psychologytoday.com/blog/go
al-posts/200911/monday-morningquarterbacking-the-case-the-hindsightbias
• Confirmation bias (NY Times activity)
• Overconfidence
Topic 2: Descriptive Techniques
• Case studies
• Surveys
• Naturalistic observation.
Case Studies
Case Study- Descriptive technique in which one
individual or group is studied in depth in the
hope that a universal principle will be
discovered.
What is the potential mole?
The person or group studied may be atypical.
Individual cases are useful, but can often lead to
mistaken judgments, and false conclusions.
Phineas Gage
• https://www.youtube.com/watch?v=_nikOxNf
jqs
• Most famous psychological case study. Taught
psychologists a lot about the brain.
Survey Activity
• Half of you will go out in the hall while the
other half answers the questions on the next
slide.
• When they are finished, the other half of you
will come in and answer questions on the
following slide.
Group 1
• Is the Mississippi River longer or shorter than
500 miles?
• 2. How many miles long is it?
Group 2
• Is the Mississippi River longer or shorter than
3000 miles?
• 2. How many miles long is it?
Activity Continued
• Both of the groups actually answered the same
second question.
• It was clearly an unambiguous question, no tricks
or anything.
• Let’s record the answers to the second question
from the first group, and then the answers for the
second group.
• What do we notice?
*The river is 2,320 miles long
Surveys
A survey- a technique for ascertaining the selfreported attitudes and behaviors of a particular
group, usually by questioning a representative,
random sample of the group.
What are the potential moles?
1. The wording of the question can influence the
answer.
2. How do you insure the sample is indeed
random, and accurately reflects the attitudes of
the group as a whole?
Good Surveying Techniques
• Carefully word your survey questions in order to
ensure you are not generating a particular
response.
• Control for any errors by asking a similar question
with different wording.
• Avoid sampling bias or a flawed sampling process
that produces an unrepresentative sample.
• Establish a random sample i.e. a sample that
fairly represents a population because each
member has an equal chance of inclusion.
Topic 3: Experimentation
• The purpose of an
experiment is to
establish a cause-andeffect relationship.
• Experiments are the
only research method
that can establish
cause-and-effect.
Example Experiment
General hypothesis: Food affects learning.
Specific hypothesis with operational variables:
Students who eat an oatmeal raisin cookie
before class each day will have higher average
scores on the semester final than students who
don’t eat a cookie.
Eating cookies before class each day will
lead to higher average scores.
Variables:
Independent (IV)
Controlled by experimenter
The “cause” variable
Dependent (DV)
Predicted by experimenter
The “effect” variable
Eating cookies before class each day
will lead to higher average scores.
What if kids get cookies and A’s?
Groups (conditions): to establish different levels
of the IV
Experimental group
Exposed to IV
Get cookie
Control group
Not exposed to IV
No cookie
Eating cookies before class each day
will lead to higher average scores.
Confounding
Variables
IV
DV
Expt. Gp.
Cookie
95%
Environmental
Cntrl. Gp.
No Cookie
82%
Expectations
Individual
differences
Control for confounding variables
• Environmental: Make the environment the
same for both groups (so it’s not a variable).
• Expectations: Utilize a blind procedure (so
nobody knows what to expect).
• Individual differences: Randomly assign
participants to groups (so the differences
have the same average impact on each
group).
Random Sampling & Random
Assignment
Random Sampling
• To select participants from population
• Allows you generalize results
Random Assignment
• To divide participants into groups
• Controls individual difference confounding
variables
Eating cookies before class each day
will lead to higher average scores.
IV
DV
Expt. Gp.
Cookie
95%
Cntrl. Gp.
No Cookie
82%
85%
93%
Statistical Significance
p value
likelihood a result is caused by chance
can be no greater than 5%
p ≤ .05
Replication
Non-replicated results are preliminary.
Linus Pauling (1970). Vitamin C prevents colds.
IV
DV
Expt. Gp.
Vit C
Cntrl. Gp.
Placebo
Expt. Gp.
45%
Fewer colds
Has never been
proven, even after
16 double blind
tests.
Importance of Operational Definitions
Students are more likely to smile for their
senior pictures if they have a friendly
photographer.
IV?
DV?
Photographer friendliness
Smiling
Operational definitions are needed for both of these variables.
To illustrate the importance of this, have students determine how man
the students on the following slide are smiling.
How Many Smiles
Importance of Operational Definitions
• If we want to be critical consumers of
research, we need to always ask how research
variables were operationalized (“What do they
mean by ‘best school,’ ‘learning,’ ‘happiness,’
etc.?”).
Research cannot be replicated without
operational definitions.
Statistics
• Our focus should be conceptual, not
computational.
• Statistics are necessary to understand the
meaning of a set of numbers.
• The importance of statistics needs to be
evident throughout the entire course, not just
in the methods unit.
Topic 1: Frequency Distributions
Putting scores in order adds meaning
Bar graphs (histograms) are visual
representations of frequency distributions.
A 40 4
39 7
38 10
37 8
36 15
B 35 8
34 8
33 8
32 7
C 31 4
30 5
29 7
28
D 27
26 2
25 1
24 2
F <24 1
45%
32%
16%
5%
1%
Topic 2: What’s the center of the
distribution?
Measures of Central Tendency
Mode
--Most common = 4
Mean
--Arithmetic avg = 20/5 = 4
Median
--Middle score = 4
Quiz Scores
4
3
5
4
4
Central Tendency: Mean vs. Median
1968 TOPPS Baseball Cards
Nolan Ryan
Billy Williams
Luis Aparicio
Harmon Killebrew
Orlando Cepeda
Maury Wills
Jim Bunning
Tony Conigliaro
Tony Oliva
Lou Pinella
Mickey Lolich
$1500
$8
$5
$5
$3.50
$3.50
$3
$3
$3
$3
$2.50
Elston Howard
Jim Bouton
Rocky Colavito
Boog Powell
Luis Tiant
Tim McCarver
Tug McGraw
Joe Torre
Rusty Staub
Curt Flood
With Ryan:
Median=$2.50
Mean=$74.14
$2.25
$2
$2
$2
$2
$1.75
$1.75
$1.5
$1.25
$1
Without Ryan:
Median=$2.38
Mean=$2.85
The median is a better measure of
central tendency than the mean when
there are extreme scores.
Topic 3: How spread out are the data?
Measures of variation
Range
•
The spread between the
highest number & the
lowest number.
•
Only considers two
numbers
Standard deviation- a computed
measure of how scores vary
around the mean score.
Calculation Example for Standard
Deviation
Punt
Distance
36
38
41
45
Mean =
160/4 =
40 yds
Deviation
from Mean
-4
-2
+1
+5
Deviation
Squared
16
4
1
25
std. dev. =
Variance =
11.5 = 3.4 yds
46
46/4 = 11.5 =
variance
Topic 4: Properties of the Normal
Curve
In a large, randomly distributed data set
•
68% of scores will be within 1 SD of the mean.
•
95% of scores will be within 2 SDs of the mean.
•
99.7% of scores will be withing 3 SDs of the
mean.
Topic 4: Properties of the Normal
Curve
Marilyn vos Savant: claimed IQ of 228.
Is it more meaningful to express her
IQ as points above average or as
standard deviations above average?
Topic 5: Correlation
• A measure of the strength of the
relationship between two variables.
• Can be positive or negative.
• Useful for making predictions.
• You can fairly easily calculate correlations
with Excel or Google Docs.
Topic 5: Correlation
What does a correlation looks like?
Scatterplots
Positive Correlation
Negative Correlation
Topic 5: Correlation
No Correlation
Topic 5: Correlation
How do you express a correlation numerically?
The Correlation Coefficient
Topic 5: Correlation
A strong correlation is not enough to
establish a cause and effect relationship.
Example: There is a correlation between TV
watching and grades.
Do you think it’s positive, or negative?
From this, what do we know about causeand-effect.
Topic 5: Correlation
Even correlations that are clearly not causeand-effect relationships can be used for
prediction.
Ex: College entrance exams and freshman
GPA.
Ex: Shoe size and vocabulary size in
elementary school children.
Ex: Ice cream sales and the rate of violent
crimes.
Topic 5: Correlation
Weird correlations:
http://www.ebaumsworld.com/pictures/view
/84284804/?autoplay=true
Topic 5: Correlation
Topic 6: Statistical Significance
• A measure of the likelihood that a result is
caused by chance.
• In an experiment, we want that likelihood
to be low so we can conclude a causeand-effect relationship exists between the
IV and the DV.
Topic 6: Statistical Significance
• P value is an estimate of the probability
that a result was caused by chance.
• In an experiment, it’s the likelihood that the
difference between the experimental and
control conditions as measured by the DV
was caused by chance.
• We want this difference to be caused by
our manipulation—the IV—not by chance.
Topic 6: Statistical Significance
• To say that the results of an experiment are
statistically significant means that there is a
small likelihood that the results were caused
by chance; that is, a high likelihood they were
caused by the IV.
• The threshold for statistical significance is no
more than a 5% likelihood the results were
caused by chance.
• We express this: p ≤ .05
Important Things to Consider In
Regards to Research
• Laboratory experiments can often illuminate certain
principles that can help us understand everyday life.
• While the laboratory is a simplified reality, the idea is
to control certain features that cannot be controlled in
the outside world.
• Experiments are conducted in this setting in order to
test theoretical principles, which can then be applied.
• Think of the laboratory as a sort of wind tunnel, like
the kind used to develop aeronautical technology.
Important Things to Consider In
Regards to Research
• It is important to consider cultural differences when
considering research.
• Culture, or the enduring behaviors, attitudes, values,
and traditions shared by a group of people and
transmitted from one generation to the next, matters.
• Gender often matters to.
• As a result, research needs to take these differences
into consideration.
• The key to remember though, is that the same
underlying processes guide people everywhere.
• We are more alike than we are different.
Important Things to Consider In
Regards to Research
• Many psychologists study animals, and much research is conducted
in this way.
• There is certainly much that can be learned from animals that can
be applied to humans, because there are fundamental similarities.
• In addition, animals are used in many other scientific experiments
that will yield results for humans.
• There is considerable debate concerning the use of animals in
research.
• The question is whether the good that may come from the research
to humans, and many times to other animals, is worth the potential
pain or stress that animal may endure in research and testing.
• It is important to reach a balance of conducting research that is
effective, but still ethical in dealing with animals. As Myers states,
“A psychology concerned for humans, and sensitive to animals
serves the welfare of both.”
Important Things to Consider In
Regards to Research
• There are important ethical principles adopted by
the APA. Psychologists are urged to:
1. Obtain informed consent from participants (be
told enough to decide whether they wish to
participate).
2. Protect participants from physical and emotional
harm and discomfort.
3. Maintain confidentiality.
4. Debrief participants after a study (inform them
of the purpose of the study, and any deceptions
that may have been used).