Transcript Types of Inferential Statistics

Research Design & Analysis 2: Class 15
• Announcements: Final exam: April 12: 2 5:00 p.m.
• Final comments on External validity
– Animal research (concluding issues)
– Human volunteers (change to schedule)
• Stress and health
• Ex-Post-Facto designs
– Prospective
– Retrospective
– DVs used in ex-post-facto designs
– Breast cancer data
Psychology as a Major
Dr. Sonya Symons
Department head
Wondering which course to take?
Interested in a career in psychology?
~~~~~
Tuesday March 14th, 2000
4:30 p.m.
Horton Hall Room 215
all are welcome
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
2
Using animals in psychological research
• Animal rights movement
• Alternatives to using animals
– in vitro studies
– computer simulations
…”there are no viable alternatives to
using animals in behavioral research”
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
3
Scientific Explanations
• Science is a self correcting process
• Single research findings are seldom
conclusive
• Problem especially in applied areas where
answers are needed quickly
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
4
Factors Influencing External Validity
Issue of generalizability beyond the
specific experimental conditions
• Population sampled
• Operational definitions
• Parameter values
• Demand characteristics
Ecological validity?
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
5
Characteristics of Volunteers
From Text Book pp. 122-128
Issues are:
• Whether volunteers differ in meaningful
ways from non-volunteers
• Do any differences affect the external
validity of research
• Do any differences affect the internal
validity of research
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
6
Table 5-4*: Characteristics of Research
Volunteers
MAXIMUM CONFIDENCE
1.Volunteers tend to be more highly educated
than nonvolunteers.
2.Volunteers tend to come from a higher social
class than nonvolunteers.
3.Volunteers are of higher intelligence, but not
when volunteering for atypical research (such
as hypnosis, sex research).
4.Volunteers have a higher need for approval
than nonvolunteers.
5.Volunteers are more social than
nonvolunteers.
*from Rosenthal & Rosnow, 1975. The Volunteer Subject.
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
7
Characteristics of Research Volunteers (cont)
CONSIDERABLE CONFIDENCE
1. Volunteers are more "arousal seeking" than
nonvolunteers (especially with stress research).
2. People who volunteer for sex research are more
unconventional than nonvolunteers.
3. Females are more likely to volunteer than males, except
when research involves physical or emotional stress.
4. Volunteers are less authoritarian than nonvolunteers.
5. Jews more likely to volunteer than Protestants;
however, Protestants are more likely to volunteer than
Catholics.
6. Volunteers tend to be less conforming than
nonvolunteers, except where the volunteers are female
and the research is clinically oriented.
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
8
From Rosenthal & Rosnow ‘75
Volunteers & internal validity
• 42 undergrad
women
Design?
•2x3 factorial
• Volunteers more
interested in
pleasing
experimenter
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
9
Fig 5-4: From Horowitz, 1969
Volunteers & external validity
Design?
•2x2 factorial
Interaction?
• Yes: Volunteers
more effected by
high fear
information (no
difference with low
fear)
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
10
From Horowitz, 1969
Volunteers & external validity
Design?
•2x2x2 factorial
• Difference
between
volunteers and
non increased
with multiple
exposures
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
11
Remedies for Volunteerism (p. 127-28)
•
•
•
•
•
•
•
•
•
•
Make the appeal interesting
Make the appeal nonthreatening
State the importance of the research
State why the target population is relevant
Pay them. Give gifts.
Have request come from a high status (preferable
female) person.
Avoid stressful research
Communicate the normative nature of participating
Make the appeal come from a a known person if
possible
Depending, commitment to volunteer might be better
made privately or publicly
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
12
Ex-Post-Facto (after the fact)
Quasi-analytic Designs (on reserve)
Much less able to determine causality than
true experiments but necessary and
important designs.
Necessary for:
1. Ethical reasons or
2. An interest in organismic variables
In either case, you cannot randomly assign
to treatment conditions
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
13
Types of Ex-Post-Facto Designs
Two main types:
1. Prospective
2. Retrospective
In both cases you find naturally occurring
groups ("after the fact") and follow them
forward (prospective) or trace their histories
(retrospective).
Even if randomly sampled from these
populations, there are potential
confounds.
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
14
Ex post facto Studies of Stress & Cancer
Solution 1: Prospective design
Solution 2: Retrospective design
Sample 100
subjects,
high stress
Sample 100
subjects,
low stress
Sample 100
subjects,
with cancer
Sample 100
subjects,
no cancer
wait:
look forward
in time
wait:
look forward
in time
look back
in time
look back
in time
Frequency
of cancer
20
Frequency
of cancer
5
Frequency
of high stress
40
Frequency
of high stress
20
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
15
Prospective ex post facto Study of Stress &
Cancer
High Stress Population:
random sample of 100 airtraffic controllers
•
•
•
•
•
•
•
average age = 35
live in city
married
high smoking rate
high-fat diet
high drug usage
high exposure to VDTs
• frequency of cancer = 20
D.V.
7/20/2015
Low Stress Population:
random sample of 100
clerks
•
•
•
•
•
•
•
average age = 35
live in city
married
low smoking rate
low-fat diet
low drug use
low exposure to VDT
• Frequency of cancer = 5
D.V.
Psyc2023 Class#15 (c) Peter McLeod
16
Prospective ex post facto Study of Stress &
Cancer
High Stress Population:
random sample of 100 airtraffic controllers
•
•
•
•
•
•
•
average age = 35
live in city
married
high smoking rate
high-fat diet
high drug usage
high exposure to VDTs
• frequency of cancer = 20
D.V.
7/20/2015
Low Stress Population:
random sample of 100
clerks
•
•
•
•
•
•
•
average age = 35
live in city
married
low smoking rate
low-fat diet
low drug use
low exposure to VDT
• Frequency of cancer = 5
D.V.
Psyc2023 Class#15 (c) Peter McLeod
17
Bivalent Experimental Study of Stress &
Cancer
Random sample of 100
people in 1980
experimental high-stress
manipulation
•
•
•
•
•
•
•
age
marital status
smoking rate
drug use
weight
hair colour
anything else...
•
•
•
•
•
•
•
• frequency of cancer = 20
7/20/2015
Random sample of 100
people in 1980
experimental low-stress
manipulation
age
marital status
smoking rate
drug use
weight
hair colour
anything else...
Frequency of cancer = 5
Psyc2023 Class#15 (c) Peter McLeod
18
Problems with Prospective Ex-Post-Facto
Designs
1. subjects not randomly assigned to
treatments
– may be inherent confounds in the
populations studied
2. sampling problems (often convenient)
– may be impossible to identify all
members of the populations
3. dropouts in prospective studies
4. detection bias
– designs assumes detection and
measurement are equal in both groups
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
19
Partial Solutions
Matching:
1) Subject for subject (preferable but
difficult) or
2) Distribution by distribution
in both cases can selectively drop individuals
and bias the sample further
Measuring:
1) Know if potential confounds (uncontrolled
or extraneous variables)
2) Statistically control for these variables
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
20
Retrospective Studies
• Retrospective studies have additional problems
in relying on memory so the partial solutions are
more difficult to employ successfully
• Have the advantage (over prospective designs)
in that they are more efficient (cheaper and
faster) May be necessary with very rare groups
or variables of interest (e.g., rare diseases)
Note: Even with measurement and matching,
internal validity is questionable.
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
21
Partial Solutions for Retrospective
Designs
Same problems, same (partial) solutions!
Matching:
1) subject for subject (preferable but
difficult) or
2) distribution by distribution
Measuring:
1) know if potential confounds
2) statistically control for these variables
But, much more difficult to employ!
Dependence upon memory and other
records
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
22
Life events and the Occurrence of Cancer
in Children (Jacobs & Charles, 1980)
Retrospective ex post facto design
• Case studies suggested experiencing
separation or loss related to childhood
leukemia
• Method: semi-structured interview
• Control group:
“a variety of physical complaints drawn
from a medical facility”
specificity of hypothesis
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
23
Life events and Cancer in Children
Matching:
– age, sex, S.E.S.
Measures:
– detailed medical history
– personality measures
– relationship measures
– psychological history
– marriage assessment
– changes in life one year prior to
disease onset
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
24
Characteristics
of Samples:
Matched variables
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
25
Measured Variables in Samples
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
26
Holmes & Rahe:
Social Adjustment
Rating Scale
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
27
Paired Data on LCU Measure
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
28
Frequency of Life-Events by Group
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
29
D.V.s Used in Ex-Post-Facto Studies
Relative risk ratio (prospective studies)
– illustrated by breast cancer data (later)
Relative odds ratio (approximates the relative
risk) in retrospective studies
Problem with both is that absolute risks are
hidden. Both absolute and relative risks
should be reported.
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
30
Calculating Relative Risk Ratio
Prospective ex post facto design
100 subjects
100 subjects
high stress group
low stress group
Frequency of cancer
Frequency of cancer
20
5
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
31
Contingency Table for Prospective
ex post facto Data
High Stress Low Stress totals
Cancer Cases
20
5
25
No Cancer Cases
80
95
175
totals
100
100
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
32
Contingency Table for Prospective ex
post facto Data and Relative Risk Ratio
High Stress Low Stress totals
Cancer Cases
20
5
25
No Cancer Cases
80
95
175
totals
100
100
7/20/2015
Relative risk ratio =
20/100  5/100 = 4
These (fake) data would
indicate that the risk of
someone with a highly
stressful life developing
cancer in the future is
four times that of
someone with a low
stress lifestyle.
Psyc2023 Class#15 (c) Peter McLeod
33
Hypothetical Data: Same Relative Risks
but Very Different Absolute Risks
Tampon
User
Nonuser
5
1
NoToxic Shock
995
999
totals
1000
1000
Tampon
User
Nonuser
Toxic Shock
Syndrome
250
50
Toxic Shock
Syndrome
NoToxic Shock
750
950
totals
1000
1000
Relative risk ratio =
250/1000 ÷ 50/1000 = 5
Relative risk ratio =
5/1000 ÷ 1/1000 = 5
In both cases, the relative risk ratio is 5
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
34
Calculating Relative Odds Ratio
Retrospective ex post facto design
100 subjects with
100 subjects
cancer
without cancer
Frequency of
Frequency of
high-stress case histories high-stress case histories
20
7/20/2015
5
Psyc2023 Class#15 (c) Peter McLeod
35
Contingency Table for Retrospective
ex post facto Data
High Stress Low Stress totals
Cancer Group
20
80
100
Noncancer Group
5
95
100
totals
25
175
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
36
Contingency Table for Retrospective ex
post facto Data and Relative Odds Ratio
High Stress Low Stress totals
Cancer Group
20
80
100
Noncancer Group
5
95
100
totals
25
175
7/20/2015
Relative odds ratio =
20/5  80/95 = 4.7
This is the odds of people
who report a high stress
past being in the cancer (vs.
noncancer) group, divided
by the odds of people who
report a low stress history
being in the cancer group.
•The relative odds ratio
provides an estimate of the
relative risk ratio.
Psyc2023 Class#15 (c) Peter McLeod
37
Cautions about D.V.s Used in ExPost-Facto Studies
• Statistically, both measures assume
random sampling (unlikely or
impossible)
• Both the Relative risk and Relative
odds ratios hide absolute risks.
Some human cancer data
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
38
Lifetime Risks of
Developing
Breast Cancer
(US study
published in
Science ‘93)
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
39
Top 10 Women’s
Cancers
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
40
Incidence of
Breast Cancer
by Stage
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
41
Incident & Mortality Rate Trends
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
42
Canadian
Data
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
43
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
44
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
45
penny
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
46
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
47
Relative Risk
Ratios for Breast
Cancer
* * * relative risk >4
* * relative risk of 2- 4
* relative risk of 1.1-1.9
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
48
Putting risk of Breast Cancer in
Perspective
One woman in 9 in whom breast cancer will
develop, has a 50% chance of receiving
the diagnosis after age 65 and a 60%
chance of surviving that cancer and dying
of other causes
Risk of breast cancer in any given decade
never exceeds 1 in 34 (at 30 is 1 in 250)
Based on editorial in NEJM
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
49
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
50
Deaths
by Age
and Type
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
51
Research Fraud in Canada
Dr. Roger Poisson
From Maclean’s, March 28, 1994
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
52
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
53
Some Levels at Which
Cancer May be
Social/Environmental
Regulated
Cultural
Psychological
from A.J. Cunningham (1985)
Nervous System
Canadian Psychology,
26(1), 13-29.
Endocrines
Immune System
Tissues
Cells
DNA
7/20/2015
Psyc2023 Class#15 (c) Peter McLeod
54