Transcript Research Design

Research Design
Designing Your Experiment
Setting the Stage
Type of Manipulation
Straightforward Manipulation
Staged or Event Manipulation
1. to create a psychological state in the participants
2. to simulate some situation that occurs in real life
frequently employs confederates
Strength of the Manipulation
Cost of the Manipulation
Experimental Group Design
There are many ways to design an experiment. Here are three of the most
common group designs:
Independent or Between-Subjects Groups Design
Repeated Measures or Within-Subjects Groups Design
Mixed-Subjects Group Design
Independent or Between-Subjects
Requirements:
Each level of each independent variable has different subjects.
participants are randomly assigned to the different groups
random assignment prevents any systematic biases
Advantages:
minimizes order effects
minimizes demand characteristics
is the easiest to statistically analyze.
Drawbacks:
requires many subjects
less statistical power
takes more time
Control
Experimental
S01 S02 S03
S04 S05 S06
S07 S08 S09
S10 S11 S12
Order Effects
order of presenting the treatments affects the dependent variable
Practice Effect
improvement in performance with repetition
Fatigue Effect
decrease in performance with repetition
Contrast Effect
second response is altered due to the influence of the first
Counterbalancing is used to correct order effects:
Counterbalancing
helps to minimize order effects
•Complete Counterbalancing
•Randomized Block Design
•Matched Pairs Design
•Latin Squares
Repeated Measures or
Within-Subjects
Requirements:
Each subject participates in all levels of all independent variables
Advantages:
uses fewer subjects than the Between-Subjects Design
greater statistical power/ lower score variability
takes less time
minimizes subject effects
Drawbacks:
is difficult to control order effects
increases demand characteristics
Control
Experimental
S01 S02 S03
S04 S05 S06
S01 S02 S03
S04 S05 S06
Mixed-Subjects
Requirements:
must be at least two independent variables
Each subject participates in all levels of just one independent variable
Advantages:
uses fewer subjects than the Between-Subjects
controls for order effects
controls for subject effects
Drawbacks:
is the most difficult to analyze statistically
Control
Experimental
Control
S01 S02 S03
S04 S05 S06
S01 S02 S03
S04 S05 S06
Experimental
S07 S08 S09
S10 S11 S12
S07 S08 S09
S10 S11 S12
Factorial Designs
Factorial designs have more than one independent variable or factor
e.g., 2 x 2, 2 x 3, 2 x 2 x 2, IV by participant variables - IV x PV designs
Main Effects:
the effect of each independent variable by itself
Interaction and/or Moderator Variables:
Interaction: when one IV level affects the DV differently than another level
Moderator Variable: a particular situation or participant characteristic
Outcome of a 2 X 2 Factorial Design:
1. Main Effect A significance/non-significance
post-hoc analysis (across individual levels)
2. Main Effect B significance/non-significance
post-hoc analysis (across individual levels)
3. Interaction significance/non-significance
if interaction is significant you must reevaluate the main effects
Interaction
The interaction is the effect of the combination of the two independent variables
on the dependent variable. It is best seen by graphing the means of all levels of
both factors. If the two lines are parallel then the interaction is typically nonsignificant. If the lines are not parallel as in the figure below, it means that there
is an interaction between the two independent variables that is having an effect
on the dependent variable.
Life Span Development Designs
Life Span Development Designs
Weaknesses and Strengths
Non-Experimental
Research Methods
•Surveys
•Descriptive Research Methods
•Correlation
Surveys
•are a common and important method of studying behavior
•provide us with a tool for asking people about themselves
•their attitudes and beliefs
•Past or intended future behaviors
•demographics or other facts
•provide a snapshot of how people think at some point in time
•are used to determine relationships between variables
•can measure how attitudes and behaviors change over time
•can serve as a compliment to experimental research
Survey Problems
•Response Set
•tendency to respond to all questions from one perspective
•can affect the usefulness of survey data
•social desirability - "faking good"
•answers in the most socially acceptable way
•big problem with sensitive questions
•False Answers
•respondents will lie if they do not trust the experimenter
•must give full disclosure and maintain confidentiality
Research Objectives
• Attitudes and Beliefs
• The way people evaluate and think about issues
• Facts and Demographics
• Things they know about themselves and their situation
• Behaviors
• Focus on past behaviors or future intended behaviors
Question Wording
• Simplicity
• Avoid jargon and technical terminology
• Double-Barreled Questions
• Avoid questions that ask two things at once
• Loaded Questions
• No leading or emotionally charged questions
• Negative Wording
• Avoid phrasing questions with negatives
• “Yea-Saying” and “Nay-Saying”
• Don’t’ ask similar questions that might get similar answers
Question Types
• Closed- Versus Open- Ended Questions
• Number of Response Alternatives
• Rating Scales
• Labeling Response Alternatives
Closed- Vs. Open- Ended
• Closed-Ended Questions
•
•
•
•
Fixed number of response alternatives
More structured approach
Easier to code
Used when the dimensions of the variable are well-defined
• Open-Ended Questions
•
•
•
•
Respondents are free to answer any way they like
Requires time to categorize and code
More costly
Can yield valuable insights into what people are thinking
Number of Response Alternatives
• 5-7 point scale is preferable
On a scale of 1 (low) – 7 (high), how would you rate your happiness? ______
strongly agree ___ ___ ___ ___ ___ ___ ___ strongly disagree
Rating Scales
• Graphing Rating Scale
strongly agree ___________________________ strongly disagree
• Semantic Differential Scale
evaluation: good ___ ___ ___ ___ ___ ___ ___ bad
activity: strong ___ ___ ___ ___ ___ ___ ___ weak
potency: active ___ ___ ___ ___ ___ ___ ___ passive
• Non-Verbal Scale for Children
Labeling Response Alternatives
Hi-Frequency
Lo-Frequency
Formatting
Your survey should:
1. appear attractive and professional
2. be neatly typed
3. be free of spelling and grammatical errors
4. have easy to identify questions and answers
5. leave space between questions
6. keep the scale format consistent
7. order the sequence of the questions carefully
8. ask the most interesting question first
9. capture the respondents attention
10. motivate the respondent to answer all of the questions
11. group similar theme questions together
12. be tested on a small sample first
Questionnaires
• Personal Administration to Groups or Individuals
• Have a captive audience
• Researcher is available for questions
• Mail Surveys
• Fairly inexpensive
• Has a potential for low responders
• Internet Surveys
• Inexpensive and growing
• Problem confirming who is responding
• Other Technologies
• Cell phones, pagers, PDA’s
Interviews
• Face-to-Face
• Expensive and time consuming
• Telephone
• Less expensive
• Data is collected quickly
• Focus Group
•
•
•
•
Group interview with 6-10 for 2-3 hours
Open ended questions with group interaction
Must facilitate communication and quell problems
Costly and time consuming
• Panel Study
• Studies changes over time
• Same people surveyed at two or more points in time
Descriptive Research Methods
Behavior is observed and measured as it occurs naturally
• Case Study
• Naturalistic Observation
• Systematic Observation
• Archival Research
Limitations
1. Direction of cause and effect
can not determine cause and effect
2. The Third-Variable Problem
some other variable may be driving the relationship
confounding variables
Case Studies
provides a description of an
individual:
•
person
•
business
•
school
•
neighborhood
“Anna O”
Psychobiography
• Case study research that applies psychological theory to
explain the life of an individual
• Usually investigates some important
historical figure:
Political figures
Literary/Artistic figures
Sports/Hollywood Notables
• Uses library research and phone interviews
• No direct observation at all
Naturalistic Observations
Observations are made:
in a particular setting and,
over an extended period of time
A variety of techniques to collect information are used
Final report includes:
the observations and,
the researcher's interpretations of the findings
Helps us to understand how people in a social/cultural
setting:
live,
work, and,
experience the setting
Data Collection
Naturalistic observation researchers must immerse themselves in the
situation and observe:
• the setting
• the patterns of personal relationships
• people's reactions to events
• other environmental factors
must keep detailed field notes
must write or dictate at least once a day everything that has happened
• Goal 1: describe setting, events, and persons observed
• Goal 2: analyze what was observed
analysis needs to be supported by multiple confirmations
must provide a complete and accurate picture
no hypotheses testing
Participation and Concealment
1. Non-Participant Observer
is an outsider who does not become an active part of the setting
could miss key interactions
2. Participant Observer
does become a part of the setting
could lose objectivity
could affect behavior (reactivity)
3. Experimenter Bias
must leave biases at home
carefully assess the observers role
Defining the Scope of the Observation
Naturalistic observation researchers may have to limit
the scope of study to focus on central issues
Too large a scope:
risk being overwhelmed
could miss more subtle effects
Too small a scope:
risk missing important data
could miss larger more important interactions
Limitations to
Naturalistic Observation
1. Do not start with a well-defined hypothesis
can affect the direction of data collection
2. Very difficult to conduct
requires a great amount of energy
ever changing pattern of events - some important, some not
must record them all
must be flexible to adjust recording as research progresses
3. Identify inconsistencies
additional research may be needed
negative case analysis
an observation that does not fit the explanation
Systematic Observations
• Careful observation of:
one or more specific behaviors in a particular setting
• Interest is in only a few specific behaviors
• Observations are quantifiable
• Researcher begin with prior hypotheses about the
behaviors
Coding Systems
• Are used to quantify behavior
• Should be simple as possible
• Should allow observers to easily categorize behaviors
• Should use coding systems devised by others
system has already been proven useful
training materials are usually available
Methodological Issues
1. Equipment
videotaping is becoming very common
2. Reactivity
the possibility that the presence of the observer will affect people's behavior
3. Reliability
the degree to which a measurement reflects a true score
rather than measurement error
reliable measures are stable, consistent, and precise
4. Sampling
samples of behavior taken over a long period
provide more accurate measure of behavior
Archival Research
Uses previously compiled information
systematic analysis of existing documents
requires research to devise coding systems that raters can use
to quantify the information in the documents
Researcher does not collect any original data
There are three types:
1. Statistical Records
2. Survey Archives
3. Written and Mass Communication Records
Statistical Records
Archival researchers use existing statistical
records that are on file
There are many sources:
US Census
APA
MLB
Public Records
marriage license applications
Survey Archives
Data from surveys stored in large databases
Available to any researcher who wants them
Internet has allowed more availability
Written and Mass Communication
Written Communication
diaries
historical letters
public documents
speeches
Mass Communication
books
magazine articles
movies
television programs
newspapers
Limitations of Archival Research
Desired records may be difficult to obtain
Can never be completely sure of the accuracy of
the data collected by someone else.
Correlation
•A Correlation:
• is a statistical test.
•demonstrates the relationship between two variables within the
same participant.
•can determine significant relationships between variables.
•does NOT show a causal relationship between the variables.
•is best represented by a Scatterplot.
•is determined by calculating the correlational coefficient.
Scatterplot
•Graphical representation of the two sets of data you are comparing.
•Shows the strength and direction of the relationship.
Correlational Coefficient
Pearson Product Moment Correlation
n = number of pairs
Sigma X = the sum of all the X variable values
Sigma Y = the sum of all the Y variable values
Sigma XY = the sum of all the X times Y values
Sigma X2 = the sum of all the X2 values
Sigma Y2 = the sum of all the Y2 values
Calculating a Correlation
X
Y
X2
Y2
XY
10
104
100
10816
1040
12
100
144
10000
1200
19
98
361
9604
1862
4
150
16
22500
600
25
75
625
5625
1875
15
105
225
11025
1575
21
82
441
6724
1722
7
133
49
17689
931
113
847
1961
93983
10805
n=8
Neutral Correlation
Positive Correlation
Correlation: Quiz/Exam
Negative Correlation
Correlation: Days Absent/Exam I
Correlation is Not Causation