Transcript Slide 1 - Pearson

CHAPTER 10
Preparing to
Conduct a Study
Copyright, 2005, Prentice Hall, Sarafino
The Developmental Phase
• The easiest part of any research is the
actual gathering of data.
– It is the easiest part because of all the
planning and effort you have put into in it to
ensure that things go smoothly.
– The slides that follow will highlight the nine –
yes 9 – phases that researchers must go
through in order to ensure an efficient and
true outcome of their research effort.
Copyright, 2005, Prentice Hall, Sarafino
Phase 1: Identify the Research
Topic and Hypothesis
• Things to consider:
– Have you reviewed the ideas of other on
this topic? Do your ideas mesh with those
others? Why or why not?
– Are there any theories relating to your topic?
– Is your hypothesis testable and falsifiable?
– Is your hypothesis theoretically supported
by previous literature?
Copyright, 2005, Prentice Hall, Sarafino
Phase 2:Determine Participants
and Their Availability
• Do you need human or animal
participants and how many do you need?
• Do your participants need to have unique
qualities (e.g., have a phobia, or prone to
develop a disease)?
–
If so, can you find enough participants?
• Will you be able to generate equal
groups?
–
If not, what are the implications?
Copyright, 2005, Prentice Hall, Sarafino
Phase 3: Choose Your Variables
• Are the variables you selected consistent with
previous research?
– How do I find reliable and valid measurement
instruments to measure my variables?
• Mental Measures Yearbook; Test Critiques
• Are you going to manipulate your IVs?
– If so, what levels will you select to maximize
variance?
• Will your dependent variable(s) produce
nominal, ordinal, interval, or ratio data?
Copyright, 2005, Prentice Hall, Sarafino
Phase 4: Choosing a Design
• What is your hypothesis asking?
– Will an experimental, quasi-experimental,
correlational, or descriptive strategy work best to
support your hypothesis.
• Will you need a control group? If so, what kind?
• Will the participants be repeatedly measured?
• What statistical analyses will need to be
performed?
– Are there assumptions associated with running these
analysis, are there limits to the interpretive power of
these statistics?
Copyright, 2005, Prentice Hall, Sarafino
Phase 5: Identify Sources of
Error, and Controls
•
•
•
Try to predict unwanted sources of variance in your
study and then implement control procedures.
–
If you think participant or experimental bias will influence
your result then implement a double blind procedure.
–
If there may be order effects, make sure you counterbalance.
How will you establish and maintain equivalent groups?
Is it possible to hold the conditions of the experiment
constant.
–
Will the different groups of participants really be treated
identically except for the IV manipulation?
Copyright, 2005, Prentice Hall, Sarafino
Phase 6: Identify Limitations on
Generalization
•
•
•
Will the types of participants used (e.g.,
students) limit the study’s external
validity?
Does the environment in which the study
is run (e.g., laboratory) limit external
validity?
How far will your statistical analyses
allow you to generalization?
Copyright, 2005, Prentice Hall, Sarafino
Phase 7: Ethical Risks and
Debriefing of Participants
• Ask yourself the following questions: Will or
might your study:
– Physically or psychologically harm?
– Use deception? Violate privacy?
– Use coercion? Fail to use informed consent?
• If you answer yes to any of these – ask yourself
if they can be avoided, if not, be sure to include
measures in your debriefing to address and/or
minimize any possible lasting effects.
Copyright, 2005, Prentice Hall, Sarafino
Phase 8: Design An Informed
Consent Form
• Informed consent is usually required in
any study with human participants.
– Informed consent involves providing
participants with the relevant details of the
study, what they might experience, and that
they can withdraw from the experiment at any
time. Once participants are informed, then
their written consent is obtained to participant
in the research.
– See example in your textbook.
Copyright, 2005, Prentice Hall, Sarafino
Phase 9: Submit a Proposal for
IRB Approval
• IRB = Institutional Review Board
• The proposal often includes:
– Detailed descriptions of the purpose,
methods to be use, the expected results,
and the implications of the results.
• The previous 8 phases should provide
you will enough information to submit a
good proposal.
Copyright, 2005, Prentice Hall, Sarafino
Pilot or Trail Runs
• Once you have received all the
necessary approvals, it is time to run your
study.
• Many studies include a consolidation
phase to debug the study before the
actual data collection begins.
– This is often called a pilot study or a trail
run of the proposed research.
Copyright, 2005, Prentice Hall, Sarafino
Pilot Study
•
•
A pilot study is a trail run of your entire experiment with
participants – the participants in your pilot usually are
not included in the actual study.
The pilot study will help you:
–
–
–
•
Isolate unforeseen problems and fix them
Figure out how long the testing takes
Identify problems with test materials and/or variables.
Any of the above may lead to a redesign of the study, a
slight change in the operational definitions of variables,
or even an altering of the hypotheses.
Copyright, 2005, Prentice Hall, Sarafino
A Crucial Problem: Sampling
• After debugging your study in the pilot phase, it
is now time to collect data.
• One of the more important aspects you’ll need to
consider is how you will recruit and select your
participants. This is called sampling.
– Sampling, if done correctly, will enhance external
validity. But if done incorrectly will significantly
decrease your study’s external validity.
– The goal of sampling is to generate a representative
sample of your target population.
Copyright, 2005, Prentice Hall, Sarafino
Types of Sampling
• There are two main categories of
sampling:
–
Probability Sampling
•
–
The odds of being selected from a population
to a research study can be calculated.
Nonprobability Sampling
•
The odds of being selected from a population
to a research study cannot be calculated
Copyright, 2005, Prentice Hall, Sarafino
Probability Sampling
•
•
Probability samples almost always start with a
“sampling frame” – a list of all individuals from which the
sample will be selected. In a best scenario, all
members of the target population will be on the list.
There are several types of probability sampling:
–
Simple Random Sample
•
–
Stratified Random Sample
•
–
Each individual in the population has an equal chance of being
selected (e.g., names out of a hat).
The sample frame is partitioned in some manner and simple
random sampling is done on each partition. Why?
Cluster Sample
•
Units or clusters of similar individuals are identified. Some
clusters are randomly selected, and each individual in each of
the selected clusters is measured.
Copyright, 2005, Prentice Hall, Sarafino
Nonprobability Sample
•
In nonprobability sampling a sampling frame is not
present.
–
•
•
As a result, nonprobability samples may not be representative
of the target population.
The most common type of nonprobability sample (and
the most common in psychology) is called a
“convenience sample”
Convenience samples are qualified volunteers who are
easily available. You were likely part of a convenience
sample if you participated in research while enroll in
your first year Introduction to Psychology course.
–
–
If a convenience sample is used, what should researchers
report regarding the convenience sample?
How will a convenience sample affect external validity?
Copyright, 2005, Prentice Hall, Sarafino
Descriptive Statistics
• Descriptive stats provide information about the
central tendencies of a group of data.
• Importance terms include:
–
–
–
–
Mean: the arithmetic average in a data set
Median: the middlemost score in a data set
Mode: the most frequent score in a data set
Variance: the degree to which scores in a data set
deviate from the mean. There are various ways to
measure variance or variability.
•
•
Range: measure of variability in a data set
Standard Deviation: most commonly used measure of
variability.
Copyright, 2005, Prentice Hall, Sarafino
Inferential Statistics
• Inferential statistics are mathematical /
statistical procedures for determining the
probability that the relationships or differences
we observe in our data actually occur in the
population.
• Inferential statistics also tell us whether the
differences we see in our data occurred by
chance or not.
• There are 2 types of inferential statistics:
–
–
Parametric statistics
Nonparametric statistics
Copyright, 2005, Prentice Hall, Sarafino
Parametric and Nonparametric
Statistics
• Parametric Statistics test hypotheses that are
based on data that allow us to estimate
parameters (e.g., means and standard
deviations).
– In other words, parametric statistics are used with
interval or ratio data.
– E.g. Pearson r, multiple regression, t-test, ANOVA
• Nonparametric Statistics test hypotheses that
do not involve parameters (e.g., when the data
are nominal or ordinal, or not normally
distributed).
– E.g., Spearman rank correlation, Chi-square.
Copyright, 2005, Prentice Hall, Sarafino
Statistical Significance
•
What do we mean when we say something is
statistically significant?
–
–
We are simply saying that there is only a small probability that
what we found was due solely to chance.
That small “chance” probability goes by various names:
•
•
•
•
Type I Error
Alpha level
And is often symbolized by an italicized p “p”
The alpha level is predetermine before the study begins
and in psychology the level is usual set to 5% or 0.05.
–
This means that if the results we gather cannot be obtained by
chance more that 5 times in 100 random trials we would say
that our results are statistically significant. It is unlikely they
occurred solely by change. So can we ever be wrong?
Copyright, 2005, Prentice Hall, Sarafino