Transcript Curriculum Alignment: A Concept Map

研究設計
Research Design
Emily Lin, PhD. (林永芬)
Department of Communication Disorders
University of Canterbury
Christchurch, New Zealand
Taiwan Academy of Physical Medicine and Rehabilitation Conference:
Current Intervention for Children with Developmental Delay
Taoyuan, Taiwan
December 2, 2006
Outline
• Basic Concepts
• Experimental Research
 Group Design
 Single-Subject Design
E. Lin
What is Research?
“Research is the process of investigating
scientific questions.”
To satisfy the need to:
1. explain events
2. solve practical problems
3. demonstrate certain effects
• legal, social, professional, and scientific considerations
(Hegde, 2003)
E. Lin
Research Process

Phase I: Identify the research question











define the research problem
review literature; provide theoretical framework
identify target population
identify variables
state research rationale
clarify objectives
state specific purposes or hypotheses
Phase II: Design the study (i.e., design protocol, select a sample)
Phase III: Methods (i.e., collect data, reduce data)
Phase IV: Data Analysis (i.e., analyze data, provide interpretation)
Phase V: Communication (i.e., report findings, suggest future
studies)
(Portney & Watkins, 2000)
E. Lin
Purpose of Research
To determine the relationship between
variables*
*Some basic terms:
1. constant (常數)
2. variable (變數):
-independent variable
-dependent variable
-extraneous variable
E. Lin
Types of Research
Descriptive
(Describe population)
e.g.,
• Case study
• Developmental
research
• Normative
research
• Qualitative
research
• Correlational
research
(Portney & Watkins, 2000)
Exploratory
(Find relationships)
Experimental
(Cause and effect)
e.g.,
• Experimental
(randomized
controlled)
• Quasiexperimental
• Sequential
clinical trial
• Single-subject
designs
E. Lin
What is an experiment (實驗)?
1.
2.
Manipulation of variables:
Independent variables are manipulated through:
-administration of treatment, or
-deliberate operation imposing predetermined
experimental conditions combined with
classification (Quasi-experiment)
Random selection/assignment:
1. Obtain a representative sample
2. Establish equivalency between comparison
groups
E. Lin
Selecting a Study Sample
Target population (reference population)
Accessible population (experimental population)
Subject selection
Participants (Study sample)
Non-participants
Group assignment
Experimental
group
Control group
(comparison groups)
(Portney & Watkins, 2000)
E. Lin
Sampling Strategy

Probability sampling:






Simple random sampling
Systematic sampling
Stratified random sampling
Disproportional sampling
Cluster sampling
Nonprobability sampling:




Convenience sampling
Quota sampling
Purposive sampling
Snowball sampling
(Portney & Watkins, 2000)
E. Lin
Compensations for Lack of
Random Sampling
Homogeneous groups
 Matching
 Control
Build in extraneous factors
 Blocking
E. Lin
Validity (效度)
•
Internal validity:
the degree a cause-and-effect inference can be made
based on the observed relationship between the
variables
“Measure what is claimed to be measured”
•
External validity:
generalization
“Generalize to other situations”
E. Lin
Threats to Internal Validity
Potential for confounding factors to interfere
with the relationship between the independent
and dependent variables:
e.g.,
•
History
•
Maturation
•
Mortality/attrition
•
Testing or test-practice effects
•
Statistical regression
•
Differential selection of subjects
•
Instrumentation
(Schiavetti & Metz, 1997)
E. Lin
Threats to External Validity
Factors limiting “the degree to which
internally valid results can be generalized”:
e.g.
•
•
•
•
Subject selection
Reactive or interactive effects of pretesting
Reactive arrangement (Hawthorne effect)
Multiple-treatment interference
(Schiavetti & Metz, 1997)
E. Lin
Measurement



Purpose: to provide a mechanism for achieving a
degree of precision in the understanding of the
characteristics of the object of interest
Key elements:
 Construct
 Rules
Evaluation of a measurement:
1. validity (效度): measuring what was intended
2. reliability (信度): yielding consistent results
E. Lin
Level of Measurement
Ratio
With absolute zero
Interval
Equal intervals
Ordinal
Nominal
Ranking
Category
E. Lin
Test Validity
1. Face validity: appears to test what is supposed to test.
2. Content validity: consists of items that adequately
sample the content that defines the variable being
measured.
3. Criterion-related validity: yields outcomes that can
be used as a substitute measure for an established gold
standard criterion test.
a.
b.
Concurrent validity
Predictive validity
4. Construct validity: the degree the test measures an
abstract construct
E. Lin
Threats to Test Validity
•
•
•
•
•
Length effect: e.g., fatigue, learning
Enabling behaviors required of the
test taker
The representativeness of the norm
Bias
Reliability: e.g., test-retest, inter-judge
E. Lin
Reliability
1. Test-retest reliability:
• Repeat the whole test or a portion of test
• Conduct a parallel test
• Split-half method (internal consistency)
2. Inter and intra-judge reliability:
•
•
•
•
Total reliability
Trial-by-trial (point-by-point) reliability
Occurrence reliability
Nonoccurrence reliability
E. Lin
Three Basic Measurements In
Descriptive Statistics
1. Central tendency: the average (“center”) score
of a distribution; mean, median, mode
2. Variability: the dispersion of scores;
range, standard deviation (SD)
3. Relative position: a score’s position within
a distribution; percentile, z-score
E. Lin
Inferential Statistics
Sample
Population
Known
Unknown
*Difference between subjective inference and statistical inference:
Statistical inference requires objective criteria to make decisions.
Inferential statistics:
• Decision-making process
• To estimate population characteristics from sample data
• Assumptions made about how well the sample represents
the larger population. The assumptions are based on
two concepts of statistical reasoning:
 Probability
 Sampling error
E. Lin
Hypothesis Testing
• Null hypothesis (statistical hypothesis; H0):
the group difference is due to sampling error
• Alternative hypothesis (research/scientific hypothesis;
H1): the research hypothesis is correct
The purpose of posing a research hypothesis:
usually with the intention to reject the null hypothesis
E. Lin
Hypothesis Testing: Type I and II errors
(Real Situation )
Reject H0
H0 is true
Type I error
(a)
H0 is false
Correct decision
(1-b)
(Power of test)
Accept H0
Correct decision
(1-a)
Type II error
(b)
(Decision)
a : significance level
Power (1-b): the probability that a test will produce a significant
difference at a given significance level
Type I error: The error that results when null hypothesis is falsely rejected.
Type II error: The error that results when null hypothesis is falsely accepted.
E. Lin
Hypothesis Testing
(Portney & Watkins, 2000)
E. Lin
Group Design
One-shot case study
One-group pretest-posttest design
Static-group comparisons
Pretest-posttest control group design
Posttest-only control group design
Solomon four-group design
Multigroup pretest-posttest design
Multigroup posttest-only design
E. Lin
Group Design -continued
Factorial designs
Single-group time-series design
Multiple-group time-series design
One-group single-treatment counter-balanced
design
Crossover design
Correlational analysis
E. Lin
Group Design: One-Way Design for Independent Groups
A1
S1
S2
S3
S4
S5
S6
S7
S8
S9
S10
A2
10
S11
S12
S13
S14
S15
S16
S17
S18
S19
S20
A3
10
S21
S22
S23
S24
S25
S26
S27
S28
S29
S30
10
5
4
Score
3
2
1
A1
A2
A3
Teaching method
Analysis Method: One-way ANOVA
E. Lin
Group Design: One-way Within-Subjects Design
A1
S1
S2
S3
S4
S5
S6
S7
S8
S9
S10
A2
10
S1
S2
S3
S4
S5
S6
S7
S8
S9
S10
A3
10
S1
S2
S3
S4
S5
S6
S7
S8
S9
S10
10
5
4
Score
3
2
1
A1
A2
A3
Teaching method
Analysis Method: One-way Repeated Measures ANOVA
E. Lin
Two-way Factorial Design
Factors:
-Factor A (teaching method): 2 levels (oral vs. visual)
-Factor B (gender): 2 levels (female vs. male)
This is a two-by-two (2X2) design.
A1
B1
B2
A2
S1
S2
S3
S4
S5
5
S11
S12
S13
S14
S15
5
S6
S7
S8
S9
S10
5
S16
S17
S18
S19
S20
5
E. Lin
Main Effect
Effect of Factor A (Teaching method)
A1
B1
S1
S2
S3
S4
S5
A2
5
S11
S12
S13
S14
S15
5
5
B2
S6
S7
S8
S9
S10
5
S16
S17
S18
S19
S20
5
4
Score 3
2
1
A1
A2
Teaching method
E. Lin
Main Effect
Effect of Factor B (Gender)
A1
B1
S1
S2
S3
S4
S5
B2
S6
S7
S8
S9
S10
A2
5
5
S11
S12
S13
S14
S15
S16
S17
S18
S19
S20
5
5
6
5
Score 4
3
2
1
B1
B2
Gender
E. Lin
Interaction Effect
Effect of Interaction between Factors A and B
A1
B1
B2
S1
S2
S3
S4
S5
S6
S7
S8
S9
S10
A2
5
5
S11
S12
S13
S14
S15
S16
S17
S18
S19
S20
5
B1 (Female)
B2 (Male)
5
7
6
5
Score 4
3
2
1
0
A1
(Oral)
A2
(Visual)
Teaching method
E. Lin
B1 (Female)
B2 (Male)
7
6
5
Score 4
3
2
1
0
A1
(Oral)
A2
(Visual)
Teaching method
No interaction
Interaction
E. Lin
Single-Subject Design
Synonym:
 Single-case design
 Single-system design
 Time series experimentation
Definition: an experimental design
involving the systematic collection of
repeated measurements of a behavioural
response over time, usually at frequent and
regular intervals
E. Lin
Single-Subject Design: Length of Phases
Two choices:
 Equal phase lengths: preset a short period
time to minimize maturation, motivational
changes over prolonged periods
 Unequal phase lengths: extend baseline or
intervention phases until stability is
achieved
E. Lin
Single-Subject Design: Structure

Repeated measurement

Two design phases:
1. Baseline phase: period prior to treatment
2. Intervention phase: Period during treatment.
(Portney & Watkins, 2000)
E. Lin
Single-Subject Design: Assumption
Baseline data reflect the ongoing effects of background
variables, such as daily activities, other treatments, and
personal characteristics, on the target behaviours.
Therefore, when treatment is initiated, changes from
baseline to the intervention phase should be attributable to
intervention.
(Portney & Watkins, 2000)
E. Lin
Single-Subject Design: Length of Phases
Why not 1 or 2 sessions only?

Stability: a minimum of 3 to 4 data points in each
phase (the greater the number of data points, the
more obvious trends will become)
Why not 100 sessions?

Efficiency: also to avoid maturation, history, and
other confounding factors.
(Portney & Watkins, 2000)
E. Lin
Single-Subject Design:
Measuring the Target Behaviour
 Frequency: the number of occurrence of
a certain behaviour within a fixed time
interval or a fixed number of trials
 Duration: how long the target behaviour
lasts
 Magnitude: some form of
instrumentation that provides a
quantitative score
E. Lin
Single-Subject Design
Why use it?
 Practical: fewer subjects are required.
 Emphasis on individual performance:
allowing for differentiation between
subjects who respond favourably to
treatment from those who are not affected
by treatment.
E. Lin
Types of Single-Subject Design
 Withdrawal design
 A-B-A design
 A-B-A-B design
 Multiple treatment designs
 A-B-C-B design
 Interactive design: A-B-BC-B-BC
 Alternating treatment design
 Multiple baseline designs
E. Lin
Withdrawal Design: A-B-A Design
(Portney & Watkins, 2000)
E. Lin
Withdrawal Design: A-B-A-B Design
(Portney & Watkins, 2000)
E. Lin
Multiple Treatment Design:
A-B-C-B Design
(Portney & Watkins, 2000)
E. Lin
Interactive Design:
A-B-BC-B-BC Design
(Portney & Watkins, 2000)
E. Lin
Alternating Treatment Design
(Portney & Watkins, 2000)
E. Lin
Multiple Baseline Designs
1. Multiple baseline design across subjects
2. Multiple baseline design across conditions
3. Multiple baseline design across behaviours
(Portney & Watkins, 2000)
E. Lin
Split-Half
Method
(Portney & Watkins, 2000)
E. Lin
Split-Half Method (continued)
(Portney & Watkins, 2000)
E. Lin
Two Standard Deviation Method
(Portney & Watkins, 2000)
E. Lin
References
Hegde, M. N. (2003). Clinical Research in Communicative Disorders:
Principles and Strategies (3rd Edition). Austin, TX: Pro-ed.
Jadad, A. (1998). Randomised Controlled Trials. London: BMJ Books.
Portney, L. G. & Watkins, M. P. (2000). Foundations of Clinical
Research: Application to Practice (2nd Edition). Upper Saddle
River, NJ: Prentice Hall Health.
Schiavetti N., & Metz, D. (2002). Evaluating Research in Communicative
Disorders (4th Edition). Sydney: Allyn & Bacon.
E. Lin