Transcript Appropriate Designs

Johan Mouton© February 2006
C Hart 2007
Exploratory questions
What are the most important variable
that have an effect on learner
achievement?
What happens in the classroom i.t.o.
teaching pedagogy?
What conceptions do different educators
have about “authentic pedagogy”?
Appropriate Designs
Exploratory surveys
In-depth case studies
Ethnographic studies in the school


Descriptive/Correlational Questions
Appropriate Designs
Bivariate: Do learners in small classes
do better than children in large classes
Multivariate: Are children from singleparent families & rural areas less likely to
perform well than…?
Cross-sectional surveys
Cross-sectional surveys


Predictive Questions
Appropriate Designs

What would the effect of x be on y?

Experiments & Field Experiments
Causal questions
What is the differential effect of parental
SES on children in large/small classes &
within rural & urban schools?
Which causal/multiple regression model
best explains differences in learner
performance?
How does an effective teacher
produce/achieve situated pr0blemsolving in the classroom?
Appropriate Designs
Analytical surveys
(theoretically driven surveys + statistical
modeling)
Experimental & field-experimental
studies
In-depth case studies incl. Extensive
participant observation


Evaluative Questions (outcome /
Appropriate Designs
impact Questions)
Will a change in teaching training
method (intervention) produce more
effective teaching (outcome)?
Multivariate
Evaluation studies
(comprehensive evaluations/quasiexperimental designs)



The what’ of your study – what object, phenomenon,
entity, process or event
Unit of analysis typically also the unit of observation
Unit of analysis – things we examine to construct
summary description of all such units
Must be clear to what your unit of analysis is
Common social science units of analysis:
1.
2.
3.
4.
5.
6.
Individuals
Groups
Organizations & Institutions
Social artifacts/cultural objects
Social actions
Interventions



Characteristics
1.
•
•
•
Individuals - Gender, age, height, marital status, deformities,
region of origin, or hearing ability
Groups & Organisations – size, structure, location & aggregated
descriptions of their members
Artifacts – size, colour, weight, characteristics of people
associating with them – social events; where they occur what type
of people involved, etc.
Orientations
2.
•
•
Individuals – attitudes, beliefs, values, personality traits,
prejudices, religious, political, intellectual, superstitious
Social Groups – purposes, policies, regulations/procedures
Actions
3.
•
Observe directly or second-hand accounts of actions i.e. voting



Rationale for research design: Plan & structure
research project for validity of findings through
minimizing and/or eliminating potential error
Validity = ‘best approximation to the truth’
Key: How does one recognize valid research?
1.
2.

Id key dimensions of validity
Argue maximizing validity if by minimizing/
eliminating foreseeable threats to validity
Validity to be applied to entire research process
Stage in research
process
Sources of error
Methodological
‘move’ / ‘strategy’
(objective research
Outcome/goal/endproduct
Epistemic (validityrelated quality or
criterion
Conceptualization
(conceptual analysis)
Complex notions,
Vagueness, Ambiguity,
Abstract concepts
Thorough literature
review,
Clear & logical
definitions.
Concepts/definitions
Theoretical validity
(clarity/scope)
Operationalisation
Poor sampling of
items, Leading
questions, Scaling
errors
Scale validation,
Face Validity,
Pilot test.
Measuring instruments
Measurement validity
(construct validity)
Sampling
Bias, Heterogeneous
populations,
Incomplete sampling
frame
Probability sampling
Stratification,
Optimal sample size
Sample
Representativeness
Data collections
Observation effects,
Interviewer bias,
Respondent bias,
Context effects
Multi-method
Proper training of
fieldworkers
Data sets
Reliability
Analysis/interpretation
Competing/rival
conclusions or
explanations
Appropriate techniques
of analysis
Thorough
understanding of
literature
Conclusion/results/
findings
Inferential
(conclusions) validity
1.
2.
3.
4.
5.
Theoretical validity
Measurement validity
Representativeness
Reliability &
Inferential validity
1.
2.
3.
4.
5.
Conceptual vagueness
Measurement error
Biased samples
Unreliable data &
Invalid conclusions



Key validity criterion for data collection is
‘reliability’
“Will the same method as used by different
researchers and/or at different times produce the
same results? Smith, 1975:58)
Reliability of data is effected by:
1.
2.
3.
4.
Researchers, experimenters, interviewers/ observers
Individuals/subjects participating in the research
Measuring instruments – questionnaires, scales, tests,
interviewing & observation schedules
Research context/circumstances under which the
research was conducted
4. Research
Context
1.1 Researcher
1.2 Researcher
characteristics
orientations
(gender, nationality, age,
socio-economic status,
educational level)
(attitudes, values,
options, preferences
1. Researcher
3. Measuring Instrument
(questionnaire, interview, schedule, observation,
categories, scale, index)
2. Participant
2.1 Participant
characteristics
(gender, nationality,
age, educational,
level, income level)
2.2 Participant
orientations
(attitudes,
preferences, opinions