Transcript ixed methods 2012

Is it good to mix methods?
Martyn Hammersley
The Open University
What is mixed methods?
Here is one definition:
‘Mixed methods’ is a term increasingly
used in social science to describe ‘the
class of research where the researcher
mixes or combines quantitative and
qualitative research techniques,
methods, approaches, concepts or
language into a single study’
(Jonson and Onwuegbuzie 2004:17)
Precursors of mixed method
• Anthropologists doing ethnography but also
drawing on village surveys, for example to
document variation in attitude towards
schooling within a community.
• Sociologists doing in-depth case studies but
also collecting and analysing quantitative
data, for example Hargreaves 1967 and
Lacey 1970.
• Survey researchers doing qualitative pilot
work, experimentalists using qualitative
debriefing interviews.
It is possible to ‘combine methods’
without mixing quantitative with
qualitative
• Using relatively unstructured observational
and interview data in the same study.
• Collecting questionnaire data and combining
these with data from official statistics.
• Using factor analysis and regression analysis
within the same study.
• Collecting data through ethnographic means
but subjecting it to discourse analysis
A
Stricter
Definition
of ‘Mixed
Methods’
Triangulation
• This term may, but need not, imply ‘mixing
methods’.
• Originally, it referred to checking validity
through using data having different threats to
validity, and relied upon an analogy with
navigation and surveying.
• Later ‘triangulation’ came to be used to mean
‘using different kinds of data to provide
information about different aspects of the
same phenomenon’ (see Hammersley 2008).
The mixed methods movement
• A response to the paradigm wars of the
1980s and 90s: recognition of the value
of both qualitative and quantitative
methods.
• A pragmatic attitude: the ‘fit for purpose’
and ‘best of both’ rationales.
• Is it a new paradigm? Is this the best
way to think of it?
Paradigm or pragmatism, or
Pragmatism as a paradigm?
• Maxwell and Mittapalli (2010) recommend
realism as the guiding methodological
philosophy
• Mertens et al (2010) propose the
‘transformative paradigm’
• Johnson and Onwuegbuzie (2004) and
others recommend Pragmatism as an
underpinning philosophy for mixed methods
research.
A rather different view
‘Mixed methods research is a Trojan
Horse for positivism, reinstalling it as the
most respected form of social research,
while at the same time — through
inclusion — neutralizing the oppositional
potential of other paradigms and
methodologies that more commonly use
qualitative methods’ (Giddings and Grant
2006:59).
The value of pragmatism
• Research is a practical activity aimed at
a particular goal and product: producing
answers to a set of research questions.
• Given this, how researchers go about
their work should be determined by
what is required to answer the research
questions they are addressing.
Not so simple!
• Even the formulation of research questions is
based upon assumptions about the nature of
the phenomena concerned and how they can
be understood.
• There are discrepant assumptions about the
social world and how it can be understood
built into different data analysis strategies.
• Much discussion of mixing methods
underplays these differences in
methodological philosophy.
Can we not just discover how the
world is through our research?
• To some extent yes.
• But we cannot avoid making
epistemological and ontological
assumptions, at least for working
purposes.
• And when it comes to what are for us
deep or central assumptions about the
world, there can be no direct test of these.
The unavoidability, but also danger,
of relying upon ‘commonsense’
• It is impossible to question or avoid all
assumptions. All research relies upon some.
• But we must try to avoid being misled by what
seems to us to be ‘commonsense’.
• Research demands that we be critically
cautious. But this cannot amount to a
thoroughgoing scepticism, otherwise…
• So, what should we assume, and what should
we doubt? In part, this is what qualitative and
quantitative researchers disagree about.
The case of eliciting accounts
Criticisms of the structured questionnaire by
qualitative researchers:
1.Assumptions built into the structure might be
false, or might apply to some informants but
not to others, introducing systematic error;
2.We will probably fail to recognise when
people are uncertain or ambivalent;
3.It is false to assume that ‘standardising the
stimuli’ across respondents/informants makes
their responses comparable.
A Questionnaire Item
Tick ONE BOX ONLY for
each question
•Is your research:
Positivist?
•
Anti-positivist?
[ ]
[ ]
Is it aimed at:
Producing knowledge?
[ ]
Improving practice?
[ ]
Getting an EdD?
[ ]
The ‘radical critique’ of interviews
• Interview data are co-constructed through the
interaction between interviewer and
interviewee: responses do not simply reflect the
attitudes of informants.
• Answers given also depend upon the sociocultural resources that the informant has
available to draw on. Different resources will
produce different answers.
• People’s behaviour is not a product of stable
beliefs, attitudes etc, or of fixed causes
operating upon them, but is context-dependent.
The uncertain relationship between
methodology and method
• Use of structured questionnaires may be
premissed upon positivist assumptions, but it
need not be.
• Use of relatively unstructured interviews may
rely upon interpretivist philosophy but need not.
• The radical critique of interviews is based upon
constructionism, and this does imply a respecification of the research goal. However, the
points the critique makes can be recognised up
to a point by both positivists and interpretivists.
A paradigmatic chasm?
• We don’t have to accept these methodological
philosophies (positivism, interpretivism,
constructionism) at face value. In fact, each has
stronger and weaker versions.
• However, integrating these philosophies in
some way, or adopting a different one, as the
rationale for using ‘whatever methods are
appropriate’ does not overcome the conflicts.
• Some philosophical assumptions have to be
made in all research, and it is important to be
aware of these.
Should all research mix methods?
• My answer to this question is definitely ‘No’.
• But if you ask me whether research should be
designed so as best to answer its research
questions I would say yes, and this may well
involve combining quantitative and qualitative
methods.
• At the same time, it’s necessary to remember
that how we frame research questions, what
inferences we draw from data, etc can rely
upon contentious philosophical assumptions.
A final point
‘The process of mixing requires distinct method
elements to mix and so, ironically, the
metaphor of mixing actually works to preserve
method schisms […]’ (Gorard 2007 p.1).
Do qualitative and quantitative approaches
represent two absolutely distinct sets of
methods? No, these terms indicate, and at the
same time obscure, a much more complex
field of decisions as regards research design,
data collection, and analysis. But this is
another story!
References
Giddings, L. S., & Grant, B. M. (2006). Mixed-methods research, positivism dressed in drag?
Journal of Research in Nursing, 11(3), 195-203.
Gorard, S. (2007). Mixing methods is wrong: An everyday approach to educational justice. Paper
presented at the British Educational Research Association
Hammersley, M. (2008) ‘Troubles with triangulation’, in Bergman, M. (ed.) Advances in Mixed
Methods Research, London, Sage.
Hargreaves, D. (1967) Social Relations in a Secondary School, London, Routledge and Kegan
Paul.
Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed Methods Research: A Research Paradigm
Whose Time Has Come. Educational Researcher, 33(7), 14-26.
Lacey, C. Hightown Grammar, Manchester, Manchester University Press
Maxwell, J. and Mittapalli, K. (2010). Realism as a stance for mixed methods research. In A.
Tashakkori & C. Teddlie (Eds.), Handbook of Mixed Methods in Social and Behavioral
Research. Second edition. Thousand Oaks, CA: Sage Publications.
Mertens, D., Bledsoe, K., Sullivan, M., and Wilson, A. (2010) ‘Utilization of mixed methods for
transformative purposes’. In A. Tashakkori & C. Teddlie (Eds.), Handbook of Mixed Methods in
Social and Behavioral Research. Second edition. Thousand Oaks, CA: Sage Publications.
On the radical critique of interviews, see Hammersley, M. (2008) Questioning Qualitative Inquiry,
London, Sage, Chapter 5.