Transcript Microsoft PowerPoint - NCRM EPrints Repository

+
Mixing science and intuition: the process of synthesising data
from a longitudinal mixed methods study of volunteering
Rose Lindsey and Liz Metcalfe, University of Southampton
Third Sector Research Centre
ESRC grant no. ES/K003550/1
+
Presentation aim

To explore the challenges encountered when combining
longitudinal qualitative and quantitative secondary data to study
volunteering across time

Key challenge: What do we mean when we talk about
synthesising, integration, combining, mixing, interweaving,
blending, merging…? (Bryman, 2008)

Do we think our methods of combining have actually worked?
+
Presentation outline
Part 1: Designing the project
Introduction to Continuity and
Change project
Discussion of
the methodological challenges faced within the
mixed-method research design
 Bringing
different data sources and findings into dialogue
Part 2: Challenges in practice
Exploring
the analytical challenges faced when putting the
design into practice:
 Working
across methodological paradigms
 Understanding

the effect of, and working across, time
Design versus practice
+
Part 1: The Continuity and Change
Project

Aim: To explore individual attitudes and behaviours
towards volunteering, and individual views on the role and
responsibility of the state towards provision for social
need, across a period of thirty years.

Design: Concurrent use of longitudinal mixed-methods to
analyse secondary data

Time-frame: 1981-2012, encompassing different periods of
economic adversity and prosperity

Project website:
http://longitudinalvolunteering.wordpress.com
+
Choice of secondary data sets
Qualitative data
 The Mass Observation Project



Aim: to capture experiences,
thoughts and opinions of
individuals
A national panel of volunteers
writing in response to themed
questions or ‘directives’ (1981 to
present day)
Longitudinal data following the
same people through thirty years of
their life-course
Quantitative data
 British Household Panel
Survey/Understanding Society

Aim: to understand individuals’
and households’ social and
economic change

A national panel of the British
population and volunteers (1991 to
2012)

Longitudinal data
 British Social Attitudes Survey

Aim: to track people‘s changing
social, political and moral
attitudes

A national survey of the British
population (1983 to 2012)

Cross-sectional data
+Why did we use mixed-methods?
Enhancing strengths and offsetting weaknesses
Our research design aimed to potentially ‘offset’ the respective weaknesses
of these two analytical methodologies by taking advantage of their joint
strengths to provide a ‘complete[ness]’, and ‘comprehensive’ picture
(Bryman, 2008, p.91)
Study strengths
Qualitative
Study weaknesses
•Provides depth and nuance relating to the
• Not representative of the
complex reasons why people behave in a certain population
way, or hold particular viewpoints
•Too much data
•Offers potential insights into how and why
perspectives change or continue over time
• Enables insights into the connection between
the life-course and routes into volunteering
Quantitative •Representative of volunteers within the
population
•Can formally test how volunteering behaviour
and attitudes change over time
•Offers potential insights into
contextual re external events affect on change
or continuity over time
•Insight into motivations and
barriers are limited
•Limitations re understanding
how individual time and the
life-course affect volunteering
Focus on individuals increases
Multi-layered picture of volunteering
behaviour
Sample size decreases
+
In-depth
Contextual:
analysis
social, of
individual
economic
and
political
events
volunteers
over time
Behaviour and attitude
analysis for volunteers within
the population
In-depth analysis
of individual
Contextual:
social, economic
and
volunteers
political
events over time
+ Bringing secondary data sources,
analyses and findings into dialogue
We aimed for three types of mixed-method dialogue:
across the lifetime of the project, described by Tashakkori and Teddlie
(2008, p.104) as a ‘continuous feedback loop’, to enable an iterative research
process;
some direct comparisons between qualitative and quantitative analyses
where there was a fit between the data;
combining substantive findings so that the sum of our joint knowledge
claims would be greater than our individual findings
Qualitative data
Substantive
findings
Quantitative data
Project beginning
Project end
+
Study design challenges-sample fit
British Household Panel
Survey/Understanding Society
The Mass Observation Project

The Mass Observation Project
(MOP)

15 directives (sets of questions)
were selected

N=38

2 samples were taken to provide
a range of ages and occupations

Sample 1, n=20 were writers
from 1981 to 2012

Sample 2, n=18 were
younger and wrote for
shorter periods of time

Sample restricted to available
volunteering data (every other
year between 1996 to 2011)

N=2067
British Social Attitudes Survey

Questions of volunteering
only asked a limited number
of times

Number of people each year
mean (sd) 3393 (711.7)
+
How the three datasets complement
each other, temporally and thematically
+
Part 2: Challenges in practice
Three main challenges were present throughout the project:
Working
across methodological paradigms
Understanding
Putting
the effect of, and working across time
the design into practice
+
Working and communicating across
methodological paradigms
Working across methodologies we encountered some challenges:

Differences in terminology

Forming definitions

Timings/speed of analysis

Methodological standpoints: differences in the
types of questions that are being addressed

Conceptions of time
+
How time fits together
The way that these multiple perceptions of time interact and intersect (or
not) was at the heart of the mixed methods effort for our research project.

the flow of personal biographical, narrative,
retrospective, life-course, individual time

chronological time, moving from one year to the next

contextual public/collective time related to chronological
Hi, I’m
Sarah
time
Children
Marriage
1981
Recession
1984
1987
1990
1993
Recession
Retirement
1996
1999
2002
2005
2008
2011 2013
Double-dip recession
Focus on individuals increases
Multi-layered picture of volunteering
behaviour
Sample size decreases
+
Biographical time
In-depth
volunteer analysis
Behaviour and attitude
analysis for volunteers within
the population
Chronological time
Changes in social, political and moral
attitudes over time
Contextual time
+ Design versus practice

Longitudinal mixed-methods are more complicated than a single
methodological approach

Over-estimation of mixed methods: It has not been possible to
answer all of the designed research questions with the data chosen, the
fit of the samples and the timing of the analysis

Did we achieve our mixed method dialogue?

Paradigm, background, and terminology differences make maintaining a
mixed-method dialogue difficult

How time fits together: in practice time does not relate directly
between different methodologies

Has the project benefited from using mixed-methods?
+
References

Bryman, C. (2008) ‘Why do Researchers
Integrate/Mesh/Blend/Mix/Merge/Fuse Quantitative and Qualitative
research?’, in M.M. Bergman (ed.) Advances in Mixed-Methods Research,
London: Sage. pp 87-100.

Tashakkori, A. and Teddlie, C. B. (2008) Quality of Inferences in Mixed
Methods Research: Calling for an Integrative Framework in in M.M. Bergman
(ed.) Advances in Mixed-Methods Research, London: Sage. pp.101-119
+
Thank you for listening, any
questions?
Contact details:
 [email protected]
 [email protected]
Project website:
http://longitudinalvolunteering.wordpress.com/
+
Back-up slides
+
The challenges of analysing
secondary data over time
Quantitative:
Variations in data collection process were difficult to uncover
The questions that were asked limits the data available
Data collected is set within the present time, only part of the lifecourse is recorded
Qualitative:
Inconsistent descriptions of the life-course at different timepoints
Lack of awareness of the what is happening within the present
time
Accuracy of
retrospective writings
+
Study design challenges
The mixed-method design framed the study, and influenced how
well the data sources fitted together. Compromises around the
following choices needed to be made:
 Choice of
secondary data sources
 Choice of
timing of analyses
 Choice of
samples and how these substantively fit
together
 Choice of
samples and how these fit together across
time (thematic and temporal bunching)
+
Concurrent mixed method design
Our research design aimed to potentially ‘offset’ the respective weaknesses of
these two analytical methodologies by taking advantage of their joint
strengths to provide a ‘complete[ness]’, and ‘comprehensive’ picture (Bryman,
2008, p.91)
Qualitative data
Project
beginning
Project end
Quantitative data
+ Cross-sectional or Longitudinal?
Synchronic or Diachronic?
The length of chronological time being researched affects our
perceptions and understandings of behaviour and attitudes
Hi, I’m
Sarah

Longitudinal/diachronic: following a person through time

Cross-sectional/synchronic: A certain point in time
Children
Marriage
1981
Recession
1984
1987
1990
1993
Retirement
1996
Recession
Volunteered
Do not
volunteer
Increasing age
1999
2002
2005
2008
2011
2013
Double-dip recession