Transcript Workshop report

A1: Surveys for Behavioural
Experiments
Peter Jones, CTS, UCL
Regine Gerike, TUD
Giorgia Servente, Polito, TO
Workshop Context
• Many countries are considering using VBC
programmes to influence (travel) behaviour
• How should such programmes be
monitored and evaluated?
• Are there methodological problems unique
to VBC?
• Are there best practice recommendations?
• What are the methodological research
challenges?
SURVEY DESIGN PROCESS
Existing Travel and
Consumption Patterns
** Need for Change:
Policy OBJECTIVES?
Voluntary Behaviour
Change Intervention
POST Travel and
Consumption Patterns
SURVEY DESIGN PROCESS
Existing Travel and
Consumption Patterns
** Need for Change:
Policy OBJECTIVES?
CONTROL
Group
Voluntary Behaviour
Change Intervention
POST Travel and
Consumption Patterns
What were the
IMPACTS of the
intervention?
- On Subjects
- On Others
SURVEY DESIGN PROCESS
Existing Travel and
Consumption Patterns
** Need for Change:
Policy OBJECTIVES?
CONTROL
Group
Voluntary Behaviour
Change Intervention
WHAT Led to
the Observed
Changes?
= PROCESS
POST Travel and
Consumption Patterns
What were the
IMPACTS of the
intervention?
-On Subjects
- On Others
Key Issues - 1
What are we trying to measure?
– What are our objectives? (reduce car use,
save energy, increase physical activity…)
– What about secondary impacts? (local
economy, traffic accidents…)
– Are we seeking just to measure change or
understand reasons for people to/not to
change?
Key Issues - 2
Ways of measuring behaviour:
– Take up of marketing information
– Reported behaviour (trips, durations, etc)
– Direct individual consumption measures
(odometers, GPS, etc) – reported or relayed
– Aggregate area measures (bus passengers,
traffic, etc)
Key Issues - 3
How to identify ‘real’ change:
– Need for ‘control’ groups – difficult!
– Potential biases in reporting, especially with
VBC programmes
 Retrospective vs. pre-determined
 Key importance of question framing
 Greater risks where respondents associate
survey with VBC programme
– Dangers of using panel surveys
Key Issues - 4
How to assess statistical significance:
– Systematic vs. random errors
– What level of significance do we really
need?
– Large samples are expensive to collect
 Possible scope for meta analysis?
– Importance of % response rate:
 Higher % => lower mean trip rate
Key Issues - 5
When should impacts be measured?
– Allow time for effects to settle down (e.g.
after expiry of free bus tickets)
– Match ‘before’ and ‘after’ seasonally
How long do impacts last?
– May increase over time (snowball effect)..?
Key Issues - 6
How can we understand causes of changes
and willingness to change?
– Trips as multi-dimensional packages
– Need to ground in psychological theory (e.g.
Theory of Planned Behaviour)
– Important role for qualitative research
…..All of which can help inform best type of VBC
interventions
Some Key Research Areas
•
•
•
•
•
•
•
•
Accuracy/role of ‘meter’ readings
How best to integrate data sources
Defining effective control groups
Cultural effects (survey and behaviour)
Deeper understanding of change processes
Importance of ‘interviewer’ effects?
Greater use of experimentation (reduced need for
precision in evaluating practical applications?)
‘Tool kit’ for VBS evaluation: low cost options for
developing countries