Transcript Registerkvalitet og prosjektet BLUE

1
Reflections about the Impact
Business Questionnaires have on
Perceived Response Burden and
Survey Quality
Gustav Haraldsen
Statistics Norway
[email protected]
1
“What we have found most striking when
reviewing these articles, is the apparent lack of
NSI efforts in actually measuring response
burden before and after the redesign processes”
Dag F. Gravem, Tora Löfgren and
Yngve Bergström
Statistic Norway
• An initiative
• QDET methods
• (Response burden indicators)
• Response rate or response qualitity change?
2
NSI’s reasons for reducing questionnaire
communication burdens
(Re)Design
Lower
Response burden
Higher quality
Lower cost
Nonresponse bias
Question elements
Cognitive burden
Design elements
Communication
mode
Response rate
Time
Time burden
Validity
Reliablility
3
Figure 2: The business survey respondent’s environment
Encode
Select
Prioritize
Communication mode
Questionnaire design
Motivate
Comprehend
Time burden
Wording
Task
Validity
No of questions
Retrieve
Low
Judge
No of minutes Low
Report
Release
Response
format
Reasonable
low
High Ineffective
High
Effective
Reliability
Reasonable
high
4
Some review results
• Changing from paper to web may rather increase than
reduce the response burden and the nonresponse.
• Web functionality may make some response tasks easier,
but still have little effect on the perceived response
burden.
• It’s an open question if changing from paper to web
reduces the time burden of filling in the questionnaire.
• Design improvements may lead to higher time and
response burden, but also to higher quality.
• Time burden affects perceived response burden and the
response rate, probably also response quality
5
How to proceed
• Be more specific about design elements, burdens and
quality improvements
• Run a three steps design
Response burden
+ Quality
QDET
Response burden
+ Quality
Thank you for your attention this far,
….. there is more to come
6
Perceived Response Burden and Sources of Response Burden. Structural Statistics 2010. Percent (n = 16 572)
Perceived Response Burden
Sources of Response Burden
Quite easy
29,3
Neither easy nor
difficult
Very easy
39,9
9,6
E-technical
2,4
5,8
3,7
E-Usability
Very difficult
15,5
Quite difficult
Other
5,7
Layout
9,8
No of questions
13,2
Wait for information
17,5
Terms
18,2
Calculations
25,2
Need of help
25,9
Mismatch
26,4
27,4
Response categories
0
20
40
60
80
100
7
Structural statistics, manufacturing industry.
Response burden and completion time 2004 and
2006
PRB -index
Paper 2004
One establishment
business
0.32
Multi establishment
business
- 0.08
Web 2006
0.18
- 0.15
Time
Paper 2004 Web 2006
79+26=
77+49=
105
126
306+148=
157+81=
454
288
8