Transcript Revisions for a sceptical audience

Revisions for a sceptical
audience
David Marder, ONS, UK
George Orwell, Nineteen Eighty-Four
“But actually, he thought, as he
re-adjusted the Ministry of Plenty’s
figures, it was not even forgery. It was
merely the substitution of one piece of
nonsense for another . . .”
The statistical trade-off
Stan the Statistician and his juggling act
A PR problem
Financial Times, 3 July 2006
Media put the boot in
Stan the Statistician gets a kick from the sceptical media
Public view of official statistics
Len Cook
• Users naively expect statistics to remain comparable over
time in a fast changing world
• In public policy, there is rarely the luxury of waiting until the
final statistical measure is available
• There are major policy risks in delaying key decisions
because timely information is not available
• Challenge for statisticians – not only what we measure is
changing but so are sources, systems standards and the
methods we use to obtain raw data
Three types of revisions
• Unplanned and regular – for example, regular
revisions that fall outside the expected range
• Planned and unusual – such as the introduction of
methodological changes
• Unplanned and unusual – including errors. These
inevitably get the media attention
Statisticians’ ‘motto’
• Clarity
• Communication
National Statistics Revisions Protocol
Statement of principle
Substantial methodological changes will be
announced before the release of statistics
based on the new methods
Additional principles
• A general statement describing its practice on
revisions from each department
• Key outputs subject to scheduled revisions will
have a published policy covering those revisions
• A statement explaining the effect of revisions will
accompany the release of all key outputs subject to
scheduled revisions
• Revisions comply with the same principles as other
new information
Additional principles
• Timeliness balanced against the need to avoid
frequent revisions
• Minimal unexpected revisions but, if they occur,
they will be released as soon as practicable and in
an open and transparent manner
• Substantial revisions accompanied by explanation
of nature and extent
• Long-term effects of revisions on key outputs
monitored to improve quality
Code of Practice – Statement on revisions
Recognising the challenge
• Communication
– Likelihood
– Reasons
• The ‘expert’ filter
– City analysts
– Media correspondents
Communication innovations
• More information on average past revisions and
planned future revisions
• Economic statistics forum
• Horizon scanning
Other lessons
• Proactive explanation
• Don’t ‘oversell’
• Think about the language
– New information = updates
– Methodological changes = improvements
– Errors = corrections
• Be less defensive
• Promote statistical literacy
Happy Stan