Transcript Relative size and predictability of revisions to GDP, Industrial

Relative size and predictability of
revisions to GDP, Industrial Production
and Retail Trade – a comparative
analysis across OECD Member
countries
Richard McKenzie OECD
Montreal, 5-6 Oct 2007
Workshop on Macroeconomic Forecasting,
Analysis and Policy with Data Revision
1
The OECD Real-Time Data and
Revisions Analysis Database

Full time series of data published every month starting from the
February 1999 edition of the Main Economic Indicators for 21
key economic variables

Access OECD revisions analysis studies for GDP, Index of
Industrial Production and Retail Trade Volume

Automated programs and detailed user guide allowing users to
perform their own revisions analysis for any country / variable
combination available in the database

Enables economists to test the performance of their econometric
models in simulated real-time (out-of-sample testing using
original first release data)

Other variables often used in econometric models that are not
revised (e.g. financial variables, exchange rates) are available in
a parallel interface
2
The OECD Real-Time Data and
Revisions Analysis Database

Project originated from needs of Euro Area Business
Cycle network

Survey of potential users from Central Banks on
variables to include

Built the database by loading old CD ROMS into an
SQL database
3
http://stats.oecd.org/mei/default.asp?rev=1
4
5
6
7
8
9
Zurich, 23-24 July 2007
Workshop on Real Time Data Analysis
10
11
Promotion of the interface

Two OECD Statistics newsletter articles
and Statistics Brief

OECD Statistics working paper
– First paper in a series, followed by additional papers for Real
Time Data Workshop in Zurich, submitted to the journal of
Business Cycle Measurement and Analysis, and now this
workshop

Emails to working groups of statisticians, academics,
central banks and economists through various networks
12
Promotion of the interface

Letter to heads of NSOs encouraging them to use the
facility to perform their own revisions analysis
– Related to promotion through the OECD Short-Term Economic
Statistics Working Party

Integrated fully with statistics portal on OECD website
– Download statistics show it is one of the most accessed
databases in the OECD (between 6000 – 9000 views per
month)

Updating is part of the monthly MEI process, automated
procedure run by the IT area
13
Future developments

Include output gap from OECD Economic Outlook

Ensure ongoing periodic review

Possibly look to expand variable list or integrate with
other sources

Hopefully IT performance and stability will continue to
improve (enabling us to load metadata for each vintage)
14
What about the quality?

Comparison with Philadelphia Federal Reserve real time
database for United States
– Quick evaluation of GDP constant prices
– OECD vintages start from February 1999 whereas the Fed
start in 1965
– OECD vintage time series back to 1960, Fed to 1947
– OECD extracts at the beginning of the month, every
month, Fed is middle of the quarter
– OECD in whole numbers, Fed in $Billion

3 vintages chosen randomly (Nov 01, May 03, Aug 06)
15
Analysis of revisions for short-term
economic statistics

Quick overview of terminology

Purpose of revisions analysis
– From both a user and producer of statistics perspective

Results from detailed analysis of GDP, Industrial
production and Retail trade
16
Terminology
Pt
Earlier estimate – often first published data
Lt
Later estimate, many time intervals can be considered
Rt  Lt  Pt
Revision, observable n times
Mean absolute revision:
1 n
1 n
MAR   Lt  Pt   Rt
n t 1
n t 1
n
Relative mean
absolute revision:
RMAR 
n
 L P  R
t
t 1
t
n
L
t 1
t

t 1
n
t
L
t 1
t
Terminology
Mean revision:
1 n
1 n
R    Lt  Pt    Rt
n t 1
n t 1
1  2 3
2

var( R) 
 ˆt   ˆt ˆt 1   ˆt ˆt 2 
n(n  1)  t 1
4 t 2
3 t 3


n
ˆt
n

=
Rt  R
n
Terminology / references

All revisions analysis is done for growth rates:
– Month-on-previous-month (MoM): (Mt/Mt-1) -1
quarter-on-previous quarter (QoQ): (Qt/Q t-1) - 1
or
– Year-on-year (YoY): (Mt/Mt-12) -1 or (Qt/Qt-4) -1

OECD approach is built on initial work done by Di
Fonzo (2005) and UK Office for National Statistics.

Other key references for revisions analysis include
Mankiw and Shapiro (1986) (new vs noise) , Rao et.
al. (1989)
19
Purpose of revision analysis

Users (policy makers, analysts, forecasters etc.)
– Robustness of first published data
– Evidence of bias
– Expected size of revisions over different time intervals
(is this changing over time?)

Producers (Statistics offices)
– Indicator of quality and reliability
– Diagnostic tool to improve compilation processes
20
OECD Revision Analyses

GDP (constant prices)
– Monthly vintages from May 1995 to June 2007
– 18 OECD countries

Index of Industrial Production
– Monthly vintages from Feb 1999 to Feb 2006
– 25 OECD countries, Brazil, India, South Africa

Retail Trade Volume
– Monthly vintages from Feb 1999 to April 2006
– 24 OECD countries and South Africa
21
Mean absolute revision to first published QoQ growth rates for GDP
1.0
1 year later
2 years later
3 years later
MAR ( % )
0.8
0.6
0.4
0.2
0.0
AUS
BEL
CAN
CHE
DEU
DNK
ESP
FIN
FRA
GBR
ITA
JPN
KOR
NLD
NOR
NZL
PRT
USA
RMAR to first published QoQ growth rates for GDP
1.0
1 year later
2 years later
3 years later
0.8
RMAR
0.6
0.4
0.2
0.0
AUS
BEL
CAN
CHE
DEU
DNK
ESP
FIN
FRA
GBR
ITA
JPN
KOR
NLD
NOR
NZL
PRT
USA
Robustness of first published growth rates

First published estimate of GDP QoQ growth rate:
– most comprehensive indicator of the current
performance of a countries’ economy

First published estimate of IIP MoM growth rate
– early indicator of the current state of the business
cycle, expansion or contraction in production activity

First published estimate of RTV MoM growth rate
– early indicator of current consumer demand
24
Robustness of first published growth rates

Relative mean absolute revision (RMAR) from
revision analysis can help us assess robustness

RMAR to first published MoM or QoQ growth rate
assessed on revisions after 1 year
– Expected proportion of the first published growth rate
that will be revised within one year
– If greater than say 0.5, should policy makers / analysts
really base decisions on these first published growth
rates?
25
RMAR to first published data after one year: QoQ vs YoY
growth rates for GDP
0.8
Quarter on quarter
Year on Year
RMAR
0.6
0.4
0.2
0
AUS
BEL
CAN
CHE
DEU
DNK
ESP
FIN
FRA
GBR
ITA
JPN
KOR
NLD
NOR
NZL
PRT
USA
RMAR to first published data after one year: MoM vs YoY
growth rates for IIP
1.4
1.2
Month on Month
Year on Year
1
RMAR
0.8
0.6
0.4
0.2
0
AUS BEL BRA CAN CHE CZE DEU DNK EMU ESP FIN FRA GBR GRC HUN
RMAR to first published data after one year: MoM vs YoY
growth rates for IIP (cont ….)
1.4
Month on Month
Year on Year
1.2
RMAR
1
0.8
0.6
0.4
0.2
0
IND
ITA
JPN KOR MEX NLD NOR NZL POL PRT SWE TUR USA ZAF
RMAR to first published data after one year: MoM vs YoY
growth rates for RTV
1.4
Month on Month
Year on Year
1.2
1.0
0.6
0.4
0.2
F
ZA
D
NO
R
NZ
L
PO
L
PR
T
SW
E
US
A
NL
JP
N
KO
R
M
EX
IT
A
FI
N
G
BR
G
RC
HU
N
IR
L
E
DE
U
DN
K
EM
U
ES
P
CZ
T
BE
L
CA
N
AU
S
0.0
AU
RMAR
0.8
Conclusion on robustness

Relative mean absolute revision (RMAR) from
revision analysis can help users assess robustness
of first published growth rates:
– If MoM / QoQ are not robust, YoY may be more
suitable for short-term analysis (but this can delay the
identification of turning points …..)
– Or may need to look at other estimators (trend
estimates? – but need to test these for revisions too)
– Put pressure on statistics office to improve their
methods
30
Predictability of revisions and assessment
of bias for mean revision

Ideally revisions should centre around zero over time
(i.e. equally likely to be + or - ).

Easy to assess the statistical significance of the
mean revision at different revision intervals

If a bias is found, what does this mean?
– Could users / analysts exploit this information to
improve on first published estimates or their
forecasting models?

Does a ‘true’ or ‘final’ value of the economic variable
we are using to assess an aspect of the economy
exist?
31
Reasons for revisions and their timing (GDP focus)

Revisions in first few subsequent releases
– Revisions to input source data (e.g. arising from
sample surveys with late respondents, corrections of
previous errors found etc.)
– Revisions to models based on partial indicators or
replacements of estimates based on models with
actual data
– Concurrent seasonal adjustment
32
Reasons for revisions and their timing (GDP focus)

Periodic revisions performed annually
– Revision of seasonal models
– Benchmarking to annual data sources (may also
involve reconciliation with aggregate level supply and
use tables)
– Annual chain linking, rolling updates to base period

Periodic revisions at other frequencies
– Benchmarking to 5 or 10 years census (may also
involve reconciliation with detailed Input Output tables
– Change to base year for constant price estimates
33
Reasons for revisions and their timing (GDP focus)

Major adhoc revisions:
– Changes to compilation methodology (e.g. annual
chain-linking)
– Changes to conceptual definitions (e.g. SNA 93)
– Changes to classifications (e.g. NAICS, NACE rev. 2)
34
When is an observed bias to the mean revision
important?

Policy makers / analysts should only be interested in
small number of subsequent revisions to first
published data
– Thus forecasters should only consider whether first
estimates are efficient or biased based on a small
number of subsequent revisions (probably not more
than one year after first published data)

Consider statistical significance of mean revision
after one year to first published GDP QoQ growth
rates
35
Country
Australia
Belgium
Canada
Switzerland
Germany
Denmark
Spain
Finland
France
United Kingdom
Italy
Japan
Korea
Netherlands
New Zealand
Norway
Portugal
United States
Average all
MR (Y1_P)
-0.004
0.05
-0.01
-0.002
-0.02
0.13**
0.01
0.16**
0.01
0.04
0.001
-0.04
-0.01
0.05
0.03
0.01
-0.04
0.05
0.02
% Y1 > P
55.6
60.0
Longer term revisions to GDP growth rates

Propose the conjecture that revision to GDP QoQ
growth rates are more likely to be upwards the longer
the period from first published data
– Due to the systematic influence of changes in
compilation methodology providing better estimates of
volume and productivity backcasted through the series
(e.g. ICT deflators and PPIs for service industries)
– Possible tendency for conceptual and definitional
changes to have a similar impact (e.g. capitalisation of
software in SNA 93)
37
Country
Australia
Belgium
Canada
Switzerland
Germany
Denmark
Spain
Finland
France
United Kingdom
Italy
Japan
Korea
Netherlands
New Zealand
Norway
Portugal
United States
Average all
MR (L_P)
0.07
0.07
0.12**
0.07
0.01
0.21**
0.16**
0.15
0.05
0.17***
0.07
0.06
0.13
0.14*
0.27**
0.15
0.14
-0.01
0.11
Country
Australia
Belgium
Canada
Switzerland
Germany
Denmark
Spain
Finland
France
United Kingdom
Italy
Japan
Korea
Netherlands
New Zealand
Norway
Portugal
United States
Average all
MR (L_Y1)
0.10**
0.02
0.13**
0.10
0.02
0.06
0.16**
0.03
0.02
0.11**
0.07*
0.08
0.13
0.13
0.20
0.13*
0.23**
-0.04
0.09
Country
Australia
Belgium
Canada
Switzerland
Germany
Denmark
Spain
Finland
France
United Kingdom
Italy
Japan
Korea
Netherlands
New Zealand
Norway
Portugal
United States
Average all
MR (Y1_P)
MR (L_P)
MR (L_Y1)
-0.004
0.05
-0.01
-0.002
-0.02
0.13**
0.01
0.16**
0.01
0.04
0.001
-0.04
-0.01
0.05
0.03
0.01
-0.16
0.05
0.02
0.07
0.07
0.12**
0.07
0.01
0.21**
0.16**
0.15
0.05
0.17***
0.07
0.06
0.13
0.14*
0.27**
0.15
0.01
-0.01
0.11
0.10**
0.02
0.13**
0.10
0.02
0.06
0.16**
0.03
0.02
0.11**
0.07*
0.08
0.13
0.13
0.20
0.13*
0.24**
-0.04
0.09
Other results

Mean of revisions after one year to first published
MoM growth rates were statistically significant at 95%
level for:
– Greece, Belgium and India for Index of Industrial
Production
– Canada for Retail Trade Volume

For each of GDP (4 countries), IIP (8 countries) and
RTV (4 countries) much higher incidence of bias to
first estimates of YoY growth rates
41
Main conclusions

Revisions analysis provides essential information to
both users and producers
– Users to understand degree of robustness for first
published data (RMAR) and any short-term bias
– Producers to understand better the quality and as a
trigger to improve processes

Assessment of bias in revisions must be treated with
caution (especially for GDP)
– Revisions to GDP growth may have a legitimate
tendency to be positive in the longer term

OECD task-force on revisions policy and analysis
42
THE END
43