Transcript Sophisticated/ special user measures

Data requirements and
summary statistics for
revisions analysis
Performing revisions analysis for sub-annual
economic statistics
Michela Gamba, Statistics Directorate - OECD
OECD Short-Term Economic Statistics Working Party
______________________________
Paris 23-24 June 2008
Scope of revisions analysis
WHY USEFUL?
– Allow users and providers to better understand quality
of published sub-annual statistics and to study
magnitude of revisions
WHAT IS NEEDED?
– A Real-time database to perform Revision analysis
– A dataset containing a separate time series for the
variable as appeared in each official past release
2
OECD input to Task Force on revisions
analysis
1. DATA & METADATA KEY ELEMENTS
»
Guidelines for data and metadata requirements
2. INVENTORY OF DATA SOURCES
»
List of existing sources available publicly
3. SUMMARY STATISTICS
» Review summary statistics used
» Include relevant statistics- how much the “story has
changed”
» Selected statistics to be included in the toolkit
» Provide description and purpose of statistics
3
DATA & METADATA KEY ELEMENTS
8 requirements to build a RT database
1 COUNTRY/REGION DESCRIPTION
-> identify country/region/ zone aggregate
2 VARIABLE DESCRIPTION
->Full description of variable (classification system,
value/volume, ref years) with links to current revision policy
associated requires well detailed METADATA
3 VARIABLE MEASURES
-> gross/ SA/ trend/ provided as published (index or level) + GR
4
DATA & METADATA KEY ELEMENTS
8 requirements to build a RT database (cont.)
4 IDENTIFICATION OF VINTAGE
-> data and metadata snapshot for each variable, for each official
release
-> Field providing the date of release (one new data point for each
vintage, forming a triangle)
5 LENGTH OF VINTAGE
-> going back as far as possible…
5
DATA & METADATA KEY ELEMENTS
8 requirements to build a RT database (cont.)
6 LENGTH OF TIME SERIES
-> ideally the full time series available for each official
release..general rule of at least one year before the starting point of
vintages to allow y_o_y growth rates
7 ONGOING UPDATING
-> store all new vintages with regular achieving
8 DATA ACCESSING
-> different formats available but..it is important to be able to
extract data in the right format ( i.e. triangle table)
6
Summary statistics
3 levels of analysis
1.
BASIC/ CORE MEASURES
Targeting users that require quick, easy to understand information
2. ADDITIONAL/ ADVANCED MEASURES
Targeting users that require more in-depth analysis
3. SOPHISTICATED / SPECIAL USER MEASURES
Information for detailed research purposes – references to formula
only
7
Summary statistics (cont.)
Default dataset information
Period of analysis
reference data points of the time series being
analysed (e.g. from 1994Q1 – 2007Q3)
Revision Interval L_P
revision interval being analysed for the vintages of the
sampled time points
n
number of observations
8
Summary statistics (cont.)
Drop-down boxes
1. USUAL SIZE AND RANGE OF REVISIONS
Basic measures
Mean absolute revision
Range that 90% of revisions lie within
Advanced/additional measures
Median absolute revision
2. ASSESSMENT OF POSSIBLE DIRECTIONAL TENDENCY IN
REVISIONS
Basic measures
Mean
Statistical significance of the mean
Advanced/additional measures
Median
% positive/ negative/ zero revisions
HAC standard deviation
Adjusted t-stat of mean revision
Critical value of t-stat for significance of mean
9
Summary statistics (cont.)
Drop-down boxes
3. VARIABILTY OF REVISIONS
Basic measures
Standard deviation
Advanced/additional measures
Root mean square revision
Quartile deviation
Min/ max/ range
Sophisticated/ special user measures
Skewness
4. IMPACT OF REVISIONS ON SIGN OF GROWTH RATES
Basic measures
% sign(later)= % sign(earlier)
Advanced/additional measures
Acceleration/ deceleration test
10
Summary statistics (cont.)
Drop-down boxes
5. EFFICIENCY ASSESSMENT
Advanced/additional measures
Correlation between revision and earlier/ later estimate
Test if revisions are noise/ news
Sophisticated/ special user measures
Mean squared revision
Mean squared revision Decomposition UM
Mean squared revision Decomposition UR
Mean squared revision Decomposition UD
11
The tool kit…