Transcript Moving into the 21st Century: A New Strategy for Disseminating IMF

IMF Approach to Storing Metadata
with Macroeconomic Statistics
UNECE Workshop on the Common
Metadata Framework
(Vienna, Austria, 4-6 July 2007)
Dissemination Standards Bulletin
Board (DSBB)
• data standards initiative (SDDS/GDDS
• countries’ dissemination practices
• information that SDDS countries provide the
IMF on their dissemination practices
• direct links to the economic and financial data
that countries disseminate under the SDDS
• information that GDDS countries make
available to the IMF on their statistical practices
• http://dsbb.imf.org
Collaboration with OECD
Dec 2006 - Agreement to use Dotstat and MetaStore to
form the basis of the IMF data warehouse
Jan 07 – software available on joint Team Foundation
Server (TFS)
Feb 07 IMF.Stat installed with the assistance of OECD
May 07 have loaded: International Financial Statistics
(IFS), World Economic Outlook (WEO), and Sub
Saharan Africa Regional Economic Outlook (REO)
June 07 – signed an MOU which supports a
collaboration approach to future enhancements for
the mutual benefit of both organizations
IMF.Stat Data Model
Data
Country Group
Referential Metadata
Concept
Country Group
Concept
CouGrpID
100
ConceptID
250
CouGrpID
100
ConceptID
250
ParentID
Null
ParentID
200
ParentID
Null
ParentID
200
Code
156
Code
NGDP
Code
156
Code
NGDP
Label
Canada
Label
Gross ...
Label
Canada
Label
Gross ...
Data Fact table
Unit Of Measure
UofMID
10
ParentID Null
Code
N
Label
Nat Curr
Metadata Fact table
CouGrpID
100
ConceptID
250
DataSourceID
3000
UnitOfMeasID
10
DatSrceID
3000
TimeFreqID
25
ParentID
Null
StatusID
2
Code
WEO
Observation
158.1
Label
World ...
Flag
E
Unit Of Measure
DataSource
UofMID
10
ParentID Null
Code
N
Label
Nat Curr
CouGrpID
100
ConceptID
250
DataSourceID
3000
UnitOfMeasID
10
DatSrceID
3000
TimeFreqID
25
ParentID
Null
StatusID
2
Code
WEO
Label
World ...
MetadataID
5487
DataSource
Time & Frequency
Time & Frequency
TimeFreqID
25
ParentID
Null
Code
200401
Label
2004 Q1
Status
StatusID
2
ParentID Null
Code
SHARE
Label
Shareable
Status
TimeFreqID
25
StatusID
ParentID
Null
ParentID Null
Code
200401
Code
SHARE
Label
2004 Q1
Label
Shareable
Metadata
MetadataID
5487
Text
Chain-linked GDP
volume measures
are expressed in ...
2
Structural metadata
• Economic Concepts -mapped as many time series as
possible to the Catalogue of Time Series and loaded them
to IMF.stat
• Countries and groups – used IFS version of Country
names and codes as the authoritative source for codes
and labels
• Unit – chose to combine unit and scale e.g. Millions of US
dollars
• Storing data in native units i.e. not trying to convert
observations to a common unit.
• Status, Source and Time and Frequency reasonably
straight forward so far. Will become more problematic
when we introduce versioning.
Referential Metadata
• Working through existing metadata from IFS publications
and production system
• Where necessary/possible cleaning it up, standardizing it
and loading it to MetaStore
• WEO – metadata sourced from the external web site,
reformatted and stored in MetaStore then exported to
IMF.stat
• All referential metadata loaded to MetaStore and then
exported to IMF.Stat
Data- IFS
All time series which were able to be mapped to the
Catalogue of Time Series (CTS)
• Includes
–
–
–
–
–
–
Exchange rates
Balance of Payments
International Investment Position
Real Sector Statistics
International Liquidity
Money and Banking non-SRF data
• Excludes
– Government Finance
– Money and Banking SRF data
– Fund Accounts
»
»
»
»
191 concepts
233 countries
39 groups
7.6 million observations
Data-WEO
WEO
• Two most recent editions
• Includes series published externally as well as other
series available internally
• Concept - generally consistent with the CTS
• Country and group – some differences in codes used
so mapped where possible. Some groups added.
• Unit - limited number of units used and mainly
consistent across countries
Data-REO
Sub Saharan Africa REO–Structural Metadata
• Concepts – virtually no codes or labels in common with
the CTS
• Able to map those series published in the REO but the
supporting series too difficult. Are now working through
them on a case by case basis to determine which if any
map to the CTS
• Country and group – country codes and labels mainly
consistent with WEO. Groups all different even though
sometimes have the same label.
• Units - mainly ratios which were added to the
authoritative list.
Sub Saharan Africa REO Referential Metadata
• Have sourced top level referential metadata only. Will
work with the Africa Department after the data are
loaded to identify any usable referential metadata.
MetaStore
• Some modifications with assistance from OECD
– Now includes
• structural metadata
• mappings to authoritative lists
• referential metadata
SchemaLogic
– In future may integrate structural metadata in
MetaStore or replace
Alignment with SDMX
– Have used 42 ‘types’ to categorize our referential
metadata
– Added one to the OECD set, which are consistent
with SDMX
Managing metadata within the IMF
•
•
•
•
•
•
Locate relevant sources of metadata
Locate potential warehouse content
Central repositories for data and metadata
Harmonizing and mapping to a preferred term
Authoritative lists
Working with Information Services Division (ISD) to
ensure information management best practice
• Assigning data stewards to manage metadata
Governance
• Establishing groups and individuals with certain roles and
responsibilities for management of metadata
– Economic Data Advisory Group
• Representation from departments across the Fund
• Includes several working groups with specific focus
– Information Services Division
• Responsible for provision of metadata
– Metadata and Standards team
• New group in the Statistics Department currently focusing on
metadata used in the data warehouse
Next Steps
•
•
•
•
•
•
Changes to work practices across the Fund
Identify a data steward for each dimension in IMF.Stat
Standardization, authoritative sources
Reuse of metadata across systems
Raise awareness of the value of quality metadata
Tie together basic schemas
EDW Top Level Diagram
Data
sources
MetaStore
Referential
and structural
metadata
Internal
User
interface
Concept
Time
Country
Data & Freq
Group
Source
IFS
WEO
ETL
External
End-users
Structural
metadata
2005;USA,GDP,548.25
2004;USA,GDP,526.25
...
111 USA
112 UK
273 MEX
...
Referential
metadata
DataStream
User
interface
Haver
IMF.stat
Data flow