Transcript Steps towards an SDMX implementation

The SDMX framework: key issues and the business case
Gabriele Becker
Bank for International Settlements
SDMX Conference: Towards Implementation of SDMX
Washington, 11 January 2007
Views expressed are those of the presenter and not necessarily those of the BIS.
1
Overview
 SDMX vision
 The statistical production chain
 Components of the SDMX framework
 Analyse the relationships
 Conclusions
2
The SDMX vision
 Facilitating data and metadata exchange
 Efficient use of technologies and standards
 Reduce reporting burden
 Enhance availability of statistical data and metadata
for the users
 Data reporting = data dissemination = data sharing
3
Statistical processing chain in an NSO or an NCB
 Requirements specification
 Data collection design
 Data collection from reporting agents
 Data quality control
 Data compilation
 Data dissemination (internal, respondents, public, int.
organisations)
 Data analysis (data discovery, navigation, search)
4
SDMX components
 Information model for data and metadata
 (Draft) content oriented Guidelines
 Meta Data Common Vocabulary
 Cross Domain Concepts
 Two syntaxes to exchange data and metadata (EDI, XML)
 SDMX Registry standards and registry interfaces
 SDMX Tools
 How do these relate to the statistical processing chain?
5
EDI and XML
syntax expressions
and tools
SDMX registry
and registry
interfaces
Content oriented
Guidelines
Information
Model
1
Requirements specification


2
Data collection design




3
Data collection from reporting agents




4
Data quality control



5
Data compilation


6
Data dissemination (to respondents, public, int.
organisations)
Data analysis (data discovery, navigation, search within the
organisation)








The statistical processing chain and the
SDMX components
7
6
Some initial conclusions
 Information model “is everywhere” in the statistical
processing chain
 Content Guidelines ”are everywhere” as well
 Technology (syntaxes, tools and registries) are enablers
when it comes to exchanging or sharing data
 How can SDMX impact the individual steps in the
statistical processing chain?
7
Requirements specification & data collection design
 Information model
 Forces us to consider metadata from the start
 Helps to organise the data when specifying new data
collections
 Examples for efficient structuring of data later today
 Applying the (same) information model from the start leads to
for economies of scale
 Content Guidelines
 Potential re-use of existing code lists
 Use of registries to share code-lists and structures
 Learning effect across the statistical organisation(s)
 First data structure definition is the most difficult ….
 SDMX Tools to help defining data structures
8
Data collection from reporting agents
 Standard formats (SDMX-EDI or SDMX-ML)
 Reflect the Information Model and Content Guidelines
 Support collection of data and metadata
 Data structures and code lists exchanged in computer readable
formats
 Support for automation
 Stepwise implementation using SDMX tools …
9
Stepwise implementation
 Currently “push” models prevail
 Reporting agents send data (files) to the collecting agency
 Can start with simple tools (eg EXCEL SDMX creator)
 Automation possibilities for reporting agents and for
collection agency already for “push models”
 Generalisation of SDMX file creation
 Creation of SDMX-EDI or SDMX-ML out of the database
 Conversion between SDMX-EDI and SDMX-ML via tools
 Move to “pull model” and use of registries possible any
time
10
Data quality control and compilation
 Standard reporting formats allow for automated checking
 Check formulas may be derived from the code lists
 eg in the case of hierarchical code lists
 SDMX supports move towards metadata driven
processing
 Generic processing systems based on information (data) model
 Decreasing marginal cost for new data collections
11
Data dissemination
 Commonalities with data collection
 Changed roles: collector (NSO / NCB) becomes provider
to international organisations or the public
 Same SDMX formats as used for data collection
 Disseminate
 Data and metadata
 Data (and meta data) structure definitions
 Push and pull models
 Use of SDMX registries
 Automation
12
Data analysis (discovery, navigation, search)
 Any navigation system is metadata driven
 SDMX information model forces us to be clear about
the metadata and the data structure from the start
 Generalised navigation system based on the model
can easily incorporate new data collections (= new
data structures based on the information model)
 Registries help to find data from remote sources
 Generic tools (stylesheets) for easy data rendering
 Economies of scale and better service for users
13
Conclusions: Information model
 SDMX is not only for IT experts
 SDMX can only be applied if statistical experts apply
the information model in their work
 Define data and metadata structures
 SDMX information model and content guidelines can …
 Affect the complete statistical processing chain
 Influence the design of our processing systems
 Influence the design of our navigation systems
14
Conclusions: technical standards and tools
 Technical standards allow efficient and generalised
application of SDMX information model
 Technology (eg registry technology) is an enabler to
make SDMX work better
 A stepwise introduction of SDMX is possible using the
SDMX tools
 SDMX is a framework
15
SDMX: Benefits
 Speed up movement of data through the processing
chain
 Reduce “time to users” for data and metadata
 Increase level of automation
 Reduce risk of errors
 Enhance facilities for metadata exchange
 Easier to ship metadata with the data
 Better understanding of data
 Reporting = dissemination = data sharing
 Reduction of reporting burden
16
Practical steps
 Review SDMX information model
 Sample data structures used by others
 Relate to your own data model as implemented in your
organisation and systems
 There will be commonalities …
 Experiment with the SDMX tools to understand and build data
structures for your own data
 Review the technical standards and formats
 Understand relationship between the information model and its
practical implementation as SDMX-EDI or SDMX-ML
 Assess impact on your own organisation
17
Towards SDMX Conformance
 Apply information model for data structure definitions
 Use cross domain concepts as much as possible when
defining data structures
 Re-use existing (internationally shared) code lists
 Use SDMX-EDI or SDMX-ML for data and meta data
exchange (push model)
 Publish data structure definitions
 Offer data in SDMX-ML on (public) website
 Offer data via SDMX registry (pull model)
18
Thank you!
Gabriele Becker
Head, Statistical Information Systems
Monetary and Economic Department
Bank for International Settlements
Basle, Switzerland
[email protected]
19