Statistical Metadata System based on SDMX

(1)

Statistical Metadata System

based on SDMX

Petko I. Yanev; Jean-François Fracheboud

Federal Statistical Office

Switzerland

(2)

Département fédéral de l’intérieur DFI Office fédéral de la statistique OFS

• Challenges

• Vision / Metadata Strategy

• Architecture and coverage

• Funtionalities

• Plug-in: Statistical Codebook

(3)

• Centralized system for de-centralized metadata management

• Unique metadata creation : multiple usage

• An integrated system for the entire life-cycle of each statistic

• The memory of statistics

(4)

• Centralized system for de-centralized metadata management

• Unique metadata creation : multiple usage

• An integrated system for the entire life-cycle of each statistic

• The memory of statistics

SMS MEETS ALL THESE CHALLENGES

Building SMS since 2010; Operational since 2012

(5)

Statistical Metadata System : Vision

- SDMX based - Modular/Flexible 1 Specify Needs 2

Design Build 3 Collect 4 Process 5 Analyse 6 Disseminate 7 Archive 8 Evaluate 9

SMS

Metadata

Data

Dissemination

- SDMX and other international

standards

- Data files CSV, SAS, PX or DB tables and cubes

- Reporting - Web access - Codebook

GSBPM

• End user

• OECD

• Eurostat

Data collector

Stat. producer

(6)

W

eb

Serv

ices

Reference Metadata

Subscription

Provisionning

Registre SDMX

Structural Metadata

Data Registration

CrossRef DB File System - data mining -

SDMX editor

SMS : Architecture and Coverage

CSV / XLS writer

PDF / Codebook writer

(7)

W

eb

Serv

ices

Reference Metadata

Subscription

Provisionning

Registre SDMX

Structural Metadata

Data Registration

CrossRef DB File System - data mining -

SDMX editor

CSV / XLS writer PDF / Codebook writer

SMS

DB / SAS export

(8)

identify describe

W

eb

Serv

ices

Reference Metadata

Subscription

Provisionning

Registre SDMX

Structural Metadata

Data Registration

CrossRef DB File System

Statistical Data

- data mining -

SDMX editor

- dat a quering - CSV / XLS writer PDF / Codebook writer DB / SAS export

(9)

W

eb

Serv

ices

Reference Metadata

Subscription

Provisionning

Registre SDMX

Structural Metadata

Data Registration

Statistical Data

- data mining -

SDMX editor

SDMX Registre SDMX Registre SDMX Registre - dat a quering - CSV / XLS writer PDF / Codebook writer - ext. metadata - DB / SAS export

(10)

W

eb

Serv

ices

Reference Metadata

Subscription

Provisionning

Registre SDMX

Structural Metadata

Data Registration

Statistical Data

- data mining -

SDMX editor

SDMX Registre SDMX Registre SDMX Registre - dat a quering - CSV / XLS writer PDF / Codebook writer - ext. metadata -

Data / Metadata Diffusion - Web portal - Data cubes (PX, SDMX) - Codebook (PDF, HTML) - dis s em inat ion -DB / SAS export

(11)

W

eb

Serv

ices

Reference Metadata

Subscription

Provisionning

Registre SDMX

Structural Metadata

Data Registration

Statistical Data

- data mining -

SDMX editor

SDMX Registre SDMX Registre SDMX Registre - dat a quering - CSV / XLS writer PDF / Codebook writer - ext. metadata -

Data / Metadata Diffusion - Web portal - Data cubes (PX, SDMX) - Codebook (PDF, HTML) - dis s em inat ion

-SMS

DB / SAS export

(12)

•

Single Entry Point via Web Services

•

Statistic life-cycle metadata management :

 Management of statistical activities (national and European)

 Creation / management of catalogues of variables per theme

 Classifications and code lists with correspondence tables

 Data description and registration / discovery

 Metadata reports (quality, questionnaires, codebook)

• H

armonization of the structure and the metadata content

•

Extensions: SAS plug-in; Questionnaire CATI;

Codebook

; etc.

•

Metadata driven data access / monitoring

•

Synergy, reutilization, redundancy reduction, integration

Statistical Metadata System : Functionalities

(13)

Statistical Metadata System in Numbers

SDMX Artefact Type

Count

CategoryScheme

10

Codelist

1849

ConceptScheme

66

DataFlow

112

DataStructure

134

HierarchicalCodelist

116

MetadataFlow

48

MetadataStructure

26

MetadataSet

163

Registered users:

148

(14)

Statistical Metadata System : Codebook

(15)

Statistical Metadata System : Codebook

SDMX

X

DDI

DDI Codebook PDF / HTML

(16)

(17)

(18)

(19)

(20)

• Finalize the described SMS architecture



The new SDMX 2.1 compatible registry



The new Web services for registering and querying data



The Codebook extension

• Build an integrated metadata thesaurus

(21)

For all inquiries concerning the

Statistical Metadata System @ FSO

Please contact :