Ton de Waal 15 March 2017 Overview Komuso ESSnet on quality of multisource statistics Basic data configurations BDCs and some work done For all BDCs literature reviews have been carried out ID: 599678
Download Presentation The PPT/PDF document "Output Quality of Multisource Statistics" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Output Quality of Multisource Statistics
Ton de Waal
15 March, 2017Slide2
Overview
Komuso
:
ESSnet
on
quality of multisource statistics
Basic data configurations (BDCs) and some work done
For
all BDCs literature reviews have
been carried out
For most BDCs suitability tests have been done
ConclusionsSlide3
Komuso: ESSnet
on
quality of multisource statistics
Komuso
is
part of
ESS.VIP
Admin
Project
During
first
Specific Grant Agreement
(January
2016 until April
2017)
four Work Packages (WPs) have been defined:
WP 1: Evaluating
the quality of input
data
WP 2: Methodology
for the assessment of the quality of frames for social
statistics
WP 3: Framework
for the quality evaluation of statistical output based on multiple
sources
WP 4: Communication
with respect to the
ESSnetSlide4
Komuso: ESSnet
on
quality of multisource statistics
Komuso
is
part of
ESS.VIP
Admin
Project
During
first
Specific Grant Agreement
(January
2016 until April
2017)
four Work Packages (WPs) have been defined:
WP 1: Evaluating
the quality of input
data
WP 2: Methodology
for the assessment of the quality of frames for social
statistics
WP 3: Framework
for the quality evaluation of statistical output based on multiple
sources
WP 4: Communication
with respect to the
ESSnetSlide5
BDC 1: The baseline
Several data sources with non-overlapping units that together cover complete population
Estimates from data sources can simply be added
Even in this “simple” case important problems occur, such as
Progressiveness of data
Unit problems, e.g. classification into domainsSlide6
Some results for BDC 1
Statistics
Netherlands
has examined the effect of errors in the NACE code classification
on
growth rates of enterprise statistics broken down by NACE
codeSlide7
BDC 2: Partly overlapping units/variables
Part of variables and units in data sources overlap
Observed value in one data source may differ from observed value in other data source
Options:
Micro-integration
Latent class models
Structural equation modelsSlide8
Some results for BDC 2
ISTAT
has examined multiple administrative and survey sources that provide the value of the same variable of interest
A
Latent Class model can be used to estimate the true
values
E
stimates
of the probabilities
, where
is the observed value in data source
and
is the true (latent) value, can be used to evaluate the accuracy of data source
Slide9
Some results for BDC 2
Statistics
Austria
has analysed a quality framework
that
can be used when several data sources with possibly conflicting values for common variables are available.
The
quality framework models errors in variables in these data sources as well as systematically uses expert knowledge. Slide10
BDC 3: Partly overlapping units/variables with under-coverage
Part of variables and units in data sources may overlap
Under-coverage occurs
Options:
Capture-recapture
methodsSlide11
BDC 4: Microdata and aggregated data
Microdata are combined with aggregated data
Inconsistencies between microdata and aggregated data should be avoided
Especially complicated if aggregated data are estimates
Example: Dutch virtual Population Census
Options:
Repeated weighting
Calibrated imputation
Macro-integrationSlide12
Some results for BDC 4
Statistics Netherlands
and
Statistics Norway
have
been working on quality measures that can be applied to
BDC 4
Many
macro-economic figures are connected by constraints (“accounting equations”)
Input estimates usually do not automatically satisfy accounting equations due to measurement and sampling errorsEstimation involves a
reconciliation step by which the input estimates are
modified
A
n
accounting equation is considered as a single entity and scalar quality measures
have been defined
These
measures capture the adjustment effect as well as the relative contribution of the various input estimates to the final estimated
accountSlide13
BDC 5: Only aggregated data
Sometimes only aggregated data are combined with each other
Example: National Accounts
Options:
Macro-integrationSlide14
Some results for BDC 5
Same method that has been developed by Statistics Netherlands and Statistics Norway for BDC 4 can be applied to BDC 5Slide15
BDC 6: Longitudinal data
Combining longitudinal data with different frequencies
Example: combining turnover data from monthly survey with (more accurate) quarterly data from Tax Office
Problem: calibrate monthly data on quarterly data while preserve month-to-month growth
Option:
Benchmarking techniquesSlide16
Conclusions
Much
work has been
done
More work is needed
simplifying
some of the quality measures, methods to compute them, and the use of these measures/methods in
practice
extending
the range of situations in which the quality measures and methods to compute them can be appliedexamining
quality measures relating to “coherence” in more detail