Some comments on methodology, especially benchmarking

One method of co-ordinating statistical output is so-called benchmarking, where one set of estimates is forced to agree with another. This is a special case of consistent estimates introduced in Section 10.2.3. Typically, short-term statistics could be benchmarked on annual statistics, if the former (after aggregation to the calendar year) are an indicator of the latter. One reason could be to simplify for the user by unifying the two time series (ensuring that the monthly series has the same annual sum as the annual series), another to improve the accuracy of the short-term statistics. For this to be meaningful, the two sets of statistics should have the same target parameters for the calendar year.

The use of procedures to make estimates consistent may influence not only one but both sets of statistics. The implementation of benchmarking of, say, short-term statistics on annual statistics, involves comparisons. These may consider not only the macro level, but also the micro level. The evaluations performed may imply further edits for both short-term and annual statistics.

In cases like benchmarking short-term statistics on annual statistics, the former have been published when the latter appear. That means a revision, may be one or two years after the first publication (longer for January than for December), or even more. Many users will react badly to revisions in their time series. Advantages and disadvantages have to be balanced. There are several methods for benchmarking, based on different approaches to the two time series as to what is fixed and what is random variation, see for example Cholette & Dagum (1994) with emphasis on survey errors, Durbin & Quenneville (1997) with emphasis on stochastic time series models (and also references therein), and the very recent Dagum, Cholette & Chen (1998).

There is a recent suggestion on co-ordination at the estimation stage by Renssen & Nieuwenbroek (1997), who call their procedure aligning estimates. Surveys with variables in common – variables that are observed in these surveys and have unknown population totals – are pooled and the common variables are used as regressors (in addition to variables with known totals). Then the estimate obtained is used as auxiliary information in the individual surveys. The procedure is interesting from both coherence and accuracy points of view. Furthermore, statistics may be related, although without clear connections in terms of, for example, units. Labour market statistics based on business surveys and on household surveys provide an example, see also Section 10.4.

10.3.4 Comparability over time

There is usually interest both in recent statistics and in long time series. Accuracy of changes is often at least as important as accuracy of levels.

Stability of definitions is important, but changes in structure should also be taken into account. For example, the use of chain-linked indices has increased, and an index with a fixed base is recommended to be rebased fairly frequently, at least every fifth year. It may be necessary to change variables to be in line with accounting practices if these change. New administrative rules may influence the BR in a way that carries over to the statistics. Comparability over time should be taken into account when choosing variables: current prices are often complemented with constant price or volume measures.

The methodology used has an influence on comparability, and there has to be a compromise between introducing for example improved estimation methods and keeping the old way with regard to time series. There is often a “jump” in a time series when a change is made. Hence, care is needed when introducing changes in methods. It may be wise to have a “double-run” period, that is to run the two methods in parallel to measure the effects and possibly link the two time series. As a minimum, explanations should be provided to the users.

When comparing short-term statistics, calendar and seasonal adjustments are important tools, with regard to corresponding periods in different years and adjacent periods. There are different methods of adjustment, building on different assumptions, like additive or multiplicative components. The appropriateness of a method is not necessarily the same in all countries. Still, for comparability reasons there should be some harmonisation of the adjustments of time series.

10.3.5 International comparability

As already indicated above in Section 10.3.1, there are many international harmonisation activities to improve comparability between countries.

Standard classifications is a typical example, with for example NACE Rev. 1 for classification of economic activities. There is a Regulation on Structural Business Statistics that includes definitions of variables. A Regulation on Short-Term Statistics has become law during 1998. There is a Regulation on statistical units – like enterprise, kind-of-activity unit and local unit – and also one on Business Registers. These regulations aim at increasing the comparability through making basic definitions equal – and also the applications similar by providing not only theory but also manuals with examples.

However, the subsidiarity approach means that each Member State may implement surveys in its own way, even when there is a regulation such as those mentioned. Similarly, regulations on for example statistical units may be interpreted and applied somewhat differently between MSs due to different traditions, prerequisites, etc. There are inherent cultural differences, like the number of working hours per full-time and part-time employee, the distribution of working hours over the year and over the week, taxation rules etc. The variable investments in fixed assets provides an example where the precise definition of the variable may vary

between countries, at least before regulations have come into use. In such a case, it may be possible to make some kind of estimate as to the effect of a different national definition in comparison with the European concept. That is an attempt to overcome the lack in comparability, but to measure the difference is a difficult task.

Among examples of methods in the direction of comparability, standardisation of death rates in population demography is an old and illustrative one. Depoutot & Arondel (1997) discuss business statistics, and they advocate econometric models. Dalén (1998) presents sources of non-comparability in a general approach to the case of consumer price indices, and he presents empirical analyses of the effects of different conceptual and technical differences based on Swedish and Finnish data.