Quality improvement and the census

1.254. The quality circle can be applied to the entire census cycle with:

(a) Performance in the previous phase being evaluated at any given level of detail;

(b) Problems with quality ranked in order of importance; (c) Root causes identified and corrective action implemented.

1.255. The dependencies in the census cycle are represented in figure 2. 1.256. It is worth noting that it is possible to start at any point in the diagram and achieve the same result.

1.257. The following sections outline the way in which the concept of a quality circle is superimposed over the census cycle. Much of the discussion on form design, enumeration, processing and dissemination is in terms of relevance and accuracy. However, these are subject to constraints of time and cost that may be established prior to commencing the census cycle. These are discussed briefly below.

(a) Topic selection

1.258. The first step in managing the quality of the product (namely, the sta- tistics to be produced) is to ensure that the product will be relevant. The key process is extensive consultation with actual and potential users of census information. The key success factor in this process is full, frank and open communication with users and all areas concerned with the census (in particular, subject matter and classifica- tion experts). As should be expected, users are reluctant to propose their needs for a future census until they have been able to assess the extent to which their current needs have been satisfied by the output from the previous census. This should be seen as an evaluation process feeding into the current cycle, the first step of quality management.

(b) Form design and testing

1.259. The next quality management task concerns the testing of each census question and the testing of the design of the form. Again, the quality circle approach is used, with the results of each test being analysed and evaluated before being fed into further design and testing. The following areas are the key internal stakeholders of the form design process and their requirements need to be taken into account:

(a) The dissemination team, to ensure that the questions asked will deliver the data to meet the needs of users;

(b) The subject matter specialist team;

(c) The team responsible for development of the processing system. For exam- ple, positioning of text and delineation of response areas will be dependent on data capture and the processing methodology to be adopted. It is critical that there is ongoing coordination between the form design and processing areas;

(d) The field operations team, which is responsible for training the enumera- tion workforce and printing the form.

Figure 2

Quality circle dependency chart

evaluation 1. Data quality 2. Process 3. Products/services topic selection Dissemination

Data processing Form design and

testing Classification

and subject matter specialists

(c) Field operations

1.260. The quality management process continues throughout the design of the census field operations. These are tested, as far as possible, in conjunction with form designs testing. The key internal client of field operations is processing. However, field operations can also impinge on other areas, such as dissemination and classifica- tion and subject matter areas where certain concepts, such as what constitutes a dwell- ing, are implemented during the field operations phase. Several components of field operations can be subject to specific quality circle mechanisms as these are likely to take some time and involve iterative processes. These components include:

(a) Demarcation of enumeration areas; (b) Map production;

(c) Form printing, where a sample of forms is rigorously tested for adherence to standards.

1.261. Quality monitoring should be established for each of these components and mechanisms put in place to ensure that the outcomes of the monitoring are used to improve processes. It is more difficult to implement the quality circle during actual enu- meration owing to the very tight time constraints. However, this can be achieved by:

(a) Clearly establishing the aims of the field operations phase; (b) Applying thoroughly documented procedures;

(c) Ensuring that the enumerators understand their role through appropriate training and providing inspection of corrupted forms;

(d) Providing opportunities for field staff to be observed operating on the job so that feedback can be given and retraining undertaken.

1.262. However, it has to be acknowledged that during the actual carrying out of the enumeration this approach tends to identify “problem enumerators” rather than systemic or process errors. This means that evaluation following collection is vital. The evaluation should attempt to capture the experiences and suggestions of a range of enumerators and other field staff so that improvements can be made to the subsequent census.

1.263. A general overview of the quality of enumeration can be obtained through:

(a) The use of techniques such as post-enumeration surveys to gauge the level of underenumeration of people and dwellings;

(b) Feedback from field staff;

(c) Measures of the quality of any coding undertaken by field staff; (d) Mechanisms that may be in place to handle queries from the public. 1.264. The effectiveness of the public communication strategy may be assessed by the amount of press coverage (positive and negative) of the census and follow-up surveys to test the reaction to particular advertising.

(d) Processing

1.265. The key clients of processing are the dissemination area and the areas of the country’s statistical agency responsible for maintaining standard classifications and those with specialist subject matter knowledge. The dissemination area depends on the processing team to obtain data in an agreed format and compiled to agreed quality standards. This is necessary so that the data can be used in dissemination sys- tems.

1.266. Since the census is part of an overall national statistical system, data from the census are likely to be used in conjunction with data from other collec- tions. Thus the classification and subject matter specialist areas, which are responsi- ble for those other collections, need to be satisfied that the coding, editing and other data transformation processes are conceptually sound and deliver data of acceptable quality.

1.267. Extensive testing of processing systems must be undertaken in advance of the census. Coding processes and training packages need to be prepared and tested using the type of staff likely to be involved in the operations. The processing phase gives the fullest scope for the use of quality improvement techniques, as many of the processes in this phase are repetitive and take a reasonable amount of time. This ena- bles the quality circle to go through much iteration. It is vital that structures are put in place not only to monitor quality but also to involve processing staff in the identifica- tion of problems with quality and in proposing solutions.

1.268. It is generally not possible for processing to improve the accuracy of census data. At best, processes such as editing may reduce some inconsistencies within the data. However, in the end, the data coming from the processing system will not be of any better quality than the information supplied on census forms. Much effort can be expended in correcting apparently inconsistent or inaccurate census data with no real improvement in the fitness for the purpose of the data. It may be a better strategy to educate users to accept slight inconsistencies in census data, rather than developing highly complex procedures that may introduce other errors and impose heavy costs in terms of delay in release of the data and cost to the community.

(e) Dissemination

1.269. Census dissemination can easily be overlooked in the chain of provid- ing a quality outcome for the census as management attention is diverted to the costly and risky enumeration and processing operations. The dissemination area is respon- sible for the timely delivery of products and services to the census data users. There- fore, insufficient planning and resources for this phase can have the effect of delaying the release of the data and thus compromising the overall achievement of the census objectives. The dissemination phase should also be regarded as an ongoing process that will service the needs of users over a long period of time.

1.270. Management of the quality in census dissemination is driven by con- cerns to (a) deliver relevant products and services while (b) maintaining accuracy of the data, and (c) timeliness and predictability of data release within agreed cost constraints.

1.271. The first of these objectives is to provide relevant products and services. This can only be done by reviewing the experiences of the previous census products and services and by user consultation processes with both current and potential users of census data.

1.272. The second objective is to ensure that the data supplied from the process- ing system are accurately transformed into output products. A quality assurance strat- egy to ensure that data tabulations and transformations are carried out accurately needs to be documented and followed. The quality circle approach to these processes needs to be applied and any gaps identified and corrected through extensive testing prior to the census and ongoing process improvement during the dissemination phase.

1.273. The third quality objective for dissemination is the timely and predict- able release of data from the census. While this is the responsibility of all phases of

the census programme, special responsibility resides with the dissemination area. The dissemination area needs to be realistic about release dates and ensure that these are communicated to clients early so as to manage client expectations. The involvement of staff actually responsible for the dissemination phase in devising these dates is recom- mended where this is possible. Dissemination systems and processes need to be avail- able, documented and tested prior to the release of data from the processing phase.

(f ) Evaluation

1.274. Evaluation is usually considered the last stage in the census cycle. How- ever, it is also possible to consider the evaluation of one census cycle as being the first step in the following census cycle. Similarly, evaluation of one process within a census cycle could be the first stage in the next process of the same census cycle. All aspects of the census programme should be evaluated. The strengths and weaknesses of each task should be identified and action points proposed for future census managers.

1.275. Evaluation of the accuracy of the census data should also be under- taken, to the extent possible, by comparing the census results with similar data from other sources. These sources can include surveys in a similar time frame or previous census results. The purposes of evaluating the accuracy of the data are to inform users of the quality of the current census data and to assist in future improvements. Future improvement may be achieved by (a) improving processes; and (b) establishing per- formance benchmarks against which the quality of the data from subsequent censuses can be measured.

1.276. Evaluation of data accuracy may have two parts. Preliminary evalua- tion will enable the identification of any problem areas that have not been previously detected through the quality management processes in earlier phases of the census. More extensive evaluation should be undertaken on data items where problems have been identified or where new questions or processes have been attempted.

1.277. The results of the evaluations should be made available to census data users.