Selection and Quality of Variables

International agencies generate numerous amounts of data on issues ranging from the economic, environment and the social. Although these permit international comparisons, often the information collated between countries is not methodologically consistent. This makes comparability difficult and can hinder the usefulness of a comprehensive index. Hence an assessment criterion on the quality of the selected variables, which follows quality framework guidelines similar to those discussed by Eurostat’s (2004)six components, has been developed.153

The current study truncates the six components into three: (i) whether the variable chosen matches the dimension it is meant to represent – partly based on the variable’s use in other well established measures; (ii) methodological consistency and appropriateness – is the approach defensible; and (iii) the frequency and reliability of the data. Thus, variables are chosen, ‘on the basis of their analytical soundness, measurability, country coverage, relevance to the phenomenon being measured, and relationship to each other’ (Freudenberg, 2003, p. 8).

This section therefore is divided into two parts. The first part will focus on justifying the selection of the variables, whereas the second part will summarise the results of the data quality in a table format.

6.3.1 Selecting Variables

The task of selecting variables needs to, as Shweder (2000) posits, deal with the issue of ‘morally mapping the world’, since measuring progress requires making decisions

153

According to the Eurostat high quality declaration (2004, p. 175), the six components are: accuracy, timeliness, relevance, availability, comparability and response burden.

regarding desirability. This subjective process leads to the researcher morally mapping the world. One example Shweder mentions is the variable life expectancy at birth, a commonly used variable representing the health status of a nation. He questions why a variable such as life expectancy at 40 is not used, or even life expectancy at conception? The results, he expects, would be far more different if this was to occur. For Shweder, the issue is whose ideals become the benchmark for a good life?

Clearly the variables selected in the RIEindex, as with all progress measurements, are subject to this same criticism. Given this, it is essential that the selection process be conducted in a transparent and appropriate manner that mirrors the key objectives of the present research.

The RIE framework differs from most progress frameworks in the sense that it does not claim to include only aspects that are, in principle, prone to ‘objective’ measurement. As mentioned in Section 3.2.3, claiming epistemic privilege over contested areas in progress measurement, of which there are many, does not do this difficult concept justice. Subsequently some variables were included despite the fact that ‘official’ statisticians would be reluctant to incorporate them. For instance, the dimensions net brain gain, biodiversity and institutional quality. However, allowing measurement in dimensions that ‘official’ statisticians may frown upon does not give the current study carte blanche when selecting variables. In fact, the variables selected still need to comply with most of the data quality criteria.

Hence by not adopting a dogmatic approach to variable selection, the present research adopts a key point of difference compared to other comprehensive progress measures. Here, the importance and usefulness of variables in dimensions recognised as contributing to progress is an imperative consideration in justifying the selection of variables.

Given that methodological differences between nations will arise, practical compromises are essential to the success of the RIE index. One such compromise involves the inclusion of variables based on their usefulness despite exhibiting some

methodological inconsistency.154 Moreover, variables are included only where international comparisons are possible thus omitting country-specific issues.

With this criterion in mind the following variables were selected for the 7 themes, 23 dimensions and 21 characteristics that comprise the RIE index. To assist the readability, the variable justification will be broken into smaller segments preceded by a summary table. The variables are discussed at the level of their smallest grouping.155

A. Human Resources

1. Health (a. Health status; b. Access to health) 2. Population (c. Demographics)

3. Food consumption

4. Education and training (d. Access to education; e. Investment and educational quality)

5. Knowledge renewal (f. Investment and stock of knowledge; g. Codifying knowledge and ideas)

6. Net brain gain (h. Net skilled migration)

Health status. The variables life expectancy at birth (years), and infant mortality rate (per 1,000 live births) are considered the most general and best-known measures of the health status of the population, and have long been used in numerous studies including the OECD’s 2005, Society at a Glance (OSG) health indicators and the HDI (UNDP, 2005). Although some discrepancy exists between methodologies there is a considerable match between the variable and the issue, as well as frequent observations.156 Thus, their inclusion is relatively straightforward.

However as Wolfson (1996) points out, the above measures are based on a death status thus ignoring the health status of a living person and their quality of life. Consequently, the WHO introduced a summary health measure that incorporated this, the health- adjusted life expectancy – HALE (years). The HALE summarises the expected number

154

For instance, the variable infant mortality rate (per 1,000 people) possesses variations insofar as some countries registering very premature deaths as live births while other countries do not. Yet despite this, it is a variable that is still included in most comparative health studies (WHO, 2000).

155

The RIE index is broken down by theme (A, B, C, etc.), dimension (1, 2, 3, etc.) and, where applicable, characteristics (a, b, c, etc.).

156

of years to be lived in ‘full health’ that is responsive to the likelihood of survival and death as well as the frequency and severity of a comprehensive set of health states between the inhabitants (Mathers, 2002).157 Issues regarding the reliability and comparability of HALE estimates do exist, with some fine-tuning still expected to improve its comparability (Williams, 1999). According to the WHO’s statistical annex section, the HALE methodology has been peer reviewed by the Scientific Peer Review Group and the methodology is now considered well advanced (WHO, 2004). Consequently, it has been used to complement the traditional measures of health status, making up one of five health indicators used by the OECD, as well as a part World Health Report(OECD, 2005a, 2006b; WHO, 2006).

This representation of health status excludes variables that measure lifestyle and water- borne diseases. These were excluded on the basis that the inclusion of the HALE estimate already incorporates the burden of disease.

Access to health. The one variable in this characteristic is considered more a contributing, rather than direct, factor dealing with health. The variable physicians (per 1,000 people) represents direct access to health and is a part of the HDR’s commitment to health segment. Although some discrepancies exist in the methodology, all three countries still have much in common (OECD, 2006a). Furthermore, the frequency of data for the specified time period is high.

The variables hospital beds per 1,000 people and health expenditure per capita (US$ PPP) are excluded. The former is excluded on the grounds that as technology improves and expands, the time spent in hospital decreases thereby diminishing the usefulness of hospital beds as an indicator of healthcare. The latter is due to the fact that health system expenditure is not reflective of outcomes, often seeming to make little difference to health status (WHO, 2000). This is reinforced by the fact that only recently the Cuban infant mortality rate fell below that of the US (WHO, 2006), highlighting how expenditure, even in real terms, may not be strongly linked to outcome. Other health- related (though less direct) variables, such as access to water and sanitation, are included elsewhere in the RIE index.

157

This measure began its conception as the disability-adjusted life expectancy (DALE) in the WHO

Demographics. The variables annual population growth rate (%) and total fertility rate (births per woman) have been used in studies such as the HDR (demographic trends), OECD Social Indicators, the WDI in the WB (2006b) and are a part of the UN’s millennium development indicators. The variables exhibit a high frequency, consistent methodology (indicating at what rate human resources are being regenerated), and are considered the best-known population indicators. Hence, they are included in the RIE index.

Food consumption. The variables selected to represent this dimension are total calories intake (calories per capita per day), total fat intake (grammes per capita per day), and sugar consumption (kilos per capita). These variables are originally part of the UN’s Food and Agricultural Organisation (FAO) database and make up over half the food consumption variables in the OECD Health Data. In fact, the combination of these variables (with others) makes up the foundation of food balance sheets. The data has been revised under the auspices of FAOSTAT, the FAO statistical database, and exhibits high frequency. It has been used for national policy setting and by the academic community (FAO, 2006).

Access to education. The variables are average school life expectancy – primary to tertiary (years) and net enrolment rate – secondary all programmes (% of corresponding population). UNESCO’s Institute for Statistics (UIS) collates both variables. The UIS issued a break in the classification system in 1997, making comparisons between pre-1998 and post-1998 unreliable, limiting the data frequency. Both variables are a part of the UIS World Education Indicators and the World Resource Institute’s EarthTrends (WRIE) amongst others. Additionally, the inclusion of net secondary enrolment rate is to complement school life expectancy which is best interpreted via a complementary indicator (UNESCO, 2005; UIS, 2005).

Although these variables are employed by major organisations such as the OECD, availability and quality of data from the UIS can vary. Hence, comparisons need to be made with caution.

Investment and educational quality. With regard to investment in education one variable is selected, public expenditure on education (% of GDP). The match between

the variable and the dimension is illustrated via its employment in many studies including the HDR’s commitment to education segment, and also Bontsis’ (2004) national intellectual capital index. Although the variable’s methodology is relatively straightforward, akin to the characteristic above, the UIS issued a break in the classification system in 1997, affecting comparisons between pre-1998 and post-1998 data, which should be avoided (UNESCO, 2005). Hence, the current study has the raw data for 1999 onwards.

As reviewed in Chapter 4, the controversy regarding appropriately measuring educational quality needs reiterating here. Briefly, studies conducted by Barro and Sala- i-Martin (1995) and Barro and Lee (1993) used student-teacher ratios as a proxy for quality of schooling. However, Hanushek and Kim’s (1995) study argued that it is an inconclusive proxy. Furthermore, Gundlach, Rudman and Wobmann (2002) argue that an assessment of student achievements in mathematics and natural sciences via standardised international tests, a direct measure of individuals’ cognitive skills, is preferable. This however is tempered by the fact that differences in education outlook, such as focusing on world matters compared to a job-oriented training, may result in misleading findings (Streeten, 1994).

In sum, the complexity of the educational quality concept requires the use of proxies that are, in their own way, unsatisfactory. Of these though, a learning outcome measure is the most appropriate (UNESCO, 2005).

Consequently, the RIE index includes the variable tertiary students in science, math and engineering (% of all tertiary students), which has been included in the HDI due to its perceived importance in education, and unlike the humanities area, its measurable impact on progress. Additionally, pupil/teacher ratio primary (students per teacher) is also included, as well as the OECD’s Programme for International Student Assessment (PISA) science mean score. PISA is a three-yearly survey that proxies learning outcomes for science, mathematics, reading and problem solving which are essential for full participation in society (OECD, 2005a). Its usefulness, the present research argues, outweighs its poor data frequency.