Availability and comparability of education data on equity dimensions

DIMENSIONS

That education in a society should be equally distributed and impartial, regardless of individual or group characteristics, is a central tenet of educational equity. The group measures introduced earlier provide ways to assess the impartiality of education systems. But they require thoughtful identification of the characteristics that are likely to make children and young people more vulnerable and, therefore, more likely to undermine impartiality and ultimately equity in education.

22 For more detailed reviews on the availability and comparability of data for the measurement of education equity, see Education Equity Research Initiative (2016) and UIS (2016).

Although the determinants of disadvantage vary by context, certain factors have emerged in international frameworks that seek to improve equity in education. For example, the Incheon Declaration (UNESCO, 2016a) identifies the following key dimensions that need to be taken into account to achieve equity: gender, disability, forced displacement, and diversity along cultural, linguistic and ethnic lines. In addition, poverty, residency, gender and disability are named in SDG 10 on reduced inequality and in Goal 4 on education. These characteristics are all often associated with resource deprivation or discrimination and have known predictive effects on education experiences and outcomes. Section 4.2 summarizes some of the disparities linked to personal and household characteristics.

Although international frameworks make clear that the education agenda must target all marginalised individuals and groups, more efforts are needed to collect disaggregated data to address different equity dimensions. It is important to note that characteristics not explicitly mentioned in the SDGs also play a role in equity. For example, individuals’ gender identity can strongly influence their educational experiences but such information remains largely absent from many data collection efforts.

The following sections explore how and whether equity dimensions are currently being measured in major international and regional sources of education statistics. The results show that, at present, the potential to assess the impartiality of education systems is constrained by the scarcity and non- comparability of data on key equity dimensions. Although some promising efforts are underway, greater investment is needed to ensure better and more widespread equity analyses in the future.22

3.4.1 Scarcity of data on key dimensions of equity

Table 3.3 documents the availability of education data by equity dimension across popular international and regional education databases. As Table 3.3 shows, education data disaggregated by sex are more widely available than by other dimensions. This is due to the relative ease of collecting data by sex through different modes of data collection, such as school censuses, household surveys, population censuses or learning assessments. Indeed, measurement of education equity with respect to gender differences tends to be limited more by the unavailability of certain education data (e.g. on topics such as learning or absenteeism) than by the inability to disaggregate available statistics by sex, as recognised by the MDGs and now by the SDGs.

Nevertheless, gaps in the availability of gender- disaggregated education statistics persist. A recent UIS survey found that only 85% of existing data for SDG 4 indicators on education can be disaggregated by gender (UIS, 2016). More and better data are also needed on the intersection between gender and other dimensions of inequality. Such intersection often compounds educational vulnerability, with poorer or more rural girls at particular disadvantage in some contexts (see Section 4.2.1). These intersecting dimensions can be measured from certain primary sources (e.g. large-scale household surveys) and are available from some databases, including the UIS. Stat online database, the World Inequality Database on Education (WIDE) and the Education Policy and Data Center (EPDC), as well as in the 2016 Global Education Monitoring Report (UNESCO, 2016b). However, improving their coverage – and coverage of

23 Whether residency data are collected by national administrative sources varies by country and over time. Such data are often but not always collected by school censuses and other national sources.

24 Although collecting information about these individual- and household-level characteristics at schools poses difficulties, it is feasible to do so. For example, UNICEF (2014) provides recommendations for measuring disability in school censuses, and the UIS and the Global Partnership for Education began collaborating in 2017 on the production of statistics on education and disability based on administrative records.

gender-disaggregated education statistics in general – must remain a focus in present and future work. Residency (urban or rural, sub-national region) is relatively simple to document, like gender. In fact, all primary data sources covered in Table 3.3 include urban-rural residency as an equity dimension.23 _By comparison, the coverage of sub-national regions is more common in household-based surveys and school surveys than in school-based student assessments, which often have smaller samples and are limited in the extent of detail they can report. While residency is widely collected as an equity dimension in primary sources, it is not always shared as an equity dimension in major international education databases. Urban-rural residency is included in 7 of 11 international databases, but sub-national regions in just 3 databases (see Table 3.3). One constraint on greater international coverage of residency information stems from the lack of comparability, an issue that will be further explored in the following section. In short, while residency is regularly collected by primary sources, its utilisation at the international level is sporadic at best.

Whereas collecting gender and residency data is straightforward, it is more difficult in school-based data collection (e.g. school censuses) to gather information about other important equity dimensions – such as wealth, ethnicity, language, disability and migration status.24 _{Indeed, as Table 3.3 shows,} apart from school censuses, all primary sources collect information on wealth or socioeconomic status, though only one-half of international and regional databases publish education statistics using wealth as an equity dimension. Information on ethnicity and language is also collected by most primary sources, at least in some countries, but

Table 3.3 Availability of equity dimension data in international and regional databases and primary sources

Equity dimension Education focus Education level

Sex Residency - urban/rural Residency - r

egion W ealth or SES Ethnicity , r eligion, or language

Disability Migration status Resour

ces/inputs

Access Participation Retention/survival Attainment Lear

ning outcomes

Literacy/skills Pre-primary Primary Lower secondary Upper secondary Post-secondary

Institution Source

International databases

EPDC - FHI 360 epdc.org P P P P P P P P P P P P P P P P

ILO, UNICEF, World Bank Understanding Children’s Work P P P

OECD OECD.stat P P P P P P P P P P P

UIS UIS.Stat - administrative data P P P P P P P P P P P P

UIS UIS.Stat - household survey data P P P P P P

UN UNGEI P P P P P P

UNESCO GEMR WIDE P P P P P P P P P P P P P P P

UNICEF data.unicef.org P P P P P P P

UNICEF TransMonEE * P P P P P P P

USAID Early Grade Reading Barometer P P P P P

World Bank EdStats/Educational Equality P P P P P P P P P P P P

Household-based surveys and assessments

ICF International DHS P P P P P P P P P P P P P P

National agencies Population censuses varies by country varies by country varies by country PAL Network members PAL Network learning assessments P P P P P P P P P

UNICEF MICS P P P P P P P P P P P P P P P

School-based student assessments

CONFEMEN PASEC * P P P P P P P P

IEA PIRLS and TIMMS P P P P P P P P

OECD PISA P P P P P P P P

SACMEQ SACMEQ * P P P P P P P

UNESCO/LLECE PERCE, SERCE, TERCE * P P P P P P P

School- or teacher-level surveys

National agencies Annual school census varies by country varies by country varies by country * Regional initiatives

is only published regularly in the WIDE database. As with residency, challenges with comparability of data on these topics, especially ethnicity and language, limit more widespread use. For disability and forced displacement, primary data collection is very limited even though these dimensions are widely recognised as factors that are likely to create educational disadvantage. It is encouraging that international efforts led by the Washington Group on Disability Statistics (subsequently referred to as the Washington Group) and UNICEF have championed measurement of disability, with more widespread collection of disability data likely to be realised in the near future. The 2016 Global Education Monitoring Report highlighted the need for – and challenges of collecting – data on forced displacement (UNESCO, 2016b), which will hopefully garner more attention for measurement on that dimension.

In sum, national agencies and international organizations recognise the importance of being able to disaggregate data by key equity dimensions. In practice, however, data are more widely available for some dimensions (e.g. gender, residency and wealth) than others (e.g. ethnicity, language, disability or migration status). In some cases, like residency, data availability is limited more by problems of comparability and availability at the international level than by data collection itself. For other dimensions, notably disability and migrant status, there are limited data available from primary sources. An important next step is therefore to mainstream the collection of data on those topics. The next section turns from the topic of data availability to the specific challenges that arise with data comparability when using key equity dimensions.

3.4.2 Comparability of data on key dimensions of equity

Even where data on equity dimensions are available, the differences between definitions, modes of data collection and culturally-specific response options make it difficult to make comparisons within and

across countries. This section explores these challenges further and provides examples from the primary data sources shown in Table 3.3.

One type of comparability issue arises from differences in how dimensions are defined across sources. For example, there are often country- specific definitions of urban and rural that are used by DHS and other surveys, with some based on population size and others based on infrastructure (DHS, 2013). Definitions of wealth also vary, and the majority of sources include a composite index of relative economic or socioeconomic wellbeing, though the specific items included in these measures vary by data source, country and time. While such discrepancies need not prevent the use of these dimensions in international comparisons, they do mean that comparisons should be understood to represent country-specific, and sometimes data- source-specific, definitions of concepts.

Comparability can also be undermined by differences in categories of responses, especially with measures of identity, such as ethnicity, religion, indigenous group or language. Identity groups are important for equity analyses, because disparities between them may suggest discriminatory policies or unfair resource allocation. However, comparability of data on identity groups within countries depends on which identity groups are considered. For some data sources, the choice is driven by the relevance of certain social divisions – all countries have some degree of ethnic, religious and linguistic diversity but those cleavages do not always drive disadvantage. In Latin America, for example, indigenous status is an important social division with ramifications for educational inequality, and the Latin American Laboratory for Assessment of the Quality of Education (LLECE) – which has conducted learning assessments across the region, as well as censuses and household surveys – seeks information about it. For learning assessments, language is of particular interest because command of the language of instruction is a strong determinant of academic performance (Brock-Utne, 2007).

Even when ethnicity, religion or language are measured repeatedly over time, the groups identified might vary. For example, 1989 and 1999 censuses in Viet Nam list roughly 50 groups, most of which are common to both surveys, whereas the 2009 census includes only three categories: Kinh (the ethnic majority group), other ethnic group and unknown. Shifting categories result from survey design or changes in the social relevance of certain categories. Modern conceptualisations of ethnicity tend to see ethnic identities as simultaneously malleable and durable, with the social or political relevance of certain categories evolving over time (Brown and Langer, 2010). In quantifying the concept of ethnicity, surveys must wrestle with the challenge of assigning concrete codes to blurry concepts and must negotiate

occasional tensions among feasibility of measuring many groups, relevance and comparability over time. Differences in cultural understandings of concepts may also complicate comparability. How to avoid such problems has been a particular focus in the measurement of disability, as the concept of disability itself is often deeply rooted in cultural norms.

Population censuses, for example, may reflect local understandings of disability and priorities for data collection and therefore may not be compatible with international standards. The Integrated Public Use Microdata System (IPUMS), which works to harmonise data from over 270 censuses, cautions that, even when responses on disability can be presented under a common variable, comparability across surveys is complicated by differences in questionnaire phrasing, what counts as a disability (e.g. some censuses include chronic diseases under disability) and how severe a condition must be to be labelled a disability. Fortunately, question sets on disability developed by the Washington Group and UNICEF are increasingly adopted in household survey programmes and have also been endorsed in the census guidelines by the United Nations Statistics Division (UNSD) (UN, 2015) and the Conference of European Statisticians (UNECE, 2015), meaning that availability and comparability of disability data should

continue to improve (see Section 4.2.1 for a more detailed discussion of the links between education and disability).

In conclusion, more comparable data on key

dimensions of inequality and greater use of such data in equity assessments are achievable. The growing consensus around the measurement of adult and child disability indicates that greater harmony in measurement of key equity dimensions is theoretically possible. International attention to the measurement of other dimensions could yield improved availability and comparability of data for those areas as well. The effects of forced migration on equity in education, for example, warrant special attention. In advocacy literature this dimension is regularly tied to severe vulnerability, yet it has been difficult to measure in many national and international surveys on education to date.

3.5 DESIGN AND SAMPLING