In 2012 the Brookings Institute and the UNESCO Institute for Statistics assembled the Learning Metrics Task Force (LMTF). In the course of 18 months, they partnered with 30 member organizations and 186 working group members from 118 countries. LMTF served two purposes: The first was a political mission to put learning on the Post-2015 Agenda. At this, they and a larger alliance behind them succeeded. In the 2014 Muscat Agreement, we find:
Target 3: By 2030, all youth and at least x% of adults reach a proficiency level in literacy and numeracy sufficient to fully participate in society, with particular attention to girls and women and the most marginalized.
Target 5: By 2030, all learners acquire knowledge, skills, values and attitudes to establish sustainable and peaceful societies, including through global citizenship education and education for sustainable development.
In the May 2014 Working Draft of Indicators for Sustainable Development Goals, the Sustainable Development Solutions Network suggested that Goal Three of the Sustainable Development Goals (SDGs) be “Ensure Effective Learning for All Children and Youth for Life and Livelihood.” The proposed Indicator 19 reads, “Percentage of girls and boys who master a broad range of foundational skills, including proficiency in reading and foundational skills in mathematics by the end of the primary school cycle (based on credibly established national benchmarks). At the time of writing , the indicator was yet to be developed.
The second aspect of the LMTF was the technical mission of trying to establish universal standards of what should be measured. One of their earliest innovations was establish seven domains of school-based learning: physical well-being, social and emotional, culture and the arts, literacy and communication, learning approaches and cognition, numeracy and mathematics, and science and technology. They then went to work establishing sub-domains across three levels of education: early-childhood, primary, and post-primary.
The first trend is that the Muscat Agreement and the Sustainable Development Goals are moving in the direction of nationally-defined learning targets. LMTF has acknowledged that 100 subdomains are too many for a global measurement framework (see Anderson, 2014). Because “there are no internationally recognized standards for defining “proficiency in reading”, “it is recommended that each country adopts and/ or defines a core set of standards that can be assessed either through school-based or household-based assessments” (SDSN 2014: 52). It is further recommended that, “that each country adopts and/or defines foundational numeracy skills standards that, while being locally relevant, are referenced in some way to international benchmarks.”
The LMTF is now in the process of working with individual countries to develop the capacity to measure and monitor learning. At the same time, LMTF, UNESCO, UNICEF and other organizations are “developing international benchmarks for these indicators, recognizing the variation of education systems and contexts across countries” (ibid). Their goal follows recommendation of a “composite measure at the end of the primary school cycle” (SDSN 2014: 52).
The second trend is the growth of existing international learning assessments. The first PISA, which was conducted in 2000, included 32 countries (28 OECD countries and four partners.) The 2012 PISA had 65 participants. In an effort to expand their presence, OECD is introducing PISA For Development, “[a] project which aims to enhance the PISA tests and background questionnaires to make them even more relevant for a broader range of contexts, particularly those found in developing countries.” Similarly, TIMSS expanded from 25 participating countries for fourth grade assessments in 1995 to 52 in 2011.
Measuring learning outcomes has become a contentious field in education policy and research. The numbers PISA, TIMSS-PIRLS, and other comparative assessments generate become used by policy makers and political entrepreneurs to either boast of Illustration 3: The Learning Metrics Task Force’s “A Global Framework of Learning Domains”
achievements or warn of losing international competitiveness. Learning outcomes in the form of standardized tests have been used in many countries as a benchmark for teacher quality.
Many also fear the inherent reductionism in assessments of this size. They tend to focus on mathematics, science, and literacy because they seem the easiest to internationally compare. Within this report, for instance, the TIMSS mathematics assessment had the broadest representation of Commonwealth countries. There are also unresolved issues over the nature of learning outcomes and curriculum. If specific set learning outcomes are prized and rewarded over others, they will tend to shape what is taught in classrooms. What learning outcomes are measured, then, can have powerful impact on shaping curriculum. The careful wording of the Sustainable Development Goal is written to empower school systems to make their own locally relevant and useful benchmarks. Following this path will make league tables more difficult. League tables, however, are in vogue and there is a risk that existing international assessments will make the effort of assisted localization more difficult.
Table 3: The Learning Metrics Task Force’s Proposed Sub-Domains for Primary-Aged Pupils
Physical Well-Being Physical health and hygiene, food and nutrition, physical activity, sexual health
Social & Emotional Social and community values, civic values, mental health and well-being
Culture & the Arts Creative arts, cultural knowledge
Literacy and Communication
Oral fluency, oral comprehension, reading fluency, reading comprehension, receptive vocabulary, expressive vocabulary, written expression/composition
Learning Approaches and Cognition
Persistence & attention, cooperation, autonomy, knowledge, comprehension, application, critical thinking
Numeracy and
Mathematics Number concepts and operations, geometry & patterns, mathematics application
Science and
Chart 37: Youth Literacy Rate By Commonwealth Region (2000-2015)
84 92 100 100 88 92 79 81 0 10 20 30 40 50 60 70 80 90 100 Asia[7 of 7] Advanced Economies [2 of 7] [4 of 9] Pacific Sub-‐Saharan Africa [18 of 18] 2000 2015 99 100 97 99 84 89 74 76 0 10 20 30 40 50 60 70 80 90 100 Very High
[3 of 8] [6 of 17] High [10 of 12] Medium [13 of 14] Low 2000 2015
Chart 38: Youth Literacy Rate By Commonwealth Human Development
Literacy
EFA Goal 4, about literacy, is also difficult to measure. On this theme, the EFA Global Monitoring Report noted four discrete understandings (UNESCO 2005: 148):
• Literacy as an autonomous set of skills, • Literacy as applied, practised and situated, • Literacy as a learning process, and
• Literacy as text.
Even the first of these, which is the most common understanding insofar as it relates to skills of reading and writing, encounters challenges in definition and measurement, particularly when comparing across very different categories of languages such as Arabic and Chinese. Analysts may not agree on the intervals in measurements of literacy or on the instruments for securing those measurements.
Youth literacy moved from 84% to 92% in Asia, remained at nearly 100% in the Advanced Economies (though only two of seven countries submit data), 88% to 92% in the Pacific (with only four of nine countries reporting data), and with mild improvement in Sub- Saharan Africa, moving from 79% to 81% but with all countries reporting data (see Chart 37 on page 66). By Human Development Index Levels, Very High and High averages remained stable and almost universal, while Medium HDI countries moved from 84% to 89%, and Low HDI countries moved from 74% to 76%. Of all the goals, EFA Goal 4 might have had some the weakest progress (see Chart 38 on page 66). Along with EFA Goal 3, it might be classified as one of the neglected if not forgotten goals.