Terms of reference

a. BackgrOund

1. UNICEF and a wide range of actors and/or partners have, throughout the past two decades or so, embarked on efforts to deliver LSE programmes to empower in-school and out-of-school children and young people for the challenges that they face as part of growing up. Life skills are determinants of behaviour. They include psychosocial capabilities for using knowledge, such as critical thinking and problem-solving; for being decisive and taking responsibility for life choices, such as decision-making; for living together, such as communication and empathy; and for completing tasks, work and other assignments, such as motivation and persistence. In line with education for sustainable development, thematic areas covered by life skills education include health promotion and disease prevention, environmental protection and disaster risk reduction, social and emotional learning and psychosocial support, human rights, citizenship and social cohesion, as well as livelihoods and financial literacy. 2. Life skills education aims to support the strengthening of skills that contribute to behaviours believed to reduce risk and vulnerability and increase opportunities in life. As a result, it has received significant attention and funding, mainly because of its importance to HIV prevention among adolescents, as can be seen from the number of programmes focusing on this thematic area, but also because of its contribution to quality education and as part of child-friendly schooling. The importance of life skills is also acknowledged in two of the six EFA goals: ensuring that the learning needs of all young people and adults are met through equitable access to appropriate learning and life skills programmes (Goal 3); and improving all aspects of the quality of education and ensuring excellence of all so that recognized and measurable learning outcomes are achieved by all, especially in literacy, numeracy and essential life skills (Goal 6).

3. In 2007, a UNICEF stocktaking exercise revealed that life skills education programmes are currently being implemented in approximately 70 countries, and that the number of countries in which national education authorities are incorporating LSE in formal school curricula is growing. A majority of countries start life skills education at the primary level, and in some cases, as early as at the pre-primary level. In only a few of the countries, life skills education is included in the curricula in one of four ways: as a stand-alone subject; integrated into one or two main carrier subjects; infused throughout the curriculum; or in a blended approach that combines integration and infusion. In a majority of countries, school-level interventions focus on short-term extra-curricular activities that are delivered only at the secondary level and do not reach a critical mass of learners. Hence several reviews have converged on the observation that the education sector has not sufficiently harnessed the strength of its core business of schooling to provide long-term and structured interventions to the large groups of children that are in their system. 4. For example, a 2009 evaluation of My Future, My Choice, a UNICEF supported extra-curricular HIV prevention life skills programme in Namibia, found that only 8.2 per cent of secondary and combined school students participate in the programme, and recommended that the programme be institutionalized into the formal school curriculum. The evaluation also recommended that rather than offering the

programme at late adolescence as is presently the case, the programme be offered as a mandatory subject to learners at the pre-adolescent or mid-adolescent stages, when they are less likely to be

179_{WHO, UNICEF, UNESCO, World Bank, UNFPA, Education Development Center, Education International, Partnership for Child}

Development, Skills for Health: An important entry point for health promoting/child-friendly schools, WHO, Geneva, 2004, <www.who.int/ school_youth_health/media/en/sch_skills4health_03.pdf>

sexually active. Several other studiesi,ii,iii,iv _{indicate, however, that inclusion in the formal curriculum does} not necessarily ensure full implementation, and that intentional life skills learning outcomes should be specified.v,vi_{They also delineate characteristics that are necessary for programmes to be effective.}179,vii,viii 5. Components that are necessary for successful LSE programmes have been analysed and used to develop five overarching programmes standards, namely that (i) LSE programmes should be rights- and needs-based, paying attention to CRC principles and girls’ empowerment in particular; (ii) LSE programmes should be results-based; (iii) life skills learning should comprise of knowledge, attitudes and skills; (iv) life skills education should be provided in protective and enabling learning environments with access to community services; and (v) that teachers should be trained on equity-oriented delivery methods and provision of psychosocial support.

6. While there is more understanding and appreciation of what the key elements of good LSE programmes should be, how they should be structured and targeted, and what would be the realistic and achievable outcomes, there has not been matching investments in monitoring and evaluation of structured, long-term and sustained interventions that are delivered through the formal system, or whether learning outcomes are achieved as a result of these efforts. For these reasons, a global evaluation could bring new information on the implementation levels and quality characteristics of LSE programmes and on the evaluation of results for children through a closer look at policy pronouncements on life skills education, curriculum and programme analyses, as well as investigating if life skills learning outcomes have been incorporated in assessment and examination systems.

B. PurPOse, OBjecTives and Overview Of The PrOPOsed

evaLuaTiOn

7. The purpose of the proposed exercise is to conduct a global evaluation of UNICEF’s support to establish sustainable and evidence-based LSE programmes. In particular, the evaluation will examine where countries are with respect to accepted knowledge about components of successful LSE programmes at formal and non-formal levels. The standards and benchmarks outlined in the UNICEF draft, Life Skills

Learning and Teaching: Principles, concepts and standards, will be used as a basis for the evaluation. The

evaluation will also assess whether LSE programmes are implemented from a right-based perspective, and make additional efforts to include the most at-risk and/or vulnerable young people.

8. In line with OECD/DAC criteria for evaluating development programmes,ix_{the evaluation will} also examine the added value of UNICEF investments in LSE programmes in terms of their relevance, coverage, efficiency, effectiveness and sustainability. For the purpose of this evaluation, these terms will be operationalized as follows:

• relevance – to determine whether programmes are consistent with beneficiaries’ requirements, country needs, policy positions, and whether their design remains appropriate given ever- changing circumstances. Relevance will also assess if the thematic coverage is in line with international commitments and the MDGs, and will focus on sustainable development. For instance, depending on the country needs, the evaluation will examine whether programmes include issues of health promotion and disease prevention, environmental protection and disaster risk reduction, social and emotional learning and psychosocial support, human rights, citizenship and social cohesion, livelihoods and financial literacy;

• coverage – to determine whether programmes are implemented at scale, with desired levels of dosage, and have reached the desired levels of participation from young people of school-going age, and subgroups of excluded boys and girls who need a more targeted approach. Coverage issues are critical in increasing the likelihood of achieving the stipulated results and/or impacts; • efficiency – to determine whether LSE programmes are planned and implemented in ways

that are coherent and recommended for effectiveness. This includes assessments of whether the standards and benchmarks are in place, meaningful partnerships have been created and resources are deployed in a cost-effective manner;

• effectiveness – to determine whether programmes stipulate plausible and feasible pathways to achieving results as represented in the results framework and programme activities, and efforts are in place to monitor implementation and measure intended outcomes; and

• sustainability – to determine if LSE programmes are planned and implemented in gender- sensitive and sustainable ways through the education system response (e.g., whether they are reflected in plans, curricula, examinations, pre-service teacher training efforts, and inspections). 9. The evaluation will be executed in two phases. The first phase will be a desk study of national efforts to examine the relevance, coverage and sustainability of programmes. This phase will focus on upstream work with education authorities. Informed by the reviews from the first phase, Phase 2 of the evaluation will examine programme coverage, efficiency and effectiveness, and implementation will be sustainable beyond the period of UNICEF support, focusing on downstream work at the national, sub- national and school levels where programmes are implemented. Phase 2 will be executed in two tracks, the first being data collection on all LSE themes that are applicable to sample countries, except HIV and AIDS, while the second track will include data on HIV and AIDS.

c.

scOPe Of The evaLuaTiOn

10. ministry of education sector responses. The evaluation will focus on the provision of life skills education as a result of a national education system response to HIV and AIDS. The overarching picture of the response (Phase I) will therefore be sought through analysis of national education policy and sector plans, formal school curricula for primary and secondary level education, pre-service teacher training curricula, inspections and examinations.

11. formal intervention in schools. Previous reviews have noted that the education sector has not harnessed the strength and reach of its core business of schooling to ensure that LSE programmes are well structured, institutionalized into the system, and reach a critical mass of in-school children. Hence a significant amount of effort will be spent on formal school-based LSE programmes in place in 2005– 2010, targeting in-school and out-of-school programmes that serve children and youth of pre-primary to secondary school age. The cut-off date of 2005 is to attempt to capture programmes that have benefited from recent recommendations for successful programming.x,xi,xii

12. non-formal and extra-curricular interventions. A significant number of LSE programmes are provided to in-school children as extra-curricular activities, delivered through school clubs and/ or community-based organizations. Other programmes are in the non-formal sector, designed for out- of-school populations to mitigate perceived risk and/or address specialized instances of vulnerability. Typically, the two forms of interventions have profoundly different reasons for being, strategies for planning, barriers for implementation and effectiveness, and groups targeted. A selection of programmes from these two categories will be included in the evaluation with the aim of examining their efficacy and gleaning from them lessons that may be applicable for large-scale implementation.

13. skills focus. The evaluation will look at the focus on psychosocial capabilities for using knowledge (critical thinking, problem-solving), for being decisive and resilient (decision-making, motivation, resilience), and for living together (communication, empathy).

14. Thematic focus. Life skills education thematic areas are in line with education for sustainable development. They include health promotion and disease prevention; environmental protection and disaster risk reduction; social and emotional learning and psychosocial support; human rights, citizenship and social cohesion; livelihoods; and financial literacy. The selection of thematic areas and their relative priority depends on the country’s situation with regard to prevalence of illnesses, violence and drug use, and high levels of unemployment, as well as whether the country is prone to natural disasters, or is experiencing an emergency from a conflict or disaster. Hence, the evaluation will examine the relevance of themes chosen in curricula, and analyse these with regard to the contextual situation. 15. implementation focus. Low implementation rates, inefficient implementation methods, and/ or lack of enabling learning environments affect access to LSE programmes. These have also been

identified as barriers for achieving results and outcomes for children. Where it is available, the evaluation will access secondary data from school visits and/or inspections to evaluate implementation capacity. 16. unit of analysis. While a number of countries will participate in the evaluation, the unit of analysis or ‘objects of study’ will be the specific life skills education and/or interventions within each country. As part of the evaluation methodology, an analysis plan will specify a set of indicators and outputs, and indicate how data will be organized, classified, compared and displayed relative to evaluation questions.

17. attribution and contribution. While it is desirable to determine if UNICEF-supported LSE programmes are being implemented, exhibit the characteristics known for effectiveness, are realistic in form of expected results, and are achieving the desired outcomes, it is often difficult to attribute results to only one source of inputs, actions or actors. A contribution approach will be taken, and the methodology of the evaluation will stipulate, a priori, how the contribution analysis will be carried out.

d.

evaLuaTiOn quesTiOns/criTeria

18. The first phase of the evaluation will focus on upstream work with education authorities. Hence evaluation questions will address the relevance of the national programme as envisaged from the perspective of the country, and in relation to the MDGs and other international commitments. The questions will also address coverage and sustainability of national programmes. A non-exhaustive list of questions include:

• Policy level – Does the MOE have a national life skills education policy, strategy or framework? Is it articulated or inferred from several documents? Does life skills education address important needs as identified in situation analyses? Does formal life skills education address important needs as shown by national commitments, policies and plans (relevance)? Is life skills learning specifically mainstreamed or referred to in the education sector plan or strategy? Does the life skills education policy, strategy or framework address out-of-school children and other disadvantaged/vulnerable children?

• school curricula – Is the role of formal life skills education interventions clearly articulated in the country curriculum framework? What thematic areas do LSE programmes address? Is life skills education provided at primary and/or secondary levels? Is life skills education mandatory or optional? Are the objectives and expected learning outcomes of the life skills-based education programmes well defined? Is life skills education provided as a stand-alone specific topic, part of a main carrier subject or infused throughout the curriculum?

• Pre-service teacher training curricula – Is life skills education part of nationally provided pre- service teacher training for primary- and secondary-level teachers?

• assessments and monitoring and evaluation – Are life skills education learning outcomes part of national examinations at the various levels? Are life skills education learning outcomes part of guidance for assessment at the various levels (intended coverage)? What is the evidence that LSE interventions are successful in achieving their intended outcomes and objectives? Have process monitoring mechanisms and instruments been developed and used to collect data on programme implementation and coverage?

19. Focusing on downstream work at the national, sub-national and school levels where programmes are implemented, the second phase of the evaluation will examine programme coverage, efficiency and effectiveness, as well as whether implementation will be sustainable beyond the period of UNICEF support. A non-exhaustive list of questions drawn from UNICEF for life skills education standards includes:

• are Lse programmes needs-based? – Did relevant gatekeepers, stakeholders and learners participate in the development of life skills programmes? Has formative research been conducted to identify risk and protective factors that influence the needs and assets of learners? Does content reflect learning needs to reduce risk and vulnerability and increase opportunities? • are Lse programmes results-based? – Do programmes identify behavioural

specific and measurable learning outcomes in the form of knowledge, attitudes and skills and are developed in line with the behavioural objectives?

• Do LSE programmes follow sound pedagogical principles? – Do teaching activities reflect established learning outcomes? Do teaching activities make use of participatory and skills- building methods that allow sufficient opportunities to practice skills development? Are teaching activities appropriate to a learner’s age, gender and experience? Are multiple teaching activities structured in a logical and cumulative sequence to support reinforcement and internalization of identified learning outcomes? Are learners informed about intended learning outcomes? Have assessment mechanisms been developed to measure individual progress towards achievement of the learning outcomes?

• are teachers trained on methods and psychosocial support? – Are teachers trained on aligning instruction with learning outcomes and assessments? Are teachers trained on life

skills content areas and sensitive subjects? Are teachers trained on the use participatory

and experiential methods for learning and teaching? Does teacher training include psychosocial support to address personal issues related to life skills education content? • are Lse programmes delivered in protective and enabling learning environments with

access to community services? – Do all learners and staff feel safe, protected, connected and valued in their learning environments? Were community service providers involved in planning of interventions? Are life skills interventions coordinated to minimize risk and maximize protection in learners’ lives? Is information collected and analysed to determine how characteristics of the learning environment and social context are helping or hindering achievement of life skills education objectives and goals?

• are Lse programmes being implemented as intended? – What, if any, are the unintended barriers to efficiency of the LSE programme? What core programmatic elements are missing, and are these being addressed by other actors and

initiatives? Are there outstanding gaps not being met at all? Have resources been used effectively to deliver high-quality outputs, in a timely manner?

• is life skills learning being assessed? – What classroom assessment methods are used for assessing student learning? How? What is the evidence that the LSE interventions are successful in achieving their intended learning outcomes? What, if any, are the unintended outcomes of the LSE programme?

e.

Overview Of meThOdOLOgy

20. The evaluation will be executed in two phases. Phase 1 will be a desk study of all country programmes with well-defined LSE programmes, and will focus on upstream work as well as examine policies, school curricula, teacher training curricula, school inspections reports, examinations and any other MoE documents that were developed in support of LSE programmes. Phase 2 will involve site visits for data collection. This phase will focus on downstream work at the national, sub-national and school levels where programmes are implemented, and will examine programme efficiency and effectiveness according to participants, beneficiaries and stakeholders.

21. sampling strategy: In the first phase (the desk study) a census study of all countries with some form of life skills-based education and/or HIV education programmes (approximately 70) will be included. Within each country, all LSE programmes that are implemented in the formal school system

In document EVALUATION REPORT GLObAL EVALUATION Of LIfE SkILLS EdUcATION PROGRAmmES EVALUATION OffIcE AUGUST 2012 (Page 145-154)