Top PDF Embedding Formative Assessment : evaluation report and executive summary

Embedding Formative Assessment : evaluation report and executive summary

Embedding Formative Assessment : evaluation report and executive summary

Somewhat 28 Very little 5 Not at all 0 During the case study visits, most interviewees (both senior leaders and teachers) were hesitant in concluding that the programme would lead to improvements in pupil attainment within the relatively short time span of two years. Some reasoned it would be hard to disentangle from other initiatives in their school which had worked in tandem with EFA, or the general upward trajectory of the school. Many wondered how it would be measured in the context of government reforms of KS4 and exam specifications. Finally, participants explained that the embedding of formative assessment principles and the improvement in pupils’ approaches to learning was a long process. In addition, teachers frequently observed that the intervention had had more impact on younger pupils who were considered more receptive and less critical to trying new things. In contrast, older pupils were described as more exam-minded and frequently questioning the purpose of specific teaching activities. A common example was peer assessment where older pupils wanted the perspective of the teacher as a subject expert rather than input from a peer. Teachers also explained that older students had not had these types of teaching techniques embedded from a young age, predicting that the new cohort of students would be more receptive to formative assessment techniques. Therefore, some suggested that it would be more appropriate to evaluate the impact on pupil attainment a number of years after delivery, among pupils who were younger at the time of the intervention.
Show more

87 Read more

ScratchMaths: evaluation report and executive summary

ScratchMaths: evaluation report and executive summary

concentrations of EAL pupils in intervention schools compared with control schools do not serve to undermine confidence in the trial and impact evaluation findings Bias and reliability in implementation and process evaluation data In Appendix J, details of the process evaluation interview and survey samples are given. Schools that did not achieve sustained participation are under-represented in the sample. Thus, reasons for low fidelity are largely inferred rather than fully established Survey data were shared with the ScratchMaths team and participants were aware this was the case. This may therefore also have led to bias in responses. However, there is broad agreement between interview data and survey data findings, and the identities of the interview sample were not shared with the ScratchMaths team. Lastly, participants were assured that steps would be taken to ensure anonymity as far as possible.
Show more

123 Read more

Mutual Evaluation Report Executive Summary

Mutual Evaluation Report Executive Summary

5. Overall, Mexico has made progress in developing its system for combating ML and FT since its last assessment by the FATF in 2004, but further work is needed to strengthen it. First, the laws criminalizing the ML and FT offenses are comprehensive but do not fully meet international standards, and there is scope to significantly improve their implementation. In particular, laws and procedures do not adequately provide for the freezing without delay of terrorist funds or other assets of persons designated in accordance with relevant United Nations Security Council resolutions (UNSCRs). Given the extent of drug trafficking, organized crime and other predicate criminal activities, the ML offenses are not being adequately investigated; the authorities have obtained only 25 convictions for ML since the criminalization of the ML offense in 1989. During the period 2004–
Show more

21 Read more

Dialogic Teaching : Evaluation Report and Executive Summary

Dialogic Teaching : Evaluation Report and Executive Summary

Data from the process evaluation supports the claim that the intervention did not last long enough to have the maximum effect on children ’ s attainment. The majority of participants (including headteachers, mentors, and teachers) reported that the Dialogic Teaching approach was having a positive effect on pupils ’ learning, as well as on pupil engagement, confidence, and motivation, but that they did not expect to see increased attainment within the scope of the project. Many teachers told us that they thought that it would take more than a year for this approach to have an effect on attainment. Similarly, headteachers, when asked about their intentions to use this approach in the future, responded by saying that they thought it needed to start in Key Stage 1 and run throughout children ’ s entire primary school experience for it to be fully effective.
Show more

78 Read more

Maths counts : evaluation report and executive summary

Maths counts : evaluation report and executive summary

‘sticky-note’ hard copy version made from folded card (see below). Privacy board or mini learning environment Maths map (sticky note version) The Maths Map is made up of three categories: ‘things I can do’, ‘things I am getting better at’ and ‘things I have stated working on’. Putting a large number of the ‘secured’ objectives from diagnostic assessment within the ‘things I can do’ category means the learner recognises immediately that they can already ‘do’ lots of maths and therefore helps give them a positive start to their programme. Learning objectives are moveable across the Maths Map and are added to as new objectives are selected for lessons. Each week the LPs take a few minutes to look at the map alongside the learner, think about progress over the last few lessons and reflect on this by moving statements across to the appropriate section of the map. LPs encourage learners to also think about ‘learning to learn’ behaviours and include these statements on the Maths Map where applicable. These include statements such as ‘I can chose a resource that will help me’ or ‘I can explain what I did’.
Show more

88 Read more

Philosophy for children : evaluation report and executive summary

Philosophy for children : evaluation report and executive summary

The evaluation team made 30 visits to treatment schools, usually one at the beginning of the intervention and one towards the end to observe changes in teacher and pupil behaviour. Schools were visited repeatedly to assess progress. The trips included observations of the initial training of teachers as well as the delivery of the programme in the classroom. Evaluators attended three training sessions as participant observers, noting the process of implementing P4C, the methods of delivery, and also teachers’ responses to the training. The observations of P4C in action were non- intrusive, with the evaluator sitting either inconspicuously at the back of the classroom or more usually as part of a circle but not taking part in the dialogue unless directly addressed. Interviews with teachers and pupils were also conducted during these visits. These interviews were very informal conversations with teachers and pupils who were involved in doing P4C intervention. In each visit a prior meeting was set up between the P4C lead and the teaching staff to discuss the lesson to be taught that day. The evaluation team members also observed the debriefing sessions after lessons in order obtain teachers’ feedback on P4C sessions.
Show more

45 Read more

Paired Reading: Evaluation report and Executive summary

Paired Reading: Evaluation report and Executive summary

In addition to the risk of treatment diffusion, the process evaluation found that there had been some variation amongst participating schools with regards to the set-up, delivery and implementation of the programme. For example, it was recommended to run the pupil training in four sessions. However, schools used a range of approaches to train the pupils involved, from not giving any advance training at all to using the first few sessions to gradually introduce them to the process. Similarly, the length of the sessions varied from school to school, with some able to use the recommended half an hour, while others were forced to condense the lesson into a shorter time period. There was also a varying level of support provided to pupils within the intervention by the teachers involved, based mainly on the reading ability of the pupils. However, these appear to be natural variations between the settings of the schools involved and are unlikely to have affected the dosage of the intervention for the pupils involved.
Show more

129 Read more

TextNow Transition Programme : evaluation report and executive summary

TextNow Transition Programme : evaluation report and executive summary

Primary coach 42 11% 27 50.0 Secondary coach 72 21% 18 58.1 Four pupil focus groups were conducted by one researcher from the evaluation team after the end of the programme. A purposive sample of participating schools was selected for the focus groups to maximise variation in terms of geographical distribution, rural/urban school location, socio-economic profile, and type of governance. Of the six schools originally approached, two declined, one due to a forthcoming Ofsted inspection, and another did not have staff capacity to make the necessary arrangements for the visit. The four schools which agreed to participate in the focus groups varied in their degree of engagement with the TextNow programme. The TextNow coordinator in each school was asked to select 6 pupils from the intervention group that were representative of the range of pupils who had received the intervention in terms of gender, reading ability, attitudes towards reading, programme outcomes and English as a second language status (EAL). Due to pupil absence, there were fewer focus group pupils at one school. In total, 22 pupils (10 girls and 12 boys) from the intervention group participated in focus groups conducted in four secondary schools. The participant characteristics broadly matched those requested by the evaluators. Pupils were asked about their experiences of the programme at both their primary and secondary schools and how they thought participation in the programme had impacted their enjoyment of reading and reading skills. Each focus group was recorded and a thematic focus group report with illustrative quotes was prepared.
Show more

95 Read more

Writing about values : evaluation report and executive summary

Writing about values : evaluation report and executive summary

The delivery of the treatment in the pilot phase was carried out in early May 2016 before the onset of the GCSE exams. Light touch process evaluation of the delivery from training of staff to implementation in the classroom was conducted in a random sample of three of the five pilot schools to test the fidelity of implementation, and assist where possible by providing suggestions and feedback. Formative feedback on the training, delivery of intervention, and teaching materials were relayed back to the project team. The pilot also helped ascertain whether the level of support and training was sufficient, identify potential hiccups and to suggest improvements needed to ensure that the main trial ran smoothly. The process evaluation was primarily in the form of participant observations, and informal chats with staff and pupils in the pilot schools to identify potential barriers to implementation, issues with data collection, possible resistance and also any potential risks of contamination. Lesson observations were very informal. More specifically we asked staff about issues relating to the delivery/implementation, resources/materials used, and if there were suggestions for improvement.
Show more

70 Read more

Hallé SHINE on Manchester : evaluation report and executive summary.

Hallé SHINE on Manchester : evaluation report and executive summary.

Overall, pupils’ experiences of SHINE were very positive and this was obvious throughout the five formal visits as part of the process evaluation. Pupils seemed to be experiencing a sense of belonging, having a good time and were kept busy throughout the day. They also showed good behaviour and satisfactory levels of participation. They had the opportunity to experience good music teaching and be creative through their involvement in often elaborate pieces of artwork in preparation for the culmination performances. They gained a sense of confidence in performing in front of an audience which resulted in an obvious sense of achievement and personal satisfaction at the end of each performance. They were also encouraged to develop an interesting view towards ‘core subjects’ such as literacy, numeracy and science as these were taught in a thematic way following the theme that was being explored. This led to an understanding of relevance and added a sense of purpose to many of the activities in which the pupils were involved. However, on the basis of the limited evidence collected regarding the exact nature of the teaching of these ‘core subjects’, the literacy, numeracy and science lessons observed did not seem to engage the learners sufficiently cognitively and critically. A further level of challenge could push learners to think harder and be cognitively engaged at a deeper level.
Show more

85 Read more

Thinking Maths: Learning Impact Fund Evaluation Report: Evaluation Report and  Executive Summary

Thinking Maths: Learning Impact Fund Evaluation Report: Evaluation Report and Executive Summary

The desired alpha was 0.05 and power was 0.8, with a minimum detectable effect size (MDES) of small (Cohen’s d = 0.2). We also needed to take into account the design effect of clustering by including an estimate for the intra-cluster correlations (ICC). This accounts for students in one school being more like each other compared to students in another school (Hutchison & Styles, 2010; Eldridge et al., 2006) when the sample is not a simple random sample, resulting in a net loss of information. In other words, from a statistical perspective, similarities between students in the same class effectively reduce the number of participants in the intervention (Torgerson & Torgerson 2013). The ‘design effect’ was used to estimate the extent to which the sample size should be inflated to accommodate for the homogeneity in the clustered data. In similar studies in Australia, Zopluoglu’s (2012) recommended an Australian ICC coefficient range of 0.2-0.3 (p.264) and the PISA 2012 Technical Report used an Australian ICC for mathematics of 0.28 (OECD 2014, p.439). Taking a conservative approach, we adopted an initial ICC coefficient of ρ = 0.3.
Show more

74 Read more

Summer Active Reading Programme : evaluation report and executive summary

Summer Active Reading Programme : evaluation report and executive summary

5.3 Interpretation The findings of this trial are in line with the few other RCTs of book gifting programmes that have been reported, which have found a limited number of significant effects. These studies have tended to examine changes to parental attitudes and the few identified effects have been small in size. In this study, there were, however, promising signs regarding the improvement of children's attitudes towards reading, particularly for children from non-FSM households. Therefore, there is potential that this programme, with refinement, could improve reading attitudes which could then act as a catalyst to encourage children to engage with linked activities or programmes that focus more directly on improving reading comprehension outcomes. For example, meta-cognition and self-regulation programmes have been found to be low cost and highly effective ways to develop comprehension and thinking skills (EEF, 2014 - http://educationendowmentfoundation.org.uk/toolkit/meta-cognitive-and- self-regulation-strategies/), which could build on the improved reading attitudes provided by an effective book gifting programme. Adopting such an approach would depend on: (a) ascertaining the effectiveness of a refined version of the Summer Active Reading programme through a further fully powered trial; (b) studies looking at the added value of linking interventions that target both reading attitudes and comprehension outcomes; and (c) a full assessment of cost-effectiveness of the Summer Active Reading programme.
Show more

68 Read more

Thinking, Doing, Talking Science: Evaluation report and Executive summary

Thinking, Doing, Talking Science: Evaluation report and Executive summary

The impact evaluation of the pupil tests used hierarchical linear modelling (HLM), a multilevel analysis in which pupils are nested within schools. Because randomisation was carried out using matched pairs, the pairings were accounted for by including them as an extra level in the analysis. This results in a 3 level model, with Pupil nested within School nested within Pair. The HLM analysis used degrees of freedom associated with the number of schools, not the number of students. Pupils in schools randomly assigned to TDTS were compared to those in the randomly assigned control group, controlling for the pre-test scores. After the main analyses across all pupils, subgroup analyses were carried out for boys and girls; for high and low achievers based on the pre-test scores (using greater or equal to the median and below the median). Although we conducted an analysis for FSM-eligible pupils (defined as ever eligible for FSM), it should be noted that there were four schools with no pupils eligible for free school meals, and nine further schools with fewer than five FSM pupils. There were a number of reasons for this low number of FSM pupils. Firstly, there was a focus on poorly-performing schools and not just those with high rates of FSM at recruitment. Secondly, more of the schools recruited through the developers’ existing contacts had low rates of FSM. Thirdly, reflecting the rural nature of parts of the county, some schools were very small and did not have many Year 5 pupils in total. Since there was such a high level of attendance at the training sessions, which acted as a proxy for implementation fidelity, it was not appropriate to perform the intended analysis by fidelity.
Show more

97 Read more

Teacher Observation : evaluation report and executive summary, November 2017

Teacher Observation : evaluation report and executive summary, November 2017

For the school-level experiment we used the sum of the two GCSE scores as the measure of pupil attainment. For the department- and teacher-level experiments we used the GCSE score for the subject taught by the particular department or teacher as the outcome variable. We also administered Year 10 maths and English tests as a secondary outcome measure of attainment for Year 10 pupils. Although some commercial maths and English tests that cover basic skills are available, we did not feel that these were adequately tailored to provide a valid assessment of the types of changes in student performance that might be expected to result from this intervention. We therefore constructed new, bespoke tests by selecting items from Key Stage 3 past papers covering the ten-year period from 2000 to 2010. We knew that these were robust test items as they were rigorously developed and trialled for the Qualifications and Curriculum Development Agency (QCDA) or the Standards and Testing Agency (STA). They also cover a wide range of ability and are appropriate to the age range and the curriculum. Our aim was to provide a valid measure that was more likely to detect subtle changes in student performance, compared with commercially available instruments.
Show more

91 Read more

Evidence for the Frontline: Evaluation report and executive summary. March 2017

Evidence for the Frontline: Evaluation report and executive summary. March 2017

The evaluation found that the development phase was instrumental in framing and formulating the implementation of E4F. This phase was important because it was consultative and invited developer schools to make an active contribution to shaping the service. There was a good level of participation by school representatives at three awareness-raising events in the development phase. They valued the fact that their role was not prescribed and that they could initiate ideas. The schools worked well together and senior leaders’ and teachers’ suggestions were incorporated into the design of the service which, through their contribution, evolved from a telephone-based to a web-based communication model. The website was a notable success: it was created in time for the E4F launch day and was a pivotal feature of the service. The evaluation ascertained that establishing and running E4F at this scale was feasible, though there were challenges in recruiting experts who collectively had a range of appropriate expertise, matching teachers’ questions to experts’ areas of expertise, and securing answers to questions. While it was reasonably easy to recruit schools to the pilot, sustaining teachers’ use of the service given their busy work schedules was a challenge. Further work on supporting research experts in their role within E4F, in order to engender their further investment, should be considered. In addition, demand was at the top end of what was expected, and hence if any further scale-up is considered, additional brokers and academics would need to be recruited. A hub model could be considered—whereby additional lead schools provide brokerage support to groups of schools.
Show more

80 Read more

Affordable online maths tuition : evaluation report and executive summary.

Affordable online maths tuition : evaluation report and executive summary.

Education Endowment Foundation 97 Appendix F – Process Evaluation: Questionnaire Intervention Schools AFFORDABLE MATHS TUITION-Autumn Survey Your students are part of the Online Maths Tutoring Project funded by the Educational Endowment Foundation. By now you are up and running using the online tutoring provided by Third Space Learning (TSL). As you have been previously notified by the York evaluation team, we are conducting a process evaluation alongside the randomised controlled trial. The purpose of the evaluation is to collect data to understand more about how online tutoring is running in order to identify issues that might need addressing if more schools opted to use this tool in the future. We are interested in knowing what has worked well for you and your students in the first few weeks and what has not. This questionnaire has 12 questions and will take less than 10 minutes to complete. The questionnaire is organised into three key sections.
Show more

136 Read more

Summer Active Reading Programme : evaluation report and executive summary

Summer Active Reading Programme : evaluation report and executive summary

5.3 Interpretation The findings of this trial are in line with the few other RCTs of book gifting programmes that have been reported, which have found a limited number of significant effects. These studies have tended to examine changes to parental attitudes and the few identified effects have been small in size. In this study, there were, however, promising signs regarding the improvement of children's attitudes towards reading, particularly for children from non-FSM households. Therefore, there is potential that this programme, with refinement, could improve reading attitudes which could then act as a catalyst to encourage children to engage with linked activities or programmes that focus more directly on improving reading comprehension outcomes. For example, meta-cognition and self-regulation programmes have been found to be low cost and highly effective ways to develop comprehension and thinking skills (EEF, 2014 - http://educationendowmentfoundation.org.uk/toolkit/meta-cognitive-and- self-regulation-strategies/), which could build on the improved reading attitudes provided by an effective book gifting programme. Adopting such an approach would depend on: (a) ascertaining the effectiveness of a refined version of the Summer Active Reading programme through a further fully powered trial; (b) studies looking at the added value of linking interventions that target both reading attitudes and comprehension outcomes; and (c) a full assessment of cost-effectiveness of the Summer Active Reading programme.
Show more

68 Read more

EVALUATION REPORT. EVALUATION OF UNICEF S EMERGENCY PREPAREDNESS SYSTEMS Executive Summary

EVALUATION REPORT. EVALUATION OF UNICEF S EMERGENCY PREPAREDNESS SYSTEMS Executive Summary

This independent evaluation was commissioned by UNICEF’s Evaluation Office (EO) to pinpoint the specific gaps in UNICEF’s EP policies and systems that need to be addressed or strengthened. Objective The evaluation’s objective is to examine, as systematically and objectively as possible, the relevance, effectiveness, efficiency 3 , connectedness, sustainability and coverage of UNICEF’s current EP systems across its global operations 4 . Its main purpose is to help UNICEF become better prepared for the wide range of emergencies it faces – and to blunt the effects of emergencies when they do occur – and thus help it save more lives and reduce human suffering in ever-better ways. In order to look both back and ahead, the evaluation considers the evolution of UNICEF’s EP systems to date as a means of tracking progress. The evaluation is both summative and formative (with emphasis on the latter).
Show more

8 Read more

TELEGEOGRAPHY REPORT EXECUTIVE SUMMARY. Executive Summary

TELEGEOGRAPHY REPORT EXECUTIVE SUMMARY. Executive Summary

The content on the preceding pages is a section from TeleGeography's TeleGeography Report The work is based on sources believed to be reliable, but the publisher does not warrant the accuracy or completeness of any information for any purpose and is not responsible for any errors or omissions. This work is for the confidential use of subscribers. Neither the whole nor any part of this publication may be reproduced or transmitted in any form or by any means, electronic, mechanical, photocopied, recorded or otherwise, without prior written consent from PriMetrica, Inc.

10 Read more

TELEGEOGRAPHY REPORT EXECUTIVE SUMMARY. Executive Summary

TELEGEOGRAPHY REPORT EXECUTIVE SUMMARY. Executive Summary

The content on the preceding pages is a section from TeleGeography's TeleGeography Report The work is based on sources believed to be reliable, but the publisher does not warrant the accuracy or completeness of any information for any purpose and is not responsible for any errors or omissions. This work is for the confidential use of subscribers. Neither the whole nor any part of this publication may be reproduced or transmitted in any form or by any means, electronic, mechanical, photocopied, recorded or otherwise, without prior written consent from PriMetrica, Inc.

11 Read more

Show all 10000 documents...