The academy programme

As the academy programme has operated since the early 2000s, with academies opening from 2002, a number of studies have previously attempted to analyse the impact of academy school status on pupil outcomes. However, as discussed within the background section of this chapter, following the election of the coalition government in 2010 and the subsequent change to the programme in the Academies Act 2010, the face of the academy school has changed dramatically. Whilst prior to 2010, academies may reliably refer to previously underperforming secondary schools that converted to sponsored academies, the alterations to policy post-2010 led to the production of both sponsored academies and converter academies, where converter academies previously attained high Ofsted ratings and typically voluntarily became academies.

Eyles et al. (2015) attempt to analyse the differences in the characteristics of Labour academies relative to the post 2010 academies using NPD data on all secondary academy conversions between the 2002/2003 and 2012/2013 academic years. The study finds that the characteristics of post-2010 academies differ significantly from Labour, pre-2010, academies; early academies are characterised by low attainment and a high proportion of disadvantaged pupils from poor family backgrounds. This is unsurprising given the remit and aims of the Labour academy policy. Coalition academies, on the other hand, have a lower share of free school meals (FSM) eligible children and experience little change in the ability composition of their intake, unlike early academies. The paper therefore warns that that generalisation and extrapolation of research findings, based on early academies, may provide a distorted and unreliable view of coalition converter academies, due to the

177 significant difference in converting school's characteristics between the two policy periods. Though it is important to review the existing literature to evaluate the methods of analysis and data adopted, the results may not be entirely meaningful in the analysis of later converter academies.

Due to the availability of data on post-2010 academies and pupil outcomes being rather restricted, due to the short length of time since the policy change, few papers have analysed the effect of the programme since 2010. It is therefore important to take into account this change in policy when both comparing results with the existing literature and when undertaking the evaluation itself within this chapter, since it is possible that converting failing schools into academies does not have the same impact upon pupil performance as in already well performing schools. In addition, the pre-2010 programme focused on underperforming secondary schools and just as few papers have analysed the academy programme post-2010, few have analysed the programme in regards to primary school pupil performance.

Furthermore, a number of early papers analysed the impact of the academy programme at the school-level, however, as discussed by Eyles and Machin (2015), it is highly likely that the pupil composition will change following academy conversion, or more generally, following a change in institution type. The results of studies at the school-level may therefore reflect the adaptation of pupil intake following conversion.

One paper with particular relevance to this chapter is Eyles and Machin (2015) which analyses the impact of academy school conversion on pupil intake and pupil performance. Using NPD pupil level data from the 2000/01 to 2008/09 academic years, the impact of academy conversion at the secondary school level is analysed. To ensure that academy enrolment is exogenous, performance effects are examined for students who are enrolled in the secondary school before it converts to an academy. The study adopts a difference- in-differences set up and defines the control group as the students enrolled in schools that become academies after the sample period ends.

The impact of academy conversion on pupil intake is firstly considered with the quality of pupil intake being measured by the KS2 score of pupils on entry to secondary school. A differences-in-differences approach is adopted to compare the intake quality of academies before and after conversion, relative to non-academies. The results suggest

178 that individuals with higher KS2 test scores are attracted to schools that convert to academies, thus suggesting significant differences in the pupil ability composition post academy conversion. This effect seems to grow over time but is only apparent for academies converters that were previously community schools.

The study goes on to examine the impact of academy status on pupil performance by observing KS4 (GCSE) attainment; since individuals must be already enrolled in the school when it converts to an academy, the study is limited to four years post conversion so that individuals may experience academy conversion when in their first of five years at secondary school; the set up allows for estimates to be obtained for each of the years’ post-conversion separately. The years of academy school attendance therefore may vary between individuals. Not all individuals who enrolled in the school remained there to take their KS4 exams, therefore an alternative approach is taken to that analysing the impact on pupil intake. In order to estimate the Local Average Treatment Effect (LATE), selection into academy conversion, defined as treatment, is accounted for by using the intention to treat (ITT) status21 as an instrument for the outcome which indicates whether the individual started in the school which later became an academy and if the pupil was enrolled in the school in the conversion year. The results indicate that the KS4 standardised score is significantly increased by attending an academy converter; this impact on pupil performance is increasing in the number of years of academy attendance. Relative to individuals who attend state schools that later become academies, individuals who attend academies therefore benefit from a higher value added. The paper also examines the impact of academy conversion on Ofsted inspection ratings and finds that on average, schools were likely to gain improved ratings after conversion relative to the control group used throughout the study. Overall, the study therefore suggests that academies are beneficial for educational outcomes.

Worth (2014) similarly analyses the impact of secondary school academies upon pupil performance at GCSE level; this study is one of only a few that attempts to analyse the impact of post-2010 academies. Academy schools are analysed based upon the year of conversion with the outcomes being measured in 2014, thus allowing for the duration of academy status to be examined. Data from a number of sources are adopted, including data from school performance tables, the DfE open academies list, Ofsted historical data

21 _{The ITT group consists of all students who are enrolled in the school prior to conversion and are in line}

179 and NFER’s register of schools; the study utilises the data from the 2009/10 academic year until the 2013/14 academic year; all academies have been an academy school for between 2 and 4 years since the academy converters of the 2009/10, 2010/11 and 2011/12 academic years are observed. The paper uses a propensity score matching methodology to compare academies to maintained non-academy secondary schools. Schools are matched based on having similar characteristics at the time they became an academy, including the proportion of pupils gaining the gold standard GCSE outcome22_{, the} proportion of pupils FSM eligible and the school Ofsted rating. A number of outcome measures in are observed and are measured in 2014, including the average KS4 point score, the percentage of pupils achieving the gold standard and the average value added. A regression analysis is also undertaken to take account of differences in converter academies and the control group in terms of individual level, pupil characteristics.

The results identify an insignificant difference in the performance of converter academies relative to characteristically similar maintained schools, though in 2014, converter academies that had opened as academies for two years were found to significantly outperform schools in the control group in the attainment of the gold standard and the value added, though differences are small. The study also fails to identify a relationship between the time open as an academy and performance, when comparing earlier and later converters.

A number of largely descriptive papers, presenting non-causal impacts of academy schools have also been presented within the literature. One example is a paper by Gorard (2014) that utilises data from the NPD and Annual Schools Census (ASC) to investigate the performance and pupil intake of academies. The study echoes the findings of Eyles et al. (2015) since it suggests that sponsored academies and converter academies are vastly different with dissimilar characteristics. Additionally, the paper highlights that converter academies are much less likely to have a high proportion of FSM children whilst high percentages of converter academies are likely to be found in areas with high levels of local socio-economic segregation.

Also of particular interest to this chapter is a descriptive report from the Department for Education (2014a) which analyses the Ofsted outcomes of both primary and secondary

180 converter academies in 2012/13. The report indicates that relative to local authority (LA) maintained primaries, primary converter academies were more likely to maintain an outstanding Ofsted rating, while those rated good or satisfactory were more likely to gain an improved rating when inspected in 2012/13. The report also indicates that the proportion of primary converter academy pupils achieving level 4 (the expected level for KS2), or above, is much higher than in LA primary schools. Of course, considering the reviewed literature, this may reflect that higher ability pupils are attracted to converter academies since the report does not report causal effects.