Much of the research on Aimhigher activities suggested an increase in levels of aspiration, changes in attitudes towards HE, improved levels of confidence and self- esteem (AHKM, 2009; HEFCE, 2009; Miller and Smith, 2011; Doyle and Griffin, 2012). For example mentoring schemes were considered to be highly effective in changing attitudes towards HE and aspiration levels (NFER, 2010; EKOS, 2007). Yet there was relatively little reliable evidence to support such claims of a positive impact (Gorard et al., 2006; Thomas, 2011). Poor research design and reporting hindered the ability to draw firm conclusions of the actual impact and effectiveness of outreach activities (Gorard et al., 2006; Thomas, 2011). Robust research has yet to explore the impact as well as the effectiveness of outreach activities to ascertain how and why activities had such a positive impact (Gorard et al., 2006; Thomas, 2011; Doyle
59 and Griffin, 2012). It is therefore difficult to conclude whether outreach activities are a suitable method to address the patterns of participation inHE.
Despite this, evident in the literature are claims such as summer schools being the crown jewel of outreach provision (Aimhigher Hampshire and Isle of Wright, 2009), as this activity was thought to increase levels of motivation towards school and positively influence attitudes towards HE (Hatt, Baxter and Tate, 2009). The residential feature of the summer schools was regarded as the aspect that was most effective in raising learners’ aspirations and awareness of HE (Hatt, Baxter and Tate, 2009). However, little research explored the effectiveness or the impact of this activity to discover what the key components were and whether they did change attitudes and raise aspirations (Gorard et al., 2006).
Ambassadors were also considered to be the key ingredient in widening participation activities to raise aspiration levels (Ylonen, 2010).The employment of undergraduate students as Ambassadors to facilitate outreach activities was extremely popular (Austin and Hatt, 2005), and continues to be so. This was because Ambassadors were thought to be able to break down barriers to progression whilst also providing a level of psycho-social support, improving levels of confidence and self-esteem (Smith, 2009; Rogers, 2009). Being closer in age to the young people the Ambassadors were also thought to be able to discuss the benefits of staying in education while alleviating any fears the young people may have regarding university (Doyle &Griffin, 2012). Their inclusion in outreach activities was to deliver the message of the widening participation agenda by demystify what HE was and to engage learners on a journey to achieve their full potential (Austin and Hatt, 2005).
Research suggested that the potential impact of an outreach activity was dependent upon the Ambassadors. The style of delivery and engagement with the activities was thought to be affected by the personality of the Ambassadors (Dismore and Smith, 2007). Lewis and Ritchie (2010) explored this by facilitating focus groups with forty- six young people who had participated in the Aimhigher Associate Scheme, which involved an Ambassador meeting with a young person for an hour a week over an academic year to provide continuous support for the young people to complete their studies. The young people thought the Ambassadors were good role models but to
60 be this they must be bubbly, understanding, a good listener, trustworthy as well as have similar interests (Lewis and Ritchie, 2010).
Lewis and Ritchie’s (2010) exploration of the Associate Scheme reported that the Ambassadors level of commitment was key. The mentees in their research discussed the importance of the Ambassadors commitment to developing a positive relationship with their mentee (Lewis and Ritchie, 2010). Where relationships had not developed in the Associate Scheme it was thought to relate to the Ambassador being either disorganised or not frequently attending the pre-arranged meetings. If an Ambassador was perceived as noncommittal, the young people discussed the scheme as a waste of time as there was no opportunity to develop a positive working relationship (Lewis and Ritchie, 2010; The Focus Group UK, 2009). The effectiveness of a mentoring scheme as explored in this research was suggested to relate to the actions of the Ambassadors. Highstandards of preparation and motivation were required for the scheme to be of impact (Lewis and Ritchie, 2010).
The enthusiasm and attitude of Ambassadors for the activities had the potential to influence the overall success of an activity (Kerrigan and Carpenter, 2008). Yet little research explored the direct impact of Ambassadors on outreach activities (Ylonen, 2010). Relatively little was known about the impact of Ambassadors in spite of their significant presence in outreach activities (Gorard et al., 2006). Further exploration of the role of an Ambassador and their experiences is therefore required to support such assumptions (Ylonen, 2010).
Research suggested also Aimhigher activities increased confidence levels, levels of self-belief as well as improved attitudes towards future studies (Evidence Plus Consultancy, 2011). Yet there was a lack of evidence to support such claims. These claims are primarily based on information gathered on the immediate impact of participating in an activity. That is, attendees were asked to complete an evaluation form immediately after the event had occurred. Therefore no baseline data was collected to measure the distance travelled to determine the impact of an activity. Moreover, due to the completion of an evaluation form taking place immediately after the event, the mood of the individual may have affected the responses provided, as after an event the young people may have had a more positive outlook on HE overriding previous thoughts, known as the 'wow effect' (Brown, 2011). This affective
61 emotional response was due to the young people observing something different (Brown, 2011). The attitudes being measured may not therefore have a sustained presence. Research has not yet explored whether the immediate impact has a sustained impact on educational decisions at the two crucial transitional stages, 16 and 18. This is due to a shortage of long-term evidence to determine the actual benefit of participation in outreach activities on attainment and educational progression (Gorard et al., 2006).
The actual benefits of participation were rarely reported. Partnerships commonly used questionnaires to evaluate activities (Younie, 2009). These were however not standardised measures to evaluate the impact of an activity (Passey and Morris, 2010). Research reports often omitted descriptions of the questionnaires used, numbers completed and information to replicate the research. This approach to the evaluation of activities provided little conclusive evidence of impact. It did not inform why anintervention had been a success (Kerrigan and Carpenter, 2009). The questionnaires used to evaluate outreach activities were neither reliable nor valid to draw firm conclusions of the actual benefits of participating in an outreach activity. Furthermore, there was relatively little research exploring the effectiveness of activities such as summer schools (Hatt, Baxter and Tate, 2009).
This shortage of evidence was somewhat related to the infrastructure of Aimhigher. It was difficult to access pupil datasets to enable the tracking of participants (Passey et al., 2009). Furthermore, the initial set up of activities did not provide the opportunity to collect baseline information to track participants (Passey et al., 2009). Due to these limitations much of the research focused on the immediate impact. Moreover, practitioners generated a wealth of evidence but lacked the skills to robustly evaluate outreach activities and carry out statistical analysis contributing to the evidence gap (Chilosi, Noble, Broadhead and Wilkinson, 2009).
Reliable evidence is required to overcome the number of limitations noted above and that also feature in the HEFCE commissioned review of the widening participation literature led by Stephen Gorard in 2006. The papers appraised in their review provided insufficient detail of the methods used, rarely included a comparison group and used datasets without a consideration for there limitations (Gorard et al., 2006). Papers reviewed explored only the perceptions of the staff and young people in
62 school-based interventions instead of exploring the actual effectiveness, echoing the limitations noted in this chapter (Gorard et al., 2006). For instance research evaluating summer schools focused on the young people's perceptions of impact rather than the actual impact of participating. Shortcomings were also detailed for the written reports as there were frequent omissions of basic information for replication such as number of participants, sample selection and research design (Gorard et al., 2006). Moreover, a major blind spot in the evidence-base of outreach activities was not evaluating the efficiency of interventions (Gorard et al., 2006). The poor quality of research and reporting questioned the reliability and validity of the current evidence. The limitations in the reporting prevented the review to draw firm conclusions of the impact and effectiveness of outreach activities.
On the contrary, it was considered the limitations of the current evidence-base of outreach activities identified were too critical, as the critique resided with a new orthodoxy approach to evidence (Doyle and Griffin, 2012). This approach is thought to restrict and modulate practices in the field of education just to produce good science and inform policy and practice (Hodkinson, 2001). Methods of data collection according to this approach should maximise objectivity and reduce subjectivity (Hodkinson, 2001). Unlike each piece being standalone, this approach would ensure findings could be replicated and generalised so to provide direction for new research. However, this approach was considered unfeasible for widening participation research. For instance experimental design advocated by the new orthodoxy approach was thought to be unethical for evaluating outreach activities (Doyle and Griffin, 2012). However, an objective approach is necessary to inform policy and practice.
Bold statements of impact and effectiveness were evident in the literature. Yet papers infrequently provided the evidence to support such conclusions. Moreover rarely were the limitations of the research design noted. Each paper discussed was standalone (Gorard et al., 2006). However, to build an evidence base, it is important to learn from previous research and provide direction for further research. The presentation of limitations and counterfactual evidence was rare, but thisis of importance to judge the merit of the research undertaken (Thomas, 2011). However, funding for outreach activities was based on impact thus the information provided by practitioners may have been tainted due to the pressures to report findings that
63 adhered to a predetermined plan (Gorard et al., 2006). To evaluate the potential impact and effectiveness of outreach activities and to limit researcher bias, an independent evaluation is necessary.
There was little evidence to demonstrate the impact of pre-entry interventions such as those provided by Aimhigher (Gorard et al., 2006). Outreach activities aim to encourage participation inHE, yet there was a significant absence of research evidence to conclude whether this has occurred (Gorard, See and Davies, 2012). To overcome the limitations noted, OFFA has stipulated that robust research needs to inform the outreach provision provided by HEIs. Therefore these flaws required addressing. The review of the Aimhigher evidence-base highlights areas in which improvements can be made to enhance the evaluation process. To improve the evaluation of outreach activities it must start with the design, which is also an area of concern (Cabinet Office, 2009).