The Pilot Study

The purpose of the pilot study was to identify lessons to be learnt in order to improve the data collection (Yin, 2009:94) in the main study. The data collection tools were piloted between November 2012 and June 2013 at a GFE college in England, where the researcher had some affiliation at a professional level. The college was not an Outstanding college at the time of the study but was seen as an invaluable opportunity to conduct a full trial of the whole set of data collection methods, namely the online questionnaire, the semi-structured interview schedule, the observation tool and the collection of relevant documentary evidence. Access to the GB was facilitated through the college Principal, and ultimately granted by the Chair of governors. In return for the generous access, the college was provided with two interim reports after analysing the questionnaire results and followed by a final report from the pilot study.

This pilot study was conducted in addition to the initial piloting of the draft questionnaire with four individuals: the two supervisors of the project, one former governor from the pilot college and a primary school chair of governors. They were requested to complete the questionnaire and check the questions for ambiguity, assumptions within questions, double questions, leading and sensitive questions and questions that might require an unreasonable amount of knowledge or hard-to-recall elements (Bell, 2007).

Repeated piloting increased the validity of the survey tool (Bassey, 2007; Bell, 2007:231-232). A presentation of the pilot study to the college board was arranged through the clerk and all ethical guidelines described for the main study were followed including the securing of informed consent.

The piloting began with the online questionnaire which was completed by eight governors at the college. After receiving the questionnaire, the following amendments to the questionnaire were made:-

 In questions about what roles ASGs had played, the phrase, "in this college" was included so that it is clear that the question was about the actual ASG roles specific to the colleges concerned as opposed to the roles ASGs were expected to perform in general.

 In ordering Likert Scale or Ranking questions, positive responses were placed towards the right end of the scale so that the Qualtrics

®

 Revisions were made to the questionnaire so that it took more or less the length of time suggested in the research information leaflet (10-20 minutes) by eliminating the need to obtain additional comments for some questions, for example, questions to which the pilot governors did not provide additional comments.

In order to trial the observation tool, a corporation meeting at the pilot college was observed for the first 2 hours of the meeting, where there were 12 attendees including the ASG, the Principal and 3 other SMT staff. No request was made to audio record the meeting and extensive notes were made. The following points describe the lessons learnt from the observation:

 Whenever possible, voice recording was seen as essential to get the full picture of meeting proceedings and for reasons associated with the difficulty in taking notes while studying the agenda and the meeting papers.

 Prior to the meeting a request was made by the researcher from the clerk to be given a few minutes to be introduced but the chair did not follow this through. As a result, in the main case study, the request was put directly to the chair in addition to the clerk.

 Given that there were ample notes made on the meeting proceedings, it was established that the table for the ASG contributions (see observation tool, ‎Appendix D:266), could be accurately completed if done immediately after the meeting.

 Due to the complex nature of the meetings, it was difficult and often impossible to make notes on some of the visual data such as body language. This was due to the fact the research relied on only one observer, the researcher himself. Therefore, in the main study there was only limited reliance on meeting participants’ non-verbal cues.  It was important to obtain the following prior to the meeting:

o Meeting agenda and papers

o List of Current/updated governors (and names so that their initials could be used for faster note-taking).

Following the pilot observation, four governors at the pilot college, including the ASG and the Chair, were interviewed for piloting purposes using the interview schedule (‎Appendix E:269). The trial’s purposes included developing the researcher’s interviewing skills; increasing clarity by removing any ambiguity in the questions; increasing the effectiveness of the schedule in obtaining evidence related to the research questions and checking the length of the interviews. The interviews were audio recorded, following which identifiable sections were removed for protecting anonymity and then transcribed. The two supervisors listened to the first interview recording and provided their comments to serve the piloting purposes. The main points from the 4 trials were:

 Questions that attempted to get details of critical incidents (Griffiths, 1998:24) in interviews worked well (for example, “could you give specific examples of any positive impact, you/the current ASG has had

on the GB or the college?”)

 Using prompt cards in interviews worked well (‎Appendix E:269)  The researcher should:-

o avoid using fillers (well, errm,) too often

o avoid repeating or rephrasing questions unnecessarily

o be less hesitant

o pay attention to the potential differences between the committees structure at various colleges.

Following the piloting of the interview schedule, the final set of questions was finalised, aiming for the interviews to last about 60 minutes each. The above improvements and other changes made to the online questionnaire, the observation tool and the interview schedule following the pilot are shown in Appendices C-E in red.