The development of the survey instrument

Having identified the nature of the nature of the demotivators, which was the first research question for this PhD study, the survey instrument was designed to investigate the remaining research questions, namely: to what extent were these demotivators common to most NQTs across the sector? Were there patterns of prevalence and did most NQTs expect to remain in teaching? In addition, the instrument needed to accommodate open comment sections to enable the addition of further pieces to the patchwork mosaic depicting the nature of demotivators.

A variety of instruments were studied to get a feel for the medium such as the number and type of questions. Two instruments were particularly useful as they had been used to elicit data from teachers. These were the ‘Survey into the impact of Further Education Initial Teacher Training Bursaries’ (York Consultancy, 2003) undertaken for the Department for Education and Skills (DfES) and ‘Becoming a Teacher’ (Hobson et al, 2003) funded by the DfES,

the General Teaching Council for England (GTCE) and the Teacher Training Agency (TTA). The first survey was aimed at FE teachers and the second was aimed at compulsory school teachers. Although my instrument eventually looked quite different, the examples were useful for deciding what biographical details would be pertinent for my instrument. However, there was a

substantial difference in size between the two examples (the FE survey had 4 pages whereas the second survey had 15 pages). Mindful that a huge

questionnaire might discourage potential respondents from completing the instrument, I aimed to keep the size down to 6 pages (excluding the cover).

My final survey instrument is attached as Appendix 4. It posed 46 questions in total and was designed in three sections: ‘about you’, ‘about your work’ and ‘your experiences in the field’ (in that order). The first section requested details to obtain profile information: respondents’ sex, age group, the highest qualification achieved and their ethnic group. Although de Vaus (1993) does not recommend starting with demographic questions first, and indeed neither of the examples outlined above started with sex and age questions, de Vaus does recommend that questions should be grouped into sections: ‘this helps structure the questionnaire and provides a flow’ (1993:95). It made sense to me to introduce the sections as I did to provide this flow for participants. Certainly, when the instrument was piloted, the presentation of the sections was not deemed a problem. However, the order of the sections did not follow the order of my research questions. The ‘experiences in the field’ section, which was presented last, elicited data to measure the extent of the demotivators. Data obtained from the ‘about you’ and ‘about your work’

sections was analysed for investigating potential patterns of prevalence and intentions to remain in teaching.

To identify the ages of the participants in the sample, I drew on the example by York described above, but amended some of the categories. The first group was 18-24 but I thought that it would be highly unlikely for someone younger than 20 to be working in the sector since people entering FE teaching would need to have experience in a vocation so they could teach, or be

qualified in an academic subject, which would require education to degree level. Indeed, this category had to be recoded and amalgamated with the 25- 34 group during the analysis as the sample was too small. I also introduced another group for over 55 (for some reason, the FE example omitted a group covering the 50+ ages). With regards to the highest qualification achieved prior to studying on the ITT course (question 3), I used the same groups as the FE example, but added GCSE/NVQ Level 2 as well as an ‘other’ option. This was because it is quite possible that some vocational teachers might only have needed to attain a NVQ Level 2 qualification in their subject, the rest of their expertise would accrue from their experiences.

In the ‘about your work’ section, there were 11 questions to establish a context for each of the respondents including the subject taught, the level, the age of their students as well as the type of institution they worked for. Given the diversity of teaching provision rather than list all the subjects that respondents might teach, the categories adopted by the Office for Standards in Education (OFSTED) were used (see OFSTED & ALI, 2003), which teachers should be familiar with.

Question 15 enquired whether the respondents expected to be teaching in 5 years’ time. Responses to this question were key for exploring the impact of the demotivators. Although this was a multiple choice question (don’t know, yes, no), there was also a section for respondents to give a reason if they did not expect to be teaching. On reflection, I should have also asked

respondents, who were unsure whether they would be teaching, an opportunity to explain why. Fortunately, some of these participants did anyway.

The section ‘your experiences in the field’ was devised from the findings of the re-interrogation of the qualitative data (see Appendix 3). Drawing on Dey’s patchwork mosaic metaphor (2004), the findings were organised into three clusters pertaining to themes around the students, teaching autonomy and the institution – the factors for this study – with five indicators for each factor. To measure the presence of poor hygiene factors and motivators, I employed an attitude scaling instrument with matched-paired statements asking

respondents to rate their feelings. These Likert rating scales were strongly agree, agree, disagree and strongly agree. I did not include a ‘middle ground’ such as neither agree nor disagree as the literature has shown that

respondents have a tendency to opt for this middle ground or central tendency (Cohen et al, 2007). Although some might argue that respondents should be given this option, I felt that participants would have a feeling one way or another and thus this scale would encourage them to voice it. Although the decision to employ match-paired statements – one positive and the other negative – would double the number of questions in this section, it would enable the reliability of responses to be tested. For example, if a respondent

answered strongly agree to one of the statements, s/he should respond strongly disagree to the paired statement, provided that the statements were worded correctly so that they were the opposite of each other. The

statements formed questions 16-45 and were randomised in the final questionnaire.

The final question (question 46) was an open section which invited participants to provide information that they felt was important.

Throughout the drafting of the questionnaire, research colleagues were invited to comment on the instrument to ensure that the questions were worded clearly and unambiguously, and also that the instrument had face validity. Although useful, face validity, a casual assessment of the item

appropriateness, is not deemed a sound measure (Litwin, 2003). Another subjective measure is content validity, but Litwin argues that an ‘overall opinion of a group of trained judges… provides a good foundation on which to build a methodologically rigorous assessment of a survey instrument’s validity’ (2003: 33). Thus teachers working in FE and trainees on an ITT programme were asked for feedback following a pilot run of the instrument (see 4.3.3). With regards to more rigorous assessment for criterion validity, in particular concurrent, measuring proved difficult. Since the studies in this area had all been qualitative, there were no established scales or ‘gold standard’, as Litwin (2003) calls it, to judge against. However, the factors and the indicators chosen for the instrument emerged from the qualitative findings so were therefore authentic. They had also been cross referenced with similar studies and the literature on motivators and demotivators affecting job satisfaction.

Measures to ensure the internal consistency of responses were taken with the data collected for the pilot and the main study (see 4.3.3 and 4.3.7).