Quantitative methods

5.4.2.1 Design of questionnaire survey of teachers’ assessment practices and beliefs

The use of surveys of teachers’ assessment practices and beliefs attempts to add a larger contextual background to the qualitative methods and ascertain any trends that are emerging in assessment practices in relation to the national strategy. According to Phillips and Burbules, the use of a quantitative approach such as surveys “does not attempt to describe the total reality about, say a classroom; rather, it seeks to develop relevant true statements” (2000, p.38, italics in original). Surveys “gather data at a particular point in time with the intention of describing the nature of existing conditions” (Cohen and Manion, 1994, p.83). Youngman states that the

126 prime source of surveys “must be the working hypotheses and the literature survey” (1984, p.157), and this is the source employed in the development of the survey for this research.

The survey devised for this project is divided into four sections. The first enquires about biographical information; the second examines teachers’ knowledge of assessment policy in their primary schools; the third investigates teachers’ assessment practices; and the fourth enquires about Continuing Professional Development and Assessment. There are four different methods of attitude scaling outlined by Oppenheim: Bogardus, Thurstone, Likert and Guttman (1992, p.188). To study attitude patterning or explore theories of attitudes, Oppenheim states that the Likert procedure is “probably the most relevant” (1992, p.189). This is the scale utilised in this research. Oppenheim also states that Likert scales will “effectively separate people within the same group” (1992, p.200). This characteristic is beneficial to this study as it aims to analyse any varying uses of assessment procedures. Munn and Drever outline the different types of questions that can be contained in a survey: open; closed; ranked responses; scaled responses (1999, p.25- 26). The survey utilised in my project includes a variety of question types.

The question of sampling to identify a research base of participants is important to consider when administering a survey. There are a number of different approaches to sampling, both probability and non-probability. The survey uses non- probability based purposive sampling, where researchers handpick the cases in the sample on the basis of their judgement of their typicality (Gillham, 2008; Cohen and Manion, 1994). Cohen and Manion (1994) describe the importance of avoiding leading or complex questions, as well as arranging the contents of the survey in such a way as to maximise co-operation. Gillham avers that six A4 pages is a maximum

127 length (2008, p.57), while Munn and Drever argue for an upper limit of 15 to 20 questions (1999, p.21). Gillham states that the main ethical requirement for surveys is that “you should make the purpose of your research clear to those involved and obtain their consent to use the information they disclose, protecting confidentiality as appropriate” (2008, p.26, italics in original). An information letter about the survey was sent to all principals and teachers of the schools involved (Appendix 1). Some of the questions in the survey used in the doctoral study are based on the national survey of assessment practices by the INTO in 2008. Questions 6, 7, 8, 9 and 10 ask similar questions regarding teachers’ understanding of assessment policy in their schools, the administration of standardised tests, and their views on the purposes of standardised tests. The responses to these questions from both surveys are compared and contrasted to examine areas of consistency or change. This enables the project to document any changes and consistencies in the past number of years (Knight, 2002). It utilises the simple descriptive approach to maximise the number of respondents (Mertens, 2010, p.177).

Commentators espouse the need for piloting of the instrument to examine whether the content needs to be modified; to investigate whether it works as intended; and to identify any difficulties in the analysis stage (Gillham, 2008; Munn and Drever, 1999; Cohen and Manion, 1994; Oppenheim, 1992). Three reasons for piloting are to see whether people can complete the questionnaire in a reasonable time, to identify problems with questions, and to encourage thought about the analysis of data (Knight, 2002, p.95). This took place in May 2013 with two teachers who are not included in the study sample. The teachers completed the survey individually with me present. I asked both participants to explain their

128 thoughts as they were reading and answering the questions to ensure that the questions were as concise as possible and that they were an appropriate instrument to collect data relevant to this study’s research question. The phraseology of some questions was clarified and edited to ease understanding as a result of the piloting.

The finalised survey is four A4 pages in length. It contains 18 questions which are a mixture of open and closed questions, some of which employ Likert scales. The surveys were administered between January and February 2014. Twelve primary schools in the Cork area were identified, which were a mixture of urban and rural schools (7 urban and 5 rural schools respectively). These schools were chosen as I had a contact teacher in each of them who would deliver the surveys to the teachers on staff and be responsible for their safe return to me. 301 surveys were delivered to teachers around the Cork area. 144 completed surveys were returned, resulting in a response rate of 48%. As there is only one administration of the survey instrument, internal consistency is calculated to determine reliability. Mertens states that statistical packages for computers calculate a reliability coefficient, such as Cronbach’s coefficient (2010, p.382). The Statistical Package for the Social Sciences (SPSS) is employed in this instance. The survey measures teachers’ assessment practices so concurrent validity is utilised (Mertens, 2010; Knight, 2002). The survey is analysed using inferential statistics to determine whether sample scores on assessment differ across teacher background. An ordinal scale of measurement is employed in the survey (Mertens, 2010). Knight (2002) argues that data analysis is continuous. He states that “(i)t begins with the research design and the research capture and it continues as repeated thinking about meanings that might be identified in the data” (Knight, 2002, p.176). The project uses this method to

129 decide upon units of coding as the research develops. A sample of the survey can be found in Appendix 2.