Questionnaire

Brown (2001, p.6) define questionnaires as “any written instruments that present respondents with a series of questions and statements to which they are to react either by writing out their answers or selecting from among existing answers”. Although questionnaires are basically used to collect information, they can have a variety of purposes. Tymms (2012) lists four purposes: exploratory work, describing a population, outcomes or controls in studies, and feedback. These are described below.

• Exploratory work: when researchers intend to investigate a phenomena but are unsure of the best way to proceed, questions can be asked of individuals taken from the target populations or colleagues can be consulted and literature read to shape what exactly is observed. Afterwards, the researcher may distribute questionnaires to collect data from a small sample to help to define a problem to follow up by more in depth methods.

• Describing a population: when researchers would like to identify a general pattern across a population through administering questionnaires to a representative sample of the population.

• Outcomes or controls in studies: when researchers intend to conduct a questionnaire as part of an intervention study or quasi-experiment, which needs its results to be measured and compared. In such a case, questionnaires are used in order to assess somebody’s understanding and knowledge in a manner that is similar to the purpose of testing but with less pressure and more diversity.

• Feedback: in this case, the questionnaires are used in in-service, courses, or during training in order to obtain information on the attendees’ experiences. Here, the purpose could be formative in order to improve for the next time, or the results could be used for summatively.

In this research, a questionnaire was used to collect teachers’ perspectives on the current teacher evaluation system and their views on what should dominate teacher evaluation. Therefore, the questionnaire was designed to obtain feedback and describe a population, as outlined by Tymms (2012) above.

There are some inherent advantages to using a questionnaire that compelled this researcher to use one in this study. Questionnaires are used in order to obtain a large volume of data; they can consist of several questions and be administered to a large sample (Mertens, 2015). Where a large sample can be gathered, questionnaires can be conducted quickly and easily (Burton & Bartlett, 2009) and at a low cost (Mertens, 2015). Moreover, questionnaire respondents might feel more comfortable when giving their responses due to the anonymity of the questionnaire and lack of face to face contact with the researcher (Cohen et al., 2007). Finally, the data collected from questionnaires are suitable for, and easy to use in, analysis and in comparison with other results (Burton & Bartlett, 2009).

However, one drawback with regard to questionnaires is that some participants may not complete them in their entirety or may not return them at all (Cohen et al., 2007). To avoid that, this researcher excluded sensitive questions from the questionnaire and ensured that it was not too long. This researcher also administered the questionnaire through the MOE. As pointed out by Edwards et al. (2002), questionnaires that are administered through organisations such as universities can positively affect the number of returned questionnaires since participants are more likely to return them to public bodies than they are to other sources, such as commercial organisations. Another issue is that some participants might not give serious attention to answering the questions or they may misunderstand some of the words in a questionnaire (Mertens, 2015). This researcher attempted to avoid some of these problems by providing explanations for any terminology that could be misconstrued.

3.3.1.1 Construction of the questionnaire

Questionnaires can consist of different types of questions, either open-ended or closed (Cohen et al., 2007; Tymms, 2012; Check & Schutt, 2012; Newby, 2014). Open-ended questions are not limited to a set list of choices and the participants write their responses in their own words (Check & Schutt, 2012). Closed questions are restricted to a list of responses from which the participant can choose. These are mainly Likert-type responses. A Likert-type question “involves presenting answers on a scale where the number of possible responses can vary from three up to seven or more” (Tymms, 2012, p.233). The questionnaire in this study was designed with closed questions, as they are useful for generating frequencies of response, enabling comparison between groups, and aiding

Before constructing the questionnaire, this researcher reviewed the relevant literature pertaining to teacher evaluation; however, the actual questionnaire was unique to the current study. The questionnaire was designed as follows: a cover sheet, some background questions and four sections with Likert-type questions. The cover sheet included the title and purpose of the study as well as the details of this researcher (name, contact, and name of programme), as recommended by Cohen et al. (2007). Then, background questions were asked about the participant’s gender, experience, educational districts and subjects taught, following Newby’s suggestion (2014) that these questions are better at the beginning of a questionnaire. These background questions were necessary to identify differences between male and female, their experiences in teaching, and the subjects taught with regard to teacher evaluation. The background question about educational district was meant to determine if the size of a district and the number of schools and teacher in that district make any difference in teacher evaluation.

The four sections using Likert-type questions consisted of the following (a complete version of the questionnaire can be found in Appendix 2 for English, and Appendix 3 for Arabic):

• Section one asked teacher participants to determine the actual purposes of the current system and compare this to their desired purposes. In this section, three purposes of teacher evaluation were given, and each purpose was rated according to “frequency” and “importance” (see Table 3.2). The questions were analysed individually and not in an aggregated scale.

Table 3.2: An example of a two-sided question in the questionnaire

Statement How often are these used?

How important are these? Ne ve r Se ld om So m et im es Of te n Al wa ys No t I mp or ta nt a t al l Un imp or ta nt Ne it he r Imp or ta nt nor uni m por ta nt Im po rt an t Ve ry I mp or ta nt

Promoting professional development of teachers

X X

• Section two had six items asking teachers about the tools of teacher evaluation. This section intended to identify tools that are used in the current system and to compare this to the tools that should be used. This section also had two sets of

questions for each tool, asking (A) to what degree the tools are used, and (B) to what degree the tools should be used. Again, each item was analysed separately not in an aggregated scale.

• Section three listed three sets of items about each evaluator (inspector, head teacher, head of department). This section intended to measure the involvement of the evaluators in the current system. Each set had questions asking teachers to assess (A) the role of the evaluators and (B) rate the value of their role. These questions were designed to create an aggregated scale to cover the range of teachers’ views about the role of evaluators in providing written feedback, engaging in discussion before and after teaching observations, and rating the value of evaluators’ role.

• Section four had 14 items that this researcher intended to use to make an aggregated scale to measure the extent to which the current teacher evaluation system supports teachers. These scales included the follow items:

A) Six items to measure the extent to which the current system supports teaching development. For example, the following items are measured: the support of the system regarding better use of pedagogies, clearer understanding of lesson planning, and clearer understanding of what constitutes effective teaching.

B) Six items to measure the extent to which the current system supports learning improvement. For example, teachers’ abilities to provide students with effective feedback, dealing with individual differences between students, and dealing with students' disciplinary and behavioural problems.

C) Two items to measure the extent to which the current system supports the awarding of promotions, and rewards (e.g., annual bonuses or salary increases, promotions, etc.).

3.3.1.2 Translation of the questionnaire

The official and native language in the Kuwaiti context is Arabic, but the questionnaire was designed in English. Hence, the questionnaire had to be translated for the participants into their mother tongue. The questionnaire was first translated by this researcher, and it was checked, English to Arabic, and Arabic to English, by two assistant professors at Kuwait University and in the School of Basic Education (Public Authority for Applied Education and Training, PAAET) in Kuwait who specialise in translation between English

and Arabic. Afterwards, this researcher asked an English teacher with 20 years’ experience in Kuwait to check it over as well.

3.3.1.3 Piloting the questionnaire

Piloting means trialling the questionnaire in order to increase its reliability and validity (Cohen et al., 2007) through making changes based on feedback obtain from individuals who complete and evaluate the questionnaire (Creswell, 2012). According to Newby (2014), the pilot study should start with the first draft of the questionnaire, which should be given to some experts in questionnaire design in order to revise and modify it. Then, researchers should ask some participants to complete the questionnaire in order to obtain their feedback. These participants’ answers are not be used in the study, so researchers should find people with similar characteristics to the population but who will not be part of the study.

This researcher conducted a pilot study as part of the present study for several reasons: to check the clarity of the questionnaire and its items; to reduce difficult and ambiguous words in the items or decide whether to add explanations; to obtain feedback from a sample on the questions and the format, such as rating scales, multiple choice, and so on; to check the time needed to complete the questionnaire; to establish whether the questionnaire is too long, too short, or too difficult; and finally to discover commonly misunderstood or incomplete answers across the pilot participants’ responses (Cohen et al., 2007).

PhD supervisors at Durham University were first used as the experts to review all the items in the questionnaire and to look over the first draft. The draft items were changed following their suggestions to add or delete some items, and to change some of the scales used to answer the questions. The questionnaire also changed from initially asking respondents to circle numbers to tick boxes instead, due to advice given by supervisors that circling numbers may create an impression of a performance evaluation.

Next, a focus group was conducted with five female teachers from one school in the Ahmadi educational district in Kuwait, in order to discuss with teachers some problems and some particular points about the teacher evaluation system. With the focus group, this researcher intended to find out if there were any particular points that were not addressed in the first draft of the questionnaire that should have been taken into account. As a result, this

researcher confirmed that the purposes, the tools, the support of teacher evaluation system, the involvement of evaluators, etc., were all covered in the questionnaire.

The questionnaire was then prepared as a final draft and piloted with a small sample of 16 teachers from different subjects with experience ranging from 2 to 20 years. They were asked to complete the questionnaire and make notes regarding its clarity, simplicity and the time needed for its completion. This researcher found that the time taken to answer the questionnaire was between 13 and 18 minutes, which was regarded as acceptable. Moreover, it was found that, with the exception of items 4 and 6 in section two, the items were not ambiguous. The sample suggested that the terms ‘formative purpose’ and ‘portfolio’ be explained by providing some examples.