Questionnaires

One of the most common data collection tools found in ESL research, which was used in our study, is the questionnaire. According to Dörnyei (2010), questionnaires’ popularity comes from the fact that they are versatile and provide information in a form that can be easily processed. Questionnaires are defined by Brown (2001) as “any written instruments that present respondents

17 _{According to Barcelos and Kalaja (2011), some other possible data collection instruments used in studies of} beliefs are learning journals, self-reports, language learning histories, video-recordings of classroom sessions, informal conversations, drawings, classroom observation, school artifacts, postings on an online discussion forum, and sentence-completion tasks.

with a series of questions or statements to which they are to react either by writing out their answers or selecting from among existing answers” (p. 6).

The two most common types of questionnaires found in the literature are those containing closed-ended items and those with open-ended items. Closed-ended questionnaires are defined as those made up of questions containing “ready-made response options to choose from, normally by encircling or ticking one of them or by putting an X in the appropriate slot/box” (Dörnyei, 2010, p. 26). Among the advantages of this type of questionnaire, Dörnyei (2010) mentions that because of the way they are developed, there is no room for subjectivity, which greatly facilitates the data analysis process. For this reason, they are commonly used in quantitative and statistical analyses. To illustrate the use of this method, we can mention a two-part study conducted by Fives and Buehl (2008) which focused on exploring pre- and in-service teachers’ beliefs about teaching knowledge and teaching ability. Through analysis of questionnaires, the researchers were able to find that the participants had different beliefs about the knowledge and abilities they would need to teach.

On the other hand, open-ended questions are defined as “items where the actual question is not followed by response options for the respondent to choose from but rather by some blank space (e.g., dotted lines) for the respondent to fill” (Dörnyei, 2010, p. 36). By allowing respondents to answer the questions using their own words, this type of format can provide rich data and elements that might not have been previously anticipated by the researcher. In addition, Vieira- Abrahão (2006) states that this type of questionnaire is usually used to explore participants’ personal perceptions, beliefs and opinions and to acquire richer and more detailed answers than those obtained through closed questionnaires.

Thus, among the many questionnaires used by researchers in the literature to identify and study teachers’ and pre-service teachers’ beliefs, we chose to adopt the one developed by James and Pedder (2006).

2.1.1 James and Pedder’s (2006) questionnaire

With the purpose of identifying and comparing their participants’ values18 _{and practices} related to assessment, James and Pedder (2006) created a 30-item questionnaire with three sections19_{. In order to fill out this questionnaire, the participants were asked to provide two types} of answers for each of the 30 items using a Likert-type scale (James & Pedder, 2006). On the left side (Scale X), participants were invited to clarify their own assessment practices by choosing whether the specified practices were “never true, rarely true, often true or mostly true” (James & Pedder, 2006, p. 114). On the right-hand column (Scale Y), participants had to express their opinion of how important a given practice was in terms of providing students with learning opportunities (James & Pedder, 2006). The options were: not at all important, of limited importance, important, crucial, or bad practice (James & Pedder, 2006). For the sake of clarity, we have included an illustration of James and Pedder’s (2006).

Figure 5. James and Pedder’s (2006) dual scale format questionnaire (James & Pedder, 2006, p. 114)

This instrument has been validated both by its creators (James & Pedder, 2006) and by Winterbottom et al. (2008). In both studies, the questionnaire was proven to be effective, as it provided researchers with relevant findings, such as gaps between teachers’ beliefs and practices. This is one of the main reasons we decided to use their questionnaire as one of our data collection instruments.

18 _{James and Pedder (2006) used the term “value” to describe teachers’ assessment-related beliefs.} 19 _{For the purpose of this study, we will only describe section A.}

2.1.2 A new questionnaire

In our own study, in order to identify pre-service teachers’ beliefs about formative assessment, we decided to adapt20 _{James and Pedder’s (2006) 30-item questionnaire, more} precisely, we transformed the statements found in section A (teachers’ assessment practices) into general statements that represented pre-service teachers’ beliefs about assessment so that these statements would be adequate to pre-service teachers’ practicum reality. In addition, the participants in our study were asked to reply to 30 statements related to assessment by choosing strongly disagree, disagree, undecided, agree, and strongly agree, and to write comments when they felt it was necessary. Finally, we decided to include a new section after each item for our participants to write comments in order to justify their answers (Appendix A). The main reason for adding this section was to identify the possible origins of our participants’ beliefs. Then, we piloted our questionnaire with two in-service teachers in order to validate our tool and identify any possible misunderstanding that our participants could have. Moreover, our questionnaires were applied on two different occasions during our data collection phase: the first moment was the beginning of their fourth year (between the months of September and October), in order to identify the pre-service ESL teachers’ initial beliefs about formative assessment; and the second occasion was at the end of fourth year (between the months of April and May), with the purpose of verifying whether there were changes in their initial stated beliefs.