Research instruments

In this section, the research instruments adopted in this study are presented. This study adopted four types of research instruments: speaking tasks, a self-report strategy questionnaire, an attitudinal questionnaire and retrospective verbal protocols. These instruments were developed to answer the research questions and research objectives of this study. A pilot study was conducted in November 2006 to see how well these instruments worked. Using the four types of instruments for collecting data strengthened the reliability and internal validity of this study. The following section describes the research instruments adopted in this study.

3.4.1 Questionnaires

In the area of second language research, a questionnaire is one of the most widely used instruments to gather data on “attitudes and opinions from a larger group of

participants” (Mackey & Gass, 2005: 92). According to Brown (2001), questionnaires are “any written instruments that present respondents with a series of questions or statements to which they are to react either by writing out their answers or selecting them among existing answers” (p.6). This study employed two types of questionnaires: a self-report strategy questionnaire and an attitudinal questionnaire. The self-report

strategy questionnaire was developed to obtain information concerning the types of CSs commonly used by the students and students’ perceptions of the usefulness of each CS. The attitudinal questionnaire was used to investigate students’ attitudes towards strategy teaching and its usefulness. These written questionnaires were

designed and developed to elicit quantitative and qualitative data from all participants.

3.4.1.1 Self-report strategy questionnaire

Research instruments typically used to collect data about students’ use of strategies include observation, self-report strategy questionnaires, speaking tasks, interviews, diary journals and verbal reports. In the present study, the self-report strategy

questionnaire was developed to investigate CSs commonly employed by the students and to explore students’ perceptions of the usefulness of each CS. To design the self- report strategy questionnaire, the researcher studied CS research conducted by Dornyei (1995), Brett (2000), Lam (2004), Nakatani (2005) and Pornpibul (2005). These researchers successfully employed questionnaires to investigate CSs. This influenced the researcher in the present study to adopt the questionnaires as one of research instruments. McDonough and McDonough (1997) suggest “a good

questionnaire is one which is relatively easy to answer, easy to record and evaluate, user friendly and unambiguous” (p.177). To investigate and measure students’ attitudes and beliefs towards their use of CSs, the rating scale of self-report strategy for this study followed the Likert technique of scale construction. The Likert-type scale is the most widely used method of scale construction due to “its relative ease of construction, its use of fewer statistical assumptions and the fact that, in contrast to other scaling techniques, no judges are required” (Doukas, 1996: 190). Based on such arguments, the researcher of this study designed the self-report strategy questionnaire as a 33 item five point Likert-scale (for 16 CSs), on which students indicated the extent to which they used CSs by responding either (1) never (2) rarely (3) sometimes (4) often and (5) most often (see Appendix A). A score approaching 1 indicates that the student reports using the strategies almost none of the time, while a score

approaching 5 indicates that the student reports using the strategies almost all the time. To measure students’ attitudes and beliefs towards the use and usefulness of CSs, a list

of CS statements was developed by using data on CSs drawn from the speaking task pilot study in November 2006 as well as modified from the set of CSs suggested by Tarone (1977), Faerch & Kasper (1983), Bialystok (1990), Dornyei (1995) and

Dornyei & Scott (1997) (see Chapter two). This process ensured that the questionnaire had theoretical support but also that the statements were relevant to the Thai

engineering students.

However, one possible problem of using a questionnaire is that “responses may be inaccurate or incomplete because of the difficulty involved in describing learner- internal phenomena such as perceptions and attitudes, for example” (Mackey & Gass, 2005: 96). To solve such a problem and to check the validity of this questionnaire, the first version of the self-report strategy questionnaire was trialled with the researcher’s supervisor. In doing so, the supervisor validated the questionnaire by matching the strategies to their descriptions. Then the researcher discussed with the supervisor and revised the descriptions. In addition, two Thai students in Southampton were asked to correct any mistakes or unclear statements and give comments on the Thai version of this questionnaire. Finally, some questions were eliminated and revised. The final self- report strategy questionnaire consisted of 33 communication strategy statements (see Appendix A) and was administered to the participants in the main study. To measure the reliability of returned questionnaires, Cronbach’s alpha reliability coefficient, a measure of internal consistency, was also used in this study. Cronbach’s alpha analyses yielded reliability coefficients for the total scale of 0.78 before the CS instruction and 0.72 after the CS instruction. These results demonstrated that all the items in the questionnaire could measure the students’ CS use with enough

consistency (see Pallant, 2007: 98). An example of CS statements in the self-report strategy questionnaire is shown below in Figure 3.2.

Figure 3.2 An example of CS statements in the self-report strategy questionnaire

a) How useful YOU think each of them is: b) How often YOU use each of them:

1= Not useful 2= Least useful 3= Neutral 4= Useful 5= Most useful 1= Never 2= Rarely 3= Sometimes 4= Often 5= Most often

1) If I do not know the English word for something, I describe it, e.g., “what it looks like”, or “what you can use it for”. 2) When I do not know how to express something in English, I

use a word that has roughly the same meaning, e.g., “boat” instead of “ship”.

3.4.1.2 Attitudinal questionnaire

Attitudes involve “evaluative responses to a particular target (e.g., people, institution, situation). They are deeply embedded in the human mind and very often not the product of rational deliberation of facts…” (Dornyei, 2003: 8). In general, attitudinal questions are employed to discover what people think and “this is a broad category that concerns people’s attitudes, opinions, beliefs, interests, and values” (Dornyei, 2003:8). In the current study, the attitudinal questionnaire was used to gain

information concerning students’ attitudes towards CS instruction and its usefulness. To develop the attitudinal questionnaire, the researcher studied the research of Dornyei (1995) involving the general attitudes towards the usefulness of the CS training. According to McDonough and McDonough (1997), many questionnaires use open-ended questions “to allow the respondents to feel that they can contribute more individual points of view and more detailed information than is elicited in closed questions” (p.176). The researcher, therefore, designed the attitudinal questionnaire as open-ended questions. Taking the precious “respondent-availability time” into account, six short-answer questions regarding students’ attitudes towards the CS instruction were included in this attitudinal questionnaire (see Appendix E). In addition, two Thai students were asked to correct any mistakes or unclear statements and give comments on the Thai version of this attitudinal questionnaire. This questionnaire was completed by all 62 participants after they finished the CS instruction programme. An example of the attitudinal questionnaire is shown below in Figure 3.3.

Figure 3.3 An example of attitudinal questionnaire

Section I: The following questions are about your views of communication strategy