The writing task for the think-aloud protocol session

In the think-aloud session, all subjects were required to write an essay on a topic that was selected by the researcher. The researcher was concerned to assign a topic about which the subjects would know something, in the belief that familiar topics would maximise the writers‘ involvement in the process (Friedlander, 1990; Manchón et al., 2005), taking into consideration that topics relating to the students‘ current interest

84

(education) and to their social life and educational background, would motivate them to be deeply involved in writing and to communicate their ideas to the reader.

Moreover, the reason behind choosing a single (uniform) topic for all subjects to write about was the researcher‘s wish to avoid discrepancy in the writers‘ perception of different topics, considering that all targeted subjects were in the same academic year (i.e., fourth year English majors), and should be at a similar level. Hence, there was no need to assign them two or more different topics in consideration of a potential variation in their level of competence. Furthermore, the researcher was intending to use this particular mode of discourse (i.e., argumentative) in order to investigate the processes and strategies used in producing this type of writing in a foreign language.

The topic assigned was:

―Success in education is influenced more by students‘ home-life and training as a child than by the quality of the teaching and the effectiveness of the educational programme. Discuss.‖ (Taken from Raimes, 1987).

It was important to confirm that students had not written about this topic already.

Therefore, the researcher checked with their teachers in order to preclude the possibility that students might be able to write on this subject from memory.

Additionally, the topic was given to the subjects on the spot, so that they did not have any chance to think about it beforehand, and had no option but to act as naturally and spontaneously as possible.

3.10.2.2.3 Think-aloud protocols

Think-aloud protocols were used in this research in order to explore what was going on in the minds of the students as they were writing, and to investigate the actual writing processes and strategies that Libyan fourth year English majors employ while composing in English.

3.10.2.2.3.1 Procedure

Each student had a think-aloud session individually, and each session began with an explanation of the think-aloud procedure. Actual writing did not start until each

85

subject felt relaxed with the procedure and expressed his/her willingness to start. The researcher adapted the warm-up instructions provided by Ericsson and Simon (1993) (see Appendix 7), in order to obtain valid think-aloud protocols. The following are the instructions the researcher gave to the subjects:

As you write your essay, I want you to think-aloud as soon as you start thinking about the topic. In other words, from the moment you look at the assigned topic throughout your writing, I want you to verbalise your thoughts and, therefore, say everything that goes through your mind loudly. As you verbalise your thoughts I want you to follow these guidelines:

1. Work on the task as you normally would: think about it, take notes, use outline, or just write.

2. Try to say aloud everything that crosses your mind, even fragments and stray thoughts. Say what you are thinking, reading, and writing, just as you did in the warm-up exercises. You do not need to explain or justify what you are doing.

3. Please think aloud throughout the writing session from beginning to end.

Speak audibly and as continuously as possible. If you stop speaking (10 seconds) I will remind you to think aloud.

4. As you write, if you change your mind please do not erase text that you do not intend to use. Simply cross once through anything you do not need.

5. Before we start I will give you two practices before we move to writing the essay.

6. Can you please multiply 24 x 36 and tell me what you are thinking while you do the calculation.

7. Now I will give you an anagram, which means a number of letters that you need to put in a particular order to form a word. I need you to tell me what you are thinking about while you perform this task.

(R,F,T,E,P,E,C= PERFECT)

The think aloud writing sessions were performed in a quiet place and the subjects conducted the writing tasks individually according to a schedule the researcher prepared with regard to the availability of the subjects and their free time.

86

The timetable prepared by the researcher provided a clear guideline for managing the writing sessions in terms of indicating which subject would turn up next, on which day and at what time. This helped the researcher to avoid time-wasting.

In each new think-aloud session, the researcher explained what to do to the student, and reminded him/her of the importance of continuing to verbalise as much as possible throughout the writing session. The researcher also read the instructions and the assigned topic together with the subject. Usually, he sat opposite to the student in order to be in a good position to observe him/her closely and write down comments about his/her behaviour. However, the researcher always asked the subjects beforehand whether sitting in front of them might cause them any discomfort, and if they said yes, the researcher then moved into a different position that was a little further away, but still being able to observe the subject and control the session.

3.10.2.2.3.2 Transcription of think-aloud reports

Eleven think-aloud reports were collected, and each audio-taped composing session was transcribed by the researcher. The reports were transcribed in the language in which they were produced. In respect of the transcription conventions employed in the current study, the researcher intended to employ a straightforward and fairly simple approach, which began with an attempt being made to distinguish between what was written and what was oral. This was done by the researcher using techniques to distinguish the writers‘ different activities while writing. In this connection, double underlined words indicate words that were being written down as they were verbalised. Single underlined words indicate written text being read by the students, whether part of the assigned topic, key words or phrases within the topic, directions, or previously written text, for example the title, part of the essay, or the whole essay. The underlined and italicised are the subjects‘ think-aloud voicing.

Non-underlined, but italicised words represent the subject‘s silently written text.

Finally, the words written in capital letters, and parenthesised, represent the researcher‘s remarks on the student‘s writing. Use of L1 by the subjects was presented in brackets and the researcher‘s translation was presented in italics and parenthesised next to the L1 text.

87 3.10.2.2.3.3 Coding the composing behaviour

By listening to the tapes of the composing sessions and simultaneously following the essays created by the subjects, it was possible to identify the specific functions of the different strategies in the composing process. When this was achieved for all composing sessions, the researcher made another complete coding of the data in which he compared his categories with others appearing in the literature.

Consequently, some categories identified by other researchers (e.g., Perl, 1979;

Sasaki, 2000; El Mortaji, 2001; Chaaban, 2010) were borrowed by the researcher.

3.10.2.2.3.4 Reliability of the coding scheme

In order to check the reliability of the coding in this study, Scholfield‘s (1997) formula was employed, and intra-judge reliability was calculated to determine how much agreement was reached. In the procedure, five (about 45%) randomly chosen protocols were coded for strategies by the researcher himself, and then put aside for five weeks without looking at them. The researcher then coded those same five protocols once more, as though they were being seen for the first time. A comparison was then made of the two codings to establish how similar the classifications were. A classification was scored as agreeing if the two codings matched. In the first coding, 205 writing strategy units were counted for the first subject, and in the second coding, 192 units of strategies were classified as similar to those units in the first coding for the same subject. This indicates that 192 units of the strategies in the second coding completely matched 192 units in the first categorisation. Exactly the same procedures were conducted with the other four subjects. In the first coding, the researcher found a total of 759 units of writing strategies for all five subjects, and in the second coding, 702 units of strategies were found. Those 702 units were categorised the same as in the first coding. Scholfield‘s formula was applied in the current investigation to obtain the intra-judge reliability coefficient:

Number of strategies coded the same by researcher in the 1st and 2nd coding ---× 100 Number of strategies coded by the researcher in the 1st coding

The overall agreement, accordingly is (702 ÷ 759) × 100 = 92.49%

88

Hence, the intra-judge reliability check reveals a good percentage agreement.

Moreover, and in order to ensure the reliability of the coding of L2 writing strategies, the five samples of the protocols were encoded by another person, an Arabic speaking PhD holder who teaches English writing at the English department at MU.

Along with the five protocols, this person was given a list of instructions adopted from Mushait, 2003, with modifications done by the researcher to fit the procedure applied in the present research in coding strategies and behaviours (see Appendix 19). The researcher met with this person to discuss in details the list of strategies coded by the researcher and to answer any questions raised. The independent coder identified a total of 732 strategy units employed by the five writers. The strategies that were coded the same as the researcher‘s were 689.Using Scholfield‘s formula, interrater reliability was 91%:

Number of strategies placed in the same category by both judges

--- × 100 Number of strategies coded by the researcher

Hence, the overall agreement is(689 ÷759) × 100 = 91%.

In addition, however, the researcher‘s supervisors suggested worthwhile points in relation to strategy coding as they read throughout the thesis and their comments were discussed together and some strategies were re-coded accordingly. This happened, for example, with the strategies of avoidance and postponements, and rehearsing.