Verbal reports

Following Alderson (1990b) both forms of verbal reports (TAPs and RDs) from th e sam e students w ere collected in o rder to maximise th e richness of information gath ered (Taylor and Dionne, 2000). The verbal reports w ere collected in o rder to address th e first research question of th e study.

It is im portant to choose students who reflect th e ta rg e t population of th e te s t as closely as possible, namely th e students who have com pleted Grade 10. However, as th e investigation was carried out at th e beginning of th e school year, it was impossible to find students who had just com pleted th e Grade. Therefore, it was decided to select th e students studying at Grade ten, th e prospective stu d en ts of th e SLC exam, and eight students (seven girls and one boy) aged betw ee n 14 -16,studying at Grade 10 ( four in a public school and four in a private school) in Nepal participated in this study. All of th e m

w ere Nepali native speakers. They had learned English at schools for an average of 12.5 years.

3.2.1.1 Think-aloud and RDs procedure

The data from th e students w ere collected in their respective schools. In o rd e r to ensure th a t th e participants w ere not disturbed by outside noise, a very quiet room w as selected and both th e sessions (think-aloud and retrospective debriefing) w ere audio recorded.

The following four major steps w ere taken to collect th e data.

1. The stu d en ts w ere told w h at was required of th e m following a standardized set of instructions introduced by Taylor and Dionne (2000: 424). The instructions w ere not tailored to th e research questions, in o th e r words, th e s tu d en ts w e re not asked to verbalize which reading strategy they w ere employing, as it was im portant not to influence th e range of cognitive processes reported. The instructions w ere given in th e native language of th e participants to g u a ra n te e com plete understanding.

2. A warm up task was conducted in o rder to overcom e th e inherent challenges of thinking-aloud while engaged in complex cognitive tasks, such as reading. W arm

up exercises w ere expected to make th e s tu d en ts feel m ore confident and less anxious (Ericsson and Simon, 1993), and reduce th e need for probing questions and prompting during th e verbal report sessions (Afflerbach and Johnson, 1984).

Therefore, using a n o th e r reading te s t of th e SLC exam, th e stu d en ts w e re given an example of how thoughts can be verbalized and th e y practised thinking-aloud until th e y felt thoroughly comfortable with th e process involved in thinking-aloud.

3. Immediately after th e warm up session, th e y w ere asked to take th e real te s t and think-aloud using th e language of their choice (i.e either Nepali or English)in o rd e r

to ensure th a t th e re was no hindrance caused by language use. However, all th e stu d en ts used English to verbalise their thoughts. No tim e limit was set for th e session in o rder to allow th e m enough tim e to think aloud; th e individual session lasted approximately 35 to 40 minutes. During each individual think-aloud session, th e researcher positioned herself next to th e students and took som e notes related to their activities. Following Cohen and Upton (2006), th e participants w ere reminded to keep thinking-aloud w henever they rem ained silent for about 30 seconds; they w ere politely prom pted: "Please say aloud w h a t you are thinking now." However, this was not necessary for m ost of th e participants. It should be noted th a t th e participants w ere individually trained and also took th e te s t individually.

4. Immediately after th e y com pleted th e test, the y w ere interviewed ab o u t how th e y solved each item in th e test. The researcher used th e notes she collected during th e think-aloud sessions for asking questions. Although all th e stu d e n ts used English to verbalise their thoughts, Nepali was mostly used in th e ir interview as m ost of th e m preferred Nepali to use in th e interview.

5. Finally, th e y answ ered a questionnaire (See appendix c) which includes som e questions related to th e s tu d en ts' background and th e test. The questions w ere adapted from Holzknecht (2012).

3.2.1.2Data analysis

The verbal reports of each participant w ere transcribed following th e convention of th e previous studies (Alderson, 1990b; Nikolov, 2006) and th e transcripts w e re organized according to th e individual te s t item. Thus, even if a s tu d e n t cam e back to an item after som e time, for example to check th e answer, this part was put to g e th e r with t h e first part

related to th a t item. To make th e analysis systematic, th e TAPs and RDs on each te s t item w ere put to gether. Then, th o s e verbal reports w ere coded using th e softw are called NVivolO (Lewins and Silver, 2014).While coding th e reports, both th e to p down and bottom up approaches w ere followed. Following th e to p down approach, 12 different codes w ere developed on th e basis of th e reading strategies specified in th e SLC curriculum, prior to th e start of coding th e data. Five o th e r codes em erged through th e data increasing th e total num b e r of th e codes to 14. Then, th e frequency of each strategy was calculated on an item basis for each participant. It should be noted th a t th e rep ea ted use of th e sam e strategy within a single te s t item by an individual was counted only once

so th a t th e num b e r of participants using th e strategy for th e sam e item could be easily identified. Finally, their responses to th e questionnaire w ere analysed. It should be noted th a t th e questionnaire consists of both th e open ended and close ended questions. Thus, while analysing th e close ended questions th e num ber of frequency for each choices was counted but in th e open ended questions, th e main points m ade by th e stu d e n ts w ere collected.