First Pilot Study

3.2.1.1 Purpose. The first pilot study compared the effectiveness of collecting LRE data in the individual mode using a) unprompted think-aloud protocols and b) prompted immediate recall. Since individual participants were studying online at home, I wanted to assess whether their LREs could be observed by asking them to independently complete tasks and record themselves thinking aloud, or if I would need them to come to the school and prompt the think-aloud. I also wished to assess whether the pilot task was challenging enough for use in the main study.

3.2.1.2 Method. Six online students participated: three in the at-home unprompted condition, with pseudonymised names beginning with U, and three in the in-school prompted condition with pseudonymised names beginning with P. The at- home unprompted participants audio recorded themselves on their mobile phones while completing a passage editing task; they were instructed to say out loud everything they were thinking as they edited the text. They then emailed me the resulting mp3 file. The in-school participants completed the task with me beside them, and were asked to say out loud everything they were thinking. I prompted them with “tell me what you’re thinking” if they began to write without verbalising. I recorded these participants using an mp3 recorder.

59

3.2.1.3 Task. Learners completed the First Pilot Passage Editing Task (Appendix 1) in which they corrected seeded errors related to forms they had studied during their course (see Table 1).

Table 1

Errors seeded in First Pilot Passage Editing Task

Language area Forms Exponents Errors seeded in First

Pilot Task

Lexis Verb-noun collocations with _“take”

to take a moment to take advantage

to catch a moment to make advantage

Lexis Two-part phrasal verbs

turn up (= arrive) go out (= leave)

turn out go about

Lexis Modals: can, might*

it might be quite late I will definitely put…

it can be quite late I might definitely put… Grammar (morphology) Prefixes** unnecessary misunderstanding innecessary disunderstanding Grammar (tense and aspect)

used to + infinitive vs. past simple

used to + infinitive vs. be used to + gerund

I said in my last email… We are used to having a late check out

.. I used to say in my last email

We used to having a late check out Grammar (tense

and aspect) Future continuous

We will be getting in We’ll be arriving We’ll getting in We’ll be arrived Discourse A total of 12 inappropriacies in register

Formal versus informal expressions to be avoided in formal letters / emails

1) Dear 2) I’m + ing 3) Very much 4) Million 5) Good 6) Excellent 7) Excellent 8) Give you a call 9) Is there any chance 10) Best wishes

11) I look forward to hearing from you

12) Do you have any recommendations 1) Hi 2) Just + ing 3) MILLION (choice of informal lexis) 4) MILLION (capitalization) 5) Cool 6) BRILLIANT (choice of informal lexis) 7) BRILLIANT (capitalisation ) 8) Give you a buzz 9) Any chance 10) Bye for now 11) See you soon 12) Any

recommendations? * modals are considered lexis rather than grammar, following Fortune & Thorpe (2001)

** morphology is considered a component of grammar, rather than lexis, following Williams (2001) and Storch (2007)

3.2.1.4 Results and discussion. Table 2 presents the numbers of LREs produced in the prompted and unprompted think-alouds in the first pilot, with the mean number of LREs per participant.

60

Table 2

LREs in prompted and unprompted conditions, First Pilot Passage Editing Task

Total Mean

Prompted (n = 3) Paola Pedro Priscila

LREs 11 30 13 54 18

Unprompted (n = 3) Uta Ursula Ugo

LREs 30 36 37 103 34.3

Unprompted participants produced almost twice as many LREs as prompted participants. My presence as a researcher therefore appeared not to aid but rather to inhibit language production. The think-aloud protocols suggested the presence of language anxiety regarding the procedure and / or making mistakes, probably as a result of my presence (Swain 2013). Paola, for example, apologised and asked questions regarding the procedure and requested me to answer form-related questions. Likewise, she was unwilling to verbalise forms she was uncertain about even when prompted. If, as Vygotsky (2012: 11) proposes, there exists a “dynamic system of meaning in which the affective and intellectual unite”, then negative emotions such as anxiety may have affected Paola and Priscila’s cognition and / or verbalisation of cognition. The unprompted learners, conversely, may have felt more comfortable unaccompanied, given this was their normal mode of studying, and this may have positively affected their languaging. If this is the case, it supports the importance of ecologically valid studies that observe learners in their natural learning context, rather than, for example, asking habitual group learners to perform tasks alone for the purposes of a lab study (as in Swain & Lapkin 1995 and Kim 2008).

Furthermore, prompted participantsmay have been less willing to talk because they did not perceive the task to be a think-aloud in which they talk to themselves, but rather one in which they were supposed to interact with the interlocutor. They may have felt uncomfortable having a perceived interlocutor who did not speak, except to prompt. Prompted participants may have had the capacity to language, but this either did not happen or was not made visible because the presence of the mainly silent interlocutor created the expectation of social interaction, which did not occur. Participants were plausibly making meaning of the presence of a mainly silent interlocutor, responding as they felt was socially appropriate – by speaking less than normal.

61

I also observed an inverse relationship between the amount of prompting and the number of LREs produced. Paola, who produced the fewest LREs, received six prompts; Priscila, who produced 13 LREs, received two prompts; Pedro, who produced 30 LREs, needed no prompting. While the increased prompting was a result of fewer LREs being verbalised, it is also plausible that the prompting was itself negatively affecting participants’ confidence in their ability to verbalise LREs, in a vicious circle.

In order to assess the degree of challenge inherent in the task, LREs were also coded for correctness of resolution. The results are presented in Table 3.

Table 3

LRE resolution in prompted and unprompted conditions, First Pilot Passage Editing Task

LREs % of total LREs Mean

Correctly resolved Prompted 34 63% 11.3

Unprompted 75 73% 25.0

Incorrectly resolved Prompted 11 20% 3.7

Unprompted 20 19% 6.7

Unresolved Prompted 9 17% 17

Unprompted 8 8% 2.7

The correct resolution rate of between 63% and 73% indicated that learners in both conditions could correctly resolve most of their LREs, which was perhaps unsurprising given that the forms seeded in this task were part of learners’ course of study. This led me to question whether the task was challenging enough for learners, so I decided to perform a second pilot study.

3.2.2 Second Pilot Study. In order toinvestigate the degree of challenge in