CHAPTER 3: Methodology
3.3 Methodologies adopted in the present study
Based on the description of the direct and indirect tests that were discussed in section 2.10.3, specifically the fact that these tests are used in SLA to distinguish between implicit and explicit knowledge (see Rebuschat, 2013 for an overview), the present study utilised both types of tests. The aim of this was to arrive at a better understanding of the nature of English users’ knowledge of the constructions under investigation. It has been argued within SLA that some tasks promote the use of implicit knowledge and that other tasks promote the use of explicit knowledge (Andringa & Rebuschat, 2015; Doughty, 2003; Dowles, 2011; Han & R. Ellis, 1998; Norris & Ortega, 2000).
Broadly defined, direct tests require participants to make direct use of their knowledge and allow enough time for the participants to complete the task. They are therefore argued to be reflective of explicit knowledge. In the present study the direct test here will be referred to as ‘the off-line task’, since these types of tasks are generally considered untimed measures. Examples of similar tasks are sentence-picture matching and the traditional grammatical judgement tasks (GJT), which ask the participant to judge whether a given sentence is grammatical or not (R. Ellis, 2005, 2008; R. Ellis et al., 2009; Godfroid et al., 2015). By contrast, indirect tests target the participants’ implicit knowledge and are thought to indirectly measure implicit knowledge by recording behavioural data,
115
such as differences in participants’ reaction times when they are confronted with regular versus irregular stimuli. Reaction time measures the speed of a participant’s response to a visual or auditory stimulus in an on-line experiment
.
An example of indirect tests, or what will be referred to in this study as ‘the on-line task’, is the self-paced reading (SPR) experiment (discussed in section 3.4.2.), which requires the use of special computer software package in order to record participants’ reaction times as they read information on a computer screen.The researcher designed the two tests based on insight from Lambrecht’s (1994) account of information structure. The direct test was in the form of a questionnaire consisting of an acceptability rating task along with a subjective measure of awareness (both are discussed in section 3.4.1.3). The indirect test was an on-line experiment, namely a self-paced reading task (discussed in section 3.4.2). This test was employed to investigate participants’ on-line processing while reading the target constructions (Jegerski, 2014).
3.3.1 Motivation for the methodological choices
From a construction grammar perspective, information structure is part of what makes the sentence grammatical (Lambrecht, 1994). As such, it was important to adopt a procedure that tapped in to English users’ grammatical knowledge of the use of optionally used syntactic means of information highlighting that share a comparatively similar function. One of the grammatical features of the focus constructions under investigation is that their meaning is context dependent (Lambrecht, 1994). Therefore, a decision was made to utilise some felicitous and infelicitous contexts, which were inspired by Lambrecht’s treatment of information structure, in the design of the two tests.
Since the focus constructions are not very frequently used in English (Biber et al., 2009), it was important to apply an acceptability rating task as a direct measure of English users’ knowledge of the appropriate contextual use of the target constructions. Most
116
importantly, it seemed unlikely that other data collection methods, such as authentic discourse or general forms of L2 learners' writing, would provide data that could be of great value for the purpose of the present study, that is, at least, not in sufficient quantities to make meaningful statistical analysis possible. In the current study, the task asks the participants to rate the four focus constructions under investigation on a 6-point Likert scale after reading a preceding context, which either evokes a felicitous or infelicitous use of the target constructions. The choice of an acceptability rating task was also motivated by the fact that theoretically it is possible for multiple options to be equally acceptable (Lambrecht, 1994). A 6-point rating scale was used instead of a 5-point Likert scale to avoid the possibility of the participants selecting the middle (3) or ‘neutral’ option (Johns, 2005).
As one of the aims of the present study is to find out whether participants are conscious of their knowledge when rating the acceptability of focus constructions in two types of context (Dienes, 2008), it was decided to apply one of the subjective measures of awareness proposed by Dienes (2008) and Rebuschat (2008). The decision was made to utilise a confidence-rating task since it is found to give insights into participants’ level of awareness of their judgement knowledge as they complete the test, given that it provides evidence of their level of confidence about their decisions (Rebuschat, 2008). Source attribution tasks that ask participants to indicate the source of their knowledge (e.g. guessing, memory, rule knowledge and intuition) were not considered since they are more likely to delve into participants’ structural knowledge, which is often tested after participants have had some sort of training before the testing phase, which is not the case in this study.
The confidence-rating task asks participants to rate their confidence level after rating each focus construction by choosing one of three options provided: guessing,
117
somewhat confident and very confident. Following Rebuschat and Williams (2012), participants were instructed to select the guess option only if they had no confidence whatsoever in their acceptability decision and believed they were guessing. As for the somewhat confident option, participants were instructed to choose this option if they had a small amount of confidence. If they were very confident and sure of their acceptability rating, they were asked to select the very confident category.
The direct and indirect tests are likely to tap into two different types of knowledge. Therefore, the decision was made to incorporate an on-line task in order to look into the nature of Saudi L2 users’ (advanced and intermediate L2 users) and native speakers’ processing of the focus constructions and hence arrive at a better understanding of the type of knowledge that they rely on. A self-paced reading task was selected as it is likely to reflect similar cognitive processes to those involved in normal reading and language comprehension (Jegerski, 2014). This task is a computerised technique that requires a special software package. The task asks participants to read individual sentences in segments, either word-by-word or phrase-by-phrase, that appear on a computer screen. Participants have to press a key on a keyboard or on a response pad to read the whole sentence and to proceed to reading other sentences (section 3.4.2). The time spent reading the sentences is referred to as the reaction time and this is recorded by the software package. Sensitivity to a grammatical violation while reading the target sentences takes the form of increased reaction times, and hence provides evidence of the participants’ knowledge of the constructions under investigation. A self-paced reading task has been used to investigate several linguistic-related features, such as filler-gap dependencies (e.g. Pickering & Traxler, 2003), pronoun resolution (e.g. Carminati, 2005), and processing of sentences with temporal ambiguity (e.g. Felser et al., 2003; Juffs, 2004). These studies presented evidence that parsing is constrained by several types of information related to lexical-semantic
118
information, contextual information and prosodic information. Many such studies that have used this type of methodology have involved adult native speakers of different languages. Also, studies concerned with second language learning and acquisition have adopted an on- line task and used it with adult second language learners (e.g. Hopp, 2006, 2009, 2015; Juffs & Harrington, 1995, 1996).
Most importantly for the selection of this task, the participants have a very limited amount of time to think about the meaning of a given utterance, which makes it less likely that they will depend on their explicit linguistic knowledge (R. Ellis, 2008). As such, the task was adopted to assess the participants’ sensitivity to the appropriate contextual use of the focus constructions as they process these constructions in the felicitous and infelicitous contexts. Participants’ sensitivity to the contextual effect is reflected in their speed of reading (i.e., participants are assumed to read the target construction faster in the felicitous context compared to in the infelicitous context). This in turn is thought to reflect their grammatical knowledge, since after all information structure is part of sentence grammar (Lambrecht, 1994). In this regard, the present study is novel with regard to using an off- line and an on-line method that targets L2 users’ sensitivity to information structure violations when reading the target constructions.
Since the two tests were designed by the researcher, it was deemed necessary to pilot the data collection methods applied in this study, in order to see whether they could provide fruitful data that could meet the aim of the present thesis. Moreover, to date, no SLA studies have utilised a self-paced reading task to investigate the processing of English focus constructions. A small-scale pilot study was conducted prior to the main study. It is important to mention here that the methodology in the main study was developed based on the insights gained from the pilot study. Therefore, the following section is devoted to
119
briefly discussing the pilot study, but with a detailed description for each of the research instruments used.