Supplementary Tests

As was discussed in the first chapter⁹, the accurate identification of proficiency in L2 poses striking challenges due to conceptual and practical concerns. For the purpose of attaining the highest accuracy, all the available sources were utilized to

7 See Appendix C.

8 Scores of relatedness were obtained via LSA that was utilized for the preparation of semantic condition for consistency.

9 See 1.4.

112

contribute in the decision phase. Correspondingly, the following formula was considered in the classification of participants according to their level of proficiency:

[(50x Standardized Test Scores) + (50x Self Ratings)]/100 + Bonus for Living Abroad]

i. Participants completed a 40-question standardized multiple-choice cloze test¹⁰ developed by Ionin and Montrul (2010) and used in many studies including Ionin et al. (2011;2012), who reported it as highly reliable (p.

126). The test was a sentence completion task in which every seventh word -except the first sentence- was removed and replaced with three choices.

Every correct choice amounted to 1.25 points to reach a maximum score of 50 and the wrong answers were not penalized. The test lasted approximately 15 minutes.

ii. In the Language Background Questionnaire described above, participants were asked to rate their overall proficiency in English as well as their qualification in four language skills on a 9-point likert scale. The sum of the ratings with a maximum score of 45 was proportioned to 30. Plus, the percentage that they stated about the extent of the daily use of English was taken into account with a maximum score of 10. Another 10 points were added to reach 50 on the basis of their self-ratings regarding how comfortable they felt understanding and using English in their daily lives.

iii. Participants were asked to state if and for how long they had ever been abroad. An extra 3 points were added to the sum of (i) and (ii) if they had lived in an English-speaking country for more than 6 months and 1 point if they had lived in other Western European (e.g. Germany, Belgium) or North American (e.g. Canada) countries where English is widely used.

Calculating an overall score out of 100 for each participant was followed by the operation to convert the obtained continuous variable into a categorical one. The median-split technique was run to observe that participants with an overall score above 74 were considered “high proficient” (n. 52) and those with or below 74 were classified as “low proficient” (n. 60).

10 See Appendix D.

113 3.2.4.2. Vocabulary Size Test

Considering the previously mentioned L1 studies that sought to correlate vocabulary knowledge with semantic and morphological processing, a supplementary test to measure the vocabulary size of the participants was administered in order to test the same possibility in L2 processing. A multiple-choice vocabulary size test¹¹ created by Nation (2004) and further developed by Nation and Beglar (2007; p. 9-13) was adopted and implemented. It was designed to measure L1 and L2 written receptive vocabulary size in English. Its original version contained 140 questions with 14 sets of 10 items, each of which belonged to a 1000-word family. The sets ranged from the most frequent 1^st 1000 to the 14^th 1000 word families in English with gradual decrease in the level of frequency. It was discrete, selective and relatively context independent (Nation, 2007). Empirical evidence was provided regarding its validity and reliability to utilize in L1 and L2 contexts (Beglar, 2010). A more recent version that included 200 questions with 20 sets of 10 items – addressing the first 20.000 word family in English – was adapted in a way to make it completely context-free by removing the sentences where the unknown words were mentioned. The rationale was to create an analogous condition to masked priming paradigm in which participants made lexical decisions on isolated lexical items. Two sets of 100 words were created in a balanced way concerning the frequency of items in each word family. Two forms were reported to have been tested for their equivalence (Nation, 2007). Participants were randomly administered to the versions of the test¹². The test lasted at most 25 minutes.

Participants were divided into two groups according to their scores on the vocabulary size test as a result of a median split technique, which revealed that those scoring above 51 were in the high vocabulary size group (n. 52) and participants scoring 51 or below were classified in the low vocabulary size group (n. 60).

3.2.4.3. Spelling Recognition Task

Proposals associating spelling ability with orthographic and morphological processing in L1 made way for a further task in this thesis to test its applicability in

11 The test is freely available on Paul Nation’s web page (www.victoria.ac.nz/lals/about/staff/paul-nation#paul_nation_vocabulary_resources_downloads).

12 See Appendix E.

114

L2. For this purpose, a spelling recognition task¹³ was administered in which participants were asked to circle the words that they thought were incorrectly spelled.

The task was created and developed by Andrews and her collaborators, who administered it in many L1 morphological processing studies including Andrews &

Hersch, (2010) and Andrews & Lo (2012, 2013), as discussed in Chapter 2. It consisted of 88 words that are highly frequent with a mean score of 32,27 per 1 million in the CELEX database; therefore, the absence of lexical knowledge about the items was quite unlikely to emerge as a confounding factor. Only correct responses were taken into account to calculate the scores. Participants completed the task right after the vocabulary size test in approximately 10 minutes.

The median split technique was run to form two categories based on the spelling ability of participants considering participants’ performance on this task. It showed that those with a score above 77 out of 100 (n. 54) turned out to have a high spelling ability while the rest who scored 77 or below were assumed to have a low spelling ability (n. 58).