Test Conditions Duration and Time Availability

The draft version of the tests (CRT-diagnostic tests 1 and 2 and 3 and 4) was trialled with a small group of 12 engineering students who had registered for a summer course prior to the commencement of the regular semester and were available to the researcher. The trial group participants were not part of the cohort of study participants. The time given to the trial group participants to complete the test was one hour for completing two tests each. Participants in the trial group completed the tests in about 50 minutes to an hour. It was anticipated that the study test participants require a minimum time of one hour to complete two CRT-diagnostic tests each. Slight revisions were made to the layout following the trialling for better readability. It was expected that these revisions would enhance clarity of each section for improved performance of the test participants. Weir, Yang & Yan (2000) reports that alterations made to timing, order of sections and layouts improved the efficiency of the AERT prototype version 1, an academic reading test developed for tertiary students in Chinese universities.

The draft versions of the tests were also trialled with native English language speakers (experienced academic staff) on campus and the results compared for timing and

ability. Feedback from the native English test takers on the appropriateness and suitability of test items were taken into consideration and incorporated into the review of the final draft version of the tests which have enhanced the quality and clarity of the final version of the tests. Alderson, Clapham and Wall (1995) advocate the practice of reviewing draft items with teams of language teachers to reaffirm needs analysis research practices and content validation of draft tests. The researchers (Alderson, Clapham & Wall, 1995) support the credence of experienced teachers in professional contexts where the items are developed for CRT-achievement tests. The final CRT-diagnostic and achievement tests have had inputs from experienced peers at the university who not only have subject expertise but also a good understanding of proposed test participants.

4.7.1 Test Items

The tests are designed to measure diagnostic vocabulary proficiency, both receptive and productive in nature. Receptive language information is obtained relatively efficiently and quickly through selected-response items, which are most suitable for testing receptive skills in language such as reading or listening (Brown & Hudson, 2002,p. 64).

“In selected-response items the respondents, choose the correct response from a set of supplied options. The most common form is the multiple choice item…” (Brown & Hudson, 2002:59). For selected-response items, the respondents are able to choose possible answers from a list called the options or alternatives, the correct answer is called the key and incorrect alternatives are called distractors ( Brown & Hudson, 2002:59). The distractors selected for the tests follow a model recommended set by National Assessment Governing Body ( NAGB, 2005 ; Salinger et.al., 2005 as cited in Pearson, Hiebert and Kamil, 2007, p. 293) as shown in the following section.

Distractors as defined in the context of the pre-tests-

a) a word that has a more common meaning of a target word, but that must be ignored in favour of the meaning in context.

b) A word that presents correct information or content from the text that is not what is meant by the target word.

c) A word that has an alternative interpretation of the context in which the target word occurs; or d) Other words that look or sound similar to target words

The target words definitions and distractors for the current pre-tests were taken from the Longman Dictionary of Contemporary English (2003).

4.7.1.1 Part- 1

The first section of the test Part-1 measures receptive skills of reading through multiple test items via which the respondents are tested for lexical knowledge. The respondents are asked to select the meaning of words most frequently used in the reading test, from a list of possible options and distracters along with an item-stem, which will serve as a prompt such as, “according to the article”.

Example: According to the article, power means: Choice of answer:

a) ability b) energy c) strength d) authority

The first task requires respondents to define terms used in the text. This exercise necessitates recognition skills and lexical knowledge correlated to receptive vocabulary proficiency. Four items from the first text and five items each from the second, third and fourth text; a total of 19 items, test recognition skills and lexical

knowledge using the most frequently used words in the texts. The items selected correspond to the frequency of their usage within the texts and are expected measure receptive skills of participants efficiently as they are the most frequently used words in the text. A number of words are tested for receptive skills using multiple texts. Vocabulary researchers recognise that testing in context require using reading or listening comprehension texts which in turn reduces the number of items that can be included (Read, 1993, p.357 ). The benefit of testing in context is that one is able to verify that the test-takers have actual knowledge of the words tested, the test is presented in a meaningful context, and that one could establish how well they knew the words (Read, 1993).

Read (2000; 2007) further advocates the practice of testing in context. He states that it is better to conduct an in-depth probe on a limited number of words in context than test vocabulary size on an overall superficial level. Alderson, Clapham and Wall (1995) state that reading tests should contain between 40 and 50 items and that each item should be worth about 1 mark. The current tests consist of a total of 53 items which strengthen their validity and permit the researcher to test participants in context. The multiple approach to testing employed in the current CRT diagnostic and achievement tests (multiple choice, gap filling, sentence completion) reduces the method effect in test participants. Method effect is defined by Alderson, Clapham and Wall (1995) as the influence of a particular type of testing which affect a test taker’s score.

The tables (4.5), (4.6), (4.7) and (4.8) show lists of most frequently used words in texts (1), (2),(3) and (4).

Words Frequency

Power 7 Grid 3 Utility 3 Solar 3