Research Instruments – the Pilot Study

As can be ssen in Table 3.1, the pilot study was conducted using three research tools: the reading habits questionnaire, the English reading test and interviews respectively. Therefore, it is useful at this point to discuss in detail what and how many questions/items each of the research tools was consisted of in the pilot study. Thus, this section discusses how the research instruments: reading habits questionnaire, the English reading test and interviews respectively, were adapted for the pilot study.

3.5.1.1 Reading Habits Questionnaire – the Pilot Study

As noted in Section 3.4.1.1, a reading habits survey questionnaire from Bensoussan (2009) was deemed to be an appropriate tool to investigate reading habits of Sindhi and Urdu students in English and in L1. This section illustrates the adaptations made to Bensoussan’s (2009) questionnaire to fit it for the pilot study (See Pilot reading habits questionnaire in Appendix 1) is given below.

The pilot questionnaire was aimed to obtain background information about demographic variables such as gender, class, age, first language (L1) of reading, second language (L2) of reading and medium of schooling. The variables such as gender, class and age were also given by Bensoussan (2009). However, unlike Bensoussan’s (2009) questionnaire, which incorporated reading preferences in L1, L2 and L3; the present study used the terms: ‘First language (L1) of reading’ and second language (L2) of reading. The addition of the phrase ‘of reading’ was more appropriate to the context of the present study in that some student groups at the University of Sindh such as Punjabi, Sriaki and Baloch have no standardized written form of their respective L1. They often read in Urdu as their first language of reading (See Section 1.6.1). Also, the pilot study questionnaire, in addition to Bensoussan (2009), incorporated three questions, such as how often the participants

89

read in English, in L1 and in L2 using six point Likert-type scale from ‘1 never’ to ‘6 daily’ with an addition of ‘Why’ to generate qualitative data.

The main focus of the pilot study questionnaire was to investigate the participants’ reading habits in relation to various reading materials such as newspapers, literature, textbooks, sacred texts, academic articles and poetry based on a six point scale from ‘1 never’ to ‘6 daily’. The reading materials and the six point scale ‘1 never’ to ‘6 daily’ were adopted from Bensoussan (2009) accepting the fact that students at undergraduate level, more or less, read the same materials. The other reason for maintaining the same materials was to provide the students more room to reveal their real life reading habits. Nonetheless, instead asking about ‘internet’ (as given by Bensoussan, 2009) an additional choice ‘others’ was offered in the pilot study reading habits questionnaire. This was done to assure that if the subjects preferred to read a material/s other than the given choices, which could then be incorporated into the main study questionnaire.

In addition, some important changes were made to the Bensoussan’s (2009) questionnaire to fit it in the context of present study. For example, instead of using the term ‘L3’ the pilot questionnaire replaced it with the word ‘English’. This change was made to investigate students' reading habits in English, in particular. Lastly, the pilot study questionnaire incorporated an additional question about the number of books the students read during the last six months in English, in L1 and in L2 respectively. This question was added to identify if the students in this study preferred lengthy texts or they did not do book reading.

3.5.1.2 English Reading Test – the Pilot Study Version

The other research tool used in the pilot study was the English reading test (See Pilot English reading test in Appendix 2). The purpose of the reading test was to measure Sindhi and Urdu students’ English reading performance (overall test score) and in terms of careful and expeditious reading types. Hence, to achieve the purpose, a well- structured and reliable English reading test was adapted. The test items were carefully selected to make sure that they incorporated the major reading types and also they were appropriate to the level of the participants in the current study. Table

90

3.2 below summarises each of the four parts of the pilot reading test together with the rationale and the number of items in each part of the English reading test.

Table 3.2: Pilot Reading Test and Rationale for each Part

Having summarised each part of the pilot reading test together with the rationale and the type of reading it measured in Table 3.2, it is now important to illustrate the selection process the pilot English reading test. This will help suggest the pilot reading test incorporated the major reading types and it was appropriate to the level of the participants in the study.

Moreover, in order to select the test items appropriate to the level of students, the researcher consulted with his colleague from the English Department who had approximately eight year English language teaching experience at the University of Sindh. In order to know what type of reading each part of the test involved and what level of proficiency was required to attempt each part of the test the Khalifa and Weir (2009) description of the cognitive construct each of the tests measured was followed. This helped to select and finalize the test items for the pilot study on careful reading and expeditious reading types.

Reading test Rationale

Part 1

Adopted from PET Reading section 4 5 Items

This part was adapted as Part 1 because: a) the participants were familiar with the multiple choice careful reading test types and b) it required limited cognitive processing. Part 2

Adopted from FCE Reading section 1 8 Items

The purpose of adapting FCE Reading section 1 as part 2 of the pilot test was that it required a higher level of processing in careful reading type.

Part 3

Adopted from PET Reading section 2 5 Items

PET reading Section 2 was adapted as Part 3 of the pilot test considering it not only appropriate to the students’ English reading proficiency on expeditious reading type but also interesting.

Part 4

Adopted from PET Reading section 3 10 Items

As this part consisted of the maximum number of items, hence, to receive maximum students’ response on other three parts of the test and it was adapted as the last to analyse students’ proficiency on expeditious reading.

91

Firstly, five reading sections from five authentic English reading tests, such as Cambridge KET (Key English Test), PET (Preliminary English Test), FCE (First Certificate in English), CAE (Certificate in Advanced English) and CPE (Certificate of Proficiency in English) examinations were scrutinized to select the test items for the pilot study. No items were selected from KET, CAE and CPE because the first one (KET) was considered too easy comparing to the level of the students in the study, whereas the other two tests: CAE and CPE were considered much more difficult in accordance with the level of the learners.

Therefore, the only available options from the Common European Framework of Reference for Languages (CEFR) regarding the selection of the test items were the PET and FCE tests of B1 and B2 level respectively. The B1 and B2 levels were considered by the researcher and his colleague to be similar to the English reading proficiency level of Grade 10 (Matriculation) and Grade 12 students respectively, in the Pakistani context. Finally, four parts of the reading test for the present study were selected from the reading sections of PET part 2, 3 and 4 and FCE Part 1. All the selected parts were considered appropriate to the students’ grade level measuring their reading performance on careful reading and expeditious reading types.

Moreover, the test was designed in a way that could help measure the students’ performance on two types of reading: careful reading and expeditious reading. As can be seen in Table 3.2 the pilot English reading test was divided into four parts. However, for data collection purposes, it was divided into two sections with two parts in each section. Section One was consisted of two parts: part one and part two respectively. Part one was taken from PET reading section 4 whereas, part two of the pilot reading test was adapted from FCE reading section 1. Section 2 of the pilot test was also divided into two parts: part three and part four respectively. Both, Part three and part four, were adapted from PET reading section 2 and PET reading section 3 respectively. Section One of the pilot test was intended to measure students’ English performance on the careful reading type whereas, Section Two was designed to analyse the students’ reading performance on expeditious reading.

92

The reason for the division of the test was to make sure that the students must respond to both reading types intended. The reason for making section 4 of the PET reading test as part one of the pilot test was to encourage the participants feel at ease because they were thought to be familiar with the multiple choice careful reading test types. Also, it required limited cognitive processing. The purpose of adapting FCE reading section 1 as part two of the test was that FCE section 1 was thought to be involving higher level of processing as compared to PET in careful reading type.

Similarly, PET reading section 2 was adapted as part three of the pilot reading test considering it appropriate to measure English reading proficiency of the students on expeditious reading type and also to maintain their interest in the test. Part four of the test, adapted from PET reading section 3, consisted of the maximum test items (n=10). Hence, in order to receive maximum response from the students on other three parts of the test it was included as the last part of the test to analyse students’ proficiency on expeditious reading. The pilot study identified that all test items were appropriate to measure the English reading performance of the learners on the careful and expeditious reading types. It also suggested that the test was also appropriate to the participants’ level of English reading proficiency at their grade level.

3.5.1.3 Interviews – the Pilot Study

In the pilot study, nine semi-structured interviews were conducted. The main purpose of the interviews was to identify how far the various instruments accurately served the purpose of the pilot study and what changes were to be made to the instruments to further develop them for the main study. Therefore, most of the interview questions asked the students mainly about the difficulties they faced while attempting the questionnaire and the reading test. The purpose of the interviews was to ensure whether the various instruments used in the pilot study were accurately measuring the reading habits in English and in L1 and English reading performance. The interviews were found a great help to develop the instruments further in the main study. Also, they confirmed the accuracy of the instruments at the level of the

93

participants. A detailed summary of the pilot study interviews will be presented in Section 3.5.5.

Having discussed the research instruments used in the pilot; it is now important to discuss how the pilot was conducted. Then there will be a detailed discussion on the procedures adopted to conduct the main study. However, the ethical considerations for both the pilot and the main study were exactly the same. Thus, they are discussed first, followed by a detailed discussion on the data collection of the pilot study.