1. Introduction
4.9 Data collection instruments
4.9.1 Questionnaires
Questionnaire as a general term “includes all data collection techniques in
which each person is asked to respond to the same set of questions in a predetermined order” (Saunders et al., 2012:679). Payne and Payne
(2004:186) define questionnaires as “the printed set of questions to be
answered by respondents, either through face-to-face interviews or self- completion, as a tested, structured, clearly presented and systematic means of collecting data”. Gray (2014:352) also defines questionnaires as
“research tools through which people are asked to respond to the same set
of questions in a pre-determined order”. Denscombe (2010) points out that
questionnaires are at their most productive when used with large numbers of participants. Jankowicz (2005:222) states that “questionnaires are
136
people to obtain data on the same issue or issues, often by posing the same questions to all”. Dawson (2009) divides questionnaires into three
types: closed-ended, open-ended, or a combination of both. On the other hand, Cohen et al. (2011) call the closed-ended questionnaires as structured questionnaires, the open-ended as unstructured questionnaires, and the combination of both as semi-structured questionnaires. They argue that the smaller the size of the sample, the less structured, more open and word-based the questionnaire is likely to be; and the larger the size of the sample, the more structured, closed and numerical the questionnaire should be.
Closed-ended/structured questionnaires are used to generate statistics
in quantitative research. These questionnaires follow a set format with boxes to tick or scales to rank. Great numbers can be produced because of the ease of analysis.
Open-ended/unstructured questionnaires are used in qualitative
research. They consist of a set of questions with a blank section for participants to write their answers.
Semi-structured questionnaires or a combination of both types of
questionnaire is used by some researchers. Such questionnaires start with a series of closed questions and finish with a section of open questions for more detailed responses.
Dawson (2009) lists some advantages and disadvantages of open and closed questions. While open questions tend to be slower to administrate, harder to record responses, difficult to code, and takes longer to answer,
137
closed questions tend to be quicker to administrate, easier to record answers, easy to code and quick to answer. However, open questions enable respondents to speak their minds and raise issues, which is not the case with closed questions, where response is stifled. Greener (2011) provides two reasons that make using open questions in some research projects more appropriate: to avoid imposing the researcher’s ideas and concepts upon the respondent, and when it is impractical to give all of the possible options, which might be too many. Nunan (2006) advises researchers to be aware of the types of questions they include in questionnaires as well as question wording so as to avoid culturally biased questions, which are likely to happen in language and education research when the researcher and respondents do not share the same culture. Dörnyei (2010) lists some of the main problem sources in questionnaires such as: simplicity of the questions and the short time that respondents spend working on a questionnaire result in superficiality of answers and limit the depth of the investigation; literacy problems of some respondents (specially those with limited L2 proficiency); unreliable and unmotivated respondents; having no opportunity to correct the respondents’ mistakes if they misread or misinterpret questions; and the social desirability or prestige bias which may affect the results when people do not provide true answers.
Dörnyei (2010) indicates some advantages and disadvantages of questionnaires. He highlights their “unprecedented efficiency” in terms of financial resources and researcher time and effort as well as being very versatile; that is, they can be used with a variety of people in a variety of
138
situations dealing with a variety of topics. However, there are some limitations that threaten the reliability and validity of questionnaire data if the questionnaires are ill-constructed. According to Cohen et al. (2011), the advantages of the questionnaire over interviews are that: the questionnaire tends to be more reliable; it is more economical in terms of time and money; it is possible to be mailed; and because it is anonymous, it encourages more honesty (although not guaranteed). But still, Cohen et al. (2011) argue that interviews can be effective and efficient, and accurate data can be obtained if the interviewer is skilful and does his or her job well and the respondent is sincere and well-motivated.
Table 4.8: The advantages and disadvantages of a questionnaire
Advantages Disadvantages
It is less expensive. Self-selecting bias.
It offers greater anonymity. Response rate can be very low. The inflow of data is quick from
many people.
Limited application. People of limited literacy may not be included in sample It provides convenience (of time,
place and speed) for respondents.
Lack of opportunity to clarify issues by interviewers and respondents.
Interviewer effects are absent. Others can influence the answer. It allows remote access to
respondents.
There is no opportunity for spontaneous response as it can be read as a whole before starting answering.
Data analysis of closed questions are relatively simple.
Respondents might give inaccurate or misleading answers.
Source: Bryman (2012), Kumar (2014), Burton et al. (2014) and Gray (2014)
Questionnaires are often used to measure the frequency of attitudes, beliefs and behaviours of individuals. This is in line with the objectives of
139
this study, which aim to investigate the students’ attitudes and perceptions towards learning and teaching of English reading comprehension at university level. Through the responses of the participating students, some of the current problems and difficulties that face Libyan students in the Department of English at Zawia University in reading comprehension can be figured out. In addition, the respondents’ replies illustrate how English reading comprehension is taught in the Department of English at Zawia University.
According to Kumar (2014), using closed-ended questions in a questionnaire helps to ensure that the data required by the researcher is obtained and the responses are easier to analyse. In addition, closed- ended questions are easier and quicker to complete and may enhance the comparability of answers and clarify the meaning of a question for respondents (Bryman, 2012). This study uses the the closed-ended questionnaire as a quantitative data collection instrument.
Dörnyei (2011:102) lists three types of data that questionnaires can produce about respondents and they are as follows:
Factual questions: which are used to find out certain
facts about the respondents such as demographic characteristics.
Behavioural questions: which are used to find out what
the respondents are doing or have done in the past.
Attitudinal questions: which are used to find out what
people think, covering attitudes, opinions, beliefs, interests, and values.
The data pursued in the questionnaire of this study are in line with Dörnyei’s (2011) list. The first part of the questionnaire seeks some
140
demographic data about respondents such as their gender, ages and academic levels. The questionnaire consists of 39 questions that are divided into four dimensions: Dimension One (3 questions) tries to investigate the attitudes of students towards reading comprehension and reading in English. Dimension Two (8 questions) tries to gauge the students’ abilities and use of the different skills and strategies of reading comprehension. Dimension Three (4 questions) tries to check students’ opinions about the facilities and resources of their university. Dimension Four (24 questions) is related to students’ attitudes towards their lecturers’ teaching strategies in the reading classes and the activities that they use before-, while-, and post-reading an English text.
The questionnaire has a five-point Likert scale, ranging from (1) ‘strongly disagree’ to (5) ‘strongly agree’. Kumar (2014) points out that of the three types of scale that measure attitude (which are the Likert, Thurstone and Guttman scales), the Likert scale is the most commonly used. Bryman (2012:166) states that the Likert scale “is essentially a multiple-indicator or
multiple-item measure of a set of attitudes relating to a particular area. The goal of the Likert scale is to measure intensity of feelings about the area in question”. According to Bertram (2007), some of the strengths of the Likert
scale are that it is simple to construct, likely to produce a highly reliable scale, easy to read and complete for participants and that a universal method of collecting data is used, which makes it easy to understand and code them. When constructing a Likert-scale, the items that take up the scale should be statements that are interrelated sharing the same object (Bryman, 2012).
141
4.9.2 Interviews
An interview is defined by Payne and Payne (2004:129) as “data collection
in face-to-face settings, using an oral question-and-answer format”.
According to Saunders et al. (2012:680), a research interview is a “purposeful conversation between two or more people requiring the
interviewer to establish rapport, to ask concise and unambiguous questions and to listen attentively”. The interview process means a
purposeful discussion between two or more individuals (Churchill and Iacobucci, 2009; Gray, 2014). Moreover, the purpose of the interview is to provide reliable and valid data that are relevant to the research objectives.
Burton et al. (2014) state that interviews are very effective in research in education. As far as reading comprehension is concerned, Rivers (2000:70) states that “...interviews have proved useful in identifying the
strategies employed by efficient readers as they extract meaning from texts”. According to Punch (2005) and Denscombe (2010), being one of
the main data collection tools in qualitative research, interviews are suitable when there is a need to collect data based on people’s perceptions, opinions, feelings and experiences, which is one of the objectives of this study. Interviews aim to explore diversity rather than to quantify (Kumar, 2014). Therefore, usually few people take part in the interviews.
Interviewers should be well-organised and well-prepared when they conduct an interview, dress and behave appropriately and show respect and establish rapport with the interviewees, be punctual and negotiate and
142
stick to a length of time for the interview, keep questions short and to the point avoiding double-barrelled questions, and repeat and summarise answers to aid clarity and understanding (Dawson, 2009).
Some of the purposes of interviews are: to test or develop a theory; to evaluate or assess a person in some respect; to gather data; and to sample respondent’s opinions (Cohen et al., 2011). This study uses interviews to gather rich descriptive data about the lecturers’ opinions about teaching reading comprehension and the factors that affect their practices and students’ performance.
How an interview should be conducted depends on the accessibility of individuals, the cost, and the amount of time available. Creswell (2012) lists four types of interview approaches: one-to-one interviews, focus group interviews, telephone interviews, and e-mail interviews. One-to-one interviews are the most time-consuming and costly approach; and focus group interviews are used to collect shared understanding from several individuals and to get views from specific people. Both telephone interviews and e-mail interviews are often used when the participants are geographically dispersed. For this study, face-to-face interviews were supposed to be conducted; however, due to the unrest and turmoil that is currently taking place in Libya, the researcher was unable to travel to Libya to conduct face-to-face interviews. As a result, the researcher held telephone interviews via Skype and phone calls.
A number of researchers (Dawson, 2009; Denscombe, 2010; Gill and Johnson 2010; Robson, 2011 and Easterby-Smith et al., 2012) distinguish
143
three different types of interviews: structured interviews, unstructured interviews and semi-structured interviews.
Structured interviews are used in quantitative research, frequently in
market research, hence the name as they are highly structured. This sort of interview consists of a series of pre-established questions, with pre-set response categories and should be delivered in a standardised manner. The interviewer, who ought to play a neutral role, asks the questions and ticks boxes with interviewee’s responses. It can be conducted face-to-face, over the phone or online.
Unstructured interviews are used for qualitative research. They are also
called in-depth interviews. Hair et al. (2007:419) point out that the in-depth interview is “an unstructured one-to-one discussion session between a
trained interviewer and a respondent”. Nunan (2006) indicates that these
interviews are usually guided by the interviewee’s responses rather than the researcher’s agenda, which makes the direction of the interview relatively unpredictable. The main strength of unstructured interviews lies in being free in terms of the structure, the question wording and order, and content (Kumar, 2014). The questions in this type of interview are open- ended and not pre-planned, and with no pre-established categories for responding. The interviewer attempts to achieve a complete understanding of the points of view of the interviewees who are free to talk about what they think important, with little directional influence from the researcher. Unstructured interviews can be difficult to analyse due to the great deal of data produced. Punch (2005) indicates some points to think about when
144
planning data collection by unstructured interviews such as understanding the language and culture of respondents, gaining trust and establishing rapport.
Semi-structured interviews lie between these two extremes (Punch, 2009).
They are guided open interviews (Easterby-Smith et al., 2012). Wilkinson and Birmingham (2003) point out that in the semi-structured interview, the interviewer sets up a general structure by deciding what area is to be covered and what main questions are to be asked. In addition, the interviewer has the opportunity to ask further questions in response to what are seen as significant replies (Bryman, 2012). Because of this flexibility, this type of interview is the most common type of interview used in qualitative social research (Dawson, 2009) and in applied linguistic research (Nunan, 2006; Dörnyie, 2007).
Interviewers produce interview schedules of a list of specific questions or topics to be discussed in order to elicit specific information. Although the same questions should be asked in each interview so that the researcher can compare and contrast the responses of the participants, the interview needs to remain flexible to let other important information arise. As pointed out by Dörnyie (2007), in semi-structured interviews, the interviewer provides direction and guidance, and follows up interesting developments to encourage the respondents to elaborate on certain issues. Kumar (2014) and Burton et al. (2014) present some advantages and disadvantages of interviews as follows:
145
Table 4.9: The advantages and disadvantages of interviews
Advantages Disadvantages
More appropriate for complex situations. Time-consuming and expensive. Useful for collecting in-depth information. Possibility of researcher bias. Questions can be explained. The quality of data is influenced by
the quality of the interviewer. Interactive as it allows the interviewer to
probe relevant issues.
The interviewer requires the skill to keep it focused and relevant. Very effective when there is trust
between the interviewer and interviewee.
The quality of data depends upon the quality of the interaction. Has a wider application May be difficult to arrange. Source: Kumar (2014), Burton et al. (2014)
According to Punch (2009) and Cohen et al (2011), interviews provide a useful way of understanding others and enable participants to express their ideas and understanding of the context that they work in. As a tool of data collection, the interview is flexible and powerful, enables interaction and gives space for naturalness (Cohen et al., 2011). In addition, Wilkinson and Birmingham (2003:44) state that ‘while other instruments focus on the
surface elements of what is happening, interviews give the researcher more of an insight into the meaning and significance of what is happening’.
Therefore, interviews are an appropriate tool for collecting data in this research. According to Rowley (2012:162), interviews are useful when “The research objectives centre on understanding experiences, opinions,
attitudes, values, and processes”. For this study, semi-structured
interviews were conducted to find out the lecturers’ views and perceptions about their students’ reading skills and the challenges facing both students and lecturers during the teaching of comprehension skills.
146
4.10 Sampling
Sampling is one of the key ways to assess the design of a research project. Dörnyie (2007:96) defines ‘sample’ as “the group of participants
whom the researcher actually examines in an empirical investigation”.
Hallebone and Priest (2009) state that the sample, which is a part of a larger population about which the study is, should be consistent with the study’s objectives, the research questions, the method(s) or technique(s) used and the intended uses of the study’s findings. Gray (2014) identifies two main approaches or procedures of sampling: probability sampling and non-probability. Probability sampling involves selecting random samples of subjects from a given population that represents the total number of possible elements as part of the study, whereas in non-probability sampling, the selection of participants in a study is non-random. Probability sampling includes simple random sampling and stratified sampling and cluster sampling while non-probability sampling includes convenience sampling, quota sampling and purposive sampling. Probability sampling designs are considered valuable due to the possibility to be precise about the relationship between a sample and the population, which allows the researcher to make firm judgement about the relationship between characteristics of a sample and the population (Easterby-Smith et al., 2012).
Random sampling is defined by Greener (2011:202) as “a sampling
process which attempts to be representative of the wider population by everyone within the group having an equal chance of being chosen”.
147
with respect to the research questions (Gray, 2014). In random sampling, a researcher includes all members of the population and selects a sample of them in a random process so as to avoid bias. According to Gill and Johnson (2010), another aim of random sampling is the generalisability of the finding through ensuring that the participants are a representative subset of the research population. As indicated by Greener (2011:202), in stratified sampling, “the population is split into groups that may be
representative of it according to a theory being tested (e.g. class, gender, ethnicity) and participants chosen randomly within them”. An advantage of
stratified sampling is that it increases the key groups in the sample while ensuring random selection (Gray, 2014). Cluster sampling acknowledges the difficulty in sampling a population as a whole and choosing a large number of small clustering units is preferable to a small number of large units. Hair et al. (2007) point out that, in cluster sampling, the target population is viewed as made up of heterogeneous groups or clusters.
Non-probability sampling designs such as convenience sampling involve selecting sample units on the basis of how easily accessible these samples