Validity and reliability

Drost (2011) suggests that research instruments for a quantitative study should be valid and reliable. The concepts of validity and reliability of research instruments are discussed below.

3.2.1 Validity

Hughes (2003) explains that validity relates to the level of accuracy of a test in measuring what it is intended to measure. Drost (2011) explains that there are different types of validity. These include construct validity, face validity, and internal and external validity.

3.2.1.1 Construct and criterion-related validity

Brown (2000) defines construct validity as a degree to which a test measures what it purports to be measuring. Thus a reading comprehension test should measure reading comprehension and not listening comprehension or writing ability. According to Hughes (2003, 26), the notion of construct validity refers to the general, overarching notion of validity. A test that purports to have construct validity may be evidenced by several forms of validity including content validity and criterion-related validity. For the current study, the reading test used as one of the research instruments had construct validity as it comprised a test passage and questions testing participants’ comprehension of the passage at the literal, inferential and evaluative levels. In the same vein, the vocabulary test used in the current study also had construct validity as it tested word knowledge at the 2000-, 3000-, 5000-, UWL and 10 000-word levels.

On the other hand, criterion-related validity looks at the relationship between a test score and a specific outcome (Sudweds 2007). For example, Form 5 reading tests in Swaziland test learner’s ability to answer literal, inferential, interpretive and evaluative questions. These are specific comprehension criteria in terms of which learners’ overall comprehension ability are assessed.

3.2.1.2 Face validity

Face validity is the notion that a test looks as if it measures what it is supposed to measure. For example, Hughes (2003) points out that a test that purports to measure pronunciation ability but does not require the test taker to speak, might be thought to lack face validity. At face value the research instruments used in the current study, namely the reading test and vocabulary test, had face validity as they clearly seemed to be testing reading ability and word knowledge in formats that the students recognised. The reading test had a 2000 word long passage where learners had to read and respond to various questions. The reading test also appeared in a format that is familiar to students as the text was followed by different question formats. In the same vein, the vocabulary test seemed to be testing vocabulary knowledge at the various word levels as the missing items were words that needed to be filled in. The participants in this study accepted the tests as they understood what was required.

3.2.1.3 Internal validity

The concept of internal validity is explained as freedom from bias in forming conclusions in view of given data (Leedy 1997).In experimental studies it refers to whether a researcher has controlled for all other factors to which test results could be attributed. If there are differences in the results, the researcher has to be sure that there were no other interfering factors that compromised the data. This study was concerned with the concept of validity not only in the test items but also in the way in which the participants’ responses were scored. Hughes (2003, 33) points out that if a reading test is meant to measure reading ability, the scoring of such a test must not take into account spelling and grammar because if that is done, the test would no longer be measuring what it was designed to measure.

To maintain internal validity of the reading test and the vocabulary levels test, I made sure that the scoring of the reading test and vocabulary test disregarded spelling and grammar as long as it was easy to understand what the participants wanted to say.

The reading questionnaire used in this study measured reading attitudes, reading habits, and access to reading resources. Although it was not easy to come up with items that tested these aspects directly, I made sure that the scoring of responses related directly to what was being tested (Hughes 2003, 34).

As the tests were taken at different intervals, I made sure that I took into account the dropout rate so that the results would not be compromised. As a result, students who took one test and abstained from the others were left out of the study. The questionnaire items were first tested on seven teachers in the English Department of the school where I teach, with the idea of making some changes in case the items were not clearly worded, and to clarify the items before they were piloted on the learners who participated in the study.

3.2.1.4 External validity

Nunan (1992, 15) describes external validity as the extent to which the results can be generalizable from sample to populations. It answers the question, ‘Can the conclusions drawn

study I was aware that its findings should give insight into the relationship that exists between reading, vocabulary, attitudes and academic performance. The study took into consideration external validity because the participants in the study were learners from four different schools representing typical rural and urban school settings in Swaziland. So all types of learners in the country could be said to be represented in the study as the participants came from a range of backgrounds including poor families, middle income families and from affluent households.