RESEARCH METHOD
F. Data Analysis
In this study, the data was analyzed quantitatively. The quantitative data analysis was done by analyzing the test items and students’ answer sheets. There are five points of item analysis; validity, reliability, item facility, discrimination power and distractor efficiency.
1. Validity
Measuring the validity of a test is not as easy as measuring the reliability, item facility, discrimination power, and distractor efficiency because the validity cannot be measured using formula. In order to ensure that
41
a test is valid, the researcher measured two types of validation; content validity and construct validity.
A test can be claimed as a valid test in term of its content if the test items can measure all the materials have been taught, not other or outside the given material. Hughes (1989: 22) stated that a test is said to have content validity if its content constitutes a representative sample of language skills, structures, etc. with which it is meant to be concerned.
To know whether the test items have good content validity or not, the researcher used the syllabus to get the clear specification of the skills or components or materials that it is meant to cover, then compared the test content andthe specification stated in the syllabus. At last, the researcher gave the percentage of skills being tested based on the specification provided.
Besides the content validity of the test, it is also necessary to know the construct validity of the test items in language testing. A test can be said to have construct validity if the test is created based on the underlying ability of each skill and component being tested. Hughes (1989: 26) added that a test, part of a test, or a testing technique is said to have construct validity if it can be demonstrated that it measures just the ability which it is supposed to measure because the word “construct” refers to any underlying ability which is hypothesized in a theory of language ability, so the researcher used the language testing theory of language ability to know whether the test has good construct validity or not.
42
In this study, the researcher analyzed the testing techniques used in a test, then connected it to the language testing theory to know whether the testing techniques used in the test are already appropriate to the language testing theory or not. For instance, writing test shown in the test items number 2, 6, 8, and 9, provided the students to complete the blank paragraph with the correct vocabularies provided, speaking test shown in the test items number 3, 4, 15, 27 asked the students to give correct response to a certain dialogue, reading test shown in the test items number 23, 12, 14, 13, was about deciding the similar meaning of the certain words. Listening test was not shown in the test item, it means that the test items were less of construct validity for listening skill, whereas listening skill was also mentioned in the syllabus material that it must also be achieved.
2. Reliability
In order to measure the reliability of the test items, the researcher used the KR-20 formula because this formula requires test administration only once and the scoring is one correct answer is given point 1, while incorrect answer is given 0, thus this formula is appropriate for calculating the reliability of multiple choice test form. In addition, Fraenkel and Wallen (2008:156) stated that KR-20 doesn’t require the assumption that all items are of equal difficulty .
43 Where:
r11 = reliability coefficient n = number of test items
2
s t = standard deviation
p1 = proportion of the right respond q1 = proportion of the wrong respond
After calculating the reliability of the test items, the researcher classified the reliability coefficient which taken from Sudijiono (1996: 209-230), as the table follows:
Table 3.1 Classification of Reliability Test Reliability Test Coefficient Classification
0.99-1.00 More highly
0.70-0.89 High
0.50-0.69 Fair
0.30-0.49 Low
<0.30 Very low
3. Measuring the Item Facility
To measure the item facility of level of difficulty of the test items, the researcher used the following formulas:
JS
P B (Arikunto, 2012: 223)
Where:
P = Item Facility (Level of difficulty)
B = Number of test-takers answering the item correctly
44
JS = number of test-takers responding to that item
To know the classification of the difficulty level, the researcher used the classification referred by Arikunto (2012:225). Here is the following classification and interpretation of difficulty level:
Table 3.2 Classification of Difficulty Indices
Difficulty Level Classification researcher needed to separate the students into upper and lower group in order to be applied in the following formula:
B
JA =Totalparticipant of top test-takers JB =Total participant of bottom test-takers
BA = Number of top test takers that have correct answer BB =Number of bottom test takers that have correct answer
A A
A J
P B = Proportion of the number of top class answering correctly
45
B B
B J
P B = Proportion of bottom class answering correctly
According to Arikunto (2012:232), here is the classification and interpretation of discrimination index:
Table 3.3 Classification and Interpretation of Discrimination Indices
Discrimination Index Classification
0.71-1.00 Excellent
0.41-0.70 Good
0.21-0.40 Satisfactory
< 0.20 Poor
Negative value on D Very Poor
5. Measuring Distractor Efficiency
The distribution of distractors means the distribution of alternative answers. The importance of calculating it is to know how well the distractors work in distracting the students’ answer. A good distractor is that it has the distribution index of more than 5% of the total examinees number.
Arikunto (2012: 238) points out that a distractor can be said to have functioned well when it is chosen by at least 5% of the total examinees. If the index of this is 0, thus the distractor should be discarded or eliminated.
46