It is clear that when an examinee fails to select the correct option in a multiple choice test, his
rejection of the correct option and his choice of an incorrect option have a significance which should be recognised in the marking of the test. Failure to do this has been singled out as a weakness of the multiple choice item.
A more sensitive method of administering a five- option multiple choice test which enables the examinee to
earn partial credit for partial knowledge about a
question has been developed by Wiley. (^0) This method, which Wiley calls the "Three-decision multiple choice test", requires the examinee to designate tv/o options
which he feels are definitely wrong as well as designating the option which he feels is the correct one. Three
marks are given to a question if the option designated as the correct one is actually the correct option. Tv/o marks are given for a question if the option which is actually correct is not designated as the correct option or one of the definitely wrong options. No marks are given to a question if an option designated as one of the definitely wrong choices is actually the correct option.
The purpose of the present investigation is to compare the results obtained from scoring a multiple choice test using four different scoring procedures. (a) The conventional multiple choice scoring system,
i.e. one mark for a; correct choice of option, no marks for an incorrect choice.
(b) The system suggested by Wiley as described.
(c) /
89
(c) A modification of Wil e y fs system, devised by the author.
(d) A second modification of Wiley1s system devised by the author.
Method
The pupils taking part in the investigation were one hundred and three first year pupils following the Integrated Science Syllabus who were about to take the test on section five of the syllabus. The test
consisted of twenty multiple choice items each with five o p t i o n s .
The pupils were instructed to answer the questions on the conventional answer grid by using a vertical
line to designate the correct option and to designate the two options which they considered to be most
incorrect by a cross. The questions were then scored by four different methods to allow a comparison of the different scoring systems to be made.
(a) The questions were scored using the conventional scoring system, i.e. 1 mark for choosing the correct option, 0 marks for choosing an
incorrect option.
(b) The questions were scored as a "Three-decision multiple choice test" with a marks allocation 3> 2, 0 as described in the introduction.
(c) The questions were scored as in fb* but with a marks allocation of 2, 1, 0.
(d) The questions were scored as in fb' but with a marks allocation of 2, 1, -1.
Results
The average mark scored by all the pupils under each of the four scoring systems is shown in Table 6.1. In each case the marks have been reduced to a mark out of a maximum of twenty marks in order that comparisons can /
can be made of the various scoring systems. Also shown in Table 6.1 are the percentage increases of the three "Three—decision multiple choice test" scoring systems over the conventional scoring system.
Table 6.1
The average marks obtained by pupils under four s coring systems and the percentage increase of the "Three-
decision multiple choice test" scoring systems over the conventional scoring system.
Scoring System Average Mark (Maximum 20) Percentage increase over Conventional scoring system (a) Conventional 9.4 Scoring System (b) W i l e y ' s ^ ^ 13.5 43.8 Scoring System (c) Modification 1 of 12.4 32.9 Wiley's System (d) Modification 2 of 10.2 8.5 Wiley's System Observations
(1) The conventional scoring system produced the lowest average mark.
(2) Modification 2 of Wiley's scoring system produced the smallest percentage increase over the
conventional scoring system. In this system one mark is deducted for an item in which an option designated as one of the definitely wrong choices is actually the correct option.
(3) Wiley's suggested scoring system 'b ' and the first modification of his system are basically the same
in that no penalty is. introduced for an item in which an option designated as one of the
definitely wrong choices is actually the correct choice. The relative weightings have been
altered merely to make the addition and consequent averaging /
91
averaging of individual item scores more managable. Therefore, scoring system 'c' will always produce a lower average mark than scoring system * b 1. In order to compare the marks on each of the three modified scoring systems with the marks obtained on the conventional scoring system Table 6.2 was
constructed which shows the average marks for all of the pupils on each of the three modified scoring systems who scored a particular mark on the conventional scoring
system. A graph of conventional test scores against average mark scored on each of the modified scoring systems was also plotted for particular conventional test scores.
Table 6.2
The average mark scored on each of the three modified scoring systems by pupils scoring a particular mark on the conventional scoring system.
Conventional Score Average Score (Wiley) Average Score (Modification 1) Average Score (Modification 2) 2 7.3 6.0 1.0 3 9.7 8.0 4.5 4 9.1 8.0 3.7 5 10.4 9.1 5.7 6 . 11.5 10.2 7.3 7 11.9 10.6 7.7 8 12.7 11.6 9.1 9 13.7 12.3 10.2 10 13.2 12.4 9.8 11 15.2 14.2 12.8 12 14.7 • 13.8 11.8 13 15.7 15.0 !3.5 14 17.1 16.4 15.5 15 17.0 16.5 15.5 16 17.3 17 16.0 17 17.7 17.5 16.5