Gender differences in test format

The effects of test format on female and male test takers’ performance in disciplines like mathematics, science, history and English as a native language, have been widely studied with rather conflicting results. One aspect that most studies have in common is that they examine the difference between multiple choice as selected response format, and free-response formats, as for instance short answer or extended response items. The present study will be one of the first in examining gender differences in more than one selected-response format, i.e. multiple choice, true-false-not given, and matching.

Interestingly, research focuses on contexts other than EFL within which comparisons across even rather different disciplines are drawn. To be more precise, some studies compare whether male and female test takers perform

English. This non-uniformity of disciplines as well as the period of time when the studies took place, which encompasses more than 20 years, might also have contributed to diverging results.

Broadly speaking, studies investigating gender differences in test format can be subdivided into the following categories according to their outcomes:

• Girls are disadvantaged by multiple choice tests • Boys outperform girls

• Boys and girls perform the same • Girls surpass boys

Geering (1993: 25) cites a report of a study by the University of London School Examinations Board undertaken in 1985 devoted to discovering biases of examination components. The report claimed that there is considerable evidence that females are disadvantaged by the multiple choice format compared to other test formats. Clark and Grandy as referred to in Geering (1993: 24) expanded on the results of this report in their research on sex differences in academic performance of women and men by examining their grades on the SAT, i.e. Scholastic Aptitude Test, and their college freshmen year grades. The SAT is a standardized college admission test in the United States. The majority of questions in the SAT take the multiple choice format (http://sat.collegeboard.com/home, 21.01.2010). According to Geering (1993: 24) Clark and Grandy argued that first-year college grades of female test takers were slightly underpredicted by their test scores. The same was discovered in a recent study by Christiane Spiel (2008: 39) investigating male and female test takers’ scores on the admission test for Austrian Medical Universities, entirely consisting of multiple choice tasks, and their scores on science subjects featured in the admission test at school. Girls outperformed boys on science subjects at school. Their superiority, however, was not reflected in their score on the admission test, where they were not only underpredicted but even outperformed by the male test takers (ibid). Thus the question arises whether female test takers’ performance can be adequately measured by means of multiple choice tests.

Geering (1993: 23) comments on a highly interesting study undertaken by Murphy in 1980. in which a Geography Examination before and after the introduction of multiple choice tasks were compared. Before the introduction of

multiple choice tasks male and female candidates were said to perform the same. After the introduction of the multiple choice tasks, however, boys achieved better scores than girls. This suggests that multiple choice tasks favor boys but underpredict girls. Geering (1993: 27) further reports on work of Breland who compared two advanced placement examinations: United States History and European History consisting of free-response and multiple choice parts. While in the free-response parts no gender differences could be found, boys were significantly and to a large effect size favored by multiple choice tasks.

Bolger and Kellaghan as referred to in Geering (1993: 32) even considered three different school subjects: Mathematics, Irish, and English, to investigate gender differences in results of multiple choice and free response tests. The researchers found that male test takers generally surpassed females in all subjects and on both test formats. In language subjects, however, gender differences diminished and boys only performed slightly better than girls. They concluded that male participants performed significantly better on the multiple choice than on the free-response formats, whereas for girls the opposite was true.

The disadvantage of female test takers on multiple choice tests in all of the above mentioned studies raises the question why boys seem to be favored by this selected-response format. One argument, cited in numerous studies (Spiel 2009, Freeman 2007, Barboza 1993, Geering 1993, Hardcastle 1991), might be that boys are more likely to guess and to employ risk taking strategies than girls. Barboza (1999: 33) expands on this further by adding that while boys tend to risk a guess, girls tend to leave a blank. Stobart, Elwood and Quinlan as referred to in Geering (1993: 39) argue that the superiority of boys is related to their employing of a so-called “eyes down” approach, which they claim is better suited for selecting a correct answer out of a set of options. Girls, however, appear to be inhibited by seeing the relative “rightness/wrongness” of the items. Ryan, as referred to by Barboza (1999: 33), points out that female test takers tend to see “unintended nuances” in the set of answers which in turn makes it harder for them to guess. A further aspect that might contribute to male test takers’ superiority on multiple choice tasks is that females are said to have

claims that while males are more likely to see a test situation as a “personal challenge”, females tend to regard a test situation as a threat, which according to Phakiti “leads to states of fear, and worry” (ibid).

While numerous studies claim that boys are favored by multiple choice tests, other studies argue that there is no significant difference between male and female test takers’ performance. Geering (1993: 27,28) refers to a study by Huntley in which no significant differences between males and females on a multiple choice geometry test, administered to 3,000 testees, could be detected. Interesting results also emerged from a recent study undertaken by Freeman (2007: 89) on gender differences in the reading test part of the Ohio Graduation Test (OGT), which contains multiple choice, short answer, and extended response items. Freeman discovered that male and female test takers’ scores did not significantly differ, which he suggests is contrary to most of the literature found on gender differences in multiple choice tests (ibid).

In contrast to the above mentioned researchers, Doolittle and Welch as quoted by Geering (1993: 28) found that female testees outperformed male testees in the multiple choice writing test of the Collegiate Assessment of Academic Proficiency (CAAP). In the mathematical part of the CAAP test, however, male test takers surpassed female test takers. This outcome poses the question whether gender differences are related to the subject content and not so much to the test format. An ESSSA (Equity in Senior Secondary Schools Assessment) project cited by Geering (1993: 38), however, contradicts this assumption. Geering quotes that the ESSSA report in the subject English indicated that female testees surpassed male testees on all types of essay questions, whereas boys outperformed girls on multiple choice questions, with the exception of questions about people and personal relationships. This result again highlights that boys might be favored by the multiple choice format, even in disciplines like languages where they could be outperformed by girls.