Inferential and descriptive statistical analysis.

As stated in Chapter 3, in this section I consider biological sex groups because that is how the data is presented. However, in interpreting the findings I am aware that they represent overall rather than individual effects. All of this study’s populations contain a substantial majority of boys (Table 4.7) in line with entry patterns for Triple Award GCSE Science (Murphy and Whitelegg, p. 41,

Table 4.7 Number and Percentage of Boys and Girls in the Study’s WJEC Populations. Population 1993 1994 1995(03) 1995(02) n % n % n % n % Boys 368 62.1 446 64.3 231 59.7 362 59.5 Girls 220 37.9 251 35.7 156 40.3 247 40.5

Willingham and Cole (p. 98,1997,) note that the relative number of boys and girls taking an examination can be an important consideration in describing and understanding ‘gender’ difference and similarity in selected groups but that ‘the nature o f the selected group depends partly on the character o f the selection and that ordinarily there is little precise information about

that process ’. Here, it is important to note that the sex sub-groups in my populations are in a sense ‘restricted samples ’ (ibid.). My populations consist of boys and girls who have chosen to study Triple Award Science GCSE. That choice will have been influenced by many factors, for example the personal wishes of the students, the advice of their teachers, and parental pressure. All of the boys and girls in my populations have been entered for the same tier of examinations but entry decisions will again have been influenced by numerous factors. There is little precise information about the selection processes for students taking Triple Award Science GCSE courses and then their GCSE tier entry, in accord with Willingham and Cole’s quotation above. These ‘selection’ processes and their consequences for comparability are an intended focus when I engage with teachers in the qualitative dimension of my research. In this Chapter my intention is first to investigate the boys’ and girls’ performances, acknowledging that I do not know what might be the effects of sample restriction on those performances. Statistically, as Willingham and Cole note and found, if there is a selected group (restricted sample) then there would be an expected difference in the standard mean difference between boys and girls in favour of the minority sample. In each of my populations girls form the ‘minority’ group, with between 38 and 40 per cent of each WJEC population in my study being composed of girls. Willingham and Cole would expect the girl’s sub-

group in my populations to perform better than that of the boys. I take their proposition into account when interpreting my analyses.

The only other published study of girls’ and boys’ performances at GCSE in Wales focuses on 1992 to 1997 data and so includes the years of my quantitative data (1993 — 1995 inclusive). This study was commissioned by the Qualifications, Curriculum and Assessment Authority for Wales (ACCAC) and published in 1999. It draws together data from across the whole of Wales covering attainment in national assessments from age 7 to 19. The statistical analysis is based on statutory assessment and examination results, namely Key Stages 1,2 and 3, GCSE and ‘A’ level. It focuses on the ‘gap’ in achievement between boys and girls at school for each subject and for each attainment and grade level. The study defines an achievement gap as the difference between the performances of boys and girls, taking account of the patterns in entry. This approach is adopted for fulfilling the study’s primary focus which is on changes in entry and achievement across time. Unlike my research the ACCAC study does not take any account o f tier entry in its examination populations and their GCSE results and bases its calculations on whole examination entries for specific GCSE subjects. Only achievement gap calculations are used and so, for example, there are no correlation, kappa or inferential statistical analyses to compare with those of my research. The study also only presents its analyses on sex differences for science as a whole, claiming that there is a small achievement gap in favour of boys at the higher grades and that

‘gender is not a clear problem in science ’ (p. 30, 1999). These claims subsume any differences in sex achievements for specific awards, for example Triple Award and Double Award and little is said about sex differences in the different science disciplines within each type of award. There are general statements that there is an achievement gap in favour of girls in biology and no

achievement gap in chemistry and physics but no statistical analyses results are presented to substantiate these statements or indicate the extent of the gap. My study attempts to control for tier entry and deals only with biology, chemistry and physics in GCSE Triple Award. Comparing my results with those of this ACCAC publication are therefore of limited use but reference is made where relevant.

My sub-groups’ achieved GCSE grades were first investigated with inferential statistics. The results were then explored with descriptive statistics. Cross-tabulations of sex with each of the variables biology, chemistry, physics, English and mathematics grades in the form of frequencies for each grade category are illustrated in bar charts (Figure 4.5a-d). This approach was repeated with each of my populations. This analysis was based on percentages of each sub-group’s population rather than the population as a whole (see Chapter 3) to increase the validity of my comparisons. This method is seen by Gorard, Salisbury and Rees (p. 7, 1999) in their ACCAC study as appropriate for identifying the presence or absence of patterns in differential achievement, although they advocate their ‘achievement gap’ calculations for determining differential sex achievements across time. My main concern is not with comparing achievements across the three years of my study to identify precise relative percentage changes but with patterns in sex

differences within each year of my data and their statistical significance. As discussed in Chapter 3 there are many influences on examination performance including many which emanate from the nature of the examination population itself. For this reason alone I question the validity of using precise percentage ‘achievement gap’ changes for reporting on boys’ and girls’ GCSE performance changes across time. Thus I looked for similarities and differences across the years but not for specific percentage changes as required by the achievement gap calculations (Gorard et ah, 1999). Furthermore, unlike the achievement gap method, my chosen calculation method is used widely for reporting on sex, social class and ethnic group differences in achievement (Robinson and

Oppenheim, 1998; Bright, 1998; Bentley, 1998; Gillbom and Gipps, 1996; Murphy and Whitelegg, 2006) and so allows comparison of my results with those of other researchers.

Using the language of examining groups, my results indicate there were no significant differences between the 1993 boys’ and girls’ achieved biology, chemistry and mathematics grades. However, the boys performed significantly (P — 0.008) better than the girls in physics and

proportionally more boys than girls achieved grades A and B (see Figure 4.5). In chemistry, the degree of positive skewness is more marked for both boys and girls than in the other two sciences, normal distribution being lost. This concurs with chemistry being overall the least severely graded or

Figure 4.5a Distribution of Boys’ and Girls’ Grades - 1993, WJEC Block of colour = Boys; Hatched colour = Girls

Biology Chemistry A B C D E F G U NRG Physics 35 0 c 30 a> o 1 1 25 = ! 20 s % 15 c 2 10 3 QJ 5 £ u- o- n 1 i FI i

i

1

^ B K I_____ NRG English a> o 4 0

J f e

- ____________ ____ C D E F G U NRG M athem atics O C Q) O 05 % 2 1 ° §■ O) Q_ O) re ® c « 0) c 50 40 30 20 10 0 “ 1 H --- __ n i - - U NRG

the science subject where students overall achieved relatively more highly in the three science subjects for the 1993 population. Girls performed significantly (P = 0.001) better than the boys in English and in their average GCSE grade scores. The greatest disparity in the proportion of achieved English grades occurs for grade A, with 38.5 per cent of the girls’ sub-group achieving grade A compared with 27.8 per cent of the boys’ sub-group.

Figure 4.5 shows that the distributions of mathematics grades for the boys’ and girls’ sub­ groups are similarly positively skewed, with a slight tendency for the boys to proportionally outperform the girls at grade B and this situation being reversed for grades A and C. There is no statistically significant difference between the girls’ and boys’ mathematics grades. From these results, it would appear that there is no sex effect between achieved GCSE mathematics and physics grades. Separate correlation studies of the mathematics and physics grades achieved by the boys and the girls in the 1993 population were carried out to explore further the significant

differences between the sub-groups’ physics grades. Both the boys’ (0.68) and the girls’ (0.62) Spearman correlation coefficient values for mathematics and physics grades are significantly (P=0.001) positive. One would therefore expect for both sexes high achievement in mathematics to be associated with high achievement in physics. The correlation coefficient values are sufficiently similar as not to offer any insights as to why the girls achieve significantly less well than the boys in physics. I interpret these technical findings as indicating population / subject interaction, in that something(s) within the physics examination and/or coursework evoked a reaction from individual students that taken collectively manifests itself as a sex sub-group effect. This interpretation challenges the notion of any of the sciences being described as more severely graded with the question ‘more severely graded for whom?’.

The inferential statistics suggest there are no significant differences in the 1994 boys’ and girls’ achieved physics grades. However, the boys achieved significantly (P = 0.008) better grades in mathematics than the girls. In turn, the girls achieved significantly (P = 0.001) better English grades and average GCSE grade scores, the latter showing an even greater difference than in 1993 (mean difference is 0.1 in 1993 and 0.27 in 1994). Technically this average GCSE grade score suggests the

Figure 4.5b Distribution of Boys’ and Girls’ Grades - 1994, WJEC Block of colour = Boys; Hatched colour = Girls

Biology U NRG C h e m istry §

in

. _ S i C D E U NRG P hysics ~ 40

, K , dBL

U NRG English c 50 o 5 40 _re 3 Q. O 20 a> S 3 •s a. 30 10 0 rS-- s s — s I r i J K J L i

EL cb

A* U NRG M athem atics I t L r—v— rd~

1 .

___

.

_____ ::

n r

D E GCSE Grade U NRG

1994 girl’s sub-group were more able at meeting the demands of their GCSE subjects than their 1993 counterpart with the technical approach’s assumption that grades will have common currency between subjects and across years. For me this is not necessarily so because of a multitude of possible influences. For example: the 1993 and 1994 examinations may have required differing skills and there could be differences in the girls sub-groups’ interactions with these; different subjects with their specific skill demands may have been taken by the two sub­ groups so that the same subjects are not being compared; marking may have been more lenient in one or more subjects in 1994, despite examining group’s arrangements that claim to stabilize this. Continuing with an examining group’s type of interpretation and within my chosen value of 5 per cent for significance, girls achieved significantly {P = 0.048) better in chemistry than the boys. There was no significant difference in achievement between the boys and girls in biology.

Biology has been identified in the subject pair method as the most severely graded science GCSE subject for the 1994 population of this study. It therefore appears that for both boys’ and girls’ sub-groups this was the case.

The significant differences in the 1994 boys and girls sub-groups’ achievements in

chemistry, English and mathematics were explored further by conducting correlation studies. Both the boys (0.54) and girls (0.52) sub-groups’ Spearman correlation coefficients for mathematics and chemistry grades are significantly (i^O.OOl) positive. Therefore, for both groups, high

achievement in mathematics is associated with high achievement in chemistry. The correlation coefficient values for the two groups are sufficiently similar to indicate that girls’ significant better performance in chemistry is not related to the relative performances in mathematics GCSE, which suggests to me that the underlying issues are more complex than just the mathematical skill demands of the examinations. The Spearman correlation coefficients were recalculated for the

1994 boys (0.52) and girls (0.48) sub-groups using the dataset that included all available

mathematics achieved grades (506 of the total 509 males; 283 of the total 283 females), regardless of examining session and examining group. The correlation coefficients differ by a similar relatively small amount (0.04) to that obtained from the incomplete mathematics dataset (0.02) providing no further insights to explain the differential sub-groups’ achievements in chemistry in terms of awarded GCSE mathematics’ grades. As noted earlier in the discussion about physics, the

tier of mathematics examination paper for which the boys and girls are entered might be one contributing factor.

In discussing the relationships between the 1994 boys and girls sub-groups’ chemistry achievements, it again seemed useful to explore the interplay between the boys’ and girls’ chemistry and English achieved grades. Consequently, Spearman correlation coefficients for the separate sub- groups' English and chemistry grades were calculated. The English grades achieved by girls correlate less positively (0.24) with their achieved chemistry grades than is the case for the boys (0.43). One way of interpreting the correlation coefficients is to say that for the 1994

population, high achievement in chemistry is more likely to be associated with high achievement in English for the boys’ than the girls’ sub-group. Nor is it valid, on the basis of the evidence

available at this point, to claim that the girls’ sub-group achieve well in chemistry because they are high achievers in English GCSE. Nevertheless, despite the girls sub-group’s relatively higher achievements in both English and chemistry compared with those of the boys’ sub-group, the positive association of achievements in these two subjects appears to be less marked for the girls than the boys.

For the 1995(03) population there were no significant differences between the boys’ and girls’ achieved biology, chemistry and mathematics grades. Overall, boys achieved significantly (P = 0.007) better grades in physics than the girls. Conversely, the girls achieved a significantly (P = 0.000) better average GCSE grade score and significantly (P = 0.018) better English grades than the boys. There are no significant differences in the boys’ and girls’ achieved biology and mathematics grades for the 1995(02) population. Boys out-performed girls in physics but not to such a significant level (P = 0.08) as their 1995(03) counterparts. Overall, the 1995(02) girls’ sub­ group achieved significantly (P = 0.042) better than the boys’ sub-group in chemistry and even more significantly (P = 0.000) in their average GCSE grade scores. The girls’ sub-group also performed better than the boys in English with the significance becoming greater from 1995(03) (P = 0.018) to 1995(02) (P = 0.000).

Correlation studies were conducted to explore further the statistically significant differences in the boys and girls sub-groups’ achievements in physics and English for Tier 03 and in physics, chemistry and English for Tier 02 of the 1995 examination session. The correlation coefficients for

Figure 4.5c D istribution of Boys’ and G irls’ G rades - 1995(03), W JE C Block of colour = Boys; H atched colour = Girls

C h e m istry A* A B C D NRG P hysics English

I

. M athem atics c _Q) O TO S I *♦- zz ° o D) <U 0) c I s I 60 50 40 30 20 10 0 C D NRG A* A B Biology GCSE Grade

Figure 4.5d D istribution of Boys’ and G irls’ G rad es - 1995(02), W JE C Block o f colour = Boys; H atched colour = G irls

Biology B C D E F G U NRG C h e m istry U NRG P hysics U NRG English E E L : U NRG M athem atics o c Q) O

11

1 ° S’ « c « <D C 50 40 30 20 10 0

i n

A U NRG GCSE Grade

English and physics (boys = 0.24; girls = 0.36) achieved grades are significantly (P = 0.001) positive. Therefore, high achievement in physics is associated with high achievement in English for both sub­ populations. However, the correlation values lie in the ‘weak’ correlation range (Coolican, 1994), with that for the girls being only slightly more positive than that for the boys. Using the language of examining groups, there is only a slight tendency for the girls’ high achievement in English to be more predictive of high achievement in physics than is the case for the boys. The values for both sub-groups are sufficiently similar so as not to offer an explanation as to why the male sub-group achieves

significantly better physics grades than their female counterpart. The same deductions apply for the physics: English correlation coefficient values for the 1995(02) male (0.28) and female (0.36) sub­ groups. Furthermore, the 1995(02) correlation values for the English : chemistry pairings are sufficiently similar (boys 0.28, girls 0.23) in their degree of positive correlation as to not provide insights into the differential performance of the two 1995(02) sub-groups. For the chemistry: physics pairings the values (boys 0.53, girls 0.60) indicate that both 1995(02) male and female sub-groups tend to have their high achievement in chemistry moderately (Coolican, 1994) positively associated with high achievement in physics, where this is slightly more so for girls than boys.

The ACCAC study showed that ‘girls gain more o f the higher attainment levels at Key Stages 1 to 4 in English ’ (p. 5, Gorard et al, 1999) but in the study’s 1999 publication no reference is made to significance levels. My findings also show girls significantly out-performing boys in English and gaining relatively more of the higher grades in all three years of my data. ACCAC also showed girls out-performing boys in biology. However, my results consistently show no statistically significant difference in the boys’ and girls’ performances in biology. Unlike ACCAC, I have found that boys significantly out-perform girls in physics. This applies for three out of four of my datasets and even in the dataset where this does not apply, boys gain more of the highest grade (A*) than girls. This finding is shown to hold in more recent studies. Data relating to examinations from 2000 to 2004 (Murphy and Whitelegg, 2006) shows that in England far fewer girls than boys are entered for physics in Triple Award and their performance relative to boys is lower across the pass grades. Indeed, overall the 2000 and 2001 GCSE results showed that physics (at Triple Award) was the only subject where boys

achieved a higher proportion of A*-C, the pass grades, than girls. Across 2000-2004 boys also gained proportionally more A* and A grades than girls in physics and mathematics. However, more boys than girls are entered for the higher tier paper in mathematics, which allows access to these two grades (Elwood and Murphy, 2002). Because of the restricted and selected sample it suggests that girls are not achieving as well in physics in relation to boys. This 2000 - 2004 performance trend is also shown in Wales where there is a similar though proportionally smaller discrepancy in entry between boys and girls and yet boys out-perform girls marginally (ibid). Scotland and Eire have different 16+ assessment systems to those in England and Wales. Nevertheless, entry to 16+ physics examinations in Scotland and Eire shows a similar gap in favour of boys to that in England and Wales. However, girls

outperform boys on the top grades, which could be expected for these restricted samples in contrast to girls in England and Wales (ibid.). Here it could be assumed that each smaller sample of girls is more highly selected and therefore more able and motivated persons predominate amongst the girls than the boys (Willingham and Cole, 1997).

Preece et al. ’s (1999) analysis of the 1996 Key Stage 3 science test results for a large representative sample of schools found that the largest gaps in performance occurred on questions assessing physics, where boys outperformed girls. When this study was repeated with Key Stage 3 2003 results with t-tests on the mean scores achieved by boys and girls, boys were again shown to significantly outperform girls on physics on both the lower and higher national science tests. The publication of Key Stage 3 test results by level and overall points achieved across papers and subject components masks these findings. In this respect the same reservation I have discussed earlier about sub-group effects being masked in an examination population’s GCSE grade distributions is raised. The international survey of students at grade 8 (approximately 14 years), Trends in International

In document Comparability and Examination Performance: Technical and Social Approaches to Its Study (Page 140-153)