Construct Validation

Assessing construct validation through Rasch analysis provided a number of interpretative data outcomes, including infit and outfit values to demonstrate fit to the model, bubble maps to depict item accuracy, and variable maps to demonstrate item difficulty. Mean infit and outfit scores measure the alignment, or fit, of the survey results to the Rasch modelled expectations. Mean square values (MNSQ) between 0.6 and 1.4 are generally deemed to be acceptable for Likert-scale survey items (Bond & Fox, 2007), with standardised values (ZSTD) between ±2. The scores for items and persons all fell within these required ranges (see Table 4.2). Participant statistics were also included because the degree to which they also fit the model provides further evidence that the survey behaved as intended (Wright & Masters, 1982). Participants provided a variety of answers to the different survey items; nonetheless mean person fit was acceptable. This indicated that participants responded to the items in consistent and expected ways.

Table 4.2

Mean Infit and Outfit Scores for Items and Persons

MEAN (logits)

INFIT OUTFIT

MNSQ ZSTD MNSQ ZSTD

ITEMS 0 1.11 0 1.14 0.1

80 Item infit was further illustrated in the Rasch bubble chart (Figure 4.3) and Item infit and outfit scores table (Table 4.4). The bubble chart shows most items falling within the required ZSTD range of ±2 (Bond & Fox, 2007). Within this chart, each

numbered bubble represents an item in the survey. The size of each bubble is reflective of the precision of the item in measuring the construct of gender related challenges for male primary teachers. Smaller bubbles such as those for the contact, workload and isolation items indicate more precise measures. As the larger size of some bubbles is reflective of the small number of participants in this study; further use of this instrument with increased participant numbers might improve the precision of item measurement.

81 Table 4.4

Item Infit and Outfit Scores

ITEM MEASURE MODLSE IN.MSQ IN.ZSTD OUT.MSQ OUT.ZSTD

Contact -1.45 0.18 1.31 1.6 1.26 1.4 Workload -1.31 0.18 0.8 -1.1 0.77 -1.2 Isolation -1.24 0.18 0.8 -1.2 0.76 -1.3 Media -0.56 0.19 0.57 -2.7 0.54 -2.5 Salary 0.02 0.21 1.47 2.1 1.63 2.1 Role Model 0.2 0.21 0.89 -0.5 0.84 -0.5 Sexuality 0.9 0.26 0.84 -0.6 1.53 1.3 Friends 1.12 0.28 1.03 0.2 1.01 0.2 Family 2.31 0.46 2.24 2.1 1.92 1.2

The salary, family and media items were just outside the accepted ±2 ZSTD range. The precision variance of the salary and family items are within range, but they indicated a slight under fit to the model. This under fit was probably caused by a small number of participants who chose an unexpected answer for these items. The media item was the only challenge that was out of the required range. This item indicated an overfit showing that it was marginally too easy for the model to predict. The majority of participants chose similar answers for this item, which is likely to have caused this overfit. Conforming too precisely to the Rasch model is not a threat to construct validity, as this item is still measuring the same construct as other items, it is just measuring it more rigorously.

82 Rasch analysis allows for the recalculation of reliability based on the removal of items. A Cronbach’s alpha (α) coefficient was recalculated excluding each of these

three items in turn; without the salary item (α=0.83), without the family item

(α=0.83), and without the media (α=0.77). None of these omissions strengthened our original alpha coefficient (α=0.82) enough to warrant the removal of any of these items from the survey.

Figure 4.3 (bubble map) also shows the relative difficulty of the items in logits, which is the unit of Rasch measurement. Logit values are distributed about a mean of 0 logits. The data presented in Figure 4.3 appears to be upside down because it uses a scale of difficulty, and the term ‘challenge’ is generally perceived as a negative emotion. Therefore, the major gender related challenges (physical contact, workload and isolation) appear towards the bottom of the scale (negative value) because it was easier for participants to agree that they were actual challenges. The family and friends items are located at the top of the scale because participants found it more difficult to agree that these items were actual challenges. This rating was likely caused by the fact that participants perceived these items as less difficult and easier to cope with.

Based on this quantitative analysis, this study specifically focused on the major gender related challenges of physical contact, workload and isolation. These challenges are represented by smaller bubbles, indicating that they are the more precise items. Further evidence for focussing on these three challenges can be found in another useful output derived from Rasch analysis; a variable map (Figure 4.5). Figure 4.5 demonstrates the relationship between participants, items and the construct of gender related challenges, and their alignment on a single measurement scale. This

83 variable map provides a clearer representation of item difficulty and similar to the logit chart in figure 4.3, items at the top of the scale are those which respondents found difficult to rate as challenges, whereas those at the lower end of the scale were the easiest for participants to rate as challenges.

Figure 4.5 shows that there were distinct ‘steps’ between items or groups of items. The items involving fear and uncertainty surrounding physical contact, increased workload, and social isolation are grouped together at the bottom of the scale. This indicates that participants found it easiest to agree that these items were real challenges for them. As these challenges were the most difficult for participants to deal with they became a key focus of the interview stage of this study.

84

Figure 4.5. Variable map showing participants and items.

The analysis described above indicated that this survey instrument had high internal reliability and construct validity. Through each of the Rasch analysis procedures it was evident that the three challenges of physical contact, increased workload, and social isolation were the most important items measured in the survey. Thus, these challenges were the key focus for the semi-structured interview stage to follow. Before potential prompt questions for these interviews are developed, a descriptive analysis of all items will be presented. All items were examined though

Logits Persons Items

3 + I I I I Family I 2 + I I I I I Friends 1 + I Sexuality X I X I I I Role Model 0 + Salary XX I XXXX I I Media XXXXX I XXXXX I -1 XXXX + X I Isolation I Workload XXXXX I Contact XX I I -2 XXXXXXXXXX + I XXX I I XXX I I -3 XXX + X I X I I I XXX I -4 XX +

85 the demographic variables included in the survey. This analysis enabled a more

layered approach to designing the questions regarding these three prominent gender related challenges.