Check items for differential item functioning

Differential item functioning (DIF) is another important potential source of misfit in the data. Items are classed as showing DIF when different groups of respondents (for example male and female) within the same sample respond differently to an individual item (Pallant and

125 Tennant 2007). This can mean that an item is biased towards one particular group of people. Although there is no consensus regarding the best way to measure DIF, running an analysis of variance is the most common way (Streiner and Norman 2008), and also the method used by RUMM2030. Two types of DIF exist; uniform and non uniform DIF. Uniform DIF is seen when each group of respondents shows a consistent systematic difference in their responses to an item, across the whole range of the attribute being measured (Teresi et al. 2000), identified by a significant analysis of variance test. Non uniform DIF is seen when differences in groups vary across levels of an attribute, and there is non-uniformity in the differences between groups (Tennant and Conaghan 2007). Uniform DIF can cancel out in a measure, for example if there is one item biased more towards females and one towards males in the same measure (Tennant and Pallant 2006a). If the uniform DIF does not cancel out, the problem item(s) can be split, for example by gender, and scored separately(Lundgren- Nilsson et al. 2005). Non-uniform DIF is more difficult to deal with, and items sometimes have to be rewritten or removed from the scale (Pallant and Tennant 2007). However, as emphasised by Streiner and Norman (2008), the impact and significance of the DIF must also be considered in relation to the concept being measured; it is important to consider whether the presence of DIF in the measure is problematic, and it is important to look further than the statistics.

The presence of DIF in the FROM was identified using one way analysis of varianc e (ANOVA) carried out in RUMM2030. The respondents are split into roughly equal class intervals depending upon their FROM score. RUMM2030 produces tables for the ANOVA tests for each item, and these are examined for significant differences (p<0.05). Item characteristic curves were also produced for each item using RUMM2030, to show the effects of DIF graphically. The different groups analysed for the FROM were age and gender. Relationship to patient and medical specialty were also considered, but the numbers in these groups were so small that no reliable conclusion could be made. It was also decided that the presence of DIF by specialty was not problematic, and could be expected; family members of patients from different specialties would be expected to answer differently and be biased towards certain questions. For example, family members of patients from gynaecology or urology may score more highly on an item regarding their sex life due to the nature of the patient’s symptoms. DIF by relationship to patient was also not seen as problematic; closer relations to patients (e.g. spouses) may score more highly on some items than more distant relatives. DIF becomes more of a problem when using Rasch analysis for measurement in education or in validating disability scores where bias would be problematic.

126

Table 5.8: The local dependency identified within the FROM and potential solutions Items involved Potential solution

Items 1, 2, 4 and 5

These four items are all measuring a similar attribute (emotions), so some local dependency could be expected. However, all of these items show good fit to the model. The endorsement level for all of the items was high, so all should be retained.

Items 7 and 15 The two items are measuring similar attributes, and this was noted by the family members during this stage of recruitment. The two items were given poor feedback by family members, and so will have to be reworded if one is retained. Item 7 is more highly endorsed so should be kept over item 15.

Items 12 and 14

These two items have similar wording, and only one should be retained. Item 12 is worded more clearly, as several family members felt that, as adults, they no longer have “hobbies”. The wording of item 14 fits well with the theme of the FROM.

Items 14 and 16 Removed item 14 Items 14 and 18 Removed item 14 Items 16 and 18

It is difficult to see from a conceptual point of view why these two items are showing correlation as they measure two very different things. Therefore retain both.

Items 12 and 23

Although it can be seen from a conceptual point of view why there is a correlation between these two items (both involve doing things together as a family), the theme of holidays was highly prevalent during the interview stage, and is clearly very important to family members so both items should be retained. Items 14 and 30 Removed item 14 Items 20 and 23

Both items emerged as important concepts during the interview stage of the study and are measuring different things, so both should be retained. Items 6, 25 and

26

These three items are all measuring similar concepts so do not need to all be retained. Item 26 is the most highly endorsed, and items 5 and 26 measure two overlapping concepts. These items need to be investigated further.

DIF was identified in twelve of the FROM items (Table 5.9). The item characteristic curves were then examined to identify which group the item was biased towards when uniform DIF was shown. Figure 5.7 shows an example of uniform DIF by age seen in item 14, and Figure 5.8 shows an example of non-uniform DIF by gender in item 15, where the lines on the graph can clearly be seen to be crossing, demonstrating non-uniformity.