Qualitative analysis

Qualitative analysis supported findings from study 1 (Chapter 4) and the quantitative analysis reported here. A reliable and simple coding framework showed that there were more positive comments and fewer negative comments about the gist leaflet than ‘The Facts’ booklet. ‘The Facts’ booklet was considered to be too long and as noted in study 1, there were problems in understanding the numerical presentation of screening outcomes. Respondents noted that the gist leaflet contained less complex terminology and would make a useful addition to the existing screening information. The comprehensibility findings provided further evidence that the gist leaflet met the comprehensibility stage of the Garner framework.

7.4.5 Strengths and limitations

A strength of the research was the randomised controlled design, which is considered the gold standard method in most contexts for evaluating public health interventions (Sackett, Rosenberg, Gray, Haynes, & Richardson, 1996). The validity of the study’s findings ere increased through the use of multiple recruitment centres, as single centre studies have been shown to inflate the effect of the intervention in both pharmacological and non-pharmacological randomised controlled trials (Bafeta et al., 2012; Dechartres, Boutron, Trinquart, Charles, & Ravaud, 2011). This study design enabled me to observe the impact of the gist leaflet against a group provided with the existing information materials. Although this may have limited the effect of the intervention, it gave strength to claims that the benefits seen in terms of comprehension were real added effects.

The study was somewhat successful in recruiting low SES participants compared with population estimates of unemployment, ethnicity, education and US estimates of numeracy (Ciampa et al., 2010). The quantitative data in this study allowed the views and opinions of all participants to be weighted equally, overcoming a limitation of study 1 and 2 where highly educated participants were seen to disproportionately contribute to the study findings. The collection of qualitative data that supported quantitative findings should also be noted as a strength.

Although the representation of low SES groups was greater than the previous studies, people living in deprived neighbourhoods were still less likely to respond to the study invitation. The ascertainment of the intervention’s effect as therefore

recorded in an unrepresentative sub-sample of the eligible population. This may go some way to explaining why there was a ceiling effect observed in the intention and knowledge outcomes. This finding highlights the need to observe the effect of the intervention among those who did not consent to participate in questionnaire-based research.

It should also be noted that the study took place in GP practices that were willing to take part in research. Centres such as these have been shown to be different to practices who do not participate in trials with regard to general achievement scores1 and composition of ethnic minorities in the area (Down et al., 2009). Ascertainment of the study effect among members of the public who are not registered with practices familiar with research is therefore needed.

Despite using methods to increase response, rates of return were lower than expected. For example in a UK-based randomised trial of a CRC risk communication intervention, 60% of participants returned a completed questionnaire (Robb et al., 2008). Questionnaire length is unlikely to explain response differences between the studies as they were of similar length. However, the communication materials used by Robb and colleagues may have induced less cognitive burden. In their study, participants were randomised to three study groups: a control group (no information); a ‘risk factors’ group (leaflet about the risk factors for CRC and incidence of the disease); and a ‘risk factors + screening information’ group (risk factors leaflet + 120 words about CRC screening tests). Both information leaflets achieved a Flesch readability score that as superior to ‘The Facts’ leaflet (Robb et al., = 75.5 vs. ‘The Facts’ leaflet = 62.4). Although the gist leaflet was superior to all of the materials (84.5), the need to read ‘The Facts’ leaflet may have reduced response rates in this study.

An additional factor that may explain differences in participation between the two studies is the deprivation scores noted at the recruitment sites. The present study purposively sampled from three practices that were based in areas with high levels of neighbourhood deprivation. Robb and colleagues did not report IMD data, prohibiting direct comparison. However, observing the locations of the two studies it is fair to assume that the General Practices reported here were based in more deprived neighbourhoods.

1

The recruitment of individuals who had not previously been screened was considered both a strength and limitation of the study. On the positive side participants were not biased by previous exposure and past behaviour. However, participants were younger than those who would normally be invited to participate in the NHS BCSP and the decision was anticipated rather than current. This forced me to rely on a proxy marker of behaviour (i.e. screening intention). Although the factors involved in intention and screening behaviour are similar, they may not always overlap (Power et al., 2008; Schwarzer, 2001; Shah, 2005). Further investigation with objective screening uptake data may be warranted.

A further issue with using participants who were naïve to the CRC screening process in this study was that they may have responded to the questionnaire differently to those with more experience. Vernon and colleagues investigated this topic by assessing whether responses to measures of barriers, benefits, self- efficacy and optimism were different according to level of experience with CRC screening (never screened, overdue for screening and currently screened) (Murphy et al., 2013). They showed that although most items and factors were similar, the never screened group responded differently to the barriers scale than the currently screened group. The screening categories may not be directly comparable to those reported here. However, it suggests that questionnaire responses may not always represent people who are of screening age. It may have been interesting to recruit a sample that had previously been screened so that this limitation could have been investigated further.

Finally, the majority of outcomes reported in this study were from valid scales. However some items (e.g. readability and usability) and scales (e.g. gist knowledge), were designed for the purposes of this study. The gist knowledge items were true or false items that were not difficult to answer. This may explain why a ceiling effect was observed for this outcome. However, the items were purposely designed so that they could assess the essential information needed to make a screening decision. Further research using samples of different ages, deprivation levels and ethnicities are required to confirm the reliability and validity of these measures.

7.4.6 Conclusion

In conclusion, this study did not support hypotheses related to the communicative effectiveness of the gist leaflet. The final stage of the Garner framework was therefore not met. The second stage (comprehensibility) of the evaluation was confirmed following positive effects for perceived readability, perceived usefulness and gist knowledge. Improvements to comprehensibility outcomes were particularly apparent among low numeracy individuals. The provision of two information materials may decrease the likelihood that they will be read, although the gist leaflet appeared to be favoured when people were given the option.

155

Chapter 8. Evaluating the communicative effectiveness of a