Generalised linear mixed model (GLMMs) were used to analyse highly skewed and non- normal data62 (knowledge and social distance sub-scales) and another variable that had a binary response63 (recognition of disorder in the vignette64). The GLMM is best suited for analyses such as these because it combines the features of linear mixed models and generalised linear models (Bolker et al., 2009) to address highly skewed and non-normal outcome distributions (Diegelmann et al., 2018) and handle binary outcomes (Benedetti et al., 2014) as in the following outcomes and variables.
5.6.1 Knowledge
Considering the skewed nature of the knowledge sub-scale, several transformations including log and square root were applied. However, even after these transformations, the normality assumption was violated. Hence, GLMM analysis of this sub-scale was conducted to deal with the non-normal distribution. The sub-scale scores were dichotomised using a cut-off point of 5, with scores ≥ 5 represented as ‘1’ and < 5 as ‘0’ as the scoring method. Selection of the cut-off point was based on the median of the scores over the two time-points. The model predicts the probability of participants from each cluster group obtaining a good knowledge score at the two time-points. The results from the GLMM are presented in Table 5.6. The table shows that the model predicted similar probabilities and standard error for both groups at baseline, with the probability and standard error for the intervention group being 0.62 (SE=0.07) and for the control group 0.60 (SE=0.06). At follow-up, the model predicted a modest increase in the estimated probabilities for both groups, although the increase was greater in the intervention group (p=0.94, SE=0.03) compared with the control group (p=0.86, SE=0.05). The GLMM analysis carried out on the binary knowledge scores showed that the main effect for time was statistically significant, F(1, 21.70)=181, p=0.00 (Table 5.7); however, main effect for group was not significant, F(1, 1.26)=181, p=0.26. The interaction of group and time
62 Data are judged to be non-normal data when they do not produce the theoretical curve which is bell-shaped and symmetrical
when plotted.
63 A binary response implies only one of two possible responses.eg yes/no, true/false.
64 Participants were required to respond to an open-ended question asking them to identify the mental disorder of ‘Yaw’ a
was also not significant, F(1, 1.070)=181, p=0.30. The analyses detected that the odds ratio (OR) of an increase in knowledge scores for the intervention group was 2.23 times greater than for the control group (OR=2.23, CI: 0.48 to 10.27). The intra-cluster correlation for this sub-scale was 0.00, suggesting that there were significant variations in the responses from the clusters.
5.6.2 Social distance
The GLMM analysis of the social distance scale was conducted to address the non-normal distribution and high skewness of the data. The social distance scale scores were dichotomised using a cut-off point of 10, with scores ≥ 11 represented by ‘1’ and ≤ 10 as ‘0’. Selection of the cut-off point was considered after exploring the median of the scores over the two time-points. In this scale, the model predicted the probability that participants in each group would score 1 at baseline and follow-up. The results from the GLMM are presented in Table 5.6. The table shows that the model predicted different probabilities and standard errors for the groups at baseline, with the probability (standard error) for the intervention group being 0.61 (SE=0.06) and the control group 0.53 (SE=0.06). At follow-up, the model predicted similar rates of reduction in the probability for each group from their baseline estimates. The model predicted a 28% decrease to a probability of 0.33 for the intervention group (p=0.33, SE=0.06) and a 26% drop to probability of 0.37 for the control group (p=0.37, SE=0.07) (Table 5.6). In summary, the model predicted that both groups would demonstrate a reduction in their desire to distance themselves socially from people with mental disorders, however, the intervention group would demonstrate a greater reduction in desire for social distance than the control group.
The GLMM analysis conducted on the social distance scale indicated that there was statistically significant change over time for both groups, F(1, 10.99)=233, p≤0.00 (Table 5.7); however, the main effect for group was not significant, F(1, 0.122)=233, p=0.73. The interaction between group and time was also not significant, F(1, 0.817)=233, p=0.37. The odds ratio of an increase in the social distance scores of the intervention group was 0.6 times greater than for the control group (OR=0.618, CI: 0.216 to 1.766). The analysis also indicated an intra-cluster correlation of 0.00, suggesting the clusters were independent of each other.
5.6.3 Recognition of disorder in vignette
The recognition of disorder in a vignette variable was created as a dichotomous variable and responses were coded as correct or incorrect. A correct response was scored ‘1’ and an incorrect response was scored ‘0’. For this variable, the model predicted that 4% of participants in the control group (p=0.04, SE=0.03) and 9% in the intervention group (p=0.09, SE=0.03) would respond correctly to this variable (Table 5.6). At follow-up, the model predicted an increase in the estimates for both groups, although the increase was higher in the intervention group (p=0.32, SE=0.06) compared with the control group (p=0.14, SE=0.05).
A GLMM analysis was conducted on the recognition of disorder in a vignette variable and indicated that the main effect for time was statistically significant, F(1, 10.86)=231, p≤0.00 (Table 5.7); however, main effect for group was not significant, F(1, 3.74)=6, p=0.10. The interaction of group and time was also not significant, F(1, 0.16)=231, p=0.69. Overall, this suggests an increase in both groups’ ability to recognise depression over time. Intervention group scores increased more than the control group, but the difference was not large enough to be statistically significant. The observed proportions for the time-points per group compared with the predicted probabilities of the model indicated that the model was a good fit for the data. The odds ratio reported from the analysis suggests that the intervention group’s odds of correctly recognising depression was 1.4 times greater than the control group’s (OR=1.43, CI: 0.25 to 8.06). An intra- cluster coefficient of 0.02 was detected in the analysis, suggesting that there was a small amount of variation between the clusters and low correlation of responses between the clusters, and this difference was not large enough to be statistically significant.
Table 5.6. Group by time estimated means, standard error and confidence interval derived from the GLMM Outcome
measure Intervention Control
Baseline Follow-up Baseline Follow-up
X1 SE2 CI3 X SE CI X SE CI X SE CI
Knowledge 0.62 0.07 0.48–0.74 0.94 0.03 0.84–0.98 0.61 0.07 0.47–0.73 0.86 0.05 0.74–0.93
Social distance 0.61 0.06 0.49–0.72 0.34 0.06 0.23–0.48 0.53 0.06 0.40–0.64 0.37 0.07 0.25–0.51
Recognition 0.09 0.03 0.03–0.19 0.32 0.06 0.16–0.53 0.04 0.03 0.01–0.13 0.14 0.05 0.06–0.28
Table 5.7: Estimates of main effects of group, time and interaction of group by time for measures of knowledge, social distance and recognition derived from the GLMM
Outcome measure F ratio Numerator
df1 Denominator df P value Odds ratio ICC Knowledge Group effect 1.264 1 181 0.262 Time effect 21.697 1 181 0.000
Group by time effect 1.070 1 181 0.302 2.228 0.00
Social distance
Group effect 0.122 1 233 0.727
Time effect 10.986 1 233 0.001
Group by time effect 0.817 1 233 0.367 0.618 0.00
Recognition
Group effect 3.743 1 6 0.100
Time effect 10.860 1 231 0.001
Group by time effect 0.164 1 231 0.685 1.428 0.02
Legend: ¹df: degrees of freedom
5.7 Summary
Overall, 140 participants were recruited into the study and 27 did not take part in follow- up data collection. The socio-demographic characteristics of both groups were similar at baseline, and occupation was the only characteristic found to be a significant predictor of missingness in the data. The results showed an increase in the estimated mean scores of the outcome measures in the intervention and control groups at follow-up. The size of the increase differed between groups, with the intervention group demonstrating a larger increase than the control group in most outcome measures. However, apart from the CMHI sub-scale, the differences between the two groups in changes over the two time- points were not large enough to be statistically significant. The strength of the differences was generally small, apart from CMHI sub-scale, which had a medium effect size.
Chapter Six
Process Evaluation of the Mental Health Literacy Programme
6.1 Introduction
In this chapter, the findings of the mixed methods (quantitative and qualitative) process evaluation of the cluster randomised controlled trial of the mental health literacy programme (hereafter, the programme) are presented. The process evaluation was undertaken following the intervention to assess the intervention cluster participants’ perspectives about the usefulness of the programme. In this chapter, a summary of the participants is presented. This is followed by a descriptive summary of the results of the quantitative process evaluation. Then, a report of the themes that emerged from the thematic analysis of the interview transcripts is presented, supported with illustrative exemplars. Finally, suggestions to enhance the achievement of the programme objectives and improve the mental health literacy of the community are presented.