Despite the overall positive results of this study, it is important to acknowledge its limitations. The results are qualified by methodological limitations that included a relatively small sample size of 50 and smaller subgroups yet. This reduced the power of statistical tests. This study used samples drawn from national branches of Schizophrenia Fellowship and Richmond Fellowship, with approximately one fourth of those initially contacted participating in the study. While there were many reasons for this, the main reason was that both a key family member and client needed to be involved. In many cases, clients were eager to participate but a family member was ill or not willing, or a family member was willing but the client was not on regular psychiatric medications. Many other families who also expressed interest could not participate because of overseas holidays, family deaths, illnesses, hospitalisations and shifting cities. It was also found that in the larger cities of Auckland, Wellington and Christchurch, family members were less interested in participating in an education programme than those from smaller cities such as Timaru, Palmerston North and Blenheim who often appeared to be eager for more information and support.
Family member reticence in participating in the intervention is in line with previous studies with community samples where it has been found that only up to 50% of family members take the opportunity of community support or family intervention (Barrowclough et al., 1999; Mueser & Jeste, 2008). The author relied upon the field-workers and key-workers of the different branches of S.F. and R.F to promote the intervention, the possible benefits that may be obtained from participating, and to recruit participants. The differing levels of motivation of staff from branch to branch may also have affected the number of participants.
All participants voluntarily participated in the study which suggests a commitment to learning more about the illness, and likely reflected increased levels of intrinsic motivation by virtue of their willingness to participate. All factors which could have led to a biased sample group and likely had an impact on the outcomes, limiting the ability to generalise the findings to all family members with a relative with schizophrenia. It is possible that a different outcome would have resulted from family members who were less enthusiastic about the process (Cassidy et al., 2001; Leff et al., 1989; Mullen et al., 2002). While use of the randomized controlled design was used for these reasons to increase confidence that the findings were not simply due to increased motivation, the ethical imperative of voluntary participation doesn’t erase concerns here. Thus, it is possible that the clients and family members who were contacted but did not participate in the programme would not have gained in knowledge about schizophrenia and coping or decreased in EE, burden of care and distress. Future research can help shed light on this issue.
As to why EE decreased in this brief intervention and a previous brief intervention (Moxon & Ronan, 2008) but not others, it is worth highlighting some possibilities. Firstly, the self- report measure used to assess EE should be discussed. Most previous studies (Cole & Kazarian, 1992; Gerlsma & Hale, 1997; Tarrier et al., 1988; Van Humbeeck et al., 2002) have used the CFI as the current optimal EE assessment measure and while the LEE Scale has been found to correlate well with the CFI, findings based on methodological differences cannot be excluded. In the present study, the LEE scale was preferred to the CFI in part because it is a quick, non- intrusive and easy to administer and measure with documented reliability and validity. Conversely, the CFI is a lengthy and involved interview in which the scoring process is difficult and extensive, something which would not easily integrate with the nature of community setting the intervention was conducted through.
However, it is possible to respond in a socially desirable manner on the LEE and create an illusion of low-EE by responding ‘False’ to items such as “I don’t blame him/her when he/she is not well”, and ‘True’ to “I am tolerant with him/her even when he/she doesn’t meet my expectations”. It is possible that the EE score that some people received did not accurately reflect their EE status. Most participants reported struggling with having to respond with a definitive True/False on some questions and recommended that improvements could be made by introducing a scale with a Likert type rating scale instead of using mere True/False responses.
It is possible though that family members and clients were accurate in their self-report ratings, but the accuracy of these self-report ratings cannot be verified from the data in the present study. Additionally, while there are difficulties with reliance on the singular use of a self-report measure to assess EE, the present study utilised using both the client and family member versions of the LEE to obtain a more robust rating. Additionally, while there are problems with the use of a self-report measure, confidence in findings is increased through the use of a multi-method, multiple informant approach, the use of independent assessors, and the randomized design. Nevertheless, other possibilities accounting for change include repeated testing and regression to the mean cannot be completely ruled out. However, other studies have not found such effects (Tarrier et al., 1988).
Two final methodological limitations which might have influenced the outcome of this study are that no chart diagnosis was obtained to confirm the diagnosis and the absence of stringently applied treatment fidelity measures makes it difficult to judge quality control within the present study. Fidelity could have been assessed more comprehensively through taping all education sessions and/or having participants complete a checklist post intervention. Having said that, selected sessions of this intervention were checked by a senior clinical psychologist to ensure that the education was delivered as intended. In addition, an outline that included some flexibly scripted elements (see Method) were carried out in every application of the educational intervention. Additionally, the addition of structured materials, notably, the take home booklet, guaranteed additional standardisation. Nevertheless, future research would benefit from increased attention to the direct measurement of fidelity.
In summary, this study has a number of limitations that restrict the interpretation of its
findings. There were methodological shortcomings and the sample was small and not
representative of all families affected by schizophrenia. The results may reflect a sampling bias. However as Pollio et al. reported in their 2006 study,“biases associated with self-selection to participate in educational workshops are inherent to these programmes as these programmes are not for everyone” (p.36).