Methodological issues and limitations

The possibility of spontaneous recovery needs to be taken into consideration when evaluating the effects of rehabilitation on people with traumatic brain injury and stroke.

Differentiating the effects of spontaneous recovery to those of the actual intervention is

not an easy task, particularly when patients are at a relatively early stage post-injury (less than 2 years). One way to control for spontaneous recovery would be to incorporate time since injury in the inclusion criteria. This is problematic, however, because the rate of recovery is multi-faceted and a consensus has not been reached regarding when such recovery slows down considerably or stops (Kennedy & Turkstra, 2006). The use of multi-aetiology groups further complicates things. According to Nair (2007), the randomisation process in the ReMind trial could possibly control for this effect, creating groups comparable regarding the levels of spontaneous recovery.

Describing participants’ basic demographic and psychometric characteristics is considered to allow future readers to evaluate the relevance of this group of patients to their sample of interest. However, it has to be noted that what is more important to qualitative researchers is not the comparability of the demographic characteristics but the comparability of the topic or the problem that is of concern (e.g. Morse, 1999). People with neurological disabilities represent a heterogeneous population. Therefore, controlling for all the variables that may affect the outcome of neurological rehabilitation is a difficult task which becomes more challenging when attempting to trace the factors that affected the subjective perceptions of this outcome. Qualitative methodologies do not permit the detection of cause and effect relationships and therefore evaluating whether a relationship existed between participant characteristics and the identified themes was out of the scope of this study. Future studies following quantitative designs could shed light to the complex interaction between personal characteristics and effects on participants’ perceived memory function, control beliefs and use of memory aids.

The constant comparison analytic process allowed the identification of similarities and differences between the three programmes and highlighted benefits specific to the two intervention programmes. It has to be noted, however that these qualitative comparisons cannot provide information on the significance of the observed differences. Rather than evaluating which programme was more effective, this study was interested in the patient-perceived effects of all the three programmes. This is important as the three programmes correspond to integral components of the memory rehabilitation interventions that are usually offered in clinical practice. As this study did not set to do quantitative comparisons between the groups, it is suggested that the observed variation of patients’ characteristics between the three programmes does not undermine the value of the findings. Instead, it may reflect the heterogeneity of the groups in clinical practice and allow capturing a range of different viewpoints.

Although not a common practice in qualitative research, in this study it was considered meaningful to present a detailed account of the number of people across the three programmes who experienced benefits in relation to each theme. Counting themes via frequencies and percentages helped in identifying patterns across the datasets and highlighted the differences between the three programmes. Although this can provide an idea of the prevalence of each theme, caution is needed when interpreting these frequencies. Given the small sample sizes, numerical findings can give a false impression of precision where none exists. As noted by Padget (2012), reporting the frequency of a theme can imply that a denominator exists when it does not. For example, to state that a percentage of participants mentioned getting more or less confident following the programme would only be accurate if every respondent commented on that topic. Even when using a structured interview schedule, a rate calculated from such a small number of individuals needs to be interpreted cautiously, without taking out of context (Padget, 2012). It is important not to read the volume of responses in each theme as a hierarchy but rather as an insight into the range of different ways in which rehabilitation affected participants. As the aim of qualitative methodology is to look at the whole picture, a point that is mentioned only once can still have great relevance and conceptual importance (Joffe & Yardley, 2004). For this reason, the present study attempted to consider variation and contradiction in the results by incorporating key isolated statements.

Methodological issues related to the design of the interviews need also to be considered. One of the limitations of the interview schedule was that it did not incorporate instructions for the interviewers to follow in case contingencies occurred during interviewing (e.g. steps to be taken if respondents provide incomplete responses or misinterpret questions). At the same time, like most structured schedules, there was little room for the interviewer to improvise or make own judgments in order to respond to the difficulties. The phrasing of the questions was such that it sometimes elicited one word or ambiguous answers. There was inadequate encouragement from the interviewer for further clarification of vague statements or elaboration of brief comments. Unfortunately there was no way to confirm the meaning and the genuiness of these statements other than looking at them within the context of the entire interview and comparing them with other occasions when the interviewee talked about the topic. If the meaning could not be verified the statement was excluded from the analysis. This may have led to the exclusion of potentially informative data, undermining the completeness of the findings.

Another limitation was that some of the questions were leading. For example, asking participants what were the effects of the groups on his/hers personal life, was in a way implying that the groups did actually have an effect on the respondent’s life. As a result, researchers may have imposed their own assumptions about the programme whereas respondents may have replied in a way that would please the interviewer. One of the items in the schedule included two questions at once, which may have caused more confusion to participants. Participants were asked about the effects of rehabilitation on their confidence and their anxiety about disclosing their problems in one question. An association was inferred between the two concepts whereas, as shown from the responses, each of them had a different meaning to participants. Moreover, the use of technical vocabulary might have led to ambiguity and misinterpretation of the questions.

Some participants found it difficult to understand terms like “problem-solving ability”,

“insight” and “assertiveness”. Some participants directly expressed their confusion and asked for clarification. Others, however, provided “yes” or “no” answers which were not followed up by the interviewer. Finally, the transition from one question to another could have been smoother by having general questions preceding more specific ones. Sensitive questions about mood were placed very early in the interview probably not allowing participants the time to relax and feel comfortable with the interviewer.

Although this was an exploratory study the structured approach did not allow the interviewer the flexibility to follow up interesting points and explore unanticipated themes that emerged from the responses. Therefore, there was little room for discoveries that were beyond the assumptions of the developer of the schedule. These methodological limitations may have significantly compromised the validity of the results. In order to confirm the findings there is a need to repeat the study using an approach that would elicit more descriptive responses and communicate the participants’ experience in all its rich detail. The characteristics and difficulties of the brain injured people also need to be taken into consideration. A more relaxed style of interviewing that would fit the respondents’

comprehension and would allow them to tell their stories at their own pace would probably be more appropriate for this specific population. The study needs to be replicated employing semi structured interviews, in order to promote conversational communication and deeper exploration of patients’ experiences.

Overall, important gains from memory rehabilitation were reported in the interviews, which were not reflected in the assessment tools used in the trial and other outcome

measures currently available. The findings of this exploratory study highlighted the need for an outcome measure sensitive to the effects of memory rehabilitation for neurologically