We conducted a process evaluation, the aims of which were to:
l assess delivery of the intervention to ensure it was provided in accordance with the protocol and delivered consistently
l establish the level of participant adherence to the intervention
l explore participants’views of, and satisfaction with, the intervention
l explore MIPs’and GFs’experiences of delivering the intervention
l test the logic model.
Intervention delivery
Motivational interviewing practitioners were asked to audio-record as many sessions (face to face and telephone) as possible, with a view to collecting a minimum sample of six sessions per MIP over the course of the study. A random sample of recordings from face-to-face sessions was assessed using the motivational interviewing treatment integrity (MITI) coding scale.120A stratified sample included sessions delivered in both
intervention arms and by all MIPs. Skill in MI delivery was assessed prior to study entry (via audio-recorded mock consultations with trained actors). In order to be recruited as a practitioner, individuals were required to reach the MITI proficiency threshold (seeChapter 6). We planned to observe group sessions to assess fidelity of intervention delivery.
Participant adherence
Data on intervention delivery and exposure to the intervention were examined. Overall attendance at intervention sessions and success of telephone contact were monitored (number of contact attempts and length of calls was recorded). MI CRFs from face-to-face and telephone sessions were examined for evidence of self-regulation, goal-setting and implementation intentions and analysed descriptively.
Participant self-regulation was examined by the frequency of self-weighing records/reports of self-weighing.
Participants’views of the intervention
Semistructured telephone interviews were carried out with participants in all arms, during the intervention (at approximately 6 months) and following the end of the intervention period (seeChapter 8). Participants were purposively sampled across key factors including trial arm, gender, age, recruitment route and attendance levels. In addition to assessing general views around intervention delivery, views on efficacy, barriers and facilitators, we also examined potential mediators not examined elsewhere such as the impact of life events on adherence; environmental influences; levels and importance of social support from family and friends as well as the professional support provided as part of the intervention; impact on their wider social network; WLM challenges; intrinsic and extrinsic motivations for WLM; health value; body image; and strategies, coping mechanisms and responses to relapses (seeAppendix 5). In addition, we planned to interview a small sample of participants who dropped out of the intervention to establish their reasons for discontinuing. Interviews continued until themes were saturated. Interviews were audio-recorded, transcribed and checked by the researcher.
Intervention staff’s views of the intervention
All MIPs, apart from one who took part in an interview, contributed to two focus groups designed to enable us to elicit their views of the intervention, the perceived challenges or barriers in implementing it and how they thought the intervention and training could be improved. Separate focus groups with GFs were also planned. The MIP focus groups and interview took place following recruitment closure. At this point the majority of MIPs had completed their face-to-face sessions and were mid-way through their telephone sessions. A focus group guide was developed to explore the MIPs’experiences. This comprised 18 items and spanned four main topics (seeAppendix 6). Up to three members of the research team were present during the focus groups, one of whom facilitated the session, and the interview was conducted by one member of the research team. The focus groups lasted around 2 hours, while the interview lasted 1 hour.
Qualitative analysis
All interviews and focus groups were audio-recorded and transcribed for analysis. Transcripts were checked and uploaded into QSR International NVivo10 software version 10 (QSR International, VIC, Australia) and analysed using thematic analysis. Thematic analysis is a systematic approach in which the data are initially coded and then collated into themes.121Themes are then analysed in more detail to map out the overall
data and examine relationships between them. Finally, themes are refined to produce an overall story of participants’views and experiences.121,122As outlined by Braun and Clark121thematic analysis consists of
five defining phases: (1) familiarisation, (2) initial coding, (3) creation of themes, (4) reviewing themes and (5) defining and naming themes. Our analysis followed each of these phases and is described in relation to these below. Data collection and analysis of interview data were conducted simultaneously and the analyses informed data collection in terms of changes to the interview schedule, for example adding new questions to probe particular areas of interest. Data collection continued until data saturation was reached. Analysis of the focus group data was conducted after data collection, but followed the same process.
Phase 1: familiarisation
During the familiarisation phase, a coding scheme was devised by the team to inform the coding
procedures. Separate schemes were developed for the participant interview and MIP focus group (plus one interview) analysis, as they had differing aims and different questions were asked accordingly. Therefore, it was not appropriate to analyse all data across a common coding scheme. The coding schemes were developed by initially reading and rereading a few of the early transcripts. Codes were mapped out by three researchers (for the participant analysis: SS, YM and CS; for the MIP analysis: SS, YM and LC) during two face-to-face meetings (following independent coding of two interviews/focus groups) and these initial codes were pulled together into a coding scheme. Following development of the initial coding scheme, it was then applied to other interviews/focus groups independently by two coders. Each of the schemes was then further refined as new codes emerged from the data during the initial coding procedure.
Phase 2: initial coding
Following the coding scheme development, the initial coding of participant interviews was completed by CS and YM, while the focus group coding was completed by YM and LC. Transcripts were closely examined and indexed according to the coding scheme. During the coding phase, the coders had regular discussions concerning any new codes identified and sought common agreement on any further amendments needed to the scheme. This ensured consistency between coders. An analysis log was kept to record coders’ discussions and track any changes made to the coding scheme as well as to record further thoughts and developments of the analysis process. A total of 10% of the interview transcripts were double coded to ensure reliability of the coding scheme and any discrepancies discussed and resolved. Both MIP focus groups and interviews were read and reread by YM and LC in preparation for the scheme development and were therefore not double coded.
Phase 3: searching for themes
Once all transcripts were coded, codes were examined in detail and broken down further into subcodes where appropriate. Moreover, some extracts were recoded where necessary. This phase was both data driven (i.e. bottom up) and theory driven as we sought to answer specific process evaluation questions and look at mediators not measured elsewhere. Commonly expressed themes, as well as unusual cases, were identified. Relationships between codes were explored by carrying out queries to identify text common to two (or more) related codes. This analysis led to the development of overarching themes.
Phase 4: reviewing themes
The themes developed in the previous phase were reviewed by the coders, who remained in close contact to further discuss and develop their thoughts. During this phase, coders referred to the key questions identified in the process evaluation model to support the final analysis. The initial codes were examined for any instances that were inconsistent with the emerging themes. The themes were further examined across groups (arm/gender/age) to explore their differences and similarities. A thematic map was developed for both the participant interview and practitioner focus group analysis to further inform this phase.
Phase 5: defining and naming themes
During the final phase, coders examined themes in detail ensuring they accurately represented the data and captured the overall story. Themes were mapped out in order to explore broader links and
relationships within the overall data and analysis. As themes were explored, a detailed narrative was written for each, which provided the structure for the final reporting of results. The qualitative findings then fed into the overall process evaluation.