Design considerations

Prior to conducting a large-scale definitive RCT to establish the effectiveness of a novel intervention it is imperative to conduct smaller scale studies to determine the feasibility of conducting a larger scale trial as well the acceptability of both the trial and the intervention (Craig et al., 2008). Problems which may beset trials such as lack of eligible participants, recruitment difficulties and poor retention can be anticipated and procedures to ameliorate these issues (Craig et al., 2008). Indeed, Clinical Trial Unit Directors identified that research into methods to boost recruitment in trials was of the highest priority followed by methods to minimise attrition (Tudur Smith, Hickey, Clarke, Blazeby, & Williamson, 2014).

The terms pilot and feasibility study are often used interchangeably. The National Institute for Health Research (NIHR) state that ‘Feasibility Studies are pieces of research done before a main study in order to answer the question “Can this study be done?”. They are used to estimate important parameters that are needed to design the main study.’ (NIHR).

However, the NIHR also state that pilot studies test whether components of a definitive trial can all work together (NIHR). The current study therefore does incorporate elements of both the definitions of feasibility and pilot studies, however the parameters investigated aim to address areas of uncertainty. The parameters reflect the recommendations for feasibility studies from NIHR and are; number of eligible patients in the population, recruitment and retention rates, acceptability of the trial, intervention and outcome measures to patients and staff. In addition the study aims to gather information on outcome data, in terms of standard deviations to inform a sample size calculation for a larger trial (see 6.2 above) (NIHR).. NIHR also advise that ‘the usual sort of power calculation is not normally undertaken. Instead the sample size should be adequate to estimate the critical parameters (e.g. recruitment rate) to the necessary degree of precision’ (NIHR). It is important in the conduct of a preliminary study to cautiously interpret any outcomes in terms of statistical significance due to limited sample size (Thabane et al., 2010).

It is important to define criteria for success of feasibility. In this study the following criteria are used;

 To estimate that it would be possible to recruit sufficient participants for a full trial within a reasonable period expected for conducting a definitive trial (12 months).

 Retention rates should be comparable with published trials within this setting.

 Protocols should be acceptable to patients and staff with no undue burden placed upon them.

 The intervention should be acceptable to patients and have the potential to have a positive impact on patients.

6.3.1.2 Pragmatic design considerations

To ensure that feasibility evaluation is based on the population and setting that a definitive trial would take place in, a number of pragmatic considerations were taken into account.

Although a highly controlled, narrowly designed RCT has good internal validity, it can result in an intervention which is detached from clinical practice and the actual patient group which the intervention aims to target, therefore reducing its external validity (Hotopf, 2002; J. Green, 2006). The following aspects of this study relate to pragmatic design considerations;

 Participant selection – exclusion criteria kept to a minimum i.e. inclusion of comorbid physical conditions, not just newly diagnosed participants, both those who have been admitted for manic and depressive episodes and participants who have experienced psychosis.

 Blinding and allocation concealment – It was impossible to blind participants to their allocated group in this study, as ethical requirements specified that participants were informed of the two study arms and after randomisation were informed of their allocation. However, measures were taken to try to ensure that members of the study team assessing outcomes remained blind to allocation. Minimisation by the researcher who was not involved in recruitment ensured that recruitment to the study remained independent from treatment allocation processes. Participants were requested at follow-up, not to disclose their treatment allocation until after the questionnaires had been completed. Questionnaire outcome measures also serve to reduce the possibility of observer bias.

6.3.1.3 Selection of a Control group

With a pragmatic trial it is important to compare a new treatment with what is currently practiced as it offers the potential to answer the question of whether the treatment gives benefit over what patients currently receive (Freedland et al., 2011; J. Green, 2006). It is difficult to determine whether any effect observed is due to the intervention or to other

non-specific effects such as therapist time. However, the content and attention participants received are both components of the intervention and a TAU comparison is appropriate.

6.3.1.4 Unit of randomisation: Individual vs Cluster RCTs

In designing an RCT the unit of randomisation must be decided upon, whether this is at the level of the individual or at the level of the site or unit in which the intervention is delivered. A summary of these considerations in relation to the IBiD intervention is presented in Table 6.1.

When interventions are delivered in settings such as hospital sites, consideration must be given as to whether there is any possibility of clustering effects i.e. will those in each cluster be independent of one another. When designing a trial where randomisation occurs at the level of clusters, a large number of clusters are required as participants within one cluster are not independent of one another (intra-cluster correlation) and this must be taken into account in the computation of statistic power (Elley, Kerse, & Chondros, 2004). A number of steps were taken to ensure that any cross-contamination between participants allocated to TAU was minimised. Specifically, the intervention resource was not provided to staff during the course of the study so could not be shared with other patients. It was ensured as much as possible that the intervention was conducted as close to the point of discharge as possible so the resource would not be shared between patients.

6.3.1.5 Process evaluation

In trials, consideration must be given to variables which might act to either ‘mediate’ i.e.

change or occur during treatment and have a significant effect on the study outcome or to

‘moderate’ i.e. exist at baseline and interacts with the intervention to have an effect on the outcome (Kraemer, Wilson, Fairburn, & Agras, 2002).

Exploring process variables helps to systematically describe aspects of the intervention which occur during delivery of the intervention and provide a description of what constitutes treatment as usual which may not be otherwise systematically recorded. Identifying potential moderating factors provides information for future definitive trials in this setting, helping to either targeting the intervention or for in stratification for randomisation. Identifying potential mediators could lead to the development of the intervention to maximise change in particular variable (J. Green, 2006). The feasibility study cannot statistically investigate these factors influence on outcomes due to not being powered to investigate these variables. However, data can be obtained on the variance of potential mediators and the feasibility of collecting data on these.

Table 6.1: Advantages and Disadvantages of Parallel group and Cluster research designs for IBiD Design Explanation Advantages Disadvantages Possible solutions Parallel within wards may share the written resource. may not be representative of patient group due to material but would not have received the tailored

Cluster Wards or sites randomised

A lack of independence leads to a loss of power.

To achieve the equivalent power of a parallel groups the sample size needs to be inflated.

Longer fieldwork period needed to achieve required sample.

Ethical issues of consent at individual level.

Potential of individual and whole-cluster drop-outs.

Sample size needs to be inflated by a factor

The ICC is estimated from previous research.