This editorial introduces a new special series on the pilot and feasibility testing of patient- reported outcome measures (PROMs) in the on-line open access journal Pilot and Feasibility Studies. Pilot and feasibility studies are typically implemented to address issues of uncertainty before undertaking a larger definitive study such as a randomised controlled trial or large scale survey. This editorial considers the role that such pilot and feasibility testing plays in relation to the development, evaluation and implementation of PROMs. This is often an essential element of PROM research but is typically overlooked – especially within current methodological guidance, reporting space and also debate. This editorial aims to open up a dialogue about the role of pilot and feasibility testing in relations to PROMs. It highlights some of the areas in PROMs research where these types of studies have been carried out and discusses the ways in which the PROMs community may be better supported and encouraged to integrate this element of the research process into their PROMs based work.
12 Read more
High-quality pilot and feasibility trials have the potential to advance the future success of clinical trials in the physical activity realm. They form the foundation for exceptional grant proposals to secure funds for large trials, as they can increase reviewer confidence regarding the team’s capacity to do the work. Further, they can still contribute to academic productivity if they are published. Although pilot or feasibility studies are informative, they are not often published due to emphasis on efficacy outcomes or they lack clear feasibility objectives and do not specify criteria for success (Table 2) . In addition, there are misconceptions regarding what is considered a pilot study. For example, studies performed with limited funding, in a short time frame, or with restricted resources, are not reasons a study should be called a pilot . Given the complex nature of physical activity interventions, pilot and feasibility studies in this field should be more abun- dant. Grant review panels and journals, however, will not recognize the value of pilot studies without rigor in design, conduct, and reporting. Accordingly, the Consolidated Standards of Reporting Trials (CONSORT) group has now created extensions specific to the reporting of pilot and feasibility trials  and of non-pharmacological interven- tions . Adhering to the CONSORT 26-item checklist can improve reporting and can also help in designing and evaluating future trials . Journals are increasingly becoming receptive to high quality pilot trials, and the journal Pilot and Feasibility Studies is an ideal destination.
As the number of submissions to Pilot and Feasibility Studies increases, there is a need for good quality reporting guidelines to help researchers tailor their reports in a way that is consistent and helpful to other readers. The publication in 2016 of the CONSORT extension to pilot and feasibility trials filled a much-needed gap, but there still remains some uncertainty as to how to report pilot and feasibility studies that are not randomised. This editorial aims to provide some general guidance on how to report the most common types of non-randomised pilot and feasibility studies that are submitted to the journal. We recommend using the CONSORT extension to pilot and feasibility trials as the main reference document — it includes detailed elaboration and explanation of each item, and in most cases, simple adaptation, or non-use of items that are not applicable, will suffice. Several checklists found on the Equator website may provide helpful supplementary guidance, when used alongside the CONSORT extension, and we give some examples.
On the basis that it is normally unwise simply to ignore information, trialists may wish to calculate and examine estimates of treatment effect from pilot or feasibility studies, to gain some informal reassurance (or not) of what might be expected in the main trial. However, if the size of the pilot or feasibility study has not been for- mally calculated to provide an appropriate level of preci- sion, these estimates are based on relatively meagre evidence and are therefore unreliable, and may result in inappropriate decisions either to proceed or not to proceed to a main trial. Moreover, it is not clear how they should most appropriately be interpreted. Such esti- mates should not therefore play a part in any formal decision-making regarding progression to a main trial, unless perhaps combined with other prior robust infor- mation, such as in a Bayesian decision model [23, 30]. Moreover, treatment effects calculated from pilot studies should not be among the parameter estimates used in sample size calculations for a main trial, as the targeted treatment effect is predominantly a matter of judgement,
Ninety-three individuals, including chief investigators, statisticians, trial managers, clinicians, research assistants and a funder, participated in the first round of the Delphi survey and 79 in the second round. Over 70% of participants in the first round felt that our definitions, the way we had distinguished between pilot and feasibility studies, and the labels ‘ pilot ’ and ‘ feasibility ’ were appropriate. However, these four items had some of the lowest appropriateness ratings in the survey and there were a large number of comments both in direct response to our four sur- vey items related to appropriateness of definitions, and in open comment boxes elsewhere in the survey. Some of these comments are presented in Fig 4. Some participants commented favourably on the definitions we had drawn up (quote 1) but others were confused by them (quote 2). Several compared our definitions to the NIHR definitions pointing out the differ- ences (quote 3) and suggesting this might make it particularly difficult for the research commu- nity to understand our definitions (quote 4). Some expressed their own views about the definitions (quote 5); largely these tallied with the NIHR definitions. Others noted that both the concept of feasibility and the word itself were often used in relation to studies which inves- tigators referred to as pilot studies (quote 6). Others questioned whether it was practically and/ or theoretically possible to make a distinction between pilot and feasibility studies (quote 6, quote 7), suggesting that the two terms are not mutually exclusive and that feasibility was more of an umbrella term for studies conducted prior to the main trial. Some participants felt that, using our definitions, feasibility studies would be less structured and more variable and there- fore their quality would be less appropriately assessed via a checklist (quote 8). These responses regarding definitions mirrored what we had found in the user-testing of the Delphi survey, the Society for Clinical Trials workshop, and differences of opinion already apparent in the litera- ture. In the second round of the survey there were few comments about definitions.
22 Read more
Stand-alone qualitative studies that are reported separately from the pilot trial, such as Hoddinott et al. and Schoultz et al. [22, 23] should follow appropriate reporting guidelines [24–26] and should provide link references to other pilot work carried out in preparation for the same definitive trial. When qualitative work is reported within the primary report of a pilot trial , it is not always possible to put sufficient detail into the methods section of the re- port to comply with reporting guidelines for qualitative studies. If this is the case, we recommend an online supplement or appendix to report the methods in detail. O’Cathain et al., Hoddinott et al., and Schoultz et al. have provided guidelines and examples for conducting qualitative feasibility studies alongside pilot trials [22, 23, 26, 28, 29]. Adapting the CONSORT statement for pilot trials The development of this CONSORT extension for pilot trials is described briefly here and in detail elsewhere . Before developing the checklist for this extension, the research team agreed on the definitions of pilot and feasibility studies. This was done by initially considering pilot and feasibility studies to be discrete types of study and therefore in need of separate checklists. However, preliminary work concluded that pilot and feasibility studies could not be defined in a mutually exclusive way, compatible with current understanding and the use of these terms among the research community. We there- fore adopted an overarching definition of feasibility stud- ies, with pilot studies being a subset, and developed a
32 Read more
procedure seven years later with a rate of unsuitable papers (30%) similar to ours of 33%. Arnold et al  initi- ally searched MEDLINE for pilot trials in critical care medicine, but then canvassed known clinical investigators because of the poor indexing of pilot trials. They describe five pilot trials fulfilling the requirements of either addressing methodological issues relating to the feasibil- ity of subsequent trials, or assessing mechanisms of inter- vention. Reviews based on published papers do not address internal pilots which continue into the main phase and are unlikely to be reported separately, though where a decision is taken not to continue the pilot phase could be written up. We chose to include only full papers in our study: different issues may arise amongst pilot or feasibility trials that are published as letters or brief reports. Sampling from all journals covered by MEDLINE and EMBASE and restricted to trials incorporating ran- dom allocation, the characteristics of the pilot/feasibility trials we found is not unexpected. The majority bear little resemblance to the recent definitions proposed for pilot/ feasibility studies.
11 Read more
trolled trials. Their search included both 'pilot' and 'feasi- bility' studies as keywords. They reported certain recommendations: having clear objectives in a pilot study, inappropriateness of mixing pilot data with main research study, using mainly descriptive statistics obtained and caution regarding the use of hypothesis testing for conclusions. Arnold et al  recently reviewed pilot studies particularly related to critical care medicine by searching the literature from 1997 to 2007. They pro- vided narrative descriptions of some pilot papers particu- larly those describing critical care medicine procedures. They pointed out that few pilot trials later evolved into subsequent published major trials. They made useful dis- tinctions between: pilot work which is any background research to inform a future study, a pilot study which has specific hypotheses, objectives and methodology and a pilot trial which is a stand-alone pilot study and includes a randomization procedure. They excluded feasibility studies from their consideration.
Feedback from participants in the pilot phase Be- tween January and September 2014, a total of 15 partici- pants gave feedback on their experience of attending pilot sessions at focus groups facilitated by an experi- enced and independent health psychologist (Table 3). The final two sessions of cohort 2 had no attendees, and so feedback was not taken for that cohort. Feedback was, however, obtained from the booster session pilot. Data derived from these meetings were analysed using principles of the constant comparative approach based on grounded theory . A pragmatic approach was taken to analyse data as it had been obtained only for the purposes of improving the intervention, rather than exploring other themes, such as the effect on partici- pants, which would be part of data gathering in the RCT. The main themes were as follows:
17 Read more
Learning mentors will be brought together at one of the schools (or an appropriate local site) for this train- ing. Such outreach training was found to be the most cost-effective implementation strategy for alcohol screening and brief intervention delivery in other set- tings . Training for learning mentors will be carried out by an experienced trainer using a simulated subject scenario within a training package developed and employed in other studies of brief interventions [25,37,38]. Learning mentors will be provided with sup- port materials and on-going support and supervision on implementing screening; paperwork relevant to the re- search will be provided by the research team, who will also act as the site study coordinator. Learning mentors will record all time spent on the project using a case diary, which will be used as part of the economic ana- lysis. Research staff and trainers will maintain regular contact with schools throughout the study period, in- cluding site visits and telephone support.
10 Read more
influencers in the community to facilitate dissemination and impact . The CWG process adopts a collaborative approach to PPI, which is defined by the NIHR (p. 22) as ‘ working more closely with members of the public, returning to ask them for further information, and de- veloping an ongoing relationship with them ’ . This entails a two-way flow of knowledge, where patient ad- visory groups will be updated routinely about pilot inter- vention delivery and asked to make recommendations for intervention refinements to be fed directly into CWG decision-making. To operationalise this process, patient advisory groups will mirror the CWG discussion process by working through a matrix prepared by the CWG Chair in collaboration with the patient advisory group Chair. The matrix will summarise the issues aris- ing in intervention delivery derived from interviews with facilitators and participants in the process evaluation (what) and will seek the patient advisory groups ’ expert opinions on the issues arising (so what) and their recom- mendations for how the CWG refine the intervention (now what). The recommendations of the patient advis- ory group will then provide a key knowledge source for the CWG meetings.
10 Read more
The management of emotional talk in patients with breast cancer during their early weekly review appointments with their TR predicts eventual FCR level some 6-8 weeks post discharge from primary treatment. Additional training to increase awareness of these features on the consultations of these patients may be of benefit. Studies need to be conducted to examine the generalisability of these findings and pinpoint whether there are particular emotional issues that may warrant close attention to prevent the development of disabling FCR levels. A new study protocol to develop a training programme for TRs has been reported.
24 Read more
control group. As this is a small-scale feasibility and pilot study, the researcher collecting the data will also deliver the intervention. Therefore, it is not possible for the researcher to be blind from the treatment group. Another member of the research team (the PI) will randomise participants and write the treatment allocation for each participant on a piece of paper. This will be placed in a sealed envelope, only to be opened by the researcher immediately before the intervention/control group visit. Consent/assent will be sought at two stages: firstly, for permission from the carer to be contacted by the researcher to arrange appointments and, secondly, written consent/assent to opt into participate in each aspect of the study (physical activity intervention component and interview component). This will be sought at the study visits, once the researcher has discussed the study in detail with participants. Written information sheets will be given to participants (young people and parents/ carers). The researcher will go over the information sheet with participants at the first visit to ensure that participants understand the study protocol and what is being asked of them.
11 Read more
Methods: Participants aged 12 – 18 years with JIA were recruited during their routine rheumatology clinic visits at a tertiary care hospital. Participants completed the 3-Day Physical Activity Recall self-reported questionnaire at baseline, 1 week and 5 week follow-up. At the 1 week follow up, participants were instructed to start wearing an activity tracker for 28 consecutive days. Participants completed a feasibility questionnaire at their end of study visit. Participant demographics, adherence rates and feasibility outcomes were summarized using descriptive statistics. The effect of wearing a tracker on moderate-to-vigorous physical activity (MVPA) and total metabolic equivalents (METs) per day were analyzed using a paired t-test.
Methods and analysis This protocol provides a description of a non-randomised feasibility study of an infant feeding intervention and implementation strategy, with an embedded process evaluation and economic evaluation. Intervention participants will be parents of infants aged ≤6 weeks at recruitment, attending a participating HCP in a primary care practice. The intervention will be delivered at the infant’s 2, 4, 6, 12 and 13 month vaccination visits and involves brief verbal infant feeding messages and additional resources, including a leaflet, magnet, infant bib and sign-posting to an information website. The implementation strategy encompasses a local opinion leader, HCP training delivered prior to intervention delivery, electronic delivery prompts and additional resources, including a training manual, poster and support from the research team. An embedded mixed-methods process evaluation will examine the acceptability and feasibility of the intervention, the implementation strategy and study processes including data collection. Qualitative interviews will explore parent and HCP experiences and perspectives of delivery and receipt of the intervention and implementation strategy. Self-report surveys will examine fidelity of delivery and receipt, and acceptability, suitability and comprehensiveness of the intervention, implementation strategy and study processes. Data from electronic delivery prompts will also be collected to examine implementation of the intervention. A cost–outcome description will be conducted to measure costs of the intervention and the implementation strategy.
15 Read more
The home-based interviews formed the second major component of fieldwork. As noted, the first contact with the family was the initial letter to all Wave 1 pilot families from the Study Team, to remind them of their previous participation in the pilot study when the child was 9 years of age and to ask them to provide details on the school the Study Child was now attending . This initial approach was followed up a few weeks later with an introductory letter to all families about the home-based component of fieldwork, including a further copy of the information leaflet. This approach was made to all families in the main pilot – the 45 per cent who had already returned details on the school being currently attended by the Study Child and the 55 per cent who had not. An interviewer then made a personal visit to each family to explain Growing Up in Ireland and to arrange an interview. At that initial visit, interviewers asked to speak to the person listed as the Study Child’s Primary Caregiver at the time of the Wave 1 interview. The interviewer reminded the parent/guardian about the letter and information leaflet which they had already received in the post, and answered any queries they may have had about the study. The interviewer asked the parent/guardian and Study Child/Young Person to sign two copies of the parent consent form (see Appendix A8) and young person’s assent form respectively (Appendix A10). The interviewer, Primary Caregiver and Study Child each kept one copy of the relevant Consent/Assent form. Only after securing signed consent/assent did the interviewer conduct interviews, tests or measurements.
122 Read more
Further research is now needed to assess the effective- ness and cost-effectiveness of the REACH-HF interven- tion. A multi-site, fully powered randomised trial (ISRCTN86234930) in patients with reduced ejection fraction heart failure  and their caregiver and a single-centre pilot randomised trial in patients with HFPEF (ISRCTN78539530) and their caregivers are cur- rently in progress. This includes a mixed methods process evaluation to assess mechanisms of change, based on testing key elements of the logic model in Fig. 2. Beyond this, further research might include (a) adaptation and evaluation of the intervention into a digital format, with more emphasis on remotely deliv- ered or online support; (b) implementation research about how the REACH-HF programme (if effective) could be integrated with existing health service models/ infrastructure; (c) assessment of the impact on effective- ness of using (more resource-intensive) formative feed- back to enhance the training of facilitators; (d) the impact of such interventions on longer term outcomes such as mortality and hospital admissions.
18 Read more
Large-scale cohort studies increasingly collect biological samples which, combined with survey data, might pro- vide valuable insights into the biological mechanisms of disease [1–4]. Storage of samples in large long-term bioarchives has become common, with the aim of using both current and anticipated future molecular technolo- gies to analyse the samples [5, 6]. Reductions in costs of molecular testing, in particular for genomics, proteomics and metabolomics applications, have made such ap- proaches attractive to public health researchers [2, 7]. However, collecting such samples poses multiple chal- lenges, including how to ensure that the proportion of participants providing samples is sufficiently high, and practical challenges associated with working in clinical settings where medical considerations often take priority over research [8, 9].
Methods: A randomised controlled pilot feasibility study, including embedded qualitative and economic components will be carried out. Two randomly selected nursing homes will take part in the study; participants ( n = 20) in one nursing home will receive a three-month PA intervention and participants (n = 20) in the other will be a usual care control. Nursing home staff will be provided with training and support to monitor participants PA programmes. Feasibility data will be collected on recruitment, randomisation, assessment and intervention procedures. Criteria for progression of the pilot feasibility study to a definitive trial will be specified. The Timed Up and Go test, Nottingham Health Profile, 10-Metre Walk test, the Investigating Choice Experiments for the Preferences of Older People Capability index and the Bangor Goal Setting Interview will be assessed at baseline, three-month and 12-month follow-up. Between-group and within-individual effects will be estimated using appropriate linear mixed models. Semi-structured interviews will be conducted with staff and participants of the intervention group within one month after the
13 Read more
Thanks to Katie Denman (coding), Sarah Davidson and Ian Porter (data collection), Adam Qureshi (regression analysis), Gemma Lough (editing), and Tim Kirkpatrick for calculating the subsequent pilot and definitive trial retention rates. Thank you also to all of those who gave of their time and experiences to be interviewed, Her Majesty ’ s Prison Service and National Offender Management Services, and the support and advice of various health and criminal justice agencies. This research was supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) South West Peninsula at the Royal Devon and Exeter National Health Service (NHS) Foundation Trust. The views and opinions expressed in this paper are those of the authors and not necessarily those of the NHS, the NIHR, or the Department of Health.
11 Read more