O1
DSMB monitoring of the therapeutic hypothermia after pediatric cardiac arrest (THAPCA) trials
Richard Holubkov1, Amy Clark1, Andrew M. Atz2, David Glidden3, Beth S. Slomine5, James R. Christensen5, Angie Webster1, Kent Page1, J. Michael Dean1
1
University of Utah School of Medicine;2Medical University of South Carolina;3University of California at Los Angeles;4University of California San Francisco;5Kennedy-Krieger Institute
Correspondence:Richard Holubkov
Trials2017,18(Suppl 1):O1
The NIH-funded THAPCA randomized trials compared efficacy of therapeutic hypothermia (target temperature 33 degrees C) to normothermia (temperature 36.8 degrees) after cardiac arrest in children >48 hours to <18 years of age. One trial enrolled children who sustained out-of-hospital arrest (target n = 250 analysis-eligible), and a second trial enrolled children with unplanned in-hospital arrest (target n= 504 due to smaller expected hypothermia benefit). The primary effi- cacy outcome was survival with favorable neurological outcome 1 year post arrest. Due to key differences in arrest etiology and post-arrest outcomes, the parallel trials were monitored separately.
An NIH-appointed DSMB monitored safety and efficacy of both trials. A charter specified meetings approximately twice yearly. Symmetric O’Brien-Fleming boundaries were used for monitoring treatment superiority, with an informal <20% conditional power criterion specified for declaring futility.
Recruitment began in September 2009. The first safety-only DSMB review in 2010 combined adverse outcome data across trials as event numbers were small. Subsequent study-specific, safety-only DSMB reviews occurred until the first interim efficacy look for both trials in November 2012.
In the out-of-hospital trial, a treatment difference was noted among 141 children with 1-year outcomes available, but the treatment com- parison (nominal p-value = 0.03) did not approach the conservative stopping boundaries. Enrollment continued, with n = 250 target sur- passed at 2012 year-end. A second interim analysis in August 2013 (n= 213) found that the previously observed treatment difference di- minished (nominal p = 0.12). Results of the‘negative’out-of-hospital trial were published in 2015.
The in-hospital trial encountered enrollment difficulties due to below- expected patient volumes. During the November 2012 interim analysis, for which 74 children had one-year outcomes available, no treatment difference was noted. Due to the slow accrual, the DSMB skipped effi- cacy monitoring during the 2013 meeting. The second efficacy analysis, in March 2014 (n= 149), again did not approach statistical significance. By late 2014, despite extended enrollment and addition of centers, only 254 eligible patients had been enrolled, indicating final enrollment would be substantially below target. Given these circumstances, the DSMB elected to consider stopping for futility during their January 2015 meeting. Conditional power for the primary efficacy outcome was found to be very limited under a range of assumed treatment effects and realistic final enrollment numbers ranging from 300–400. However, the DSMB decided to continue enrollment and reconvene after add- itional information about potential utility of the trial was available. In a supplemental February 2015 meeting, the DSMB reviewed conditional power estimates for exploratory outcomes, some ad hoc with postu- lated treatment effects based on out-of-hospital trial trends, and subse- quently recommended stopping enrollment. In-hospital trial results are under journal review as of November 2016.
The THAPCA trials provide an interesting study of long-term DSMB review of two parallel studies with differing enrollment patterns. For example, the DSMB elected to remain masked to treatment arm identity in both trials throughout all reviews, recognized the mar- ginal benefit of repeated efficacy monitoring under slow enroll- ment, and considered implications of futility stopping beyond the primary trial outcome. Our experience may help optimize strategies for successful DSMB involvement in randomized trials with long- term follow-up.
O2
Review of pilot and feasibility studies from a registry website; an overview of recent practice
Nicola Totton, Andrew Brand, Rachel Evans, Zoë Hoare, Nia Goulden, Paul Brocklehurst
Bangor University
Correspondence:Nicola Totton
Trials2017,18(Suppl 1):O2 Background
Pilot and feasibility studies are increasing in popularity, as demon- strated by the launch of the focussed Journal of Pilot and Feasibility Studies in January 2015. Pilot studies are defined as a mini replica- tion of a proposed full study, used to identify any potential issues. A feasibility study is similar but considered as more exploratory and is used to judge the suitability of a proposed study by assessing the viability of each of the elements individually. In this review, we aim to assess the prevalence of pilot and feasibility studies in recent prac- tice and determine whether the two differ on key characteristics. We envisage this work will then guide future research into the use of pilot and feasibility studies to better inform full studies.
Methods
Data on pilot and feasibility studies have been collected from the International Standard Randomised Controlled Trial Number (ISRCTN) registry website. As the website does not provide a downloadable database, a web scraping methodology was adopted and imple- mented using the R programme. Pilot and feasibility studies were identified using a keyword search on the main title. Unique trial re- cords were extracted using regular expressions and collated in an Excel spreadsheet.
Results
Out of the 658 unique trials that were found within the ISRCTN data- base, 456 (69%) of these were labelled as pilot studies, 196 (30%) as feasibility studies and six (1%) considered both a pilot and feasibility study. The overall prevalence of all pilot or feasibility studies does not seem to be showing any particular trend over time. Initially, pilot studies were much more common; however, in recent years feasibil- ity studies have increased in popularity and are now on a par with pilot studies. The median number of target participants for each trial was 50 (range: 5–130,000). When considered separately, pilot studies had a target of 44 (range: 5–130,000) with feasibility studies being slightly larger with a target of 60 (range: 6–1,600).
On average, the trials ran for a period of 18 months with pilot studies reporting a slightly shorter time (17 months) than feasibility studies (18.5 months), which aligns with the larger sample size target. Mental and behavioural disorders accounted for the highest propor- tion of studies (20%), followed by cancer studies (13%).
The majority of the studies within the data were interventional (96%) and defined as randomised controlled trials (85%). A hospital envir- onment is the most likely setting for the studies (45%) and in most cases, the recruitment was in a single country (97%). This is consist- ent between the pilot and feasibility studies within the database. Conclusions
In recent years, the prevalence of feasibility studies has begun to match that of pilot studies but the characteristics found for the two are very similar. Due to their similar nature, there could be an argu- ment that pilot and feasibility studies should considered as one. Alternatively, if these studies are to be separate more education is required to outline the differences and provide guidelines for each to help shape future research projects.
O3
Using routinely recorded data in a clinical trial: the feasibility, agreement and additional benefits compared to standard prospective data collection methods
Graham Powell1, Laura Bonnett1, Catrin Tudur-Smith1, Dyfrig Hughes2, Tony Marson1, Paula Williamson1
1University of Liverpool;2Bangor University Correspondence:Graham Powell
Trials2017,18(Suppl 1):O3
There are a number of administrative datasets that routinely record information on individuals in the UK. Such routine or ‘Big Data’ sources record data for a specified primary purpose and are regu- lated for security, confidentiality and disclosure by The Data Protec- tion Act 1998 and Freedom of Information Act 2000. Access for secondary uses such as clinical research is permitted when health and social care benefit can be demonstrated. Routine sources include clinical datasets such as Hospital Episode Statistics (HES) and the Clinical Practice Research Datalink (CPRD) and non-clinical datasets such as data recorded by the Department of Work and Pensions (DWP), HM Revenue and Customs (HMRC) and The Driving and Vehicle Licensing Authority (DVLA).
Routinely recorded data used in clinical research is potentially cost and resource-use effective. Clinical sources of data such as HES and CPRD are commonly accessed to provide data for retrospective observational and record-linkage studies and provide a valid dataset for this purpose. Routinely recorded data may also present advan- tages to prospective clinical research such as randomised controlled trials (RCTs). For example, routinely recorded data have been used to inform judgements about the feasibility of sample size and recruit- ment targets and measure participant outcomes. Pragmatic cluster RCTs have even been conducted through routine data sources in- cluding patient recruitment, randomisation, administration of inter- vention and trial assessments, such as through The Clinical Practice Research Datalink (CPRD). The majority of RCTs incur health service costs as clinicians assess participants, record outcomes and complete Case Report Forms (CRFs) - hence using routinely recorded data may provide an efficient alternative method for data collection in addition to reducing the burden on participants. Furthermore, data from non- clinical routine sources may inform outcomes beyond the standard RCT assessments of clinical efficacy and effectiveness. For example, cost data (such as use of healthcare resources) and socio-economic data (such as employment and means-tested benefits data) may in- form health economic analyses and the assessment of the broader societal impact of healthcare interventions.
However, limitations with accuracy of coding, confidentiality, owner- ship and access have previously been identified as significant barriers to accessing routinely recorded data for prospective research. This study has assessed the feasibility of accessing both clinical and non-clinical routinely recorded data within the context of a RCT and will assess the agreement and additional benefits of routinely recorded data compared to data collected using standard prospective methods in a RCT.
O4
Using re-randomisation in clinical trials to increase patient recruitment
Brennan Kahan
Queen Mary University of London
Trials2017,18(Suppl 1):O4 Background
Patient recruitment is often a major challenge for randomised trials. Reviews of publicly funded UK trials have found that 45 to 69% fail to recruit to target. This increases costs, delays results, and adversely impacts on the feasibility of conducting trials for conditions where there is a limited patient pool.
For some conditions, patients may require treatment on multiple occa- sions. For example, patients with sickle cell disease require pain relief for each new pain crisis, and women using fertility treatment may
undergo multiple treatment cycles until they become pregnant. The current norm for trials in these conditions is for patients to be enrolled for only one treatment episode.
An alternative approach is the re-randomisation design. This design al- lows patients to be re-enrolled and re-randomised for each new treat- ment episode. The number of times each patient is re-randomised is determined by the number of treatment episode each patient ex- periences, rather than by the trial design. Because each patient can contribute multiple treatment episodes, this design can facilitate an increased recruitment rate, potentially making trials easier and quicker to conduct.
Methods
We describe some properties of the re-randomisation design, such as the conditions required to obtain unbiased estimates of the treatment effect and control type I error rate. We also evaluate the likely impact of re-randomisation on the recruitment rate is several clinical areas. Results
The re-randomisation design can provide unbiased estimates of treat- ment effect and control the type I error rate under some very simple conditions. Furthermore, in some instances this design will have the same or even higher power than a parallel group with an equivalent number of observations. Based on a modelling study across three dif- ferent clinical areas, we estimated that re-randomisation could reduce the time to complete recruitment between 19-45%.
Conclusions
The re-randomisation design can increase the recruitment rate compared to parallel group designs, which could reduce costs and make trials more feasible to conduct. If used appropriately, it can provide unbiased esti- mates of treatment effect, control the type I error rate, and maintain or even increase power compared to a parallel group design.
O5
Adaptive non-inferiority margins under observable non-constancy Brett Hanscom1, Deborah Donnell1, Brian Williamson2, James P Hughes2 1
Fred Hutchinson Cancer Research Center;2University of Washington Correspondence:Brett Hanscom
Trials2017,18(Suppl 1):O5 Background
A central assumption in the design and conduct of non-inferiority (NI) trials is that the active-control therapy will have the same degree of effectiveness in the planned non-inferiority trial as it had in the prior placebo-controlled trials used to define the non-inferiority mar- gin. This is referred to as the‘constancy’assumption. If the constancy assumption fails, the chosen non-inferiority margin is not valid and the study runs the risk of either approving an inferior product or fail- ing to approve a beneficial product. The constancy assumption is unlikely ever to be met completely in practice, and it cannot be validated in a trial without a placebo arm. However, it is often the case that there exist strong, measurable predictors of constancy, such as dosing and adherence. Such predictors can be used to identify appropriate non-inferiority margins during the planning phase, as well as adjust the margin during the monitoring and analysis phases. Methods
We propose using meta-analysis regression to model the association between population characteristics and the effectiveness of an active-control therapy, and assume that the model provides an un- biased estimate of effectiveness. Together with expected population characteristics, the fitted model parameters are used to specify a non-inferiority margin targeted to the planned study population. During interim and final analyses, observed population characteristics are used in combination with the initial meta-regression results to adjust the margin. Two methods of adjustment are proposed: one that maintains a pre-planned minimal clinically important difference (MCID) over an inferred placebo, and a second that maintains a fixed proportion of the estimated active-control benefit over placebo. In the second scenario, sample size adjustment may be necessary. Results
We consider a hypothetical NI trial of an experimental HIV Pre-exposure Prophylaxis (PrEP) drug versus a standard PrEP drug (active control)
designed to provide one-sided alpha equal to 2.5%, and 90% power. If, due to lack of adherence to the standard drug, the constancy assump- tion fails and the active-control therapy is 10% less effective than planned, the probability of a false non-inferiority finding rises from 2.5% to 16%. If the active control therapy is 10% more effective than planned (for example, if adherence were higher than planned), power falls from 90% to 52%. By revising the NI margin according to the pre- specified meta-regression model, and maintaining the pre-specified MCID, both alpha and power can be corrected to planned levels with- out modification to the planned sample size. If the allowable effective- ness of the experimental therapy is permitted to vary depending on the estimated active-control effect, alpha and power can be partially corrected by updating the margin, and fully corrected by updating both the margin and the sample size.
Conclusion
If prior placebo-controlled trials provide evidence of an association between population characteristics and the effectiveness of an active- control therapy, non-inferiority margins can be adjusted based on ob- served population features, effectively maintaining pre-specified levels of Type-I error and power.
O6
A revised tool for assessing risk of bias in randomized trials (RoB 2.0) Jalena Savovic1, Matthew Page2, Roy Elbers2, Asbjorn Hróbjartsson3, Isabelle Boutron4, Barney Reeves2, Jonathan Sterne2, Julian Higgins2 1University of Bristol; NIHR CLAHRC West University Hospitals Bristol NHS Foundation Trust;2University of Bristol;3University of Southern Denmark; 4University Paris Descartes
Correspondence:Jalena Savovic
Trials2017,18(Suppl 1):O6 Background
The Cochrane risk of bias tool for randomized trials seeks to determine whether the findings of a randomized trial can be believed. First released in 2008, and revised slightly in 2011, it is the most widely used risk of bias tool in both Cochrane and non-Cochrane reviews on the effects of interventions. However, evaluations of the tool have highlighted some problems. Objective: To introduce a revised tool to assess risk of bias in randomized trials (RoB 2.0), which builds on the established Cochrane risk-of-bias tool as well as the thinking behind the recently developed tool for non-randomized studies (ROBINS-I). Methods
Over the last year, we assembled collaborators from across the world to develop RoB 2.0. We held an initial development meeting in Au- gust 2015 where the main structure of the tool was agreed. Working groups were formed and tasked with developing signalling ques- tions, criteria for reaching a judgment and full guidance. Working groups’contributions were collated and the draft version of the new tool was extensively piloted by individuals with varying degrees of experience, at a three-day event held in Bristol in February 2016 and remotely. The piloting feedback was considered at a second develop- ment meeting in April 2016, where refinements to the tool and to the written guidance that accompanies it were made. The working groups were further tasked with developing algorithms for reaching a domain-level judgment and creating worked examples. Further pre-release piloting took place in September 2016.
Results
Key changes in RoB 2.0 compared with the 2011 version of the tool are:−simplification of issues into fewer (mandatory) bias domains;− clearer focus on risk of bias in a specific result from the randomized trial;−introduction of signalling questions - which are reasonably fac- tual in nature - to facilitate risk-of-bias judgements; −algorithms to reach risk of bias judgements;−clarification of differences between the review team's interest on the effect of assignment to intervention (the intention-to-treat effect) versus the effect of starting and adhering to intervention: issues of blinding, implementation and adherence differ importantly between these; − clarification that selective reporting should be assessed only when a result is available (whereas selective non-reporting should be assessed at meta-level);−separate templates for parallel group trials, cluster-randomized trials and cross-over trials.
Conclusions
We believe the new tool will offer considerable advantages over the existing tool. Once programmed into software, we expect the tool will be easier to use than the first version. Some issues remain to be re- solved, however, such as how many results should be assessed for each study, and how best to integrate the assessment into the data extrac- tion process. This presentation will provide an introduction to the tool. Further details of RoB 2.0 will be available from riskofbias.info. Funding
MRC Network of Hubs for Trials Methodology Research (MR/L004933/ 1- N61); MRC ConDuCT II (MR/K025643/1), NIHR CLAHRC West.
O7
Simulation of various strategies for optimal selection of randomization methods in multicenter clinical trials Zhibao Mi, Rebecca A. Horney, Eileen M. Stock, Kousick Biswas VA Cooperative Studies Program Coordinating Center Correspondence:Zhibao Mi
Trials2017,18(Suppl 1):O7
The random allocation and masking of participants to treatment are procedures in a study design essential to minimizing bias and the suc- cess of a clinical trial. The essence of the randomization process is to ensure an equal probability for each participant to be assigned to ac-