ExamplE of a comparativE dEvicE tEst accuray study (parallEl randomisEd dEsign)

Title: Acetic acid compared with i-scan imaging for detecting Barrett’s esophagus: a randomized, comparative trial.

Reference: Hoffman A, Korczynski O, Tresch A, Hansen T, Rahman F, Goetz M, Murthy S, Galle PR, Kiesslich R. Gastrointest Endosc. 2013 Aug 13. pii: S0016-5107(13)02129-9. doi: 10.1016/j.gie.2013.07.013. [Epub ahead of print]

Source: MEDLINE (through PubMed)

Abstract: BACKGROUND: Traditional surveillance in patients with Barrett’s esophagus (BE) has relied on random biopsies. Targeted biopsies that use advanced imaging modali- ties may significantly improve detection of specialized columnar epithelium (SCE). OBJECTIvE: We compared the efficacy of targeted biopsies that used i-scan or acetic acid to random biopsies in the detection of SCE.

DESIGN: Patients with visible columnar lined epithelium or known BE were randomized at a 1:1 ratio to undergo acetic acid application or i-scan with targeted biopsies.

SETTING: Targeted biopsies were performed based on surface architecture according to the Guelrud classification followed by 4-quadrant biopsies.

PATIENTS: A total of 95 patients were randomized.

INTERvENTION: A total of 46 patients underwent acetic acid staining, and 49 underwent i-scan imaging. Random biopsies were performed in 86 patients.

confirmed by histologic assessment.

RESULTS: The diagnostic yield for SCE was significantly higher with targeted biopsies than with random biopsies in both groups combined (63% vs 24%; P = .0001). The yield of targeted biopsies was significantly greater with both i-scan (66% vs 21%; P = .009) and acetic acid (57% vs 26%; P = .012) technologies and did not differ between these groups. The accuracy for predicting SCE was 96% (k = .92) for i-scan and 86% (k = .70) for acetic acid analysis.

LIMITATIONS: No dysplastic lesions were found.

CONCLUSION: The i-scan or acetic acid-guided biopsies have a significantly higher diagnostic yield for identifying SCE, with significantly fewer biopsies, as compared with a pro- tocol of random biopsies. Acetic acid and i-scan showed comparable results diagnosing SCE in our study. (Clinical trial registration number: NCT01442506.).

C2. Prognostic (and monitoring) device accuracy studies

In accuracy studies for prognostic and monitoring test devices (or biomarkers), the interest is in the relationship between the information from the index test device and the occurrence of an health outcome (or health event) in the future – in other words, how information from the index test device can predict a future health outcome. Such outcomes can be an objective event such as a particular disease progression or recurrence, a new event, or a treatment response, but can also include more subjective measures such as decreased pain experience or improved quality of life. The future may be measured in hours or days (e.g. devices to measure blood loss during surgery in order to predict the need for postoperative blood transfusion), weeks, months or even years.

The time sequence between the index test device and outcome occurrence requires a different type of study design than diagnostic or screening device accuracy studies. The biggest difference between diagnostic and screening device accuracy studies and prognostic device accuracy studies is time. More specifically, the time interval between the index test result and the outcome of interest. In prognostic test accuracy studies, subjects must undergo the test device and are tracked for a certain amount of time to establish whether or not they develop the targeted study outcome(s) in the future. This time may range from hours, days, weeks, months or even years, depending on the working mechanism of the device. Such prognostic test device study thus have to follow a longitudinal design, such as a prospective or retrospective observational cohort study (see above). Also, data from randomised studies may be used to evaluate the predictive accuracy of prognostic test devices.

In studies of prognostic test devices, the population of interest can be patients diagnosed with a particular condition for whom a particular future health outcome is to be predicted, but may also involve predicting whether healthy individuals will develop diabetes type 2, for example. The overall question in such studies is whether the information provided by the test device is helpful in labelling or stratifying

patients by their risk of developing a certain outcome in the future. Study individuals or patients should again be representatives of the intended spectrum of individuals (patients) and not only a specific small subgroup. Important choices must be made about which relevant health outcomes to predict and thus observe in a study, and what the relevant duration of the follow-up is. Studies with longer follow-ups are usually more relevant for patients, professionals and society at large, but they are also more time-consuming, expensive, and run a higher risk of losing the study individuals. The accuracy of a prognostic device can be expressed in terms of how well the device pre- dicts actual observed outcomes and how well it differentiates between those who do or do not experience the health outcome.

C2.1 Single prognostic device studies

Like single diagnostic and screening test device accuracy studies, prognostic test devices can also be evaluated individually (in isolation) for their predictive prognostic accuracy. Such studies often apply the classic cohort design, with follow-up of the subjects using the prognostic test device until the predefined time period for developing the health outcome. One important quality item of any prognostic device study, whether single or comparative (see next section), is the completeness of the follow-up; ideally, all the study follow up all the subjects for the predefined time period. Like single diagnostic and screening test accuracy studies (C1.1), the main result of a single prognostic test device study is the strength of the association between the test device information and the occurrence of the health outcome of interest. And like single diagnostic and screening test accuracy studies, single prognostic test accuracy studies are not suitable for determining whether a new prognostic device has added predictive value beyond the accuracy already obtained by existing prognostic tests or factors; they are also not suitable for identifying the order in which prognostic devices can best be used (see C.2.2) or for determining whether actual use of the prognostic device or use of the diagnostic or screening device will lead to actual (or added) health benefits in individuals (see C.3).