Quality assessment by NICE methodology checklists

Quality Assessment of systematic reviews by NICE methodology checklists

Below are the summarised responses to the checklist questions both for systematic reviews as collected using the NICE methodology checklists (Appendix 4), followed by the evidence summary produced by the reviewing author without use of the GRADE pro software.

Question addressed

The two systematic reviews included both addressed a focused question that is directly relevant to the guideline PICO.

Applicability

Both systematic reviews refer to populations similar to that of the guideline PICO.

That is asymptomatic patients with colorectal cancer who have been treated with curative intent. Patients may or may not have had adjuvant treatment. The populations are not identical as in the population of the studies no patients with metastatic disease were included although the PICO of this systematic review also aimed to address patients with metastatic cancer as long as their treatment was with curative intent. It was considered that this difference in the populations was not significant enough to affect the applicability of the study results.

Methodology

Both systematic reviews included clear and thorough descriptions of their methodology, which was appropriate. Only RCTs were included.

Literature searches of the two systematic reviews were appropriate and rigorous.

Study quality assessment

Both reviews assessed the quality of each individual study they included.

Heterogeneity

There is clinical heterogeneity of the follow-up regimen used by the different trials and this affects both systematic reviews equally. Some of the trials compared

intensive follow-up with less intensive follow-up and some with no follow-up at all. In addition the intensity of the intensive follow-up regimen was variable and indeed the intensive regimen of one study was equal to the less intensive regimen of another study. In addition the trials span a considerable time period in which the surgery and oncological management of colorectal cancer has changed. In some studies patients were receiving adjuvant chemotherapy and radiotherapy and in others not. This adds to the clinical heterogeneity.

Overall Assessment

Both systematic reviews fulfilled all the criteria of the quality assessment checklists.

None of the quality issues identified were thought likely to alter the conclusions of the reviews.

Quality Assessment of the randomised controlled trial by NICE methodology checklists.

Below are the summarised responses to the checklist questions for RCT as collected using the NICE methodology checklists (Appendix 5), followed by the evidence summary produced by the reviewing author without use of the GRADE pro software.

Question addressed

The RCT by Wang et. al. addressed a focused question that was directly relevant to the guideline PICO.

Recruitment / Applicability

The population of the study was similar to that of the PICO but excluded patients with metastatic CRC. There were clearly stated inclusion and exclusion criteria that were appropriate for the question being addressed.

Randomisation

All consecutive patients with newly diagnosed colorectal cancer who consented were randomised to the two trial groups. This was clearly explained and appropriate.

Allocation concealment

This was described clearly and was appropriate by means of sealed envelopes that contained cards allocating patients to one or the other arm of the study.

Maintenance

The status of the two study groups was comparable. The patients in both groups received similar management throughout the study period and the treatment they received was the only addition to their management.

The follow-up period was also clearly addressed in the study. The drop out rate was mentioned and within a normal range for a study of this nature. The analysis was on an intention-to-treat basis.

‘Blinding’

In the study write up there was no reference to any ‘blinding’ having being undertaken.

Outcomes

All outcomes measured were relevant and measured in a standard, valid and reliable way. Complications were included in the outcomes.

Overall Assessment

This RCT fulfilled most criteria of the quality assessment checklist. It was not considered likely that the criterion which was not fulfilled (‘blinding’) would alter the conclusions of the study, although it potentially introduces bias.

Evidence Summary (without the use of the GRADE pro software)

Intensive versus less intensive follow-up:

There is significant overall survival benefit at 5 years with intensive follow-up.

The number of all recurrences detected is similar with both intensive and less intensive follow-up.

The number of curative procedures attempted for recurrence is significantly more with intensive follow-up.

There is no disease-specific survival benefit with intensive follow-up.

Significantly more asymptomatic recurrences were detected in the group which received intensive follow-up.

The time to recurrence is significantly less in patients receiving intensive follow-up.

Generally:

CEA blood test offers survival advantage.

Liver imaging offers survival benefit, CT of the liver improves survival, ultrasound scanning of the liver versus none offers survival advantage.

Colonoscopy leads to survival benefit versus no colonoscopy. Intensive colonoscopy versus less intensive colonoscopy does not offer survival advantage.

Clinic visits for follow-up versus no clinic visit offer survival benefit.

Complications:

1 study reported adverse events from follow-up relating to colonoscopy: 2 perforations and 2 GI bleeds from a total of 731 colonoscopies.[239]

Quality of Life (QoL):

1 study showed a small but significant increase in QoL with follow-up.[240]

1 study showed no difference in QoL in patients followed up in GP versus hospital setting.[241]