Investigating the use of Hospital Episode Statistics data to measure variation in Performance and Quality in Colorectal Surgery

(1)

Investigating the use of Hospital Episode

Statistics data to measure variation in

Performance and Quality in Colorectal Surgery

Miss Elaine Burns BSc.Edin Hons (Anatomical Sciences) MBCHB MRCS (Glasg)

DIVISION OF SURGERY, DEPARTMENT OF SURGERY AND CANCER,

IMPERIAL COLLEGE LONDON

Doctor of Philosophy (PhD) 2012

(2)

Does a relationship exist between consultant team or Trust volume and outcome following colorectal cancer surgery? 233 Is there a relationship between total resection workload (i.e. benign and malignant) and outcome? 248 Is volume relationship mitigated in part by structural factors? 252 Is there a relationship between surgeon laparoscopic caseload and outcome? 252 Is there a relationship between caseload and outcome in subspecialist care? 260 Summary ... 265

Chapter 10-‐ Discussion ... 267

Main findings ... 267

Role of caseload in determining outcome ... 268

Quality improvement ... 277

New outcome measures ... 280

Laparoscopic versus open approaches ... 287

Study limitations ... 291

Conclusion ... 297

(6)

List of figures

Figure 3.1: Schematic of inclusion following literature search. 74 Figure 5.1: Variation in admission type (elective and emergency) by Trust for all colorectal resections (percentage of elective admission with 95% confidence interval error bars). 114 Figure 5.2: Variation in Trust diagnosis for all colorectal resections by Trust of all patients

undergoing a colorectal resection. 115

Figure 5.3 Variation in type of resection per Trust of all patients undergoing a colorectal resection.

116

Figure 5.4: Variation in Charlson scoring per Trust of all patients undergoing a colorectal

resection. 118

Figure 5.5: Variation in social deprivation by Trust of all patients undergoing a colorectal

resection 119

Figure 5.6: Variation in patient age by Trust of all patients undergoing a colorectal resection 120 Figure 6.1: Variation in total volume of colorectal resection over the study period (8 years) by 156

individual NHS Trust 130

Figure 6.2: Variation in elective colorectal cancer resection caseload per consultant team per year

over the study period. 133

Figure 6.3: Increase in mean consultant team elective colorectal cancer caseload over time with

95% confidence interval error bars. 135

Figure 6.4: The percentage of patients undergoing RPC in each provider volume category (LV, MV,

HV) from 1996 to 2008 140

Figure 6.5: The percentage of patients undergoing RPC in each surgeon volume category (LV, MV,

HV) from 2000 to 2008 141

Figure 7.1: Funnel plot of consultant team caseload and APE rates amongst patients with rectal

cancer 150

Figure 7.2: Funnel plot of NHS Trust caseload and APE rates amongst patients with rectal cancer

(7)

Figure 7.3: Overall percentage APE use amongst rectal cancer patients over time 152 Figure 7.4: Uptake of laparoscopic approach for all elective resections and elective colorectal

cancer resections 154

Figure 7.5: Variation in minimal access surgery rate amongst patients undergoing an elective

colorectal resection across SHA in 2007 158

Figure 8.1: Funnel plot of adjusted reoperation rates for elective colorectal resectional procedures for individual consultant teams by volume of elective surgery. 180 Figure 8.2: Funnel plot of adjusted reoperation rates for both emergency and elective colorectal resectional procedures for individual NHS Trusts by volume. 181 Figure 8.3: Variation in incisional hernia rates following colorectal resection for each NHS Trust

with 95% confidence intervals 189

Figure 8.4: Variation in adhesion rates following colorectal resection for each NHS Trust with 95%

confidence intervals 190

Figure 8.5: Funnel plot of pouch failure rates by NHS Trust. 196 Figure 8.6: Variation in 30 day mortality following colorectal resection for each NHS Trust with

95% confidence intervals 200

Figure 8.7: Mean readmission rates following colorectal resection with 95% confidence intervals

per NHS Trust. 202

Figure 8.8: Variation in LOS amongst colorectal resection patients by NHS Trust with 95%

confidence intervals. 205

Figure 9.1: Funnel plot of adjusted 30-‐day mortality and Trust caseload. 240 Figure 9.2: Funnel plot of Trust postoperative return to theatre rates and caseload. 241 Figure 9.3: Funnel plot of consultant team caseload and 30-‐day postoperative mortality 242 Figure 9.4: Funnel plot of adjusted consultant team return to theatre rates and caseload. 243 Figure 9.5: Confidence intervals of odds ratios of 30 day mortality following rectal cancer resection at incremental increases in rectal cancer surgical caseload to demonstrate critical

(8)

Figure 9.6: Confidence intervals of odds ratios of 30 day mortality following rectal cancer resection at incremental increases in rectal cancer surgical caseload to demonstrate critical

volume of surgery (increments of 2.5). 247

Figure 9.7: Confidence intervals of odds ratios of 30 day mortality following cancer resection at incremental increases in total surgical caseload to demonstrate critical volume of surgery

(increments of 10). 250

Figure 9.8: Confidence intervals of odds ratios of 30 day mortality following cancer resection at incremental increases in total surgical caseload to demonstrate critical volume of surgery

(increments of 5). 251

Figure 9.9: Increase in number of consultant teams using the minimal access approach from 2002 to 2008 with dates of publication of randomised control trials and guidelines. 253 Figure 9.10: Kaplan Meier curve showing variation in failure rates according to institutional

volume groups 263

(9)

List of Tables

Table 2.1: Advantages and disadvantages of quality metrics classified according to the Donabedian

approach. 38

Table 3.1: Advantages and disadvantages of HES data 65 Table 3.2: Assessment of quality of studies examining data accuracy of routinely collected data in

comparison to case note review 73

Table 3.3: Summary of included studies examining data accuracy of routinely collected data in

comparison to case note review 78

Table 3.4: Summary of included studies comparing routinely collected data with clinical registries

80

Table 5.1: Patient characteristics and type of surgery for all patients undergoing colorectal

Table 5.2: Characteristics of patients undergoing RPC between 1996 and 2008 122 Table 6.1: Patient characteristics and type of surgery for all patients undergoing colorectal

Table 6.2: Patient characteristics and type of surgery for all patients undergoing colorectal cancer

Table 6.3: Patient characteristics in each institutional volume category operated on between 1996

and 2008 * 137

Table 6.4: Patient characteristics in each surgeon volume category of pouch patients operated on

between 2000 and 2008* 138

Table 7.1: Number of elective laparoscopic colorectal cancer resections by SHA in 2007. 156 Table 7.2: Multiple logistic regression of use of minimal access surgery in elective colorectal cancer resection including those patients undergoing resection in 2007. 157 Table 7.3: Pouch laparoscopic use by volume category 160

(10)

Table 8.2: Patient characteristics and type of surgery by approach for patients undergoing a

colorectal resection. 173

Table 8.3: Reoperation following colorectal resections for patients undergoing laparoscopic and

open procedures. 175

Table 8.4: Patient characteristics of those patients who necessitated a reoperation in the

postoperative period. 177

Table 8.5: Multiple regression analysis for reoperation and laparotomy amongst elective and

emergency patients. 178

Table 8.6: Descriptives of elective patients undergoing colorectal resection between 2002 and 2008 included for analysis of medium term adhesion and herniae rates. 184 Table 8.7: Rates of re-‐intervention for adhesions and incisional hernia repairs following laparoscopic and open resection at one, two and three year post surgery. 186 Table 8.8: Multiple regression analysis of operative reintervention for incisional hernia and

adhesion 187

Table 8.9: Characteristics of restorative proctocolectomy patients who experienced failure (372/5771) compared with those who did not 192 Table 8.10: Unifactorial and Multiple Cox regression analysis for pouch failure 194 Table 8.11: Variation in 30 day and 365 day mortality amongst elective and emergency colorectal

resection patients (unadjusted analysis). 198

Table 8.12: Variation in 30 day and 365 day mortality amongst elective and emergency colorectal resection patients (multiple regression analysis). 199 Table 9.1:Previous research examining surgical provision and impact of caseload on outcome

using HES data. 218

Table 9.2: OPCS codes used to classify colorectal resection. 223 Table 9.3: Volume thresholds for elective colorectal resection. 227 Table 9.4: Volume categories and number of patients in each category for restorative

(11)

Table 9.5: Patient characteristics and type of surgery by approach for patients undergoing an

elective colorectal resection. 234

Table 9.6: Unadjusted outcome of elective colorectal cancer resection according to volume

categories. 237

Table 9.7: Multiple regression of 30-‐day postoperative mortality, 28-‐day unplanned readmission, reoperation and length of stay for elective colorectal cancer resection. 238 Table 9.8: Patient characteristics of all patients undergoing an elective resection. 255 Table 9.9: Patient characteristic of cancer patients undergoing a laparoscopic elective colorectal resection between 2002 and 2008 in each volume category. 256 Table 9.10: Outcome following elective laparoscopic colorectal cancer resection in each volume

category. 258

Table 9.11: Multiple logistic regression for reoperation, 30 day medical morbidity, 365 day medical morbidity and LOS for patients undergoing an elective laparoscopic colorectal cancer

Table 9.12: Outcome of RPC according to institutional volume and surgeon volume 261 Table 9.13: Linear regression analysis for the logarithmic transformation of length of stay (LOS) with institutional and surgeon volume considered separately for pouch surgery 262

(12)

Abstract

This thesis provides a comprehensive overview of general and subspecialist colorectal surgery in England. It examines variation in provision and outcome of colorectal surgery in structure, process and outcome factors at the patient, consultant team and NHS Trust levels. Finally, this thesis examines the potential role of increasing surgical caseload to reduce any demonstrable variation and improve outcome. To address these questions, current issues in surgical quality as well as the coding accuracy reported in the published literature have been reviewed. Colorectal resection and the more specialised procedure of restorative proctocolectomy were examined from the Hospital Episode Statistics dataset. Novel outcome measures were derived using longitudinal analysis. Regression analysis was used to understand the predictors of process factors and outcome measures.

I have defined new outcome measures and demonstrated considerable variation in these new measures and in more traditional accepted measures in both general and subspecialist colorectal surgery from routinely collected datasets. Routinely collected data offer an exciting potential data source for measuring performance and quality. If data accuracy can be assured, measures such as reoperation may be used alongside established measures of quality in a meaningful way to benchmarking performance of surgical providers. The methods described in this thesis can be applied to a broad range of surgical specialties.

Though volume may have a role in determining outcome and reducing variation in subspecialist colorectal surgery, it is by no means the panacea to improve quality across all providers in general colorectal care. The impact of volume on outcome in more general

(13)

colorectal surgery is less clear. Though centralisation is likely to have benefits, further evidence of the optimum way to implement such changes is needed rather than indiscriminately increasing volume across all providers for general colorectal surgical care.

(14)

Acknowledgements

This PhD was undertaken in the Department of Surgery and Cancer in St Mary’s Hospital under the supervision of Mr Omar Faiz, Dr Paul Aylin and Professor Ara Darzi.

I am indebted to Omar Faiz for his excellent supervision throughout my PhD. As well as finding a mentor, I have learnt many skills from him that will stand me in good stead throughout the rest of my career.

Paul Aylin has been an amazing supervisor. He has always been supportive and a welcome ear to listen to my PhD moans.

I’d like to thank Professor Darzi for his support clinically and career wise, for giving me this wonderful opportunity and for helping me achieve my career and research goals.

Without the statistical support and advice from Dr Alex Bottle, this PhD would not have been possible. I am also grateful for the support of all members of the Dr Foster unit. I’d like to acknowledge Professor John Nicholls who has provided invaluable insights throughout this research.

I’d like to thanks the others who have helped me survive and enjoy research. Ravi Mamidanna and Alex Almoudaris were fantastic collaborators in our burgeoning reseach group. The current and previous residents of the Room 1029 have provided laughs, joy, tea and wine at times of need. Thanks guys.

Finally but definitely not least, I like to acknowledge and sincerely thank my parents, Mary and Martin, the rest of my family, my flatmate Caroline, and all my friends for keeping me sane throughout this journey. They have been a constant source of distraction and happiness and have gone way beyond the call of duty to support me in the last three years,

(15)

Abbreviations

AAA Abdominal Aortic Aneursym

ACPGBI Association of Coloproctologists of Great Britain and Ireland ACS American College of Surgeons

APE Abdominoperineal Excision

AUGIS The Association of Upper Gastrointestinal Surgeons of Great Britain and Ireland

CRM Circumferential Resection Margin EBHR Evidence-Based Hospital Referral

ERP Enhanced Recovery Programmes

HES Hospital Episode Statistics IBD Inflammatory Bowel Disease IPAA Ileal anal-pouch anastomosis ICU Intensive Care Unit

IQR Interquartile Range

LOS Length of Stay

NBOCAP National Bowel Cancer Audit Project NHS National Health Service

NICE National Institute for Health and Clinical Excellence NSQIP National Surgical Quality Improvement Programme PbR Payment by Results

PROMS Patient Reported Outcome Measures

(16)

SEER Surveillance Epidemiology and End Results SHA Strategic Health Authority

SD Standard Deviation

UK United Kingdom

US United States

(17)

Publications and presentations resulting from this thesis

Publications directly resulting from thesis (See Appendices)

Burns EM, Bottle A, Aylin P, Darzi A, Nicholls RJ, Faiz O. Variation in reoperation after colorectal surgery in England as an indicator of surgical performance: retrospective analysis of Hospital Episode Statistics. BMJ. 2011 Aug 16;343:d4836

Burns EM, Rigby E, Mamidanna R, Bottle A, Aylin P, Ziprin P, Faiz OD.

Systematic review of discharge coding accuracy. J Public Health (Oxf). 2011 Jul 27.

Burns EM, Bottle A, Aylin P, Clark SK, Tekkis PP, Darzi A, Nicholls RJ, Faiz O. Volume analysis of outcome following restorative proctocolectomy. Br J Surg. 2011 Mar;98(3):408-17.

Burns EM, Mamidanna R, Currie A, Bottle A, Aylin P, Darzi A, Faiz OD. The role of caseload in determining outcome following laparoscopic colorectal cancer resection. (Under review Colorectal Dis 2012)

Burns EM, Currie A, Bottle A, Aylin P, Faiz O. An evaluation of incisional hernia and adhesion-related outcomes following laparoscopic and open colorectal surgery in England. Br J Surg. 2012 Nov 12. [Epub ahead of print]

(18)

Burns EM, Bottle A, Almoudaris A, Mamidanna R, Aylin P, Darzi A, Nicholls RJ, Faiz OD. The role of volume in elective colorectal surgery (under review BMJ 2012)

Associated with work from thesis

Burns EM, Naseem H, Bottle A, Lazzarino AI, Aylin P, Darzi A, Moorthy K, Faiz O. Introduction of laparoscopic bariatric surgery in England: observational population cohort study. BMJ. 2010 Aug 26;341:c4296

Almoudaris AM, Burns EM, Bottle A, Aylin P, Darzi A, Faiz O. A colorectal perspective on voluntary submission of outcome data to clinical registries. Br J Surg. 2011 Jan;98(1):132-9.

Burns EM, Faiz O. Evolution of the surgeon--volume, patient outcome relationship. Ann Surg. 2010 May;251(5):991-2

Burns EM, Faiz O. Response to Khani et al., centralization of rectal cancer surgery improves long-term survival. Colorectal Dis. 2010 Oct;12(10):1054.

Burns EM, Mayer EK, Faiz O. Surgeon volume does not predict outcomes in the setting of technical credentialing: results from a randomized trial of colon cancer. Ann Surg. 2009 May;249(5):866;

(19)

Faiz O, Haji A, Burns E, Bottle A, Kennedy R, Aylin P. Hospital stay amongst patients undergoing major elective colorectal surgery: predicting prolonged stay and readmissions in NHS hospitals. Colorectal Dis. 2011 Jul;13(7):816-22.

Faiz O, Brown T, Bottle A, Burns EM, Darzi AW, Aylin P. Impact of hospital institutional volume on postoperative mortality after major emergency colorectal surgery in English National Health Service Trusts, 2001 to 2005. Dis Colon Rectum. 2010 Apr;53(4):393-401.

Mamidanna R, Burns EM, Bottle A, Aylin P, Stonell C, Hanna GB, Faiz O. Reduced risk of medical morbidity and mortality in patients selected for laparoscopic colorectal resection in England: A population based study. Arch Surg 2011 Nov 21.

Almoudaris AM, Burns EM, Mamidanna R, Bottle A, Aylin P, Vincent C, Faiz O. Value of Failure To Rescue as a marker of the standard of care following reoperation for complications after colorectal resection. Br J Surg. 2011 Dec; 98 (12): 1775-83.

Presentations

Burns EM, Mamidanna R, Currie A, Bottle A , Darzi A, Faiz OD The role of caseload in determining outcome following laparoscopic colorectal cancer resection. European Society of Coloproctology Copenhagen 2011.

Burns EM, Currie A, Bottle A, Aylin P, Darzi A, Faiz O. Minimal Access colorectal surgery is associated with fewer subsequent adhesion-related admissions and operations than the conventional approach. European Society of Coloproctology Copenhagen 2011.

(20)

Burns EM, Bottle A, Aylin P, Darzi A, Faiz O. Does surgeon or institution volume influence abdominoperineal excision rate? European Society of Coloproctology Copenhagen 2011.

Burns EM, Bottle A, Aylin P, Darzi A, Faiz O. Does surgeon or institution volume influence abdominoperineal excision rate? Tripartite Colorectal meeting Cairns 2011.

Burns EM, Bottle A, Aylin P, Darzi A, Faiz O. The role of surgeon volume in Colorectal Cancer Surgery. Tripartite Colorectal meeting Cairns 2011.

Burns EM, Bottle A, Aylin P, Darzi A, Nicholls RJ, Faiz O. Variation in return to theatre following colorectal surgery in England – an indication of inequality of surgical performance? Tripartite Colorectal meeting Cairns 2011.

Currie A, Burns EM, Darzi A, Faiz O, Ziprin P. Has shortened surgical training led to poorer colorectal cancer outcomes? Tripartite Colorectal meeting Cairns 2011.

Burns EM, Bottle A, Aylin P, Darzi A, Faiz O. Return to theatre in Crohn’s Disease in the biological era. European Society of Coloproctology Sorrento 2011.

Burns EM, Bottle A, Aylin P, Darzi AW, Faiz O. The impact of reoperation following colorectal resection on postoperative mortality. European Society of Coloproctology Sorrento 2011.

Burns EM, Bottle A, Aylin P, Darzi A, Faiz OD. Does surgeon or institution volume influence abdominoperineal excision rate? Association of Coloproctology of Great Britain and Ireland Birmingham 2011.

(21)

cancer surgery. Association of Coloproctology of Great Britain and Ireland Birmingham 2011.

Burns EM, Bottle A, Aylin P, Darzi A, Nicholls RJ, Faiz OD. Variation in return to theatre following colorectal surgery in England – an indication of inequality of surgical performance? Association of Coloproctology of Great Britain and Ireland Birmingham 2011.

Burns EM, Bottle A, Aylin P, Tekkis PP, Clark S, Darzi AW, Nicholls RJ, Faiz O. National Outcomes Following Restorative Proctocolectomy In England. Association of Great Britain and Ireland Liverpool 2010.

Burns E, Naseem H, Aylin P, Faiz O, Moorthy K. Trends in Laparoscopic Bariatric Surgery and Comparisons of Outcomes with Open Surgery: a National study in England 2000-2008. Association of Great Britain and Ireland Liverpool 2010.

Burns EM, Bottle A, Aylin P, Tekkis PP, Clark S, Darzi AW, Nicholls RJ, Faiz O. Examining Differences In Postoperative Outcomes Between Hospitals with differing surgical caseload following Restorative Proctocolectomy. Association of Great Britain and Ireland Liverpool 2010.

Burns EM, Bottle A, Aylin P, Tekkis PP, Clark SK, Darzi AW, Nicholls RJ, Faiz O. Examining differences in case selection between surgeons with differing surgical caseload in pouch surgery. Association of Great Britain and Ireland Liverpool 2010.

Burns E, Bottle A, Aylin P, Faiz O, Moorthy K. The role of volume in Bariatric Surgery. Association of Great Britain and Ireland Liverpool 2010.

(22)

Surgery in England. Association of Great Britain and Ireland Liverpool 2010.

Burns EM, Bottle A, Jing SS, Aylin P, Darzi AW, Faiz O. National Provision Of Laparoscopic Pouch Surgery In England. Association of Coloproctology of Great Britain and Ireland Bournemouth 2010.

Burns EM, Bottle A, Aylin P, Tekkis PP, Clark SK, Darzi AW, Nicholls RJ, Faiz O. National Outcomes Following Restorative Proctocolectomy In England. Association of Coloproctology of Great Britain and Ireland Bournemouth 2010.

Burns EM, Bottle A, Aylin P, Tekkis PP, Clark SK, Darzi AW, Nicholls RJ, Faiz O. Examining differences in case selection between surgeons with differing surgical caseload in pouch surgery. Association of Coloproctology of Great Britain and Ireland Bournemouth 2010.

Burns EM, Bottle A, Aylin P, Tekkis PP, Clark SK, Darzi AW, Nicholls RJ, Faiz O. Examining Differences In Postoperative Outcomes Between Hospitals with differing surgical caseload in restorative proctocolectomy. Association of Coloproctology of Great Britain and Ireland Bournemouth 2010.

Burns E, Bottle A, Aylin P, Kennedy R H, Hanna G, Faiz O. Disparity in use of laparoscopy for colon cancer in England. Association of Great Britain and Ireland Glasgow 2009.

Faiz O, Haji A, Burns E, Bottle A, Kennedy R, Darzi AW, Aylin P. Hospital stay amongst patients undergoing major elective colorectal surgery: predicting prolonged stay and readmissions in NHS hospitals. Association of Great Britain and Ireland Glasgow 2009.

(23)

Burns E, Aylin P, Kennedy R H, Hanna G, Faiz O. Disparity in use of laparoscopy for colon cancer in England. Association of Coloproctology of Great Britain and Ireland Harrogate 2009.

Faiz O, Brown T, Bottle A, Burns EM, Darzi AW, Aylin P. The influence of hospital provider volume on postoperative mortality and intestinal continuity rates after major emergency colorectal surgery in England. Association of Coloproctology of Great Britain and Ireland Harrogate 2009.

Faiz O, Haji A, Burns E, Bottle A, Kennedy R, Darzi AW, Aylin P. Hospital stay amongst patients undergoing major elective colorectal surgery: predicting prolonged stay and readmissions in NHS hospitals. Association of Coloproctology of Great Britain and Ireland Harrogate 2009.

Statement of Contribution and Declaration of originality

The work contained in this thesis is my own and was performed by myself. In the process of carrying out this work, other individuals were involved. My supervisors, Mr O Faiz, Dr P Aylin and Professor A Darzi provided direction and advice with conception, design and editing of the studies included in this thesis. In addition, Dr A Bottle aided with data cutting of the pouch database, statistical support and editing of the other chapters. Specifically in Chapter 4, Dr E Rigby and Mr R Mamidanna acted as second and third reviewer for the articles included in the systematic review of coding accuracy and helped with the drafting and editing fo the final manuscript (See appendix). Mr A Almoudaris compiled the structural data that were analysed by myself in Chapters 6 and 9. Though I carried out the analysis and interpretation of the data, R Mamidanna derived the algorithm and provided the data for

(24)

Chapter 1- Introduction

‘Variation in the utilization of health care services that cannot be explained by variation in patient illness or patient preferences.’

Jack Wennberg (Dartmouth Atlas of Variation)

Over the last seventeen years, since the publication of the Calman Hine report, there has been an increasing recognition that variation exists within the National Health Service (NHS) (Expert Advisory Group on Cancer to the Chief Medical Officers of England and Wales, 1995). There has been a consequent impetus towards uniform practice across England and Wales. The report, High quality care for all: NHS next stage review, went further in making quality and standard setting central to the future of healthcare in the NHS in England (Darzi, 2008).

Variation in healthcare has been previously demonstrated in both the NHS and in other healthcare systems such as in the United States (US). Patient factors such as ethnicity and socioeconomic deprivation have been demonstrated to have a negative impact on both access to services and outcome following treatment (Bagger et al., 2008, Smith et al., 2006). Variation at an institutional level also exists. NHS Trusts vary in terms of the provision of services and other structural factors such as caseload. These factors may or may not impact on the healthcare provided and consequent outcome. In the NHS, Primary Care Trusts (PCT)

(25)

commission services from Acute Trusts. PCTs differ in terms of which procedures that they will or will not permit leading to the suggestion of a ‘postcode lottery’ (Henderson, 2009) i.e. where the services available for individual patients depend on their geographical location.

It is generally agreed that there is unacceptable regional variation in NHS services throughout England (Darzi, 2008). This variation persists in surgical practice. Geographical disparities have been examined using routinely collected hospital data in England. Dixon et al, found marked variation in age and sex adjusted total hip and knee replacement rates (Dixon et al., 2006). As a first step to improving quality in clinical care, this variation must be described and demonstrated. Such descriptions can be used to direct quality initiatives. The NHS has begun to publicly report this variation in the NHS Atlas of Variation (NHS Atlas). This Atlas is similar to the Dartmouth Atlas in the US (Dartmouth Atlas of Variation). It describes variation in a range of variables including bariatric surgery, spending on cancer and stroke care.

Colorectal surgery

Colorectal surgery represents a significant portion of the elective and emergency general surgical workload. In the financial year 2007/2008, there were over 35,000 admissions for colorectal resections in England. It is a diverse speciality encompassing a wide range of diseases including colorectal cancer, inflammatory bowel disease and diverticular disease. Given the ageing population, the workload of this speciality is likely to increase over time. One study predicted a 42% increase in the number of colonic resections between 2000 and

(26)

Colorectal surgery is associated with marked morbidity (Alves et al., 2005) and, in the emergency setting a significant risk of mortality (Faiz et al., 2010a). Considerable variation between institutional and individual colorectal surgical practice and outcome has been demonstrated previously (Morris et al., 2008, McArdle and Hole, 1991). Such variability is increasingly unacceptable to clinicians, healthcare managers, commissioners and patients. Describing this variation in performance is an important step to allow quality assessment and subsequent improvement.

In the United Kingdom (UK), nationwide administrative data that include all patients admitted to National Health Service (NHS) hospitals are collected. The current availability and comprehensive coverage of these datasets makes them an attractive potential source for measuring performance. Reducing variation to improve outcome is desirable. Several quality improvement measures have been suggested to improve outcome. These include centralising services, increasing individual provider caseload, clinician revalidation or surgeon credentialing to ensure adequate technical proficiency. Surgeon credentialing involves assessing surgeons as competent in performing a particular operation or set of interventions. Revalidation represents a regulatory process incorporating regular audit and assessment to ensure clinicians meet nationally agreed standards (General Medical Council, 2010). This process may reduce variation within the health service and improve outcome. For revalidation to be effective, reliable performance measures are necessary. In the US, in addition to outcome measures such as mortality, length of stay and readmission, other measures have been used to examine variation in colorectal surgery such as return to theatre and complication rates (Billingsley et al., 2007, Billingsley et al., 2008, Morris et al., 2007a).

(27)

As yet colorectal reoperation rates have not been examined at a national level using UK based data.

Previous studies have demonstrated variation in colorectal surgical practice in England. Using Hospital Episode Statistics (HES) data, Tilney and colleagues found considerable inter-centre variation in Abdominoperineal Excision (APE) rates (Tilney et al., 2008). Using HES data linked to cancer registry data, Morris and co workers suggested that surgeon-to-surgeon variation in the use of APE persisted despite case mix adjustment (Morris et al., 2008). As well as differences at an institutional level both these studies found increased rates of APE use amongst those patients from more socially deprived areas. Tilney and colleagues in a study using a clinical database also found a relationship between lower socioeconomic status and increased rates of APE use (Tilney et al., 2009).

Alongside colorectal excisional surgery, it is necessary to describe variation following more specialist surgery. Restorative proctocolectomy (RPC) or ileal pouch anal anastomosis surgery (IPAA) is widely accepted as the operation of choice for patients with Ulcerative Colitis (UC) or Familial Adenomatous Polyposis (FAP) who require colectomy. It is one of the most challenging operations in colorectal surgery and requires a high level of both technical skill and clinical acumen in choosing the appropriate patients to undergo the procedure. There have been many large series from single institutions describing outcome following RPC in the UK and US (Fazio et al., 1995, Tulchinsky et al., 2003, Chapman et al., 2005). Published audits of outcome at a national level are, however, limited. One previous

(28)

et al., 2010). This audit took place over thirty years with ten contributing centres in the UK. Examination of variation in outcome or the role of surgeon or institutional caseload in determining outcome was beyond the remit of this study. Kennedy and colleagues examined the role of caseload in determining volume using administrative data in a single region in Canada (Kennedy et al., 2006). This study found increased reoperation and failure rates amongst lower volume institutions. The results of this study may not be applicable to the UK setting. Furthermore this study did not examine the role of individual surgeon’s caseload in determining outcomes.

This thesis seeks to assess whether it is feasible, given accurate data, to measure performance and quality in surgery from HES data. As part of assessing the HES database’s use to measure performance and quality, we review current issues in surgical quality and existing measures of quality. This thesis will examine variation in structure, process and outcome measures at the NHS Trust, and consultant team levels. This thesis, however, goes further in seeking to ascertain whether this variation can be negated through increasing institutional or surgeon caseload.

(29)

Aim

1. Define current issues in surgical quality and performance with particular reference to colorectal surgery

2. Examine current variation in colorectal surgery and contributing factors to this variation from routinely collected datasets

3. Derive new possible measures of colorectal surgical performance from routinely collected datasets

4. Explore use of administrative datasets to benchmark performance and quality in colorectal surgery

(30)

Chapter 2- What is surgical quality and how do we measure it?

Introduction

Healthcare accounts for a significant proportion of government and individual spending. In the US, in 2008, total healthcare expenditure was $2.4 trillion -17% of the Gross Domestic Product (2009d). In the UK in 2009 healthcare was responsible for £102.6 billion of public spending (www.ukspending.co.uk, 2009). Policy-makers, healthcare funders and patients demand high quality medical care to justify current spending levels in a period of financial austerity. Central to efficiency in healthcare is a fundamental need to demonstrate quality of service provision. Although it is useful to speak about improving quality in broad terms, what does quality in healthcare actually mean?

This chapter seeks to define surgical quality, discuss the reasons for measuring quality, and describe existing quality metrics.

The demand for high-‐quality healthcare

In the report, ‘Crossing the Quality Chasm: A New Health System for the 21st Century’, the Committee on Quality of Healthcare from the Institute of Medicine in America recognised that there was a significant gap between current healthcare performance and the quality that is possible (Committee on Quality Health Care in America, 2001). This document advocated redesign of the American healthcare system in order to achieve superior quality (Committee on Quality Health Care in America, 2001). In England, the 2008 government report, ‘High

(31)

This document, along with the NHS Constitution, (NHS Constitution, 2009) and the recently formed National Quality Board, underline the health service’s commitment to delivering a high quality service for all patients (National Quality Board). Other countries similarly overtly promote high quality healthcare. Australia, for example, has a well established Commission on Safety and Quality in Healthcare. This commission has wide-ranging remit to drive quality improvement, disseminate knowledge and publicly report performance against national standards (Australian Commission on Safety and Quality in Healthcare, 2009).

The definition of surgical quality

The definition of quality in surgical care is debated. It should reflect all aspects of care; not just outcome including mortality and morbidity but also patient centred factors such as patient experience and postoperative quality of life. The National Institute of Medicine in the US defines quality as the ‘degree to which health services for individuals and populations

increase the likelihood of desired health outcomes and are consistent with current professional knowledge’ (Institute of Medicine, 1990). This definition recognises the need for

care to meet current standards and the requirement for healthcare professionals to deliver up to date treatment. The above description of quality does not however define ‘desired health outcomes’.

Surgical quality is deeply intertwined, but not synonymous, with both performance and safety. Performance is the way in which a person or organisation functions whereas quality is a measure of excellence. Safe practice must be followed to deliver quality care. High quality service provision requires excellent clinical performance. Improving safety is a focus

(32)

for many quality initiatives. As such, quality initiatives may begin with improving surgical safety. The World Health Organisation (WHO) recognised the need for global action to improve safety in surgery with the Safe Surgery Saves Lives initiative. The latter resulted in a simple checklist that could be used globally in many types of surgery to attempt to reduce adverse events. Initial pilot studies demonstrated that this checklist resulted in a 47% reduction in mortality after its introduction (Haynes et al., 2009). It also illustrated that such a simple intervention can be applied internationally irrespective of differences in healthcare systems.

Reporting of surgical performance and quality

Since Florence Nightingale, attempts have been made to measure performance and make comparisons between institutions. Nightingale together with William Farr first openly published hospital mortality rates in 1857 (Spiegelhalter, 1999). They used these statistics to galvanise public opinion to improve quality in poorly performing institutions. Similarly, Ernest Codman encouraged the public reporting of surgical outcome in the early 1900s (Neuhauser, 2002). Both Nightingale and Codman were however subject to criticism for their methods - most notably because their data did not account for variation in case-mix.

Even today controversy exists regarding the use of publicly reported outcome measures (Mohammed et al., 2009). One frequently cited concern is that surgeons may be less willing to operate on high-risk patients if outcomes are openly reported (Schneider and Epstein, 1996). The Cardiac Surgery Reporting System has been collecting and publishing risk-adjusted cardiac surgery mortality data for New York State since 1989 (Chassin et al., 1996).

(33)

Initial analyses suggested that, three years following the introduction of the public reporting scheme, mortality after cardiac surgery decreased by 41% (Hannan et al., 1994). Some authors, however, questioned whether this decrease in mortality reflected true improvements in quality. Omoigui suggested that the apparent improvements may have occurred as a result of changes in surgical case-mix (Omoigui et al., 1996). This study examined the referral pattern of patients from New York to the Cleveland clinic in Ohio from 1989 to 1993. The 482 patients referred from New York were sicker than the other referral cohorts in the clinic and had higher risk scores than those patients who remained in New York. Relative to the Ohio referral group, those referred from New York State had an increased risk of mortality [odds ratio of 1.7 (Confidence Interval 1.1-2.7), p=0.030]. Chassin and co-workers refuted the data in the Omoigui study by arguing that the time period used by Omiogui and colleagues was premature given that the first publicly reported outcomes were issued in late 1989 (Chassin et al., 1996). In addition, the referral numbers to the Cleveland clinic were small in comparison to the total number of procedures performed in the comparable period in New York. Ghali and colleagues argued that the dramatic improvement in mortality seen in New York was not necessarily due to the reporting of outcome data (Ghali et al., 1997). Ghali found a similar reduction in mortality despite the absence of statewide reporting in Massachusetts. It is unknown whether this reflects a baseline improvement in mortality due to medical advances or whether the positive effects from New York State diffused to other regions. The concern that public reporting of outcome measures can lead to a change in surgical case mix was further contested in a recent study reviewing surgical practice following the publication of cardiac surgery mortality data in the UK (Bridgewater et al., 2007). This study observed an increase in the number and proportion of high-risk patients

(34)

outcome. Despite concerns regarding the potential hazards of public reporting of outcome data, this type of programme is likely to take on a wider role especially in the UK given patients’ right to choose their treatment location (NHS Constitution, 2009). Hibbard and co-investigators studied the effects on hospital performance of public reporting of quality of care data versus reports for internal consumption and finally no feedback. Hibbard found that public reporting and to a lesser extent internal feedback led to not only increased quality improvement activity but also to improvements in performance in poorer performing hospitals (Hibbard et al., 2005). The use of public reporting of outcome measures for quality improvement purposes, however, remains questionable. A recent systematic review found that although public reporting leads to quality improvement activity at a hospital level, it is unclear whether this increase in activity translates into genuine improvements in quality (Fung et al., 2008).

An alternative to public reporting is the production of reports for internal monitoring. The National Surgical Quality Improvement Programme (NSQIP) provides an example of an intervention that uses risk-adjusted quality metrics to drive quality improvement (ACS/NSQIP 2009). This programme started amongst the Veterans Affairs (VA) hospitals, following a legal mandate in 1985 for the VA to report their risk-adjusted outcomes benchmarked against national figures (Davis et al., 2007). The VA was in a unique position to create risk-adjusted models and began to inform sophisticated outcome measure reporting through their own progressive information technology systems and centralised organisation. Using this monitoring system the VA created NSQIP to encourage surgical quality monitoring across all VA hospitals. With support of the American College of Surgeons (ACS), this system is now disseminating to private hospitals as a means of reporting

(35)

outcomes. The ACS/NSQIP system uses a combination of semi-annual reports, ad hoc reports and online reporting to feedback to individual hospitals. The reports are designed for internal review rather than for public reporting, and are used as a basis to identify areas where quality improvement initiatives can be implemented to improve outcomes and track the results from such initiatives. Internal reporting of outcome data for quality measurement and improvement is used in the UK based initiative, Dr Foster Intelligence. This is an online tool that is used for timely analysis of outcome data through case mix adjusted cumulative sum charts at a provider level to allow early identification of potentially poor performance and prompt remedial action to improve the quality of care (Bottle and Aylin, 2008).

Irrespective of whether external or internal reporting is favoured, performance measurement in surgery demands identification of appropriate and measurable outcomes that can be accurately risk-adjusted and used to distinguish between good and substandard performance. The risk-adjustment model should only adjust for those factors that are outside the control of the system being measured. In the case of repair of fracture neck of femur, when assessing surgical technique in isolation, adjustment should be made for the preoperative clinical state of the patient as this is beyond the control of a surgeon’s technical skills. In contrast, for a hospital level assessment, adjustment should be made for the state of the patient at admission, but not for the degree of medical stabilisation prior to surgery or preoperative care as this will impact on their survival, and is within the control of the system.

Mortality, although useful and relevant in elective cardiac and some abdominal surgery, is an insensitive quality measure for most other elective procedures. As such, it is necessary to

(36)

Outcome measures are not necessarily the best way of defining surgical quality. Iezzoni described outcome as the sum of patient factors, effectiveness of care and also random variation between individual patients and providers (Iezonni, 2003). It follows that institutional or surgeon-specific performance may be determined by random variation if complete reliance on outcome measurement alone is used to define and benchmark quality in surgery.

Measuring quality in surgery

The Agency for Healthcare Research and Quality defines a quality measure as ‘a mechanism

that enables the user to quantify the quality of a selected aspect of care by comparing it to a criterion’ (Center for Health Policy Studies, 1995). A useful quality metric is reproducible,

easily measured over time, objective, reliable and relevant to clinicians, health service providers and patients (Bergman et al., 2006). These metrics must reflect the complexity of the underlying processes and be resistant to the ‘gaming effect’ where an organisation manipulates a particular metric rather than improving the overall quality (Smith, 1995). When used for comparison between surgeons or providers, a metric should be adjusted to reflect variation in case mix, and be shown to be a valid proxy of overall quality. Furthermore, the National Quality Forum in the US, in outlining their proposed criteria for quality metrics, suggested that metrics should address a high impact area of healthcare, be evidence based, relevant, feasible, reliable and credible. The measure should be usable by the intended audience and be useful to distinguish between good and poor quality. In addition, the National Quality Forum underlines the importance of risk-adjusting metrics where possible. Using the Donabedian approach, (Donabedian, 1980) quality metrics may be

(37)

subdivided according to whether they represent structural, process or outcome measures (Table 2.1).

(38)

Table 2.1: Advantages and disadvantages of quality metrics classified according to the Donabedian approach.

Factor Example Advantages Disadvantages

Structural Surgical caseload Easily measured May not reflect changes in outcomes

Can be improved through system changes

Over simplistic

Process Number of lymph nodes harvested in oncological colorectal resections

Reflect agreed standards of care

Need to be directly related to improved outcomes to be useful

Reduce inequality in care when used effectively

Restrict the provision of individualised patient care Usually not available from routine data

Outcome Postoperative mortality Easily understood by healthcare professionals and patients

Usually infrequent events

Can be collected through existing data sources

Can be crude measures of quality

Can be risk-adjusted to reflect case-mix

(39)

Structural factors

Structural factors reflect characteristics that relate to the surgeon or the organisation. Examples of structural factors include the number of subspecialist surgeons at an institution, the nursing-to-patient ratios, the number of surgical beds, (Brook et al., 1996) and the operative volume carried out at a given institution. Structural measures are potentially attractive quality metrics as they are easily measured. The impact of operative volume, or caseload, of a given procedure on performance has been subject to significant scrutiny (Dimick et al., 2005). Indeed, a relationship between poor operative outcome and both low surgeon or low institution volume has been demonstrated for a range of surgical procedures, (Luft et al., 1979, Begg et al., 1998) including pancreatic resection, (Sosa et al., 1998, Bentrem and Brennan, 2005) coronary artery bypass grafting, (Hannan et al., 2003, Wu et al., 2004) orthopaedic, (Katz, 2001) and vascular surgery (Killeen et al., 2007, Holt et al., 2007a, Holt et al., 2007b). The Leapfrog Group, an American alliance aimed at using employer purchasing power to improve surgical quality and outcomes, has used the volume-outcome relationship to determine payment and referral policy. (Leapfrog Group, 2007) For seven elective procedures the alliance promotes ‘Evidence-Based Hospital Referral’ (EBHR) that is largely centred on hospital caseload. Surgical volume as a proxy measure of quality is also being used in the English healthcare system to justify centralisation of upper gastrointestinal cancer services with the aim of improving quality. Regional oesophagogastric cancer centres in England should presently serve a population base of greater than 1 million people and perform a minimum of 100 oesophageal and gastric cancer resections per year (Department of Health, 2001). Volume as a quality metric is also being applied to colorectal surgery in the UK. It is currently recommended that a colorectal surgeon should carry out a minimum of

(40)

that the volume-outcome relationship may represent an oversimplification. It is possible that volume identifies and remedies the problem associated with poor performers carrying out low surgical volumes. However, this comes at the expense of surgeons who demonstrate good performance but with low surgical caseload. A recent study, based on data from Clinical Outcomes of Surgical Therapy randomised control trial, highlighted the possible role of credentialing (i.e. establishing the proficiency of surgeons at a given technique) in determining quality (Larson et al., 2008). This may account for good outcomes amongst some low-volume surgeons. Discriminating between the relative independent impact of credentialing and volume in determining quality within surgery requires further research given the growing emphasis being placed upon caseload in service planning in many countries. In addition to surgical caseload, outcomes following complex surgery have been found to improve with better staffing in intensive care units and optimum nurse-to-patient ratios (Dimick et al., 2001).

Process measures

Structural measures yield little information regarding the actual care received by individual patients. In contrast, process measures seek to reflect the interactions that lead to effective care. Process metrics frequently measure adherence to treatment standards or guidelines. As quality metrics, these measures are attractive as they can be standardised and provide information about the areas in which quality improvement may be focused. Surgical-specific process measures are currently being sought both in the US and UK (ACS/NSQIP 2009, Care Quality Indicators, 2009). The American Society of Clinical Oncology have derived a range of surgical quality process measures for breast and colorectal cancer (Desch et al., 2008). These metrics include provision of Tamoxifen or Aromatase Inhibitor in breast cancer, lymph

(41)

node harvest for colonic and rectal cancer resection and the timing of radiotherapy and chemotherapy for breast and colorectal cancer. They aim to reflect the quality of operative treatment as well as adjuvant care.

Process measures should, however, be directly linked to outcome to confirm their use as a valid quality marker. This however can prove difficult (Evans et al., 2001). Pitches and colleagues in a systematic review concluded that there was little relationship between process measures and mortality. This conclusion was controversial as 51% (26/51) studies included in their review did demonstrate such a relationship (Pitches et al., 2007). Exclusion of outlying institutions from individual studies negated any relationship. Recently, there has also been a growing appreciation of the variable use of some clinical processes in the perioperative period such as early nutrition, (Fearon et al., 2005) epidural analgesia, (Gendall et al., 2007) as well as goal-directed fluid administration (Noblett et al., 2006). Protocols, such as enhanced recovery programmes (ERP), which incorporate a number of processes that are each independently associated with high-quality care, have been shown to be clinically effective at accelerating recovery in patients undergoing colorectal surgery (King et al., 2006, Wind et al., 2006). More importantly, implementation of evidence-based care through ERPs may also facilitate standardization of high-quality postoperative care and thus improve outcome.

A single process measure is unlikely to capture all the main aspects of quality and may only capture a fraction of the variation in outcome. This requires the combined use of several measures, which can be problematic. One option is to use the proportion of patients for whom

(42)

one process may adhere very poorly to another, even in the same group of patients (Peterson et al., 2006). Combining measures into one, for example by the use of a composite measure (like a star rating system), requires a set of weights for this combination, and the choice of composite measure has been shown to affect the relative performance (rankings) of hospitals (Jacobs et al., 2005). An often-quoted advantage for process measures is that they do not require case mix adjustment because the protocol to which they relate should be followed in

all patients. While this is true in theory, in practice the sicker and disadvantaged patients have

been suggested to have such protocols followed less often than other patients (Peterson et al., 2006). Other than poorer standards of care, possible reasons for this could include greater levels of contra-indications and patient refusal.

In addition to clinical processes, effective teamwork is associated with superior outcomes in many different industries, including healthcare. Team working has certainly been associated with better outcomes in intensive care units, (Shortell, 1991) and primary care (Bower et al., 2003). Furthermore, it is central both to the delivery of healthcare and to its improvement. Group information processes such as multi-disciplinary ward rounds have also been found to be associated with lower mortality and morbidity rates, (Gurses and Xiao, 2006, Plantinga et al., 2004) as well as a reduced incidence of prescription errors (Leape et al., 1999). In addition, in the surgical context, Young and colleagues found that low-morbidity hospitals had higher levels of team communication and coordination (Young et al., 1998). These examples all suggest that teamwork is a critical mediator for the delivery of both safe and high quality surgical care.

(43)

Outcome

Outcome measures are frequently used as quality metrics as they can be collected from existing data sources. They seek to reflect the endpoint of the cumulative processes of a patient’s care. Furthermore they may be objective (e.g. mortality) or subjective (e.g. patient reported outcomes). When outcome measures represent either rarely occurring events (such as postoperative mortality, surgical site infection rates (Fry, 2008), reoperation rates (Morris et al., 2007a), or leak rates following elective gastrointestinal surgery), these can represent blunt instruments with which to measure quality. Moreover, use of inaccurate data, inadequate case-mix adjustment or even chance variation between patient outcomes may lead to difficulty in interpretation of performance ranking. In turn, this may invalidate, or render erroneous, subsequent service or policy decision-making.

Given that measures such as mortality and anastomotic leaks are infrequent events in most surgical populations, consideration should be given to the sample sizes required to judge whether outcome differences reflect true variation in performance or whether they reflect patient factors or chance alone. Singh and co-workers calculated the necessary population sizes required to detect underperformance using mortality as the outcome measure in radical cystectomy (Singh et al., 2003). They found that, given an acceptable mortality rate of 3.5%, a cohort of 211 patients would be necessary to demonstrate that a surgeon with an 8% mortality rate was underperforming rather than this high mortality being due to chance or case mix. Depending on the specialty, this may represent several years worth of patients. Given the large number of patients required to detect significant variation in performance, it may be necessary to combine performance indicators to derive a meaningful composite

Investigating the use of Hospital Episode Statistics data to measure variation in Performance and Quality in Colorectal Surgery