The Evaluation of Screening
Policies for Diabetic Retinopathy
using Simulation
R. Davies1, S.C. Brailsford1,
P.J. Roderick2 and C.R.Canning3
Diabetes mellitus
• Common disease affecting 1 – 2% of population of UK
• Accounts for 4 – 5% of total health care expenditure in UK
• Exists in two forms: insulin-dependent (IDDM) and non-insulin-dependent (NIDDM)
– IDDM (type 1 or acute-onset) – usually diagnosed under age 21
– NIDDM (type 2 or older-onset) – usually diagnosed over age 40
IDDM (Type 1)
• IDDM accounts for ≈ 15% of all cases of diabetes
• Incidence in Europe varies: highest in north (e.g. Finland
≈ 43 cases per 100,000 per year) & lowest in south:
UK ≈ 17 cases per 100,000 per year)
• Onset is sudden: obvious symptoms (e.g. excessive thirst, tiredness, or even coma)
• No evidence of higher prevalence in UK ethnic minorities • Incidence in children may be increasing
NIDDM (Type 2)
• NIDDM accounts for ≈ 85% of all cases of diabetes
• Estimated equal numbers of diagnosed and undiagnosed cases
• About 2 million cases in the UK (diagnosed and undiagnosed)
• Prevalence of diagnosed NIDDM in UK estimated between 4 and 9 %
• Onset is insidious; may already have complications at diagnosis
• Prevalence of NIDDM is 2 to 4 times higher in Asians and Afro-Caribbeans
Diabetic retinopathy
• One of the most serious complications of
both types of diabetes: sight-threatening eye disease
• DR is the major cause of blindness in the developed countries
• Retinopathy can be detected before the patient is aware of any symptoms ….
Photograph of the retina,
What is diabetic retinopathy?
• Raised blood glucose levels cause
microvascular damage throughout the body • In the eye, some small retinal blood vessels
close off and others dilate : called
Background Diabetic Retinopathy (BDR) • Can lead to two types of disease
Diabetic Macular Oedema
• Swelling of vessels spreads to macula – area where best vision is concentrated • Can lead on to Clinically Significant MO
(CSMO), when it becomes sight-threatening • Most common in elderly: gradual onset, slow
Proliferative Retinopathy (PDR)
• Growth of new vessels off the back of the retina
• Causes visual loss through bleeding and scar tissue formation
• Sudden, catastrophic and total sight loss (usually due to a bleed)
Screening for DR: methods
• Fundoscopy: direct examination of the retina using an ophthalmoscope
• Photography: with or without mydriasis (eye drops to enlarge the pupil); various types of camera (Polaroid, digital, etc)
Screening for DR: settings
• In hospital
• At local doctor’s surgery
• In high-street optometrist shop • In mobile van
Screening for DR: screeners
• Family doctor (General Practitioner) • Diabetic specialist
• Eye specialist (ophthalmologist) • High-street optometrist
• Trained technician takes photographs,
subsequently reviewed by various grades of doctor
Screening issues
• By whom?
• By what technique? • Whom to target?
• How often, and where? • Cost-effectiveness
The research project
• Many different types of screening programme
currently in use: no general consensus about form of an ideal screening policy
• Project funded by UK National Health Service
• Our task: evaluate and compare range of screening policies, using discrete-event simulation
• Develop model of the natural history of diabetic retinopathy in a population, incorporating new cases as they are diagnosed
• Superimpose different screening policies on this baseline model and compare outcomes
Working with the model
• Work with clinicians and epidemiologists in collecting and analysing data and designing model
• Design interface for population, epidemiological and run time data
• Validate model against published data • Derive cost data
The simulation model(s)
• Population representing a UK Health District (approx 250,000)
• People classified by age, sex & ethnicity • Simulate 25 years, including new cases
• Data on incidence, prevalence, progression,
mortality, screening and treatment from published literature
• Investigate different screening policies, with associated costs
No retinopathy Background Diabetic Retinopathy (BDR) Proliferative Diabetic Retinopathy (PDR) Blindness from PDR No retinopathy Diabetic Macular Oedema (DMO) Clinically Significant Macular Oedema (CSMO) Blindness from CSMO NIDDM only
Disease progression
Discrete Event Simulation
• Models individuals moving through a system (queuing network)
• Characteristics influence progression
• Random numbers used to sample from distributions
• Can incorporate delays or interactions • Can incorporate constraints
Advantages of DES
• Flexible building blocks • Can use any distributions
• Individual variability and uncertainty
• Takes account of patients characteristics • Can take account of limited resource
Drawbacks of DES
• Greedy for computer memory and time consuming (declining problem!)
• Needs statistical techniques to analyse results
• Tempting to make models complex • Need OR and programming skills for
Modelling Problems
• Events happen to patients in different orders
• Groups of events may have different timescales • Different processes go on at the same time,
and interact, e.g. disease progression, death and screening
• Need to interrupt and reschedule processes: e.g. resample time of death, since the
POST - Patient oriented
simulation technique
• Davies, O’Keefe and Davies (1993)
• Can cope with simultaneous, interacting processes
• Entities can take part in several activities, and be in several queues, at the same
time
• One event can interrupt and change others
Three simultaneous processes
Central vision loss Severe vision loss PDR False +ve Background retinopathy Diagnose diabetes Natural history Screening PDR + DMO PDR + CSMO CSMO Policy 1 Screen for any retinopathy Detect background retinopathyFurther disease to detect
Death from any state
Detect and treat PDR or CSMO Visit ophthalmology clinic DMO
POST - Patient oriented
simulation technique
• Uses set of procedures and functions in Delphi • Patient can be set to take part in more than one
event - the earliest one takes precedence over the others
• Patients can queue while taking part in events
• Future events can be extracted from the calendar and terminated or changed and returned
Verification and Validation
• Does model progress smoothly over time?
• Do screening and treatment episodes accord with experience?
• Does prevalence of disease accords with research findings? Problems can arise if there is a mismatch with incidence data. • Is it recognisable to the experts?
0 0.2 0.4 0.6 0.8 1 Prevalence of retinopathy 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36
Years from diagnosis
Simulation WESDR Proliferative Retinopathy Any retinopathy
PDR
DMO
Screening policies
• Policy 1: initial screen in primary setting, with follow-up screening in hospital on detection of BDR (or worse)
• Policy 2: continue to screen in primary setting, i.e. no follow-up screen, until treatable retinopathy detected
The screening structure PDR or CSMO + New arrivals BDR or DMO + Yes No Primary screen Stop screening Treated for both PDR and CSMO?
OP1 + = Positive test result
= Negative test result
+ Follow-up screen Treatment OP2 +
Screening Policy 1
Screening policy 2
+ New arrivals Yes No Stop screening Treated for both PDR and CSMO?+ = Positive test result = Negative test result
+ = Positive test result = Negative test result
+
Treatment OP
3 6 1 (98, 100) Assumed gold standard (ophthalmologist, 6 12 2 (61, 85) Mobile camera (1 photo, reviewed by diabetologist) 6 12 1 (52, 84) GP fundoscopy 6 12 1 (81, 95) Diabetologist fundoscopy 6 12 1 (73, 93) Optometrist fundoscopy Screening interval after detection of retinopathy Initial screening interval (months) Policy (Sensitivity, specificity) (%)
Costs in Retinopathy study
Ophthalmic clinic £53 Other OP clinic £52 Optician visit £29 GP visit £28 Mobile van £16Experimentation
• Need adequate replications: we did 500 • Need to cover key scenarios e.g. vary
screening interval, screening age, method, compliance etc.
• Vary more than one together • Test with different populations
1479 184 392 58 588 92 Total costs (£000) 163.6 29.9 143.8 28.0 150.7 28.2 Courses of treatment for PDR 156.2 16.0 137.8 14.6 143.7 15.0 Courses of treatment for CSMO 24052 2986 1504 308 6739 1253 Ophthalmology OP visits 0 0 9142 1190 3941 202
Visits to primary screener
22.7 6.1 19.7 5.7 20.6 5.8
Patients saved from blindness 138.2 46.9 116.9 41.1 125.1 43.1
Years of sight saved
NIDDM IDDM NIDDM IDDM NIDDM IDDM Gold standard Mobile camera (Policy 2) Optometrist (Policy 1) Average annual results
Results (i)
Results (ii)
• Standard methods of screening save up to 50% of the potential sight years lost.
• They give up to 85% of the sight years saved by an idealised gold standard
programme using mydriatic 7-field photography reported by an
Results (iii)
• The mobile camera, used for annual
screening and six month follow-up after the detection of background retinopathy, had an estimated cost of £449,200 per year with £2,842 per sight year saved.
• It is less efficient to screen Type 2, rather than Type 1 diabetes mellitus patients, but they contributed to almost three quarters of
Years of sight saved per year in a district of 250,000: Policy 1, 12m/6m 0 10 20 30 40 50 60 70 NIDDM IDDM
Cost per year of sight saved: Policy 1, 12m/6m £1,000 £2,000 £3,000 £4,000 £5,000 £6,000 £7,000 IDDM NIDDM
120 125 130 135 140 145 150 155 160 165 170 12m/6m 24m/6m 12m/6m 24m/6m 12m/12m 24m/12m 24m/24m Intervals Y e ar s o f si g h t saved p e r year Policy 1 Policy 2
The effects of screening sensitivity
and patient compliance
130 135 140 145 150 155 160 165 170 ear s of s ight s av ed per year Policy 1, 12m/6m Policy 2, 12m/6m Policy 1, 24m/6m Policy 2, 24m/6m Policy 2, 12m/12m Policy 2,
Different screening intervals
for the two policies
£ £100 £200 £300 £400 £500 £600 £700 £800 C o st s o f p ro g ra mme ( '000s) 120 140 160 Y ear s o f si g h t saved p e r year Policy 1 Policy 2
Total costs and years of sight saved
for the two policies
Effects of limiting the upper age of screening NIDDM patients in a population of 250,000, using Mobile Van
40 60 80 100 120 140 160 180
Conclusions
• Simulation models can aid policy-makers • Screening is clearly worthwhile
• Not much difference between methods as long as compliance is high (> 80%)
• Trade-offs between screening intervals, sensitivity and compliance
• Important to keep sensitivity > 60%
Further work
• Discounted cost-benefit analysis (work currently in progress)
• Increase in incidence of diabetes due to lifestyle changes (?)
• New screening methodologies
• Incorporate other complications of
References
1. Davies R, O’Keefe R M and Davies HTO (1993). Simplifying the modeling of multiple activities, multiple queuing and interruptions: a new low-level data structure. ACM
Transactions on Modeling and Computer Simulation 3: 332-346.
2. Davies R, Sullivan P and Canning CR (1996). Simulation of diabetic eye disease to compare screening policies. British Journal of Ophthalmology 80: 945-950
3. Brailsford SC, Davies R, Canning CR and Roderick P (1998). Evaluating screening policies for the early detection of retinopathy in patients with non-insulin-dependent diabetes. Health Care Management Science 1: 1-10
4. Brailsford SC, Davies R (1999). University of Southampton Website, Screening for
Diabetic Retinopathy. www.management.soton.ac.uk/retinopathy/
5. Davies R, Brailsford SC, Roderick PJ, Canning C, Crabbe D. (2000) Using simulation modelling for evaluating screening services for diabetic retinopathy. Journal of the
Operational Research Society 51:476-484
6. Davies R, Roderick PJ, Canning C, and Brailsford SC, (2002). The evaluation of screening policies for diabetic retinopathy using simulation. Diabetic Medicine (to