The Evaluation of Screening Policies for Diabetic Retinopathy using Simulation

(1)

The Evaluation of Screening

Policies for Diabetic Retinopathy

using Simulation

R. Davies1_{, S.C. Brailsford}1_,

P.J. Roderick2 _{and C.R.Canning}3

(2)

Diabetes mellitus

• Common disease affecting 1 – 2% of population of UK

• Accounts for 4 – 5% of total health care expenditure in UK

• Exists in two forms: insulin-dependent (IDDM) and non-insulin-dependent (NIDDM)

– IDDM (type 1 or acute-onset) – usually diagnosed under age 21

– NIDDM (type 2 or older-onset) – usually diagnosed over age 40

(3)

IDDM (Type 1)

• IDDM accounts for ≈ 15% of all cases of diabetes

• Incidence in Europe varies: highest in north (e.g. Finland

≈ 43 cases per 100,000 per year) & lowest in south:

UK ≈ 17 cases per 100,000 per year)

• Onset is sudden: obvious symptoms (e.g. excessive thirst, tiredness, or even coma)

• No evidence of higher prevalence in UK ethnic minorities • Incidence in children may be increasing

(4)

NIDDM (Type 2)

• NIDDM accounts for ≈ 85% of all cases of diabetes

• Estimated equal numbers of diagnosed and undiagnosed cases

• About 2 million cases in the UK (diagnosed and undiagnosed)

• Prevalence of diagnosed NIDDM in UK estimated between 4 and 9 %

• Onset is insidious; may already have complications at diagnosis

• Prevalence of NIDDM is 2 to 4 times higher in Asians and Afro-Caribbeans

(5)

Diabetic retinopathy

• One of the most serious complications of

both types of diabetes: sight-threatening eye disease

• DR is the major cause of blindness in the developed countries

• Retinopathy can be detected before the patient is aware of any symptoms ….

(6)

Photograph of the retina,

(7)

What is diabetic retinopathy?

• Raised blood glucose levels cause

microvascular damage throughout the body • In the eye, some small retinal blood vessels

close off and others dilate : called

Background Diabetic Retinopathy (BDR) • Can lead to two types of disease

(8)

Diabetic Macular Oedema

• Swelling of vessels spreads to macula – area where best vision is concentrated • Can lead on to Clinically Significant MO

(CSMO), when it becomes sight-threatening • Most common in elderly: gradual onset, slow

(9)

Proliferative Retinopathy (PDR)

• Growth of new vessels off the back of the retina

• Causes visual loss through bleeding and scar tissue formation

• Sudden, catastrophic and total sight loss (usually due to a bleed)

(10)

Screening for DR: methods

• Fundoscopy: direct examination of the retina using an ophthalmoscope

• Photography: with or without mydriasis (eye drops to enlarge the pupil); various types of camera (Polaroid, digital, etc)

(11)

Screening for DR: settings

• In hospital

• At local doctor’s surgery

• In high-street optometrist shop • In mobile van

(12)

Screening for DR: screeners

• Family doctor (General Practitioner) • Diabetic specialist

• Eye specialist (ophthalmologist) • High-street optometrist

• Trained technician takes photographs,

subsequently reviewed by various grades of doctor

(13)

Screening issues

• By whom?

• By what technique? • Whom to target?

• How often, and where? • Cost-effectiveness

(14)

The research project

• Many different types of screening programme

currently in use: no general consensus about form of an ideal screening policy

• Project funded by UK National Health Service

• Our task: evaluate and compare range of screening policies, using discrete-event simulation

• Develop model of the natural history of diabetic retinopathy in a population, incorporating new cases as they are diagnosed

• Superimpose different screening policies on this baseline model and compare outcomes

(15)

Working with the model

• Work with clinicians and epidemiologists in collecting and analysing data and designing model

• Design interface for population, epidemiological and run time data

• Validate model against published data • Derive cost data

(16)

The simulation model(s)

• Population representing a UK Health District (approx 250,000)

• People classified by age, sex & ethnicity • Simulate 25 years, including new cases

• Data on incidence, prevalence, progression,

mortality, screening and treatment from published literature

• Investigate different screening policies, with associated costs

(17)

No retinopathy Background Diabetic Retinopathy (BDR) Proliferative Diabetic Retinopathy (PDR) Blindness from PDR No retinopathy Diabetic Macular Oedema (DMO) Clinically Significant Macular Oedema (CSMO) Blindness from CSMO NIDDM only

Disease progression

(18)

Discrete Event Simulation

• Models individuals moving through a system (queuing network)

• Characteristics influence progression

• Random numbers used to sample from distributions

• Can incorporate delays or interactions • Can incorporate constraints

(19)

Advantages of DES

• Flexible building blocks • Can use any distributions

• Individual variability and uncertainty

• Takes account of patients characteristics • Can take account of limited resource

(20)

Drawbacks of DES

• Greedy for computer memory and time consuming (declining problem!)

• Needs statistical techniques to analyse results

• Tempting to make models complex • Need OR and programming skills for

(21)

Modelling Problems

• Events happen to patients in different orders

• Groups of events may have different timescales • Different processes go on at the same time,

and interact, e.g. disease progression, death and screening

• Need to interrupt and reschedule processes: e.g. resample time of death, since the

(22)

POST - Patient oriented

simulation technique

• Davies, O’Keefe and Davies (1993)

• Can cope with simultaneous, interacting processes

• Entities can take part in several activities, and be in several queues, at the same

time

• One event can interrupt and change others

(23)

Three simultaneous processes

Central vision loss Severe vision loss PDR False +ve Background retinopathy Diagnose diabetes Natural history Screening PDR + DMO PDR + CSMO CSMO Policy 1 Screen for any retinopathy Detect background retinopathy

Further disease to detect

Death from any state

Detect and treat PDR or CSMO Visit ophthalmology clinic DMO

(24)

POST - Patient oriented

simulation technique

• Uses set of procedures and functions in Delphi • Patient can be set to take part in more than one

event - the earliest one takes precedence over the others

• Patients can queue while taking part in events

• Future events can be extracted from the calendar and terminated or changed and returned

(25)

Verification and Validation

• Does model progress smoothly over time?

• Do screening and treatment episodes accord with experience?

• Does prevalence of disease accords with research findings? Problems can arise if there is a mismatch with incidence data. • Is it recognisable to the experts?

(26)

0 0.2 0.4 0.6 0.8 1 Prevalence of retinopathy 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36

Years from diagnosis

Simulation WESDR Proliferative Retinopathy Any retinopathy

PDR

DMO

(27)

Screening policies

• Policy 1: initial screen in primary setting, with follow-up screening in hospital on detection of BDR (or worse)

• Policy 2: continue to screen in primary setting, i.e. no follow-up screen, until treatable retinopathy detected

(28)

The screening structure PDR or CSMO + New arrivals BDR or DMO + Yes No Primary screen Stop screening Treated for both PDR and CSMO?

OP1 ₊₌_{Positive test result}

= Negative test result

+ Follow-up screen Treatment OP2 +

Screening Policy 1

(29)

Screening policy 2

+ New arrivals Yes No Stop screening Treated for both PDR and CSMO?

+ = Positive test result = Negative test result

+

Treatment OP

(30)

3 6 1 (98, 100) Assumed gold standard (ophthalmologist, 6 12 2 (61, 85) Mobile camera (1 photo, reviewed by diabetologist) 6 12 1 (52, 84) GP fundoscopy 6 12 1 (81, 95) Diabetologist fundoscopy 6 12 1 (73, 93) Optometrist fundoscopy Screening interval after detection of retinopathy Initial screening interval (months) Policy (Sensitivity, specificity) (%)

(31)

Costs in Retinopathy study

Ophthalmic clinic £53 Other OP clinic £52 Optician visit £29 GP visit £28 Mobile van £16

(32)

Experimentation

• Need adequate replications: we did 500 • Need to cover key scenarios e.g. vary

screening interval, screening age, method, compliance etc.

• Vary more than one together • Test with different populations

(33)

1479 184 392 58 588 92 Total costs (£000) 163.6 29.9 143.8 28.0 150.7 28.2 Courses of treatment for PDR 156.2 16.0 137.8 14.6 143.7 15.0 Courses of treatment for CSMO 24052 2986 1504 308 6739 1253 Ophthalmology OP visits 0 0 9142 1190 3941 202

Visits to primary screener

22.7 6.1 19.7 5.7 20.6 5.8

Patients saved from blindness 138.2 46.9 116.9 41.1 125.1 43.1

Years of sight saved

NIDDM IDDM NIDDM IDDM NIDDM IDDM Gold standard Mobile camera (Policy 2) Optometrist (Policy 1) Average annual results

Results (i)

(34)

Results (ii)

• Standard methods of screening save up to 50% of the potential sight years lost.

• They give up to 85% of the sight years saved by an idealised gold standard

programme using mydriatic 7-field photography reported by an

(35)

Results (iii)

• The mobile camera, used for annual

screening and six month follow-up after the detection of background retinopathy, had an estimated cost of £449,200 per year with £2,842 per sight year saved.

• It is less efficient to screen Type 2, rather than Type 1 diabetes mellitus patients, but they contributed to almost three quarters of

(36)

Years of sight saved per year in a district of 250,000: Policy 1, 12m/6m 0 10 20 30 40 50 60 70 NIDDM IDDM

(37)

Cost per year of sight saved: Policy 1, 12m/6m £1,000 £2,000 £3,000 £4,000 £5,000 £6,000 £7,000 IDDM NIDDM

(38)

120 125 130 135 140 145 150 155 160 165 170 12m/6m 24m/6m 12m/6m 24m/6m 12m/12m 24m/12m 24m/24m Intervals Y e ar s o f si g h t saved p e r year Policy 1 Policy 2

The effects of screening sensitivity

and patient compliance

(39)

130 135 140 145 150 155 160 165 170 ear s of s ight s av ed per year Policy 1, 12m/6m Policy 2, 12m/6m Policy 1, 24m/6m Policy 2, 24m/6m Policy 2, 12m/12m Policy 2,

Different screening intervals

for the two policies

(40)

£ £100 £200 £300 £400 £500 £600 £700 £800 C o st s o f p ro g ra mme ( '000s) 120 140 160 Y ear s o f si g h t saved p e r year Policy 1 Policy 2

Total costs and years of sight saved

for the two policies

(41)

Effects of limiting the upper age of screening NIDDM patients in a population of 250,000, using Mobile Van

40 60 80 100 120 140 160 180

(42)

Conclusions

• Simulation models can aid policy-makers • Screening is clearly worthwhile

• Not much difference between methods as long as compliance is high (> 80%)

• Trade-offs between screening intervals, sensitivity and compliance

• Important to keep sensitivity > 60%

(43)

Further work

• Discounted cost-benefit analysis (work currently in progress)

• Increase in incidence of diabetes due to lifestyle changes (?)

• New screening methodologies

• Incorporate other complications of

(44)

References

1. Davies R, O’Keefe R M and Davies HTO (1993). Simplifying the modeling of multiple activities, multiple queuing and interruptions: a new low-level data structure. ACM

Transactions on Modeling and Computer Simulation 3: 332-346.

2. Davies R, Sullivan P and Canning CR (1996). Simulation of diabetic eye disease to compare screening policies. British Journal of Ophthalmology 80: 945-950

3. Brailsford SC, Davies R, Canning CR and Roderick P (1998). Evaluating screening policies for the early detection of retinopathy in patients with non-insulin-dependent diabetes. Health Care Management Science 1: 1-10

4. Brailsford SC, Davies R (1999). University of Southampton Website, Screening for

Diabetic Retinopathy. www.management.soton.ac.uk/retinopathy/

5. Davies R, Brailsford SC, Roderick PJ, Canning C, Crabbe D. (2000) Using simulation modelling for evaluating screening services for diabetic retinopathy. Journal of the

Operational Research Society 51:476-484

6. Davies R, Roderick PJ, Canning C, and Brailsford SC, (2002). The evaluation of screening policies for diabetic retinopathy using simulation. Diabetic Medicine (to