TeleOphta: Machine learning and image processing methods for teleophthalmology

Disponibleenlignesur

www.sciencedirect.com

IRBM34(2013)196–203

Original

article

TeleOphta:

Machine

learning

and

image

processing

methods

for

teleophthalmology

E.

Decencière

,

G.

Cazuguel

,

X.

Zhang

,

G.

Thibault

,

J.-C.

Klein

,

F.

Meyer

,

B.

Marcotegui

,

G.

Quellec

,

M.

Lamard

,

R.

Danno

,

D.

Elie

,

P.

Massin

,

Z.

Viktor

,

A.

Erginay

, B.

La¨y

,

A.

Chabouis

a_{Centerformathematicalmorphology,systemsandmathematicsdepartment,MINESParisTech,35,rueSt-Honoré,77300Fontainebleau,France} b_{Serviced’ophtalmologie,hôpitalLariboisière,AP–HP,2,rueAmbroise-Paré,75475Pariscedex10,France}

c_{Directiondelapolitiquemédicale,parcoursdespatientsetorganisationsmédicalesinnovantes–télémédecine,Assistancepublique–HôpitauxdeParis,3,}

avenueVictoria,75184Pariscedex04,France

d_{InsermUMR1101LaTIM,bâtimentI,CHRUMorvan,29200Brest,France} e_{ADCIS,3,rueMartin-Luther-King,14280Saint-Contest,France} f_{TélécomBretagne,InstitutMines-Télécom,ITIDepartment,29200Brest,France} g_{UniversitédeBrest,InsermUMR1101LaTIM,SFRScInBioS,29200Brest,France}

Received21January2013;receivedinrevisedform21January2013;accepted21January2013 Availableonline15March2013

Abstract

Acompleteprototypefortheautomaticdetectionofnormalexaminationsonateleophthalmologynetworkfordiabeticretinopathyscreening ispresented.Thesystemcombinespathologicalpatternminingmethods,withspecificlesiondetectionmethods,toextractinformationfromthe images.Thisinformation,pluspatientandothercontextualdata,isusedbyaclassifiertocomputeanabnormalityrisk.Suchasystemshould reducetheburdenonreadersonteleophthalmologynetworks.

©2013ElsevierMassonSAS.Allrightsreserved.

1. Introduction

Diabetic retinopathy is one of the major causes of visual impairment and blindness in the working population. Early detectionofdiabetic retinopathyimproveshealingchancesor helpsstoppingorslowingdownitsprogress.Inordertoachieve early detection, national and international guidelines recom-mendannualeyescreeningfor alldiabetic patients.However, annualfundusexaminationisnotperformed sufficiently.One of the factors which explains this situation is the increasing number of diabetic patients (a worldwide phenomenon) and thedecreasingnumberofophthalmologists(atleastinFrance). Ascreeningprogrambytelemedicinecanimprovetheregular annualfundusexaminationofdiabeticpatients.

The first teleophthalmology networks, such as OPHDIAT [1,2],aredevotedtotheearlydetectionofretinalretinopathy.

∗_{Correspondingauthor.}

E-mailaddress:[email protected](X.Zhang).

Currently,theresultingfundusphotographsaremanuallygraded by ophthalmologists. However, ophthalmologists are already overwhelmed with work, and their numbers are decreasing. Giventheincreasing incidenceofdiabetes[3],newscreening centersareexpectedtojointheexistingtelemedicinenetworks, andnewnetworksshouldbecreated.Therefore,thissituation shouldnotimproveinanearfuture.Sotheinterpretationof fun-dusphotographswillrepresentahugeburden.Itshouldbenoted that,amongthepatientsscreenedeveryyear,about75%haveno diabeticretinopathyandonlyabout26%werereferredtoan oph-thalmologistforeitherdiabeticretinopathyorotherpathologies [1].Amethodtoautomaticallydetecthealthycasesin telemed-icalnetworkswouldfosterthedevelopmentanddeploymentof thesenetworks.

ItwasinthiscontextthatthepartnersofTeleOphtaproposed todevelopasystemtoclassifytheexaminationsacquiredina teleophthalmologynetwork,inordertoclassifythemintooneof twocategories:“normal”or“tobereferredtoaspecialist”.These examinationscontaincoloreyefundusimages,aswellas contex-tualdata,includingpatientdata.TheFrenchNationalResearch 1959-0318/$–seefrontmatter©2013ElsevierMassonSAS.Allrightsreserved.

Fig.1.IllustrationoftheTeleOphtastrategy.Patientrecordclassificationworkflow. Agency(ANR),throughitsTECSANprogram,acceptedtofund

theprojectin2009.

The TeleOphta project has benefited froma largeamount of data extracted from OPHDIAT. It has developed image processingandmachinelearningmethodstoclassifyeach exam-inationinto“normal”or“tobereferred”.Themainpathology thatistakenintoaccountisdiabeticretinopathy,buttheproject objectivesincludethedetectionofotherretinalpathologies.

Theobjectiveofthispaperistogiveageneraloverviewof TeleOphta.Itisnottheauthors’intentiontodelveintotechnical details.Moreover,eveniftheprojectdevelopmentsareintheir lastphase,somearenotcompletelyfinishedorvalidated. Read-ersinterestedbythedetailsshouldconsultpreviouslypublished articles,andstaytunedforfuturepublicationsbytheTeleOphta consortium.

2. Generalstrategy

Thedevelopmentofimageprocessingmethodsforthe analy-sisofeyefundusimageshasbeenaveryactiveresearchdomain forthelast15years.Naturally,thefirststudiesconcerned rela-tivelysmallandhomogeneousdatabases.However,inorderto bridgethegapbetweenacademicsolutionsandclinical applica-tions,especiallyintheframeworkofteleophthalmology,aneye fundusanalysismethodshouldbeabletodealwithalargevariety of images.Indeed, imagequalityandsize,acquisition condi-tions,amongothercharacteristics,canchangegreatlyfromone examinationtoanother.OneofthemajorchallengesTeleOphta hadtodealwithwasthisimageheterogeneity.

However,imagesarenottheonlysourceofinformationtaken intoaccountbythespecialisttoproduceadiagnostic.Contextual information,aspatientageordiabeteshistory,isalsoconsidered. Therefore,in the framework of TeleOphta, machinelearning methodshavebeenused tofusion informationobtained from theimages withcontextual information.This wasthe second majorchallengetheprojecthadtosolve.

Finally,evenifTeleOphtahasbeendesignedtoprocessdata acquiredonanetworkdevotedtodiabeticretinopathyscreening, the proposed solution should be able to detect all abnormal examinations,includingotherretinalpathologies.

Inordertorisetothesechallenges,theTeleOphtapartners proposedtocombinegenericimagesignatures,aswellas spe-cificlesiondetectionmethods, withclassificationmethods.In ordertodevelopandoptimize thesemethods,alargeamount ofanonymizeddatawasextractedfromtheOPHDIATnetwork. ThisgeneralstrategyisillustratedinFig.1.

3. Databases

Thee-ophthadatabaseresultsfromtheanonymized extrac-tionoftheexaminationsgatheredduringyears2008and2009 throughtheOPHDIATnetwork.Thisrepresents25,702 exami-nations,eachonecontainingatleastfourimages(twopereye) as wellas contextualdata,aspatientage, numberof yearsof diabeticretinopathy,etc.Finally,eachexaminationis accompa-niedbythe textannotationsoftheOPHDIATreader:diabetic retinopathy level for each eye, as wellas “coursetofollow” (suchasoneyearfollow-upexamination,orreferraltoan oph-thalmologist).Itshouldbenotedthattheimagesarecompressed withtheJPEGstandard,forpracticalreasonslinkedtonetwork bandwidth.Thewholedatabaseweights50GB.

Five hundred examinations were chosen from e-ophtha in order toproceed totwo supplementaryreadings. Thus, three expertopinionswillbeavailableforeachexaminationonthis database,callede-ophthaDR(“double-reading”).

Inordertodevelopthelesiondetectionmethods,lesionswere manuallyoutlinedonsomeimagesbyanophthalmologistusing softwaredevelopedbyADCIS(Annex).Theseannotationswere afterwardscheckedbyasecondophthalmologist.

Twosuchdatabasesshouldbemadeavailablebeforetheend oftheproject.Thefirst,e-ophthaEX,contains47imageswith 12,278exudates,aswellas35healthyimages.Severalimages

198 E.Decencièreetal./IRBM34(2013)196–203 amongthehealthyonescontainstructureswhichcaneasily

mis-leadexudatedetectionmethods,suchasreflections,andoptical artifacts.SomeimageexamplesareshowninFig.2.Thesecond of thesedatabases,callede-ophthaMA,contains 148images with1306microaneurysms,aswellas 233healthyimages.A thirdlesionsdatabase,containinghemorrhages,shouldbe avail-ableshortlyafterwards.

Theauthorsbelievethat thesedatabasesconstituteamajor contributionfor the researchcommunity. They gathera huge amountofpreciousinformation,usefulforthedevelopmentof automaticscreening methods. Wewill seefor instanceinthe followingsectionhowe-ophthaisusedtodesignpathological patternminingmethods.

4. Miningpathologicalpatternsinimages

Wepresentinthissectionageneralsolutiontoroughlydetect thesignsofDR,andofotherretinalpathologies.Theproposed solutionreliesonwavelet-basedimagecharacterizations devel-opedinpreviousworks[4].

Each image in a patient record is divided into patches and a vector of image characterizations, called signature, is extractedfromeachpatch.Amachinelearningalgorithmwas designed to recognize those signatures that only appear in pathological patientrecords [5]. Thisalgorithm relies onthe multiple-instancelearningparadigm.Inordertodetectvarious pathologicalpatterns,severalsizesofpatchesareused simulta-neously[6].Aglobalpathologicalindexisthenderivedforthe patientrecordasawhole:itcombineslocalpathologicalscores computedinimagepatchesindividually[7].Thealgorithm is summarizedinFig.3.Inordertopushtheclassification perfor-mancefurther,theshape ofthe waveletfiltersusedtoextract imagecharacterizationsistunedbyageneticalgorithm[8].

Notethatthissignature-baseddetectorisnotsupervisedby manualsegmentations.Instead,itissupervisedbythedecision attachedtopatientrecordsasawhole:whetherornotthepatient shouldbereferredtoanophthalmologist.Asubsetcontaining 800recordsfrome-ophthawasusedfortraining.

Thisgenericapproachallowsautomaticallydetecting patho-logical patterns in images. Therefore, we can expect that pathologieswhichareconvenientlyrepresentedinthedatabase willbedetectedwithahighprobability.Perse,thisapproach alreadyprovidesmeaningfulinformationtothepatientrecords classifier,resultinginaninterestingtrade-offbetweensensitivity andspecificity.However,fromascientificandapplicationpoint ofview,itwasinterestingtoseeifaddingspecificinformation, broughtbymoreclassicallesionsegmentationmethods,would improvetheperformanceofthesystem.

5. Specificlesiondetectionmethods

Thefinalobjectiveoftheimageprocessingmethods devel-opedintheframeworkofTeleOphtaistoextractmeaningfuldata fromtheimagesinordertoprovideinformationtothepatient recordsclassifier.Wegivebelowabriefdescriptionofthemain methods.

Fig.2.Imageexamples:a:imagefrome-ophthaEXcontainingexudates;b: detailof(a),showingmanuallydrawnexudatecontours;c:healthyimagefrom e-ophthaEX– noticereflectionsontheretina,whichcanmisleadexudates detec-tionmethods.

Fig.3.Miningpathologicalpatternsinimages. 5.1. Preprocessing

Preprocessingisparticularlyimportantforourapplication, duetotheclinicalnatureofthedatabase.

Giventheheterogeneityoftheimageswehavetodealwith,a specificspatialnormalizationmethodhasbeendeveloped[9,10]. Thismethodusesthefieldofview(FOV)of thefundus pho-tographsassizeinvariant(ahypothesiswhichhasproventobe robustenoughthankstothefactthatalltheimagesareacquired withthesamefieldofviewangle,i.e.40◦).Usingthissize invari-ant,allparametersrelatedtosizecanautomaticallybedefined, oncetheyhavebeenchosenforagivenimageresolution.Inthe diagramgiveninFig.4,theoutputofthismoduleiscalled“pixel size”.Thismustbeunderstoodasarelativepixelsize.

Finally,borderreflectionsaresegmented. Thesereflections appearas bright, moon-shaped regionsalong oneside of the fieldofview.Theycanperturbthedetectionofstructuressuch as the opticdisk.This segmentation hastwo steps.First, the residuebetweenthe originalbluechannel anditsresponse to alargemeanfilterleavesonlythemainbrightregions.Thena reconstructionfromtheborderoftheFOVgivesthereflection mask.AsshowninFig.4,thismoduletakesasinputnotonly theoriginalcolorimage,butalsothepixelsizeobtainedfrom apreviousstep.Itshouldbenotedthattakingintoaccountthis kindof artifactis, tothe limitof the authors’ knowledge,an originalcontribution.Indeed,untilnowimagesconsideredinthe literaturearerelatively“clean”,withoutopticalorotherartifacts, whicharefoundinpractice,andcanbeaserioussourceoffalse positivesornegatives.

Fig.4.Preprocessingstepsdiagram;oricorrespondstotheoriginalcolorimage oftheretina.

Oncethepreprocessingstepsare finished,wecanmoveto thedetectionofanatomicstructures.

5.2. Anatomicstructuresdetection

Weneedtodetectthemainanatomicalstructuresoftheretina inordertoimprovethedetectionoflesions.AsshowninFig.5, wefirstsegmentthemainvessels,thenwedetecttheopticdisk, andfinallythemacula.

Forourcurrentpurposes,weonlyneedaroughsegmentation oftheretinalvessels.Wedonot(yet)botheraboutsmallvessels. Theresultingvesselsmaskmightevencontainlesions,suchas hemorrhagesandmicroaneurysms.Later,thevesselsmaskwill beanalyzedtoextractfromitthosestructures.Fromthetime being,amorphologicalalternatesequentialfilterisusedtoerase vessels,aswellasotherstructuresfromtheoriginalimage.The positiveresidueofthisoperator(i.e.thepositivepartof ASF(f)-f,wherefisthegreenchanneloftheoriginalcolorimage,and ASFisthealternatesequentialfilter)isthresholdedtoobtainthe vessels,alongwithnoise.Thenalengthopeningremovessmall structuresandkeepsthemaskwearelookingfor.

TheOpticDisc(OD)isthebrighteststructureinnormaleye fundusimages.Besidesitshighintensity,theODisalsothemain convergencepointoftheretinalvesselnetwork.Thedetection starts witha thresholdto get somelarge brightstructures as ODcandidates.Thepreviouslycomputedvesselmaskisused toselecttheexactpositionandtocheckthepresenceoftheOD. Theskeleton,width,densityandorientationofthevesselmask arecomputed.TheODregionissupposedtocontainthehighest densityofwideverticalvessels.

In contrast,themaculaappearsmostof thetimeasadark region.ItsdistancetotheODisapproximatelyknown.Inorder tocombine thesetwo hypotheses, we thresholdthe imageto detectdarkregions,andthenkeeponlythosewhosedistanceto theODverifythesecondhypothesis.

200 E.Decencièreetal./IRBM34(2013)196–203

Fig.5.Anatomicstructuressegmentationanddetection.

5.3. Lesionsdetection

Anew exudatesdetection methodhas been developedfor heterogeneousdatabases[11].Itsmainoriginalityliesprobably onthefactthatitexplicitlydetectsartifactsorstructureswhich canbeerroneouslyconsideredas exudates.TheODand bor-derreflectionshavealreadybeendetectedinthepreprocessing phase.Inclinicalpractice,reflectionsinthemiddleofthevessels andalongthevesselsareverycommon.Brightartifactscaused bycameralensshouldalsobeconsidered.Wehaveshownthat thesaturationchanneloftheimageallowsdetectingthese arti-facts.Themorphologicalultimateopening[12]isthencombined withalocalvariance thresholdtoextract exudate candidates. Finally,about30characteristicsareextractedfromeach candi-datetotrainarandomforestmodel[13].Theclassifierassociates toeachcandidateariskvalue,whichcanbeinterpretedasthe probabilityofthecandidatebeingarealexudates.

Detectingmicroaneurysmsonthee-ophthaMAdatabasehas proventobeextremelydifficult.Basedonpreviousapproaches [14,15],we have simplifiedthe candidate detection step and shiftedmostoftheworktothefeature extractionstep,sothat atrainedclassifiermakesthefinaldecision.Thecandidatesare obtainedwiththesamealternatesequentialfilterasthevessel segmentation.Butthistime,onlysmallstructuresarekept. Fea-turesareextractedfromeachcandidate,bymeansofamax-tree [16]decomposition,includinglocal,geometricalandcontextual properties.Asfortheexudates,arandomforestclassifierisused tocomputethefinalriskassociatedtoeachcandidate.

Hemorrhagesdetectionisaverydifficultchallenge,seldom treatedintheliterature.Themaindifficultiescomefromtheir variableshape,sizeandcontrast.Theycanbealmostassmallas microaneurysmsoraslargeastheOD.Flame-hemorrhageshave the sameshapeas vessels.Wehaveproposedanewmethod, whichbegins by roughly segmentingthe vessel network and otherdarkstructures.Thisnewvesselmaskcontainsvesselsas wellas hemorrhages. Vesselswidth andorientationare com-putedontheskeletonofthevesselmask.Eachskeletonbranch (i.e.skeletonsection betweentwoconsecutiveskeleton bifur-cations) is labeled, while all bifurcation and end points are detected.Thus,usingthecontextualandgeometricalproperties ofthevesselnetwork,abnormalpointsaredetectedas hemor-rhagecandidates.Thefinalriskisagainobtainedwitharandom forestclassifier(Fig.6).

ItshouldbenotedthatintheframeworkofTeleOphta previ-ouslyexistingmicroaneurysmandexudatesdetectionmethods have also been used. Indeed, as stated before, the system is openandmodular,andthereforesupplementarylesiondetection methodscanbeeasilyadded.

The lesion detection methods presented until now focus on the most current lesions of diabetic retinopathy. What

aboutotherlesions,suchascotton-woolspots,neo-vessels,and intra-retinalmicrovascularabnormalities,whichareoftenvery importantfromthediagnosispointofview?Giventhegeneric pathologicalpatternminingapproachdescribedintheprecedent section,webelievethatmostoftheselesionswillbetakeninto accountinthefinaldecision.Indeed,wewillseethatthecurrent versionoftheTeleOphtasystemalreadyprovidesvery interest-ingresults.However,addingmorespecificdetectionmethodsis animprovementpaththatwearegoingtoexploreinthefuture.

6. Classifyingpatientrecords

Nowthatthevisualcontentofimageshasbeencharacterized, wepresenthowimagecharacterizationsarecombinedwith con-textualdata(age,weight,diabetestype,etc.)inordertodecide whetherornotapatientshouldbereferredtoan ophthalmolo-gist.Thisclassificationproblemhastwomainchallenges.First, we needtoprocessavaryingnumberof lesiondetectionsper patient record.Second, we needto processsparse contextual data.TheworkflowissummarizedinFig.1.

6.1. Lesion-basedpathologicalscore

Toaddressthefirstchallenge,wefirstcomputeasingle patho-logicalscoreperpatientrecordandperlesiondetector.Inthat purpose,thejointcumulativedistributionfunction(CDF)ofthe lesionprobabilitiesandofthelesionsizesisbuiltforeachpatient record.ThisCDFisthenmappedtoasinglepathologicalscore usingalineardiscriminantanalysis[17].Themappingprocess istunedtomaximizeclassificationperformanceinthetraining subsetofe-ophtha.

6.2. Heterogeneousinformationfusion

The second step consists in classifyinga sparse vector of heterogeneous descriptors: one pathological score per lesion detector, one signature-based pathological score, six quality metrics,uptoninedemographicinformationfieldsandupto18 diabetes-relatedinformationfields.Solutionsbasedondecision treesandtheDezert-Smarandachetheoryhavebeenproposedin previousworks[18].AnewsolutionbasedonApriori,themost popularalgorithmforassociationrulemining[19],wasproposed inthisstudy. Rulesassociating asubsetof descriptorswitha referraldecision(e.g.{70≤age<75,diabetes type=NIDDM, MAscore>0.75}→refertoophthalmologist)areminedinthe trainingsubsetofe-ophtha.Theirsensitivityandspecificityare measuredinthetrainingsubset.Toclassifyanewpatientrecord, alltherelevantassociation rulesareselected.Then, areferral decisionisinferredbasedonthe sensitivityandspecificity of the selectedrules. Thissolution isparticularly wellsuited to sparsedata.

7. Integrationandpreliminaryresults

Algorithms developed by both research groups were deliveredtoADCISforintegrationintothesamesoftware envi-ronment,andforvalidationon1,000examinations.Algorithms originallydevelopedinproprietarylanguagesandenvironments (i.e. C++, Python, Java) were rewrittenby the ADCIS engi-neering group to make them compatible with the company environment,andtobereusedinfutureprojects.

Thetrainingof informationfusionruleswasperformedon halfofthee-ophthadatabase,i.e.12,500examinations.Thetest wasperformed onthe otherhalfof the database.The system givesasfirstoutputa“Tobereferred”probability.Thresholding thisvalueleadstoareferraldecision.ThenaROCcurvewas computed toevaluate the automatic grading system(Fig. 7), withOPHDIATophthalmologistsdecision(tobereferredtoan ophthalmologistornot)asgroundtruth.Twomicronaneurysm detectionmethods, aswellas two exudate detectionmethods wereused,ontopofpatientdataandimagesignatures.

TheROC curveshows that for thesame sensitivityas the humanexpert(bluestar,sensitivityof80,9%),oursystemhas alowerspecificity(70% insteadof 81,5%).Butit meansthat itis possibletoremove70% of “Nottobe referred”patients fromthepatientsrecordssenttotheOPHDIATnetwork oph-thalmologists.Knowingthat theyrepresent74%of thecases, theproposedsystemwill beable,inthisexample,toremove about50%ofrecordswhichhavetobeactuallyexaminedinthe OPHDIATnetwork.Withasensitivityof90%,thisvaluegoes downto35%ofthetotalrecords,butitisstillveryinteresting: itallowstosave35%ofophthalmologisttime.

Nevertheless,althoughthisROCcurveshowstheabilityof thesystemtoclassifyexamsintwogroups(normalandtobe referred toa specialist), an additionalvalidation will be per-formedbythemedicalpartnerson1,000examsofyear2012to showtheeffectivenessofthedevelopedmethod.Thisvalidation iscurrentlyunderprogress.

Fig.7.ROCcurveobtainedonthee-ophthadatabase(inred).Patientdata,image signature,twoexudatedetectionmethods,andtwomicroaneurysmsdetection methodshavebeenusedtofeedthepatientrecordsclassifier.Thebluestar,in comparison,showsthesensitivity(80.9%)andspecificity(81.5%)obtainedby ahumanexpertonasubsetof500examsextractedrandomlyfromthee-ophtha databaseNotethatthishumanexpertwasnotexactlyinthesameconditionsof interpretationthanOPHDIATexperts.

Concerningfuture clinicaltrials,the medical partners will choose the system sensitivity, taking into account double-readingresultsonthe500 casesof thee-ophtha DRdatabase (inconstruction).

8. Conclusion

The TeleOphta project has developed a novel strategy to relievetheburdenonteleophthalmologynetworks,throughthe combinationofanoriginalpathologicalpatternminingmethod, withnewspecificlesiondetectionmethods.Thefinal classifica-tiontakesalsointoaccountcontextualinformation.

Oneofthemainresultsoftheprojectconsistsinthedatabases. Giventheirclinicalnature,theywillbringvaluabledatatothe researchcommunity,andshouldcontributetotheimprovement ofophthalmologyscreeningmethods.

Concerning imageprocessing methods, astrong emphasis hasbeen putonthe developmentofrobust methods,inorder totackle thechallenge of the heterogeneity of the databases. Theresultingexudatesandmicroaneurysmsdetectionmethods arebeing validatedontheprojectdatabases.The detectionof hemorrhagesrepresentsanevengreater challenge,giventheir potentialresemblancetobloodvessels.

Thesystemisopen,anditsperformancecouldbeimproved byaddingnewspecificlesiondetectionmethods,especiallyfor neo-vessels,intra-retinalmicrovascularabnormalitiesandother abnormalitiesofthevesselnetwork.Aprecisevessel segmenta-tionmethod,abletodetectevensmallvesselsonless-than-ideal qualityimageswillbenecessaryforthesetasks.

202 E.Decencièreetal./IRBM34(2013)196–203

Fig.8.OverviewoftheTeleOphtaannotationsoftware,inViewermode.Allimages,organizedinfolders,appearintheleft-handpane.Theselectedimageis displayedintheright-handpane,withannotationsinoverlay(severalexudatesappearinorange).

The different methods developed in the framework of TeleOphtahavebeingintegratedintoasinglesystem.Thefirst results showthat the performanceof thissystemshould lead toan interesting trade-offbetween sensitivityandspecificity. BasedontheobtainedROC curve,it willbeeasytotunethe finalsystemforclinicalevaluation.

Finally,theproposedsystemwillallowsaving ophthalmolo-gisttimeinscreeningnetworks,andthuswillhelprisingtonew populationhealthchallenges.

Acknowledgements

This work has been supported by the French National ResearchAgency(ANR)throughTECSANprogramme (ANR-09-TECS-015).

Annex. Annotationsoftware

An image annotation software was especially designed to facilitate the communication between ophthalmologists and imageprocessingresearchers,eveniftheyareremotelylocated. Usersofthisspecificsoftwarearedividedintwocategories.

Ontheonehand,theGrader(anophthalmologist)outlines objectsof interestor marksthembyacrossusingthe mouse pointer,andthenassignstheconsideredobjectstoaclassfroma pull-downlistofpredefinedclasses.Predefinedclassesconsisted of:microaneurysms,dot-hemorrhages(markedbyacross), blot-hemorrhages,flame-hemorrhages,exudates,cotton-woolspots, intra-retinalmicrovascularabnormalitiesandneo-vassels (out-lined).Optionally,commentsmaybeaddedtothewholeimage. A setof tools are available tohelp inthe diagnostic, and to

betterseesmalldetailsintheimageofinterest:zoomin/out, dis-play/undisplayexistingannotations,displaygreenimageband only,gammacorrection.

Onceanannotationorcommentissavedbythegrader,itis automaticallyuploadedtoaremoteserver.Ifdifferentgraders annotatethesameimage,eachgradercanonlyseeher/his anno-tations,toavoidanybias.Noprogrammingorimageprocessing skills are needed on Graders side. During the course of the project,theannotationsoftwarewasusedbyagraderfromthe AP–HPinParis.

Ontheotherhand,Viewers(imageprocessingresearchers) canseetheannotationsmadebyallGraders.Synchronizationof thesoftwarewiththeremotedatabaseishandledautomatically. Oncethedatabaseissynchronized,Viewerscanexport annota-tionstoTIFFfiles,soastoeasilycomparethemwithdetections producedbytheiralgorithms.

Toquicklyadapttothediversityofpossibleapplications,the annotationsoftwaredevelopedforTeleOphtagavebirthtoImage Annotator,afully-configurablesoftware thatisnowdelivered byADCIStoitscustomercommunity(Fig.8).

References

[1]MassinP, Chabouis A,ErginayA,Viens-BitkerC, Lecleire-ColletA, MeasT,etal.OPHDIAT:a telemedicalnetworkscreeningsystemfor diabeticretinopathy in the Île-de-France. DiabetesMetab 2008;34(3): 227–34.

[2]ErginayA,ChabouisA,Viens-BitkerC,RobertN,Lecleire-ColletA, MassinP.OPHDIAT:quality-assuranceprogrammeplanandperformance ofthenetwork.DiabetesMetab2008;34(3):235–42.

[3] WildS,RoglicG,GreenA,SicreeR,KingH.Globalprevalenceof dia-betes:estimatesfortheyear2000andprojectionsfor2030.DiabetesCare 2004;27(5):1047–53.

[4]QuellecG,LamardM,JosselinPM,CazuguelG,CochenerB,RouxC. Optimalwavelettransformforthedetectionofmicroaneurysmsinretina photographs.IEEETransMedImaging2008;27:1230–41.

[5]QuellecG,LamardM,CazuguelG,AbràmoffMD,CochenerB,RouxC. Weaklysupervisedclassificationofmedicalimages.In:ProcIEEEISBI. 2012,p.110–3.

[6]QuellecG,LamardM,CochenerB,DecencièreE,LayB,MassinP,Roux C,CazuguelG.AGeneralframeworkfordetectingdiabeticretinopathy lesionsineyefundusimages.In:ProcIntConfIEEECBMS.2012.p.1–6. [7] QuellecG,Lamard M,Abràmoff MD,DecencièreE, LayB, Erginay A,etal.Amultiple-instancelearningframeworkfordiabeticretinopathy screening.MedImageAnal2012;16(6):1228–40.

[8]QuellecG,LamardM,CazuguelG,CochenerB,RouxC.Wavelet Opti-mizationfor content-basedimage retrieval in medicaldatabases.Med ImageAnal2010;14:227–41.

[9]ZhangX,ThibaultG,DecencièreE,GuellecG,GazuguelG,ErginayA, etal.Spatialnormalizationofeyefundusimages.In:IEEEISBI.2012. [10]ZhangX,ThibaultG,DecencièreE.Procédédenormalisationd’échelle

d’imagesophtalmologiques.1253929(Patent)2012.

[11]ZhangX,ThibaultG,DecencièreE,GuellecG,DannoR,La¨yB,etal. AutomaticDetectionofColorRetinalImages.FortLauderdale,Florida, USA:ARVO;2012.

[12]BeucherS.Numericalresidues.In:ProcISMM.2005,p.23–32. [13]BreimanL.Randomforests.MachLearn2001;45(1):5–32.

[14]WalterT,MassinP,ErginayA,OrdonezR,JeulinC,KleinJC.Automatic detectionofmicroaneurysmsincolorfundusimages.MedImageAnal 2007;11:555–66.

[15]ZhangX,ThibaultG,DecencièreE. Applicationofthemorphological ultimateopeningtothedetectionofmicroaneurysmsoneyefundusimages fromaclinicaldatabase.ICS,Oct2011,Beijing,China.

[16] Salembier P, Oliveras A, Garrido L. Antiextensive connected opera-tors for image and sequence processing. IEEE Trans Image Process 1998;7(4):555–70.

[17] QuellecG,LamardM,CochenerB,DrouecheZ,LayB,ChabouisA,etal. Studyingdisagreementsamongretinalexpertsthroughimageanalysis.In: ProcIntConfIEEEEMBS.2012,p.5959–62.

[18]QuellecG,LamardM,CazuguelG,BekriL,DaccacheW,RouxC,etal. Automatedassessmentofdiabeticretinopathyseverityusingcontent-based imageretrievalinmultimodalfundusphotographs.InvestOphthalmolVis Sci2011;52:8342–8.

[19]AgrawalR,SrikantR.Fastalgorithmsforminingassociationrulesinlarge databases.In:ProcIntConfVLDB.1994,p.487–99.