Digital Retinal Images and
Teleophthalmology for Detecting and
Grading Diabetic Retinopathy
FRANCISCOGO´MEZ-ULLA,MD1 MARIAI. FERNANDEZ,MD1 FRANCISCOGONZALEZ,MD1,2 PABLOREY,PHD3 MARTARODRIGUEZ,MD4 MARIAJ. RODRIGUEZ-CID,MD1 FELIPEF. CASANUEVA,MD5 MARIAA. TOME,MD5 JAVIERGARCIA-TOBIO,MD3 FRANCISCOGUDE,MD6
OBJECTIVE— Detecting and grading of diabetic retinopathy (DR) by means of digital retinal images sent via the Internet.
RESEARCH DESIGN AND METHODS— Four nonstereoscopic digital retinal images (45° field each) of 126 eye fundus images from 70 diabetic patients were obtained with a nonmydriatic camera at two peripheral units. The images were sent via the Internet using a web-based system to a reference center, where they were diagnosed and graded by one ophthal-mologist. These results were compared with those obtained by two other ophthalmologists, one at each peripheral unit, after direct examination of the patients. A modified severity scale of Airlie House was used for grading DR in all cases. Agreement between observers was assessed using unweightedfor categorical data and the intraclass correlation coefficient (ICC) for continuous data.
RESULTS— Presence of DR was detected in 69 eyes (55%). All eyes with DR (69 of 69, 100%) were correctly identified ( ⫽1) by inspecting the digital images. In 118 eyes (118 of 126, 94%), 57 with no DR and 61 with DR, there was an agreement between the gradation made after the direct examination and the gradation made after the inspection of the images (ICC⫽0.92). In eight eyes with DR (8 of 126, 6%), there was disagreement in the grading made with both techniques.
CONCLUSIONS— Inspection of digital retinal images sent via the Internet allowed diag-nosis and grading of DR. The presence or absence of DR was correctly assessed by inspection of the images in all instances. We also found agreement, in most cases, between retinopathy gradation made from the images and the gradation made by direct examination of the eyes. These findings suggest that this technique is suitable for screening procedures.
Diabetes Care25:1384 –1389, 2002
T
he number of people worldwide who have diabetes is expected to in-crease to almost 300 million by the year 2025 (1). A primary objective of the St. Vincent Declaration was to achieve a significant reduction of blindness due to this disease (2). Early diagnosis of diabetic retinopathy (DR) before visual loss has occurred is essential to fulfil this goal (3– 5). DR is the leading cause of blindness and visual impairment in adults in devel-oped societies (6 –9). Approximately 2% of all individuals who have diabetes be-come blind after 15 years, and severe vi-sual impairment develops in ⬃10% of individuals with diabetes. Laser photo-coagulation in both the Diabetic Retinop-athy Study and the Early Treatment Diabetic Retinopathy Study (ETDRS) was beneficial in reducing the risk of severe visual loss from proliferative DR (PDR) and moderate visual loss (ETDRS) from clinically significant macular edema, but in general, it was not beneficial in revers-ing already diminished acuity. This pre-ventive effect and the fact that patients with PDR or macular edema may be asymptomatic provide strong support for a screening program to detect DR (10 – 13).Fundus examination of both type 1 and type 2 diabetic patients should be re-peated annually by an ophthalmologist, and these examinations will be required more frequently if retinopathy is pro-gressing (10). The recommendation of an annual examination to any diabetic pa-tient would cause an imposing surcharge in medical care. In the Galician Commu-nity, in which the estimated population is
⬃2,800,000 inhabitants, this would im-ply ⬃140,000 annual examinations. Moreover, there is an important geo-graphical dispersion of the target popula-tion (31,326 entities of populapopula-tion in an area of 29,434 km2), with a high
percent-age of rural settings having uneasy access to ophthalmologists. The demand of the population exceeds the possibility of ad-equate attention by the ocular diabetes units, and therefore, it causes long waiting
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● From the1
Ocular Diabetes and Medical Retina Unit, Division of Ophthalmology, Department of Surgery, School of Medicine, University of Santiago de Compostela and Complejo Hospitalario, Universitario de Santiago de Compostela, Santiago de Compostela, Spain; the2
Department of Physiology, School of Medi-cine, University of Santiago de Compostela and Complejo Hospitalario, Universitario de Santiago de Com-postela, Santiago de ComCom-postela, Spain; the3
Supercomputation Center of Galicia (CESGA), Santiago de Compostela, Spain; the4
Department of Ophthalmology, Hospital Meixoeiro, Vigo, Spain; the5
Division of Endocrinology, Department of Medicine, School of Medicine, University of Santiago de Compostela and Complejo Hospitalario, Universitario de Santiago de Compostela, Santiago de Compostela, Spain; and the 6
Clinical Epidemiology Unit, School of Medicine, University of Santiago de Compostela and Complejo Hospitalario, Universitario de Santiago de Compostela, Santiago de Compostela, Spain.
Address correspondence and reprint requests to Francisco Go´mez-Ulla, Department of Ophthalmology, Hospital Provincial de Conxo, C/Ramo´n Baltar s/n, E-15706, Santiago de Compostela, Spain. E-mail: ciulla@usc.es or fspaco@vsc.es.
Received for publication 16 October 2001 and accepted in revised form 1 February 2002.
Abbreviations:DR, diabetic retinopathy; ETDRS, Early Treatment Diabetic Retinopathy Study; HRC, high-risk characteristic; ICC, intraclass correlation coefficient; JPEG, Joint Photograph Experts Group; NDR, non-DR; NPDR, nonproliferative DR; PDR, proliferative DR.
A table elsewhere in this issue shows conventional and Syste`me International (SI) units and conversion factors for many substances.
lists for patients with severe affectation. It would then be useful to undertake screen-ing programs or develop techniques capa-ble of filtering the affected cases and diverting urgent conditions.
Screening is a simple diagnostic pro-cedure applied to a whole population at risk to detect lesions that should be fur-ther investigated and treated. It is not a complete clinical assessment but a method to identify patients at risk who will require further examination (14). Al-though there is consensus concerning the cost-effectiveness of screening (15–20), the standards by which screening pro-grams should be assessed have not been defined in the European protocols. An al-ternative method to the traditional exam-inations would be to obtain retinal images in situ and send them to a specialist for examination. Nonmydriatic fundus cam-eras allow the acquisition of high-quality digital images with no need for medically induced mydriasis. It has been shown that fundus photographs are more reliable than ophthalmoscopy in the diagnosis of diabetic retinal lesions (21–24). There-fore, the use of these cameras might be considered for screening, diagnosis, and grading of DR (25–27). Telemedicine is an emerging application of telecommuni-cation technology to provide health care services in regions where access to physi-cians may be difficult (28 –31). Digital fundus photographs have the advantage that they can be obtained at minimal cost and inconvenience for the patient (26,32–34) and can be transmitted through the Internet to distant experts (35,36). Although these technologies may eventually allow retinopathy screening via telemedicine, there is no consensus in the methodology and validation of an ap-propriate system. The role of the ophthal-mologist might be assessment and/or treatment of the patients after the screen-ing made by other professionals.
The Fundusnet project was an inno-vative telemedicine project supported by the Spanish Agency Commission Inter-ministerial de Ciencia y Tecnologı´a from 1998 to 2000. The aim of the present study was to test a teleconsultation sys-tem, developed within the frame of the Fundusnet project, intended for detecting and grading DR.
RESEARCH DESIGN AND METHODS— A total of 140 eyes of 70 consecutive diabetic patients
attend-ing either an endocrinology unit or an ophthalmology unit from January to March 2000 were initially included in this study. Patient inclusion was made regard-less of age, sex, ethnicity, socioeconomic status, or any known ocular disease.
Digital eye fundus images were ob-tained by a trained technician at two pe-ripheral units (the endocrinology unit at the Conxo Hospital and the ophthalmol-ogy unit at the Meixoeiro Hospital) and electronically sent via the Internet to the Supercomputation Center of Galicia (CESGA) for storage. The images were then downloaded by one ophthalmologist at the reference hospital (Ophthalmology Unit at the Complejo Hospitalario Uni-versitario de Santiago de Compostela [CHUS]). A web-based software program was used to transfer the images. This arrangement is shown schematically in Fig. 1.
The ophthalmologist at the reference center inspected the downloaded digital images, evaluated them for the presence or absence of DR, and graded the retinop-athy using a modified version of the Airlie House classification (37), as indicated be-low. Similarly, one ophthalmologist at each peripheral unit also evaluated the images for the presence or absence of DR and graded the retinopathy by direct ex-amination of the eye fundus of the pa-tients. Direct examination was performed under mydriasis, a slit lamp, and a 90-diopter lens. The ophthalmologists were not aware of the grading performed by the others.
The digital images were obtained with a nonmydriatic funduscopic camera (model CR5-45NM; Canon, Amstelveen, the Netherlands). A video camera (model DXC950; Sony, Tokyo, Japan) attached to the funduscopic camera and a frame
grab-Figure 1—Four 45° field, nonstereoscopic digital fundus images of each eye were obtained with a nonmydriatic camera from diabetic patients at peripheral units. The images were sent through the Internet for storage to a server based on the Active Server Pages technology to provide a secure environment. An ophthalmologist graded the stored digital retinal images at the reference center and made a diagnosis. These results were compared with those obtained by the ophthalmologists by direct examination of the patients at the peripheral units.
ber (model MV-300; Teac Deutschland, Wiesbaden-Erbenheim, Germany) were used to digitize the images. Various com-pression settings balanced resolution and file size using the Joint Photograph Ex-perts Group (JPEG) compression method. Typically, the final mean file size of a single image after compression was
⬃140 KB, which resulted after perform-ing a 1:3 compression ratio. Four images were taken of each eye, which covered the macula, the nasal retina and optic disc, and the superior and inferior temporal areas.
The images were then transferred to a conventional personal computer and sent via the Internet for storage in a server based on the Active Server Pages technol-ogy to provide a secure environment. The stored images were then downloaded by the ophthalmologist at the reference cen-ter and visualized on a 17-inch conven-tional computer monitor with a video board set at 24 bits per pixel. To improve image analysis, a sharpen filter was ap-plied in some instances using commercial image processing software (Photoshop; Adobe, Seattle, WA). Contrast, bright-ness, and zoom facilities were also used for enhancement of the images whenever necessary.
According to a modified version of the Airlie House classification (37), eight grades of severity were established: non-DR (NDR), minimal nonproliferative DR (NPDR), moderate NPDR, severe NPDR, very severe NPDR, PDR without high-risk characteristics (HRCs), PDR with HRCs, and advanced PDR. Because the presence of retinal thickening with no hard exudates was difficult to detect in
our nonstereoscopic digital images, grad-ing of maculopathy was based on the presence of exudates in the macular re-gion. The existence of hard exudates at
⬍500m from the fovea was considered macular edema. No distinction was made between small hemorrhages and micro-aneurysms, because this difference does not modify the classification. Notes on other additional examinations that were recommended (fluorescein angiography, laser photocoagulation) to the patient were recorded for treatment purposes but were not considered in this study.
Comparisons were made between stages graded by both ophthalmologists at the peripheral by direct examination and those made by the ophthalmologist at the reference center by inspecting the digital images. Agreement between the existence or absence of DR was analyzed by using unweighted kappa () for categorical data (38), where ⫽0 defines no correlation and ⫽1 defines total correlation. The interclass coefficient correlation (ICC) (38) was used to determine the level of agreement on the stage of DR.
RESULTS— In seven eyes (7 of 140, 5%), no attempt was made to obtain any fundus image because of the existence of an opaque cataract. A total of 532 digital retinal images from 133 remaining eyes were analyzed in this study. A total of 7 of these 133 eyes (5%) were excluded be-cause the quality of the images did not allow any grading or detection of retinop-athy. The causes of poor image quality in these seven eyes were media opacities (vitreous hemorrhage in one, lens opacity in three) or small size of the pupil (three
eyes). Therefore, the final data set of this study was 126 eyes from 70 diabetic pa-tients. The results obtained in this study are summarized in Table 1.
After inspecting the images, diabetic lesions were not observed in 57 eyes (57 of 126, 45%) and DR was diagnosed in 69 eyes (69 of 126, 55%). Similar figures were also obtained after direct examina-tion of the eye fundus. Therefore, there was an agreement of 100% ( ⫽1) on the presence or absence of DR between both techniques (100% of sensitivity and spec-ificity).
The Airlie House classification of the 126 eyes made after direct examination showed 57 eyes with NDR (45%), 28 with minimal NPDR (22%), 28 with moderate NPDR (22%), 7 with severe NPDR (6%), 5 with PDR without HRCs (4%), and 1 with PDR with HRCs (0.7%). After in-spection of the digital images, 57 eyes were classified as NDR (45%), 29 as min-imal NPDR (23%), 30 as moderate NPDR (24%), 9 as severe NPDR (7%), and 1 as PDR with HRCs (0.7%).
Agreement between the results ob-tained from retinal images and those from direct examination was found in 118 eyes (118 of 126, 94%), 57 with NDR and 61 with several degrees of DR. There was dis-agreement in the remaining eight eyes (8 of 126, 6%). The level of agreement be-tween both techniques, as determined by the ICC, was 0.92 (95% CI 0.90 – 0.95). Disagreement in the grading of five eyes occurred because intraretinal neovessels were not seen in the digital images due to an important hemorrhagic component. Four of these five cases were classified as severe NPDR, and one case was classified
Table 1—Comparison of clinical level of DR assessment from digital retinal images and from direct examination of the eye fundus
Stage by digital images
Direct examination stage NDR Minimal NPDR Moderate NPDR Severe NPDR Very severe NPDR PDR without HRCs PDR with HRCs Advanced PDR Total eyes NDR 57 57 Minimal NPDR 28 1 29 Moderate NPDR 27 2 1 30 Severe NPDR 5 4 9 Very severe NPDR 0 PDR without HRCs 0 PDR with HRCs 1 1 Advanced PDR 0 Total eyes 57 28 28 7 0 5 1 0 126
Data aren. There was complete agreement between the results obtained by examination of the digital retinal images and direct examination of the eye fundus in 118 eyes (118 of 126, 94%). There was disagreement in the remaining eight eyes (8 of 126, 6%).
as moderate NPDR based on the digital images, whereas all of them were classi-fied as PDR without HRCs after direct ex-amination. Two cases classified as severe NPDR by direct examination were classi-fied as moderate NPDR based on the dig-ital image. The remaining case classified as moderate NPDR by direct examination was considered as minimal NPDR based on the digital image.
CONCLUSIONS— In diseases in which diagnosis is based mainly on an image, as in DR, the contribution of new imaging technologies is essential. The possibility of using nonmydriatic cameras and sending digital images through the Internet allows interaction between the different health care professionals exam-ining diabetic patients. Incorporation of these new technologies in the field of te-leophthalmology presents a wide range of possibilities in a time when specialized examination is exceeded and when there is a demand for improving the quality of medical care in a diabetic population with a high probability of increasing in number during the next quarter of the century.
The main goals when screening for DR are the detection of the first signs of early retinopathy to evaluate the progres-sion of retinopathy and, above all, to de-tect severe treatment-requiring lesions. With early identification, prompt incor-poration into the health care system, ad-equate education of the patient, regular lifelong evaluation, appropriate referral, and timely treatment, the vast majority of severe visual loss can be prevented. We have developed and tested a teleophthal-mological system that fulfils these re-quirements.
The JPEG file type is widely used for transferring images on the Internet be-cause they have a small size and, there-fore, can be transferred rapidly from one computer to another and do not take up much storage space (36). However, one problem with JPEG files is finding the highest compression of the image that preserves adequate definition of the le-sions. We used a compression ratio of 1:3, which produced files of ⬃140 KB. This compression ratio allowed enough defini-tion without significant impairing of the transmission of the image through the network. It has been shown that when the original retinal image size of 1.5 MB is reduced to 29 KB (compression ratio 1:52), using JPEG compression, there
was no serious degradation in image qual-ity and that this compression method is an excellent alternative to reducing the image file size (39).
Our study shows that the use of digi-tal retinal images transmitted through the Internet may be highly effective for screening of DR. We found that the study of four retinal fields was enough to dis-card or confirm the presence of diabetic lesions in all eyes studied. The validation we made demonstrates that the detection of DR using nonmydriatic digital color retinal images taken with a 45° fundus-copic camera is in substantial agreement with direct examination of the eye fundus ( ⫽1). For continuous data, one mea-sure of reliability agreement is the ICC, as described by Bartko (38). Aor ICC of
ⱖ0.81 was defined as excellent agree-ment (40). Our analysis showed that there was excellent agreement (ICC⫽0.92) be-tween the clinical level of DR assessed from the undilated digital image and di-rect examination of the eye.
The assignment of a clinical level of DR represents an integration of the pres-ence and location of different lesions and the relative severity of these lesions. Based on the known risks for progression from each level of DR, a referral frequency with respect to repeat digital retinal images, ophthalmic evaluation, or both can be recommended. In our study, we found that 57 eyes (57 of 126, 45%) did not show any retinal lesion, and therefore, these patients would be advised to repeat photography in 1 year, thus reducing the waiting list for a specialized examination. Teleconsultation could reduce referral to ophthalmology specialists, which is im-portant in countries with public health systems covering a large number of people.
Approximately 50% of the eyes exam-ined had DR lesions (69 of 126, 55%). In those eyes, instead of making a detailed account of retinal lesions, we made a clas-sification (Airlie House clasclas-sification) by stages (37). This has the advantage that it indicates whether a patient should be re-ferred for specialized ophthalmic evalua-tion. This technique does not replace the need for comprehensive eye examinations but detects patients who require prompt referral to specialty ophthalmology ser-vices. To classify the retinopathy, we made no distinction between microaneu-rysms and small hemorrhages, because they did not imply differences in the
stage. In our study, eight eyes (8 of 126, 6%) were infraclassified in the digital im-ages. The reason was that it was difficult to differentiate among new vessels, large intraretinal hemorrhages, and intraretinal microvascular abnormalities in those DR forms with high hemorrhagic compo-nents.
There were no difficulties in reaching an agreement between image analysis and direct examination up to the severe NPDR stage. However, the proliferative forms, specially made with intraretinal new ves-sels and high hemorrhagic components, presented some difficulties for classifica-tion. This problem can be addressed by using more sophisticated equipment that will gradually obtain digital images of higher quality. However, because part of our patient sample was taken from a gen-eral patient pool of an endocrinology clinic that included only a few patients with PDR, we cannot conclude that the technique is not valid for visualizing intraretinal new vessels. In all cases in which disagreement was observed, the analysis of the digital images underesti-mated the retinopathy when compared with the gradation made by direct exam-ination. We believe this may be explained by the fact that high-grade DR has image features that may be lost when inspection of the digital images is made. In some in-stances, manipulation of the image with a sharpen filter (in our case, the filter in-cluded in the Adobe Photoshop program) allowed us as well as other authors (41) to improve the diagnosis.
We were able to perform effective de-tection of DR and derive only the serious cases with a wide margin of confidence, avoiding specialized examinations for the less severe forms. A challenge for this sys-tem, however, was associated with media opacities or the small pupil size found in some eyes of patients with long-term dia-betes or significant retinal disease. In seven eyes in our sample, we did not even attempt to obtain images, and in another seven cases, although images were ob-tained, they did not allow detection or grading of DR because of their poor qual-ity caused by the problems mentioned above.
Due to the initial difficulty in distin-guishing retinal thickening without hard exudates in our nonstereoscopic digital images, we considered that all eyes with hard exudates⬍500m from the fovea had macular edema. However, because
patients with macular edema have impor-tant reduction of visual acuity, a combi-nation of eye fundus image and visual acuity assessment would help improve the accuracy of the technique. Moreover, use of the Amsler test would improve de-tection of macular edema in the context of telemedical screening. It is also possible that, in the near future, stereoscopic im-ages may be available to solve this prob-lem. A recent report shows that it is feasible to obtain such images with a non-mydriatic funduscopic system similar to the one we have used (42).
In summary, we demonstrated that sending digital retinal images via the In-ternet is a suitable method for detecting and grading DR in a nonselected diabetic population. However, it has some limita-tions for diagnosis of macular edema with no hard exudates and for diagnosis of vas-cular lesions in hemorrhagic forms of DR. In these cases, a study of longer series would be needed to evaluate the degree of specificity of the method.
Our results show good agreement be-tween nonmydriatic digital retinal images and dilated direct examinations and sug-gest that the digital technique we have used may be an effective telemedicine tool for remote grading of DR. It is also useful for suggesting when the next retinal eval-uation should be made and for determin-ing the need for prompt referral to an ophthalmologist. The use of this system, however, should not be regarded as a par-adigm that would replace the need for comprehensive eye examinations.
The use of teleophthalmology among health professionals may represent an ef-fective alternative for treating diabetic pa-tients at low cost, with high possibilities for expansion and adaptability (43). Moreover, hospital diabetes specialists, ophthalmologists, and primary care doc-tors could readily share the digital retinal images of their patients, which will cer-tainly improve the treatment of the dia-betic patient.
Acknowledgments— This work was par-tially supported by grants from FIS (99/0567), D G S I C ( P B 9 7 - 0 5 2 1 ) , a n d X U G A (00PXI90202PR). The Fundusnet project (TL-980372) was supported by the Spanish agency CICYT.
We thank Teresa Lord for her help in this study.
References
1. King H, Aubert RE, Herman WH: Global burden of diabetes, 1995–2025: preva-lence, numerical estimates, and projec-tions.Diabetes Care21:1414 –1431, 1998 2. World Health Organization/International Diabetes Foundation Europe: Diabetes care and research in Europe: the Saint Vincent declaration. Diabet Med 7:360, 1990
3. Kohner EM, Porta M: Protocols for screening and treatment of diabetic reti-nopathy in Europe. Eur J Ophthalmol 1:45–54, 1990
4. Gibbins RL, Owens DR, Allen JC, East-man L: Practical application of the Euro-pean Field Guide in screening for diabetic retinopathy by using ophthalmoscopy and 35 mm retinal slides.Diabetologia41: 59 – 64, 1998
5. Ba¨cklund LB, Algvere PV, Rosenquist U: New blindness in diabetes reduced by more than one-third in Stockholm County.Diabet Med14:732–740, 1997 6. National Diabetes Data Group:Diabetes in
America. 2nd ed. Bethesda, MD, National Institutes of Health, National Institute of Diabetes and Digestive and Kidney Dis-eases, 1995
7. Government Statistical Service:Causes of Blindness and Partial Sight Among Adults in 1976/77 and 1980/81 in England and Wales. London, HMSO, 1988
8. Klein R, Klein BE, Moss SE: Visual impair-ment in diabetes.Ophthalmology91:1–9, 1984
9. Ferris F, Patz A: Macular edema: a com-plication of diabetic retinopathy. Surv Ophthalmol28:452– 461, 1984
10. American Diabetes Association: Diabetic retinopathy. Diabetes Care 21:157–159, 1998
11. Diabetic Retinopathy Study Research Group: Photocoagulation treatment of proliferative diabetic retinopathy: the sec-ond report of diabetic retinopathy study findings. Ophthalmology 85:82–106, 1978
12. Early Treatment Diabetic Retinopathy Study Research Group: Photocoagulation for diabetic macular edema: ETDRS re-port number 1. Arch Ophthalmol 103:1796 –1806, 1985
13. Schachat AP: The debate over diabetic ret-inopathy screening and follow-up. In Controversies in Diabetic Retinopathy: Sub-specialty Day 2000 Retina. Dallas, TX, AAO, 2000
14. Retinopathy Working Party: A protocol for screening for diabetic retinopathy in Europe.Diabet Med8:263–267, 1991 15. Javitt JC, Aiello LP: Cost-effectiveness of
detecting and treating diabetic retinopa-thy.Ann Intern Med124:164 –169, 1996 16. Dasbach E, Fryback D, Newcomb PA,
Klein R, Klien BEK: Cost-effectiveness of strategies for detecting diabetic retinopa-thy.Med Care29:20 –39, 1991
17. Javitt JC, Aiello LP, Chiang Y, Ferris FL, Canner JK, Greenfield S: Preventive eye care in people with diabetes is cost-saving to the federal government: implications for health-care reform.Diabetes Care17: 909 –917, 1994
18. Aiello LP, Gardner TW, King GL, Blan-kenship G, Cavallerano JD, Ferris FL 3rd, Klein R: Diabetic retinopathy. Diabetes Care21:143–156, 1998
19. Porta M, Kohner E: Screening for diabetic retinopathy in Europe.Diabet Med8:197– 198, 1991
20. Javitt JC: Cost savings associated with de-tection and treatment of diabetic eye dis-ease.Pharmacoeconomics8 (Suppl. 1):33– 39, 1995
21. Kinyoun JL, Martin DC, Fujimoto WY, Leonetti DL: Ophthalmoscopy versus fundus photographs for detecting and grading diabetic retinopathy.Invest Oph-thalmol Vis Sci33:1888 –1893, 1992 22. Harding SP, Broadbent DM, Neoh C,
White MC, Vora J: Sensitivity and speci-ficity of photography and direct ophthal-moscopy in screening for sight threatening eye disease: the Liverpool Di-abetic Eye Study.BMJ 311:1131–1135, 1995
23. von Wendt G, Heikkila¨ K, Summanen P: Assessment of diabetic retinopathy using two-field 60 degrees fundus photogra-phy: a comparison between red-free, black-and-white prints and colour trans-parencies. Acta Ophthalmol Scand 77:638 – 647, 1999
24. von Wendt GC, Ro¨nnholm P, Heikkila¨ K, Summmanen P: A comparison between one- and two-field 60 degree fundus photography when screening for diabetic retinopathy. Acta Ophthalmol Scand 78: 14 –20, 2000
25. Taylor R: Practical community screening for diabetic retinopathy using the mobile retinal camera: report of a 12 centre study.Diabet Med13:946 –952, 1996 26. George LD, Halliwell M, Hill R, Aldington
SJ, Lusty J, Dunstan F, Owens DR: A com-parison of digital retinal images and 35 mm colour transparencies in detecting and grading diabetic retinopathy.Diabet Med15:250 –253, 1998
27. Liesenfeld B, Kohner E, Piehlmeier W, Al-dington S, Porta M, Bek T, Obermier M, Mayer H, Mann G, Holle R, Hepp K-D: A telemedical approach to the screening of diabetic retinopathy: digital fundus pho-tography. Diabetes Care 23:345–348, 2000
28. Marcus DM, Brooks SE, Ulrich LD, Bassi FH, Laird M, Johnson M, Newman C: Telemedicine diagnosis of eye disorders by direct ophthalmoscopy: a pilot study.
Ophthalmology105:1907–1914, 1998 29. Sanders JH, Bashshur RL: Challenges to
the implementation of telemedicine. Telemed J1:115–123, 1995
30. Li HK: Telemedicine and ophthalmology. Surv Ophthalmol44:61–72, 1999 31. Lee P, Durham NC: Telemedicine:
oppor-tunities and challenges for the remote care of diabetic retinopathy.Arch Ophthalmol 117:1639 –1640, 1999
32. Ryder RE, Kong N, Bates AS, Sim J, Welch J, Kritzinger EE: Instant electronic imag-ing systems are superior to Polaroid at detecting sight-threatening diabetic reti-nopathy.Diabet Med15:254 –258, 1998 33. Kerr D, Cavan DA, Jennings B,
Dunning-ton C, Gold D, Crick M: Beyond retinal screening: digital imaging in the assess-ment and follow-up of patients with dia-betic retinopathy. Diabet Med 15: 878 – 882, 1998
34. Young S, George LD, Lusty J, Owens DR: A new screening tool for diabetic retinop-athy: the Canon CR5 45NM retinal
cam-era with Frost Medical Software RIS-Lite digital imaging system. J Audiov Media Med20:11–14, 1997
35. Williamson TH, Keating D: Telemedicine and computers in diabetic retinopathy screening.Br J Ophthalmol82:5– 6, 1998 36. Ryder REJ: Screening for diabetic retinop-athy in the 21st century.Diabet Med15: 721–722, 1998
37. Bonafonte S, Garcia CA, Davis MD: Clasificacio´n de la retinopatı´a diabe´tica. InRetinopatı´a Diabe´tica. 1st ed. Bonafonte S, Garcia CA, Eds. Madrid, Spain, Har-court Brace de Espan˜a S.A., 1996, p. 63–90
38. Bartko JJ: The intraclass correlation coef-ficient as a measure of reliability.Psychol Rep19:3–11, 1966
39. Eikelboom RH, Yogesan K, Barry CJ, Con-stable IJ, Tay-Kearney M-L, Jitskaia L, House PH: Methods and limits of digital image compression of retinal images for telemedicine.Invest Ophthalmol Vis Sci41: 1916 –1924, 2000
40. Landis JR, Koch GC: The measurement of observer agreement for categorical data. Biometrics33:159 –174, 1977
41. George LD, Lusty J, Owens DR, Ollerton RL: Effect of software manipulation (Pho-toshop) of digitised retinal images on the grading of diabetic retinopathy.Br J Oph-thalmol83:911–913, 1999
42. Bursell S-E, Cavallerano JD, Cavallerano AA, Clermont AC, Birkmire-Peters D, Ai-ello LM, Joslin Vision Network Research Team: Stereo nonmydriatic digital-video color retinal imaging compared with Early Treatment Diabetic Retinopathy Study seven standard field 35-mm stereo color photos for determining level of dia-betic retinopathy. Ophthalmology 108: 572–585, 2001
43. Gonzalez F, Iglesias R, Suarez A, Gomez-Ulla F, Perez R: Teleophthalmology link between a primary health care center and a reference hospital.Med Inform Internet Med26:251–263, 2001
