In previous chapters, we discussed the development of systems for adverse event and medication-related named entity recognition (component 1), inferring causality by automating Naranjo Causality Assessment Probability Scale (component 2), generation of figure evidence by extracting relevant figures from biomedical literature and summarizing them with a figure summarization system (component 3), and processing narrative text to identify complex medical jargon and provide definitions and explanations to better comprehend the text (component 4). The final goal of the study was the development of a user interface integrating all four components. The interface helps users to see the named entities recognized, connectives identified and the Naranjo Score calculated automatically. ADEView also allows the user to view figures related to the ADE and view the text processed by NoteAid to better comprehend the EMR notes.
System Implemented
The final objective of this study is to combine these systems to develop a comprehensive application that would be made freely available online so that the general public can access and make use of this application. In this section, we describe the system developed for this dissertation – ADEtector.
ADEtector
The ADEtector application integrates all the system components and provides a common way to see the output of all the systems implemented in this dissertation through
Named Entity Recognizer, Causality Inference Engine, Figure Evidence Generator and NoteAid system. Given an input report, the application processes the report and shows the output of each of component in individual tabs as shown in Figure 21 below.
The Named Entity Recognizer identifies all the entities and the UI displays all the entities that appear in the text using techniques as discussed in Chapter 2. Each entity is highlighted in a different color to distinguish between various entities as shown in Figure 21.
Figure 21: Screen shot of the ADEtector system showing the output of the named entity recognizer component. Each of the entities recognized is highlighted in different colors. We also display the output of a discourse connective identifier with sense detector, which is used as features for NER task and can be used for automation of remaining elements of Naranjo Scale. Figure 22 shows the output of the discourse connective identifier. The interface shows the connective as hyperlinked text and when the user hovers the mouse over it, the class-wise sense of the connective is shown. The second component Causality Inference Engine consists of automated Naranjo Causality Assessment Probability Scale, which calculates the score of an adverse event related to a drug. Figure 23 shows the
output of the Naranjo Scale. The tool identified “aspirin” caused “GI bleed” and assigned a score of 3 to it.
Figure 22: Screen shot of the ADEtector showing the output of the discourse connective identifier module. The interface shows the connective identified as hyperlinked text and when the user hovers the mouse over the text, a pop-up box shows the class-wise sense of the connective identified.
Figure 23: Screen shot of ADEtector showing the output of the Naranjo Causality Assessment Probability Scale. The tool identified GI bleed is related to aspirin and assigned a score of 3.
The third component, Figure Evidence Generator, searches evidence for ADE detected, from the biomedical literature and then presents the user with figures along with their
summaries generated by the figure summarizer. Figure 24 below shows the screen shot of the output from the Figure Evidence Generator. It shows a grid of all the figures related to the ADE extracted from the biomedical literature. When a user clicks a figure, it shows the figure with its caption and summary along with other article-related information such as title, author information and abstract of the article as shown in Figure 25.
Figure 24: Screen shot of the ADEtector interface showing the figure evidence of the ADE that was detected by the previous components. The interface shows all the figures related to the ADE. When the user clicks on a figure, then the system shows the figure along with its summary.
Figure 25: Screen shot of the interface showing the figure along with its caption, summary and other article information.
The fourth and last component is NoteAid. This component identifies complex medical jargon in the text and provides explanations to them. Figure 26 below shows the interface with medical concepts identified as hyperlinked text. When the user hovers the mouse over the concept, the explanation of the concept appears in a pop-up box. The figure below shows the explanation of the concept “Paraplatin” in the pop-up box when the user hovers the mouse.
Figure 26: Screen shot of the ADEtector showing the output of the NoteAid component. The interface shows the medical concepts identified as hyperlinked text and when the user hovers the mouse over a concept, its explanation is shown.
Conclusion and Possible Improvements
The ADEtector application integrates various components such as the named entity recognition, causality inference, figure evidence through summarization and noteaid components. We hypothesize that such a system will help researchers and regulatory agencies discover adverse events quickly and easily. It also helps physicians to explain ADEs to patients and improve patient-physician communication.
There are several improvements that can be made so that these applications are more useful and attractive to researchers. For the Named Entity Recognizer, we could explore semi-supervised machine-learning approaches to further improve the performance of the system. Then we can normalize the identified entities to ontology. After normalizing, we can provide more information about the entity such as its synonyms and its position in the ontology to help researchers and regulatory agencies understand the ADE.
The Naranjo Score is currently shown as a table, and we can enhance the user experience by graphically linking the drug and the adverse event and showing the information about the ADE and the Naranjo Score on the link.
The figure evidence component can be further improved by extracting words appearing in the figure itself as keywords. This requires the use of sophisticated optical character recognition (OCR) software because many figures have poor resolution and text can often appear mixed with the biological sample image. Future work should also focus on retrieving more relevant figure to the ADE.
Currently, the NoteAid system only provides users with the definition, but we can further improve user experience by providing users with more useful information regarding the clinical concepts depending on the task, such as providing information about pharmaceutical properties of drugs and causes of adverse events to help researchers make better decisions regarding the ADE.
References
[1] L. T. Kohn, J. M. Corrigan, M. S. Donaldson, and others, To err is human:
building a safer health system. A report of the Committee on Quality of Health Care in America, Institute of Medicine. Washington, DC: National Academy Press,
2000.
[2] “E 2 A Clinical Safety Data Management: Definitions and Standards for Expedited Reporting - WC500002749.pdf.” .
[3] D. W. Bates, D. J. Cullen, N. Laird, L. A. Petersen, S. D. Small, D. Servi, G. Laffel, B. J. Sweitzer, B. F. Shea, R. Hallisey, and others, “Incidence of adverse drug events and potential adverse drug events,” JAMA: the journal of the American
Medical Association, vol. 274, no. 1, p. 29, 1995.
[4] J. R. Nebeker, P. Barach, and M. H. Samore, “Clarifying adverse drug events: a clinician’s guide to terminology, documentation, and reporting,” Ann. Intern. Med., vol. 140, no. 10, pp. 795–801, May 2004.
[5] A. J. Forster, R. B. Halil, and M. G. Tierney, “Pharmacist surveillance of adverse drug events,” American journal of health-system pharmacy, vol. 61, no. 14, p. 1466, 2004.
[6] E. J. Thomas, D. M. Studdert, H. R. Burstin, E. J. Orav, T. Zeena, E. J. Williams, K. M. Howard, P. C. Weiler, and T. A. Brennan, “Incidence and types of adverse events and negligent care in Utah and Colorado,” Medical Care, vol. 38, no. 3, p. 261, 2000.
[7] L. L. Leape, T. A. Brennan, N. Laird, A. G. Lawthers, A. R. Localio, B. A. Barnes, L. Hebert, J. P. Newhouse, P. C. Weiler, and H. Hiatt, “The nature of adverse events in hospitalized patients,” New England Journal of Medicine, vol. 324, no. 6, pp. 377–384, 1991.
[8] D. C. Classen, S. L. Pestotnik, R. S. Evans, J. F. Lloyd, and J. P. Burke, “Adverse drug events in hospitalized patients,” JAMA: the journal of the American Medical
[9] D. J. Cullen, B. J. Sweitzer, D. W. Bates, E. Burdick, A. Edmondson, and L. L. Leape, “Preventable adverse drug events in hospitalized patients: a comparative study of intensive care and general care units,” Critical care medicine, vol. 25, no. 8, p. 1289, 1997.
[10] D. J. Cullen, D. W. Bates, S. D. Small, J. B. Cooper, A. R. Nemeskal, and L. L. Leape, “The incident reporting system does not detect adverse drug events: a problem for quality improvement.,” The Joint Commission journal on quality
improvement, vol. 21, no. 10, p. 541, 1995.
[11] D. W. Bates, N. Spell, D. J. Cullen, E. Burdick, N. Laird, L. A. Petersen, S. D. Small, B. J. Sweitzer, and L. L. Leape, “The costs of adverse drug events in hospitalized patients,” JAMA: the journal of the American Medical Association, vol. 277, no. 4, p. 307, 1997.
[12] “FDA Adverse Events Reporting System (FAERS) > FDA Adverse Event Reporting System (FAERS) Statistics.” [Online]. Available: http://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Surveillan ce/AdverseDrugEffects/ucm070093.htm. [Accessed: 18-Apr-2013].
[13] N. P. Tatonetti, P. P. Ye, R. Daneshjou, and R. B. Altman, “Data-driven prediction of drug effects and interactions,” Sci Transl Med, vol. 4, no. 125, p. 125ra31, Mar. 2012.
[14] C. Naranjo, U. Busto, E. Sellers, P. Sandor, I. Ruiz, E. Roberts, E. Janecek, C. Domecq, and D. Greenblatt, “A method for estimating the probability of adverse drug reactions,” Clinical Pharmacology & Therapeutics, vol. 30, no. 2, pp. 239– 245, 1981.
[15] S. Belknap, H. Moore, S. Lanzotti, P. Yarnold, M. Getz, D. Deitrick, A. Peterson, J. Akeson, T. Maurer, R. Soltysik, and others, “Application of software design principles and debugging methods to an analgesia prescription reduces risk of severe injury from medical use of opioids,” Clinical Pharmacology &
Therapeutics, vol. 84, no. 3, pp. 385–392, 2008.
[16] S. Belknap, C. Georgopoulos, D. West, P. Yarnold, and W. Kelly, “Quality of methods for assessing and reporting serious adverse events in clinical trials of cancer drugs,” Clinical Pharmacology & Therapeutics, vol. 88, no. 2, pp. 231– 236, 2010.
[17] D. C. Classen, S. L. Pestotnik, R. Evans, and J. Burke, “Description of a computerized adverse drug event monitor using a hospital information system,”
Hospital Pharmacy, vol. 27, pp. 774–774, 1992.
[18] R. S. Evans, S. L. Pestotnik, D. C. Classen, S. D. Horn, S. B. Bass, and J. P. Burke, “Preventing adverse drug events in hospitalized patients,” Annals of
Pharmacotherapy, vol. 28, no. 4, pp. 523–527, 1994.
[19] H. Yu and M. Lee, “Accessing bioscience images from abstract sentences,”
Bioinformatics, vol. 22, no. 14, pp. e547–556, Jul. 2006.
[20] H. H. Hiatt, B. A. Barnes, T. A. Brennan, N. M. Laird, A. G. Lawthers, L. L. Leape, A. R. Localio, J. P. Newhouse, L. M. Peterson, K. E. Thorpe, and others, “A study of medical injury and medical malpractice,” New England Journal of
Medicine, vol. 321, no. 7, pp. 480–484, 1989.
[21] T. A. Brennan, L. L. Leape, N. M. Laird, L. Hebert, A. R. Localio, A. G. Lawthers, J. P. Newhouse, P. C. Weiler, and H. H. Hiatt, “Incidence of adverse events and negligence in hospitalized patients,” New England journal of medicine, vol. 324, no. 6, pp. 370–376, 1991.
[22] A. K. Jha, G. J. Kuperman, J. M. Teich, L. Leape, B. Shea, E. Rittenberg, E. Burdick, D. L. Seger, M. V. Vliet, and D. W. Bates, “Identifying Adverse Drug Events,” Journal of the American Medical Informatics Association, vol. 5, no. 3, pp. 305 –314, May 1998.
[23] H. J. Murff, A. J. Forster, J. F. Peterson, J. M. Fiskio, H. L. Heiman, and D. W. Bates, “Electronically screening discharge summaries for adverse medical events,”
Journal of the American Medical Informatics Association, vol. 10, no. 4, pp. 339–
350, 2003.
[24] S. H. Hwang, S. Lee, H. K. Koo, and Y. Kim, “Evaluation of a computer-based adverse-drug-event monitor,” Am J Health Syst Pharm, vol. 65, no. 23, pp. 2265– 2272, 2008.
[25] A. Tinoco, R. S. Evans, C. J. Staes, J. F. Lloyd, J. M. Rothschild, and P. J. Haug, “Comparison of computerized surveillance and manual chart review for adverse events,” J Am Med Inform Assoc, vol. 18, no. 4, pp. 491–497, Aug. 2011.
[26] R. M. Gardner, T. A. Pryor, and H. R. Warner, “The HELP hospital information system: update 1998,” Int J Med Inform, vol. 54, no. 3, pp. 169–182, Jun. 1999.
[27] L. Ohno-Machado, P. Nadkarni, and K. Johnson, “Natural language processing: algorithms and tools to extract computable information from EHRs and from the biomedical literature,” J Am Med Inform Assoc, vol. 20, no. 5, pp. 805–805, Sep. 2013.
[28] B. Hazlehurst, H. R. Frost, D. F. Sittig, and V. J. Stevens, “MediClass: A system for detecting and classifying encounter-based clinical events in any electronic medical record,” J Am Med Inform Assoc, vol. 12, no. 5, pp. 517–529, Oct. 2005.
[29] B. Hazlehurst, A. Naleway, and J. Mullooly, “Detecting possible vaccine adverse events in clinical notes of the electronic medical record,” Vaccine, vol. 27, no. 14, pp. 2077–2083, Mar. 2009.
[30] X. Wang, G. Hripcsak, M. Markatou, and C. Friedman, “Active computerized pharmacovigilance using natural language processing, statistics, and electronic health records: a feasibility study,” J Am Med Inform Assoc, vol. 16, no. 3, pp. 328–337, Jun. 2009.
[31] K. Haerian, D. Varn, S. Vaidya, L. Ena, H. S. Chase, and C. Friedman, “Detection of pharmacovigilance-related adverse events using electronic health records and automated methods,” Clinical Pharmacology & Therapeutics, 2012.
[32] G. B. Melton and G. Hripcsak, “Automated detection of adverse events using natural language processing of discharge summaries,” J Am Med Inform Assoc, vol. 12, no. 4, pp. 448–457, Aug. 2005.
[33] C. Friedman, P. O. Alderson, J. H. Austin, J. J. Cimino, and S. B. Johnson, “A general natural-language text processor for clinical radiology.,” J Am Med Inform
Assoc, vol. 1, no. 2, pp. 161–174, 1994.
[34] E. Aramaki, Y. Miura, M. Tonoike, T. Ohkuma, H. Masuichi, K. Waki, and K. Ohe, “Extraction of Adverse Drug Effects from Clinical Records,” Studies in
health technology and informatics, vol. 160, no. Pt 1, pp. 739–743, 2010.
[35] C. Yang, P. Srinivasan, and P. M. Polgreen, “Automatic adverse drug events detection using letters to the editor,” AMIA Annu Symp Proc, vol. 2012, pp. 1030– 1039, 2012.
[36] H. Gurulingappa, A. Mateen-Rajput, and L. Toldo, “Extraction of potential adverse drug events from medical case reports,” J Biomed Semantics, vol. 3, no. 1, p. 15, 2012.
[37] K. W. Fung, C. S. Jao, and D. Demner-Fushman, “Extracting drug indication information from structured product labels using natural language processing,” J
Am Med Inform Assoc, vol. 20, no. 3, pp. 482–488, May 2013.
[38] B. W. Chee, R. Berlin, and B. Schatz, “Predicting adverse drug events from personal health messages,” AMIA, page 10pp, Washington, DC, 2011.
[39] R. W. White, N. P. Tatonetti, N. H. Shah, R. B. Altman, and E. Horvitz, “Web- scale pharmacovigilance: listening to signals from the crowd,” J Am Med Inform
Assoc, Mar. 2013.
[40] K. D. Shetty and S. R. Dalal, “Using information mining of the medical literature to improve drug safety,” Journal of the American Medical Informatics Association, 2011.
[41] S. M. Meystre, J. Thibault, S. Shen, J. F. Hurdle, and B. R. South, “Textractor: a hybrid system for medications and reason for their prescription extraction from clinical text documents,” J Am Med Inform Assoc, vol. 17, no. 5, pp. 559–562, Oct. 2010.
[42] I. Spasic, F. Sarafraz, J. A. Keane, and G. Nenadic, “Medication information extraction with linguistic pattern matching and semantic rules,” J Am Med Inform
Assoc, vol. 17, no. 5, pp. 532–535, Oct. 2010.
[43] H. Xu, S. P. Stenner, S. Doan, K. B. Johnson, L. R. Waitman, and J. C. Denny, “MedEx: a medication information extraction system for clinical narratives,” J Am
Med Inform Assoc, vol. 17, no. 1, pp. 19–24, Feb. 2010.
[44] S. Doan, L. Bastarache, S. Klimkowski, J. C. Denny, and H. Xu, “Integrating existing natural language processing tools for medication extraction from discharge summaries,” J Am Med Inform Assoc, vol. 17, no. 5, pp. 528–531, Oct. 2010.
[45] S. M. Meystre, J. Thibault, S. Shen, J. F. Hurdle, and B. R. South, “Automatically detecting medications and the reason for their prescription in clinical narrative text documents,” Stud Health Technol Inform, vol. 160, no. Pt 2, pp. 944–948, 2010.
[46] Ö. Uzuner, I. Solti, and E. Cadag, “Extracting medication information from clinical text,” Journal of the American Medical Informatics Association, vol. 17, no. 5, p. 514, 2010.
[47] M. Lindquist, “VigiBase, the WHO Global ICSR Database System: Basic Facts,”
Drug Information Journal, vol. 42, no. 5, pp. 409–419, Sep. 2008.
[48] M. Hauben, D. Madigan, C. M. Gerrits, L. Walsh, and E. P. Van Puijenbroek, “The role of data mining in pharmacovigilance,” Expert Opin Drug Saf, vol. 4, no. 5, pp. 929–948, Sep. 2005.
[49] M. Hauben and A. Bate, “Decision support methods for the detection of adverse events in post-marketing data,” Drug Discovery Today, vol. 14, no. 7–8, pp. 343– 357, Apr. 2009.
[50] R. Harpaz, W. DuMouchel, N. H. Shah, D. Madigan, P. Ryan, and C. Friedman, “Novel Data-Mining Methodologies for Adverse Drug Event Discovery and Analysis,” Clin Pharmacol Ther, vol. 91, no. 6, pp. 1010–1021, Jun. 2012.
[51] A. Bate and S. J. W. Evans, “Quantitative signal detection using spontaneous ADR reporting,” Pharmacoepidemiol Drug Saf, vol. 18, no. 6, pp. 427–436, Jun. 2009.
[52] J. S. Almenoff, E. N. Pattishall, T. G. Gibbs, W. DuMouchel, S. J. W. Evans, and N. Yuen, “Novel statistical tools for monitoring the safety of marketed drugs,”
Clin. Pharmacol. Ther., vol. 82, no. 2, pp. 157–166, Aug. 2007.
[53] S. J. Evans, P. C. Waller, and S. Davis, “Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports,”
Pharmacoepidemiol Drug Saf, vol. 10, no. 6, pp. 483–486, Nov. 2001.
[54] E. P. van Puijenbroek, A. Bate, H. G. M. Leufkens, M. Lindquist, R. Orre, and A. C. G. Egberts, “A comparison of measures of disproportionality for signal detection in spontaneous reporting systems for adverse drug reactions,”
Pharmacoepidemiol Drug Saf, vol. 11, no. 1, pp. 3–10, Feb. 2002.
[55] G. N. Norén, R. Sundberg, A. Bate, and I. R. Edwards, “A statistical methodology for drug-drug interaction surveillance,” Stat Med, vol. 27, no. 16, pp. 3057–3070, Jul. 2008.
[56] R. Harpaz, S. Vilar, W. DuMouchel, H. Salmasian, K. Haerian, N. H. Shah, H. S. Chase, and C. Friedman, “Combing signals from spontaneous reports and electronic health records for detection of adverse drug reactions,” J Am Med
Inform Assoc, vol. 20, no. 3, pp. 413–419, May 2013.
[57] T. Sakaeda, K. Kadoyama, and Y. Okuno, “Adverse event profiles of platinum agents: Data mining of the public version of the FDA adverse event reporting system, AERS, and reproducibility of clinical observations,” International Journal
of Medical Sciences, vol. 8, no. 6, p. 487, 2011.