BRIEF REPORT
Secondary use of electronic health record data: spontaneous
triggered adverse drug event reporting
Jeffrey A. Linder1,2*, Jennifer S. Haas1,2, Aarthi Iyer1, Michael A. Labuzetta1, Michael Ibara3,
Michael Celeste3, George Getty4 and David W. Bates1,2
1
Division of General Medicine and Primary Care, Brigham and Women’s Hospital, Boston, MA, USA 2
Harvard Medical School, Boston, MA, USA 3
Worldwide Safety and Regulatory Operations, Pfizer Inc., New York, NY, USA 4
Claricode, Waltham, MA, USA
SUMMARY
Purpose Physicians in the United States report fewer than 1% of adverse drug events (ADEs) to the Food and Drug Administration (FDA), but frequently document ADEs within electronic health records (EHRs). We developed and implemented a generalizable, scalable EHR-based system to automatically send electronic ADE reports to the FDA in real-time.
Methods Proof-of-concept study involving 26 clinicians given access to EHR-based ADE reporting functionality from December 2008 to May 2009.
Measurements Number and content of ADE reports; severity of adverse reactions (clinician and computer algorithm defined); clinician survey.
Results During the study period, 26 clinicians submitted 217 reports to the FDA. The clinicians defined 23% of the ADEs as serious and a computer algorithm defined 4% of the ADEs as serious. The most common drug classes were cardiovascular drugs (40%), central nervous system drugs (19%), analgesics (13%), and endocrine drugs (7%). The reports contained information, pre-filled from the EHR, about comorbid conditions (207 reports [95%] listed 1899 comorbid conditions), concurrent medications (193 reports [89%] listed 1687 concurrent medications), weight (209 reports [96%]), and laboratory data (215 reports [99%]). It took clinicians a mean of 53 seconds to complete and send the form. In the clinician survey, 21 of 23 respondents (91%) said they had submitted zero ADE reports to the FDA in the prior 12 months.
Conclusions EHR-based, triggered ADE reporting is efficient and acceptable to clinicians, provides detailed clinical information, and has the potential to greatly increase the number and quality of spontaneous reports submitted to the FDA. Copyright#2010 John Wiley & Sons, Ltd.
key words— adverse drug reaction reporting systems; product surveillance; post-marketing; medical record systems; computerized; ambulatory care
Received 9 December 2009; Revised 28 June 2010; Accepted 5 July 2010
INTRODUCTION
About a quarter of outpatients experience adverse drug events (ADEs) and 13% of these are serious.1 ADEs also represent common reasons for emergency depart-ment visits for adults and children.2,3 However, because of barriers to reporting, physicians report as few as 0.3% of ADEs to the Food and Drug Administration (FDA).4,5 A low reporting rate has contributed to the widespread agreement that it takes
too long to identify problem drugs and quantify risks of older drugs.6–8
Consistent with recommendations from the Institute of Medicine,9 proactive, automated reporting imbedded within electronic health records (EHRs), which are now used by about 35% of ambulatory physicians,10may improve public health by facilitating the submission of ADE reports. We developed the ADE spontaneous triggered electronic reporting system (ASTER), to submit ADE reports directly from EHRs—using data collected as part of routine care—to the FDA in a manner that was scalable and independent of a specific EHR. We conducted an initial pilot implementation of ASTER to assess the 2010; 19: 1211–1215
Published online 11 October 2010 in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/pds.2027
* Correspondence to: J. A. Linder, Division of General Medicine and Primary Care, Brigham and Women’s Hospital, 1620 Tremont Street, BC-3-2X Boston, MA 02120, USA. E-mail: jlinder@partners.org
feasibility of sending automated, real-time spon-taneous, triggered reports from an EHR to the FDA and evaluated clinicians’ attitudes toward the system. METHODS
We designed ASTER for use with any EHR that has a coded ADE documentation event or ‘‘trigger.’’ ASTER was remotely hosted by CRIXTMInternational, which could serve as a central aggregator of reports from multiple sources. For this proof-of-concept study, we used the Longitudinal Medical Record (LMR), an ambulatory EHR developed by Partners Healthcare, an integrated healthcare delivery system in Eastern Massachusetts. The LMR is an internally developed, web-based, fully functional, Certification Commission for Health Information Technology-certified EHR that uses a services-oriented architecture and includes notes from primary care and subspecialty practices; coded problem lists; a ‘‘health maintenance grid’’ for documenting preventive counseling and health habits; medication prescribing; and medication allergies.11
ASTER used a flexible technical standard for data collection known as ‘‘Retrieve Form for Data Capture’’ (RFD) from the Clinical Data Interchange Standards Consortium (CDISC) and Integrating the Healthcare Enterprise (IHE).12 RFD allowed remote hosting at CRIX, local aggregation and presentation of data within the EHR, and direct electronic submission to the FDA using the Individual Case Safety Report (ICSR) standard. ASTER was automatically triggered in the LMR when clinicians discontinued a medication due to an ‘‘adverse reaction.’’ When triggered, the LMR stored a unique identifier for the patient, clinician, and suspect medication. This identifier—the key for which was retained within the LMR—was passed to the ASTER application. ASTER then retrieved medication, demo-graphic, vital sign, medical problem, and laboratory information from the LMR to populate an ADE reporting form. The ADE reporting form was available for the clinician to review within the LMR. Before submitting the form, the clinician had to provide two additional pieces of data: (1) the outcome of the ADE and (2) the earliest date of occurrence of the ADE. The clinician-defined outcomes were derived from the FDA MedWatch form.13
We considered two types of ‘‘seriousness:’’ clinician coded and computer algorithm defined. For clinician coding, we considered the outcomes of death, life threatening, hospitalization, disability or incapacity, congenital anomaly or birth defect, and other serious outcomes as ‘‘serious’’ ADEs. For computer algorithm defined seriousness, events were automatically coded
by CRIX using the Medical Dictionary for Regulatory Activities (MedDRA).16The event was then compared to a predefined list of designated ‘‘always serious medical events’’ created by the FDA.
The study ran from 9 December 2008 to 22 May 2009. We selected, invited, and gave access to ASTER to 26 clinicians who frequently discontinued medi-cations due to adverse reactions. As a comparison group, we used Partners Healthcare clinicians who wrote a prescription or had had at least one patient visit in the prior year. We compared these clinicians with the ASTER participants based on age, length of LMR use, and, in the year prior to the pilot, medication prescribing, and discontinuations due to adverse reactions. We classified drugs using First Data Bank’s Enhanced Therapeutic Classes.14
The main outcome of interest was the number of ADE reports received by the FDA. Additional measures of interest included the content, detail, and quality of the ADE reports and the speed, usability, and value of ASTER as reported by clinicians in a post-pilot survey.
We discussed the original concepts for ASTER with the FDA, which had input on the initial design. The design team met with the FDA a month prior to going live to review the project and processes. The Institutional Review Board and Health Information Systems Group of Partners Healthcare required signed data use agreements with CRIX and the FDA and approved the study protocol.
RESULTS
The 26 selected pilot clinicians, compared to 2606 non-participating clinicians, were older (mean age in years, 52 versus 45; p<0.001); had used the EHR longer (mean years, 6.2 versus 4.8;p<0.01); and, in the year prior to the pilot, prescribed more medications (mean, 4404 versus 578; p<0.001), discontinued more medications (mean, 1422 versus 154; p<0.001) and, by design, discontinued more medications due to adverse reactions (mean, 35 versus 2; p<0.001).
During the 5-month study period, the 26 pilot clinicians discontinued 319 medications due to ADEs. Reports for some of these ADEs were not submitted through ASTER because they were submitted before the system was saving data (13); errors in transmitting the data early in the study period (21); and either the clinician closed the pop-up form before submitting data or the form timed out because of delays (68). The FDA received 217 reports.
The mean time from triggering to submitting the form was 53 (standard deviation [SD], 28) seconds.
The mean time between the ADE and report submission was 10 days (SD, 10).
The most common suspect drug class was cardio-vascular therapy agents (Table 1). The most common specific suspect drugs were lisinopril (n¼34), sim-vastatin (n¼10), nortriptyline (n¼7), tramadol (n¼6), and alendronate (n¼5). Clinicians coded 168 (77%) of outcomes as non-serious and the majority of the remainder as ‘‘other.’’ The automatic algorithm coded 4% of reactions as serious.
Reports contained information about demographics, comorbid conditions, concurrent medications, vital signs, and labs (Table 2). The mean number of comorbid conditions listed, on reports that had at least one comorbid condition, was 9 (SD, 6). The mean number of concurrent medications listed, on reports that had at least one concurrent medication, was 8 (SD, 5). The most commonly listed concurrent medications were lisinopril (n¼26), aspirin (n¼26), simvastatin
(n¼25), atorvastatin (n¼15), and atenolol (n¼12). Over 95% of reports had weight and laboratory data. After the pilot period, 23 of the 26 participants responded to the survey. Of the 23 respondents, 21 (91%) reported they had submitted no MedWatch forms in the prior year and two (9%) reported submitting three or more. Overall, 18 (78%) said they would like to view aggregate national data from the FDA about similar events. Importantly, 21 (91%) felt that ASTER would improve the care of their patients and 22 (96%) said that ASTER would improve their ability to accurately report drug risks. Among the free-text comments provided, 17 respondents (74%) stated that ASTER was either fast or easy to use and eight (35%) mentioned loading failures or ASTER taking time away from clinical care.
DISCUSSION
In this proof-of-concept study, clinicians who generally reported submitting no ADE reports in the prior year submitted 217 reports to the FDA in 5 months using an EHR-based, triggered, spontaneous reporting system. Despite initial technical challenges, the system met many of the requirements for a successful ADE reporting system: ease-of-use, acceptability to clin-icians, and generation of reports containing a wide variety of data types.15 Potential advantages of EHR-based, spontaneous triggered ADE reporting include secondary use of data collected as part of clinical care; integrating ADE reporting into clinicians’ workflow; the promptness and speed with which ADEs are reported; the ability to know the denominator of drug
Table 1. Suspect drug characteristics and outcomes (N¼217)
Characteristic N(%)
Drug class
Cardiovascular therapy agents 86 (40) Central nervous system agents 41 (19) Analgesic, anti-inflammatory or antipyretic 28 (13)
Endocrine 14 (7)
Anti-infective agents 13 (6) Chemical dependency, agents to treat 9 (4)
Other classes 26 (12)
Clinician-assigned outcomes
No serious outcome 168 (77) Other serious outcome 36 (17) Inpatient hospitalizationy 7 (3) Disability or incapacityz 4 (2)
Life-threateningx 2 (1)
Congenital anomaly or birth defect 0 (0)
Death 0 (0)
Automatically coded outcomes
Serious reactions 8 (4)
Non-serious reactionsyy 158 (72)
Not codedzz 53 (24)
Most common ‘‘other’’ outcomes were musculoskeletal and connective
tissue disorders (n¼7), respiratory, thoracic, and mediastinal disorders (n¼6), and psychiatric disorders (n¼5).
yInpatient hospitalization outcome reactions were flu-like syndrome,
sei-zure, heart block, rectal bleeding, myalgias, mental status change, and drug-induced lupus (n¼1 for all outcomes).
zDisabling/incapacitating outcome reactions were flu-like syndrome,
breathing difficulties, lung nodules, and rash (n¼1); hepatotoxicity (n¼1); insomnia (n¼1); and chronic edema (n¼1).
xLife-threatening outcomes were bradycardia (n¼1) and gastrointestinal
intolerance (n¼1).
Serious reactions were angioedema (n¼4), hyperkalemia (n¼2), and
myositis (n¼2).
yyMost common non-serious reactions were respiratory, thoracic, and
mediastinal disorders (n¼27); gastrointestinal disorders (n¼26); psychia-tric disorders (n¼21); and nervous system disorders (n¼20).
zzMost common non-coded reactions were gastrointestinal disorders
(n¼30), nervous system disorders (n¼5), and hepatobiliary disorders (n¼4).
Table 2. Example content of electronic adverse drug event reports (N¼217)
Characteristic Value
Patient age, mean (SD) 58 (15)
Women,N(%) 159 (73)
Reports with any comorbid condition,N(%) 207 (95) Total number of comorbid conditions,N 1899 Report with any concurrent medication,N(%) 193 (89) Total number of concurrent medications,N 1687
Weight,N(%) 209 (96)
Height,N(%) 177 (82)
Reports with any laboratory data,N(%) 215 (99) Specific laboratory results,N(%)
White blood cell count 211 (98)
Creatinine 205 (95)
Alkaline phosphatase 205 (95)
Total bilirubin 185 (86)
ALT 201 (94)
Most common comorbid conditions were musculoskeletal problems
(n¼297), surgical procedures and tests (n¼168), cardiovascular disease (n¼159), lung disease (n¼152), and cancer (n¼111).
prescriptions from which ADEs emerge; and increas-ing reports from clinicians rather than manufacturers, who presently submit over 80% of reports.16
Potential disadvantages of EHR-based, spontaneous triggered ADE reporting include many of the biases inherent to spontaneous reports—reporting bias, availability bias, channeling, and the Weber effect (the tendency of newly approved drugs to generate more ADE reports);17the potential volume of reports; the complexity of the data received; data incompat-ibility;18 and the frequent submission of well-known, non-serious ADEs.19 Finally, any reporting system must protect the privacy of patients and clinicians, who may be concerned about medical liability.20
ASTER complements many other ongoing and emerging pharmacovigilance efforts such as the FDA Sentinel Initiative or the European Adverse Drug Reaction Project;21,22 prospective and retrospective observational studies;23,24 population-based outcome studies;25 patient monitoring studies;26 automated ADE detection techniques;27 and independent report-ing systems.28,29Compared to other methods of ADE identification, an EHR-based, spontaneous, triggered reporting system has several advantages. Clinicians, as opposed to researchers, identify and define potential ADEs.30–33ASTER is not dependent on pre-identified safety questions, has the potential to identify unex-pected associations,34,35 and takes advantage of the existing spontaneous reporting system.19 ASTER could also be used to report vaccine-related or device-related adverse events to the FDA.36
This study has limitations. First, by design, the clinicians we selected and who agreed to participate were different from the broader population of clinicians at Partners Healthcare. In addition, given that it is academically affiliated, our results may not generalize to broader community practice. Third, technical problems, now resolved, in conducting the pilot limited clinician satisfaction with ASTER. Finally, as a proof-of-concept study, the number of clinicians and reports was small.
To address these limitations, we plan to continually monitor and quickly correct technical problems, expand participation to a larger, more representative set of clinicians, and expand participation beyond a single health system and EHR. During the project, we kept FDA staff apprised of our work and we have presented pilot results to the FDA. As we expand the number of clinicians with access to ASTER, we hope to continue working with the FDA to ensure that EHR-based, spontaneous triggered ADE reports provide high quality information. To that end, we may refine the ADE identification method, ensure the appropriate
data elements are included, and, if the volume of reports becomes too large, limit submissions to only serious events.
The true public health value of a system such as ASTER, will emerge when thousands of clinicians are submitting reports. We designed and implemented ASTER in a scalable way: a central aggregator allows collection of reports from multiple EHRs or electronic systems and can pre-process reports before sending them to the FDA. Such a system could also be bidirectional—the FDA could request additional or follow-up information about specific ADEs—and could serve as part of a national distributed health data network with functions other than ADE report-ing.37 Like any pharmacovigilance system, EHR-based, spontaneous triggered reporting should help identify new ADEs and accurately define the risks associated with drugs.8,26
CONFLICT OF INTEREST
Dr Ibara and Mr Celeste are employees of Pfizer and were involved in the planning, implementation, data collection, data analysis, manuscript drafting, and the decision to submit the manuscript for publication. Dr Linder had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
ACKNOWLEDGEMENTS
This study was supported by a grant from Pfizer.
REFERENCES
1. Gandhi TK, Weingart SN, Borus J, et al. Adverse drug events in ambulatory care.N Engl J Med2003;348(16): 1556–1564. 2. Budnitz DS, Pollock DA, Weidenbach KN, Mendelsohn AB, Schroeder
TJ, Annest JL. National surveillance of emergency department visits for outpatient adverse drug events.JAMA2006;296(15): 1858–1866. 3. Cohen AL, Budnitz DS, Weidenbach KN,et al. National surveillance of
emergency department visits for outpatient adverse drug events in children and adolescents.J Pediatr2008;152(3): 416–421.
KEY POINTS
EHR-based, triggered ADE reporting is efficient and acceptable to clinicians, and provides detailed clinical information.
EHR-based, triggered ADE reporting has the potential to greatly increase the number and quality of submitted spontaneous reports.
4. Health, Education, and Human Services Division. Adverse Drug Events: The Magnitude of the Health Risk Is Uncertain Because of Limited Incidence Data (Report GAO/HEHS-00-21), US General Accounting Office, Washington, DC, January 2000.
5. Lopez-Gonzalez E, Herdeiro MT, Figueiras A. Determinants of under-reporting of adverse drug reactions: a systematic review.Drug Saf
2009;32(1): 19–31.
6. Bates DW. Drugs and adverse drug reactions: how worried should we be?JAMA1998;279(15): 1216–1217.
7. Strom BL. How the US drug safety system should be changed.JAMA
2006;295(17): 2072–2075.
8. Avorn J, Schneewiess S. Managing drug-risk information—what to do with all those new numbers. N Engl J Med 2009; 361(7): 647–649.
9. Committee on the Assessment of the US Drug Safety System.The Future of Drug Safety: Promoting and Protecting the Health of the Public, National Academy of Sciences, Washington, DC, 2007. 10. Hing E, Hsiao CJ. Electronic medical record use by
office-based physicians and their practices: United States, 2007.National health statistics reports; no 23, National Center for Health Statistics, Hyattsville, MD, 2010.
11. Certification Commission for Healthcare Information Technology. Partners HealthCare System Longitudinal Medical Record (LMR) 5.1.1. 2006. Available at: http://www.cchit.org/products/2006/ambulat-ory/1441 [8 December 2009].
12. Integrating the Healthcare Enterprise. Technical Framework Supple-ment Retrieve Form for Data Capture. Available at: http://www.ihe.net/
Technical_Framework/upload/IHE_ITI_TF_Supple-ment_RFD_TI_2009-08-10.pdf [December 2009].
13. MedWatch: The FDA Safety Information and Adverse Event Reporting Program. Available at: http://www.fda.gov/Safety/MedWatch/default. htm [December 2009].
14. First Data Bank. Enhanced Therapeutic Classification System. Available at: http:// www.firstdatabank .com / Products/ therapeutic-classification-system-nddf.aspx [December 2009].
15. Wu JH, Shen WS, Lin LM, Greenes RA, Bates DW. Testing the technology acceptance model for evaluating healthcare professionals’ intention to use an adverse event reporting system.Int J Qual Health Care2008;20(2): 123–129.
16. Moore TJ, Cohen MR, Furberg CD. Serious adverse drug events reported to the Food and Drug Administration, 1998–2005.Arch Intern Med2007;167(16): 1752–1759.
17. Stephenson WP, Hauben M. Data mining for signals in spontaneous reporting databases: proceed with caution.Pharmacoepidemiol Drug Saf2007;16(4): 359–365.
18. Glass LM. Data incompatibility of adverse event reports electronically submitted to the AERS database.Pharmacoepidemiol Drug Saf2008;
17(2): 212–213.
19. Wysowski DK, Swartz L. Adverse drug event surveillance and drug withdrawals in the United States, 1969–2002 : the importance of reporting suspected reactions. Arch Intern Med 2005; 165(12): 1363–1369.
20. Lee LM, Gostin LO. Ethical collection, storage, and use of public health data: a proposal for a national privacy protection.JAMA2009;302(1): 82–84.
21. Platt R, Wilson M, Chan KA, Benner JS, Marchibroda J, McClellan M. The new Sentinel Network—improving the evidence of medical-product safety.N Engl J Med2009;361(7): 645–647.
22. Trifiro G, Pariente A, Coloma PM,et al. Data mining on electronic health record databases for signal detection in pharmacovigilance: which events to monitor?Pharmacoepidemiol Drug Saf2009;18: 1176–1184. 23. Psaty BM, Vandenbroucke JP. Opportunities for enhancing the FDA
guidance on pharmacovigilance.JAMA2008;300(8): 952–954. 24. Bennett CL, Nebeker JR, Yarnold PR,et al. Evaluation of serious
adverse drug reactions: a proactive pharmacovigilance program (RADAR) vs safety activities conducted by the Food and Drug Admin-istration and pharmaceutical manufacturers.Arch Intern Med2007;
167(10): 1041–1049.
25. Booth CM, Mackillop WJ. Translating new medical therapies into societal benefit: the role of population-based outcome studies.JAMA
2008;300(18): 2177–2179.
26. Harmark L, van Grootheest AC. Pharmacovigilance: methods, recent developments and future perspectives.Eur J Clin Pharmacol2008;
64(8): 743–752.
27. Wang X, Hripcsak G, Markatou M, Friedman C. Active computerized pharmacovigilance using natural language processing, statistics, and electronic health records: a feasibility study.J Am Med Inform Assoc
2009;16(3): 328–337.
28. Zafar A, Hickner J, Pace W, Tierney W. An adverse drug event and medication error reporting system for ambulatory care (MEADERS).
AMIA Annu Symp Proc.2008: 839–843.
29. Miller MR, Clark JS, Lehmann CU. Computer based medication error reporting: insights and implications.Qual Saf Health Care2006;15(3): 208–2213.
30. Bates DW, Evans RS, Murff H, Stetson PD, Pizziferri L, Hripcsak G. Detecting adverse events using information technology. J Am Med Inform Assoc2003;10(2): 115–128.
31. Honigman B, Light P, Pulling RM, Bates DW. A computerized method for identifying incidents associated with adverse drug events in out-patients.Int J Med Inform2001;61(1): 21–32.
32. Brown JS, Kulldorff M, Petronis KR,et al. Early adverse drug event signal detection within population-based health networks using sequen-tial methods: key methodologic considerations.Pharmacoepidemiol Drug Saf2009;18(3): 226–234.
33. Brown JS, Kulldorff M, Chan KA,et al. Early detection of adverse drug events within population-based health networks: application of sequen-tial testing methods.Pharmacoepidemiol Drug Saf2007;16(12): 1275– 1284.
34. Platt R, Madre L, Reynolds R, Tilson H. Active drug safety surveil-lance: a tool to improve public health.Pharmacoepidemiol Drug Saf
2008;17(12): 1175–1182.
35. Poluzzi E, Raschi E, Moretti U, De Ponti F. Drug-induced torsades de pointes: data mining of the public version of the FDA adverse event reporting system (AERS).Pharmacoepidemiol Drug Saf2009;18(6): 512–518.
36. Hinrichsen VL, Kruskal B, O’Brien MA, Lieu TA, Platt R. Using electronic medical records to enhance detection and reporting of vaccine adverse events.J Am Med Inform Assoc2007;14(6): 731–735. 37. Maro JC, Platt R, Holmes JH,et al. Design of a national distributed
