Preventing Provider Errors: Online Total Parenteral Nutrition Calculator
Christoph U. Lehmann, MD*; Kim G. Conner, RPh, MAc‡§; and Jeanne M. Cox, MS, RD§
ABSTRACT. Objective. To describe the development of a pragmatic low-cost medical information system that reduces errors in the ordering of total parenteral nutri-tion (TPN) in the newborn intensive care unit at the Johns Hopkins Hospital.
Methods. We designed an online total parenteral nu-trition order entry system (TPNCalculator) using Internet technologies. Utilization, impact on medical errors, and user satisfaction were evaluated prior to and immedi-ately after introduction of TPNCalculator (intervention 1) and after 2 years (intervention 2).
Results. Total software development time was 3 weeks. The number of orders was similar during the 3 periods: 0.39 orders per patient per day (Nⴝ557) were received compared with 0.35 and 0.43 orders per patient per day (Nⴝ471 andNⴝ656) in 2 intervention periods. During the control period, an average of 10.8 errors were detected per 100 TPN orders compared with 4.2 per 100 orders in the first intervention period (61% reduction of error rate) and 1.2 per 100 orders after 2 years and some redesign of TPNCalculator (89% reduction of error rate). We found a reduction in the following types of problems (intervention 1; intervention 2): calculation errors (100%; 100%), osmolality outside the allowed range (88%; 91%), and other knowledge problems (84%; 100%). There was a 35% increase in the number of incomplete forms in the first intervention period and a 100% reduction in the second. Rapid cycle development was used in the devel-opment of this application. Users of the system were enthusiastic and supportive and compared it favorably to the previous paper-based system.
Conclusion. Low-cost, pragmatic approaches using Internet technology in the design of medical information systems can reduce medical errors and might pose a viable option for the prevention of adverse drug events.
Pediatrics2004;113:748 –753;medication errors/classifica-tion, medication systems, hospital/organization and ad-ministration, software design, medical informatics com-puting.
ABBREVIATIONS. ADE, adverse drug event; NICU, neonatal intensive care unit; TPN, total parenteral nutrition; JHH, Johns Hopkins Hospital.
A
ccording to an estimate by the Institute of Medicine in 1999, ⬎1 million injuries and almost 100 000 deaths may be attributed to medical errors annually.1 Most errors that occur in the prescription, dispensing, and administration of medications could have been prevented2by redesign of the systems used to deliver medications to pa-tients. Practical interventions that attempt to change system processes rather than people were found to be most successful in the prevention of adverse drug events (ADEs).3 Unfortunately, the underlying sys-tem failures are rarely identified and corrected,4so that physicians, pharmacists, and nurses are often unwitting participants in the recurrence of a well-known error. The rate for potential ADE is 3 times higher in children than adults and substantially higher still for neonates in the neonatal intensive care unit (NICU).5Adequate nutritional support for premature or sick neonates requires the daily planning, calculat-ing, and ordering of parenteral nutrition.6The order-ing of parenteral nutrition is associated with a high incidence of medical errors and a significant poten-tial for patient harm.7–10and is very time-consuming (⬃10 minutes per patient).6
By focusing on this known problem area, the or-dering of parenteral nutrition, we aimed to develop a low-budget medical information system (using an existing model and quality assurance process), eval-uate its effectiveness in error reduction, and assess its acceptance by the ordering providers. We hypothe-sized that a pragmatic and inexpensive approach to ADEs could be found by correcting system failures with the help of Web technology.
METHODS
Using a model of failure mode and effects analysis,11we had identified the completion of the paper order form as the main source of errors. Using the paper form as a model, we designed an online total parenteral nutrition (TPN) order entry system (TPN-Calculator) in 2000 and introduced it in the NICU at the Johns Hopkins Hospital (JHH) as a quality improvement instrument. TPNCalculator was created using a routine web development tool (Macromedia Cold Fusion). Its interface was designed to resemble closely the TPN order form that had been in use for⬎10 years at JHH and that was replaced by TPNCalculator. This allowed TPN-Calculator to tap into the existing order system and quality im-provement process. It also reduced the risk for potential system integration issues. Using a participant/observer as the main pro-grammer eliminated the need for a specification process and re-duced total development and testing time to⬃3 weeks.
Process
Residents received a brief 10-minute training before the initial use of the TPNCalculator. Through a public workstation or a desktop computer, the user (resident, nurse practitioner, pharma-From the *Division of Neonatology, Department of Pediatrics, and Division
of Health Sciences Informatics, Johns Hopkins University, Baltimore, Mary-land; ‡Division of Pediatric Pharmacy, Johns Hopkins Hospital, Baltimore, Maryland; and §Division of Pediatric Gastroenterology/Nutrition, Johns Hopkins Hospital, Baltimore, Maryland.
Received for publication Jan 21, 2003; accepted Jul 25, 2003.
Reprint requests to (C.U.L.) Eudowood Neonatal Pulmonary Division & Division of Health Sciences Informatics, Johns Hopkins University, 600 N Wolfe St, CMSC 210, Baltimore, MD 21287-3200. E-mail: clehmann@ jhmi.edu
cist, nutritionist) accesses the TPNCalculator using individual log-in identification and password. All interactions between the TPNCalculator Server and the user’s browser are 128-bit en-crypted to be compliant with the Health Insurance Portability and Accountability Act.
The user may select to enter a TPN Order for a new patient or to modify/renew an old order for an existing patient. TPNCalcu-lator performs all necessary fluid and component calculations on the basis of provider input via an Internet browser. It contains nutritional guidelines, an osmolality calculator, and 62 rule-based alerts and reminders (Fig 1). These alerts and reminders include a check for age/weight ratio (if the child is outside the normal weight range for the calculated age, then the user is prompted), dose range alerts based on total dose, dose per weight, concentra-tion (as indicated), and percentage of total calories (as indicated) for the included components such as glucose, lipids, protein, electrolytes, heparin, and ranitidine. Other features include auto-matic selection of the appropriate protein additive based on age and an osmolality alert, which is triggered only in patients with-out a central line. Users are reminded to add phosphorus and calcium if they have not done so and to add heparin for central line TPN. Reminders can be complex based on previous user selections; for example, if a user chooses not to order trace ele-ments, then not only is he or she prompted to add selenium, zinc, and chromium, but also the dose ranges of these additives will now be checked. Users are prevented to submit an order unless it contains patient name and medical record number, physician name, and identification number. Other data entered by the user are validated against a possible range (eg, infusion time cannot exceed 24 hours). Patient data are automatically populated using a link to the hospital admission, discharge, and transfer system.
Data collected during a designated quality improvement project included the number of parenteral nutrition orders and the frequency and type of errors as identified by the pharmacist. Data were collected during a control period (October 2, 2000, to No-vember 14, 2000) immediately before and during 2 intervention periods after (intervention 1: November 15, 2000, to December 31, 2000; intervention 2: August 27, 2002, to October 13, 2002) the implementation of TPNCalculator. Data analysis was done with2
test. Prescribers, pharmacists, and nurses were surveyed online on their experiences and opinions in the course of using TPNCalcu-lator.
RESULTS
During the control period, average patient census was 32.3. During this time, 557 TPN orders were written compared with 471 orders and 656 orders in the 2 intervention periods (average census: 30.2 and 32.4). The number of TPN orders per patient per day was not significantly different between the time pe-riods (control: 0.39; intervention 1: 0.35; intervention 2: 0.43 orders per patient per day). During the control and the first intervention periods, only half of the orders were available to the pharmacy by order deadline at noon (51.7% control, 48.6% intervention). Prescribers most commonly cited lack of time as the cause for the delay (86%). During the first interven-tion period, lack of time (54%) and reduced staffing during the holidays (31%) were given as a frequent cause of delays.
During the control period, a total of 60 errors (10.8 errors per 100 orders) that required the pharmacist to contact the provider were detected compared with only 20 errors (4.2 errors per 100 orders) in the first and 8 errors (1.2 errors per 100 orders) in the second intervention period. This represents a 61% reduction in error rate (P⬍ .01) in the first and an 89% reduc-tion in error rate (P⬍.001) in the second intervention period.
Error type distribution was significantly different among the 3 groups (Fig 2). During the control
riod use of the paper form, prescribers were signifi-cantly more likely to order fluids that exceeded the osmolality guidelines, make calculation errors, and generate orders that demonstrated other knowledge deficiencies (eg, no heparin ordered for central line TPN). No statistical difference was noted in the num-ber of orders with insufficient fluid amounts to allow all additives to be dissolved between the control period and both intervention periods, although there was a trend toward a reduction in intervention 2. There was an increase from 1.7 incomplete orders per 100 TPN orders in the control period to 2.3 in the intervention period (not significant). Omission of the page number on the order form (order identifier) contributed to 92% of all incomplete orders in the intervention period (Fig 3). In the second interven-tion period, there was a 100% reducinterven-tion of omissions (P⬍.001).
After a 6-month period, when all stakeholders had an opportunity to interact with the TPNCalculator, 84 providers, nurses, and pharmacists were surveyed using an online questionnaire. Twenty-eight pre-scribers responded, including 22 postgraduate year 1 residents (100% of interns) and 6 neonatal nurse practitioners (100% of neonatal nurse practitioners). Seven percent had used TPNCalculator⬍5 times per week, 21% used it 5 to 9 times, 36% used it 10 to 19 times, and 36% used it ⬎20 times per week. On a scale from 1 (very easy) to 5 (very difficult), TPNCal-culator was rated 1.5. On a Likert scale (1⫽strongly agree, 3⫽neutral, 5⫽strongly disagree), prescribers were asked to compare the TPNCalculator with the paper form (Table 1).
In comparison with the paper form, TPNCalcula-tor was found to be easier to learn and to use. Pre-scribers believed that it protected against errors, Fig 2. Error type distribution.
saved time, was helpful, and constituted an improve-ment. Users were neutral to the statement that the TPNCalculator provided a better learning experi-ence.
As part of the survey, prescribers were asked about the series of potential problems associated with an ordering tool. Prescribers were neutral to-ward the statements that TPNCalculator carried the risk of inaccurate programming (3.3), had limited scope and was not designed to handle all clinical cases (3.0), had potentially incomplete information/ missing data (3.0), redirected provider priorities (3.6), and generated false expectation such as being alerted to all problems (3.0). Prescribers disagreed that TPNCalculator caused data overload in the user (4.1).
Using separate online questionnaires, we also sur-veyed 8 pediatric pharmacists (73% of pharmacists) and 43 neonatal nurses (42% of nurses). Comparing TPNCalculator and the paper form on a Likert scale from 1 (strongly agree) to 5 (strongly disagree), phar-macists and nurses agreed that TPNCalculator was easier to read, was easier to learn, and resulted in less need for order clarification. Pharmacists reported that TPNCalculator resulted in fewer errors, saved time, and was helpful for data entry (Table 2).
DISCUSSION
ADEs are common, affecting as many as 1 in every 25 hospitalized patients.12 The ADE Prevention Study Group reported that few risk factors are pa-tient related, suggesting that rather than target ADE-prone individuals, prevention strategies should fo-cus on improving medication systems.13
After identifying a medication ordering process with significant risk for ADEs, we designed,
imple-mented, and evaluated a web-based medical infmation system (TPNCalculator) to improve the or-dering process and reduce ADEs. We hypothesized that underlying system failures could be corrected with inexpensive, pragmatic problem-solving ap-proaches. A deliberate attempt was made to address only the “broken” link, thereby preserving the ma-jority of the existing ordering process. Furthermore, we based the interface design on existing paper forms to provide users with a familiar format and maximum comfort level. This minimized required training (usually ⬍10 minutes) and permitted the existing order-processing flow to remain intact. The use of participant/observers (prescribers and phar-macists) as programmers, designers, and testers eliminated the need for extensive life cycle assess-ment (systematic set of procedures for compiling and examining the inputs and outputs of materials and energy and the associated environmental impacts directly attributable to the functioning of a product or service system throughout its life cycle), including evaluation and analysis of usage, resource utiliza-tion, patient care implicautiliza-tion, and failure modes.11 Using this approach, the complete design process (1 Cold Fusion programmer [C.U.L.]) and quality as-surance (1 nutritionist [J.M.C.] and 1 pharmacist [K.G.C.]) took 3 weeks (⬃200 man-hours), which starkly contrasts with other ongoing clinical medical informatics projects at JHH. Modifications of TPN-Calculator require minimal intervention, because any change made on the web server is immediately available to all users throughout the system.
In this study, we were able to use a close-held, multitalented, small development group to develop a rapid solution in cooperation with local informa-tion services. This model was highly effective in cre-ating utilities for in-house support of clinicians and has in the meantime been duplicated in several other JHH projects. It must be pointed out that a develop-ment model such as ours is frequently not available to those outside the clinical process. Reasons for this may include Health Insurance Portability and Ac-countability Act concerns, institutional policies and interests, lack of access to data from legacy systems, and the inability to assemble a small team with all of the required skills.
Evaluation of error rates and user surveys revealed an overwhelming success. We reduced the total number of errors by 89%. Calculation errors were eliminated through TPNCalculator, and knowledge problems drastically reduced. Users of the system (including those who were involved only peripher-ally, eg, nurses) were enthusiastic and supportive and compared it favorably with the previous paper-based system. We experienced significant pressure to make TPNCalculator available to all units in the children’s hospital and to an affiliated hospital NICU once residents had used it during their NICU rota-tion. As a result, 6 months after the development of TPNCalculator, it was deployed throughout the JHH Children’s Medical & Surgical Center.
In our survey, users did not estimate the risks of TPNCalculator to be higher than in the paper system; TABLE 1. Prescriber Survey (N⫽28)
Average Rating Range
Easy to use 1.2 1–3
Easy to learn 1.3 1–2
Protects against errors 1.3 1–2
Saves time 1.1 1–2
Helpful 1.2 1–2
Improvement 1.2 1–2
Better learning experience 2.5 1–4
Comparing TPNCalculator with paper from on a 5-point Likert scale (1⫽strongly agree, 5⫽strongly disagree).
TABLE 2. Nurse/Pharmacist Survey
Nurses (n⫽43)
Pharmacists (n⫽8)
Easier to read 1.7 1.6
Easy to learn 1.9 1.4
Reduced transcribing errors 2.1 1.6
Saved time 2.4 1.8
Helpful for data entry 2.5 1.8
Significant improvement 1.9 1.3
Less need for clarification 1.9 1.9
Contained less errors N/A 1.3
Improved order timeliness N/A 2.5
however, experience14 has shown that the deploy-ment of a new software tool may introduce a new type of failure mode in the form of programming/ software errors. An analysis of 3140 medical device recalls conducted by the Food and Drug Administra-tion between 1992 and 1998 revealed that 242 (7.7%) were attributable to software failures.15Of those, 192 (79%) were caused by software defects that were introduced when changes were made to the software after its initial production and distribution. Although TPNCalculator corrected a number of system prob-lems, in its initial version, it had its own system failure: a drastic increase in the TPN orders without order identification (page number). Once this (small and insignificant for patient safety) problem was identified, TPNCalculator was first modified to prompt the user for a page number, and in a later version, an automatic page number creation rule was added. Data from the second intervention period demonstrated that omissions had been eliminated without adding new errors. The example of the “missing page number” emphasizes that rapid cycle evaluation is able to handle software design prob-lems through frequent series of design, evaluation, and redesign. An attempt to design a “perfect” and completely error-free application from the start would have required considerable planning, consult-ing, and review before implementation, leading to long cycle times, which might have derailed the pro-cess long before completion. In addition, the desire to design the “perfect” system might have resulted in an overengineered product with additional features, complicating its use and making it unattractive to users.
Federal guidelines require that medical software systems become subject to validation requirements (Title 21 Code of Federal Regulations §11.10[a]). Such systems must be validated to ensure accuracy, reli-ability, and consistent intended performance. Reduc-ing total development time to 3 weeks inherently means that validation features such as requirement and feature lists are mainly implicit in the design and not necessarily expressed in detail. In our case, this was possible because the software was modeled on an existing paper form in use at JHH. The paper form implicitly determined system inputs and outputs, functionality, interface definition, user interactions, safe ranges, limits, and default values. Using domain experts for the testing allowed for testing of all po-tential clinical scenarios using patient data available during the development period. The robustness of the system was tested in a clinical environment by allowing users to use TPNCalculator while for a period of 3 months all calculations were duplicated in the pharmacy to ensure safety. During that period, users generated ⬎1800 orders, which spanned the complete range of possible inputs. The existing paper order form served as a safety back-up but was never used.
Developing TPNCalculator would not have been possible without certain information infrastructure already in place at JHH. We relied on the availability of public workstations in all clinical areas where
providers might order TPN. If access to computers would have required traveling (even a short dis-tance) or a waiting period, then we suspect that users’ enthusiasm for this application would have been drastically diminished. Furthermore, the deci-sion by JHH management to provide an Internet browser on all workstations was crucial to imple-mentation. Allowing Internet browsing on public clinical workstations is not a universal practice, as demonstrated by differences even among the hospi-tals within the Johns Hopkins Health System.
We have demonstrated that pragmatic approaches to ADEs involving simple medical information sys-tems can be successful in both reducing the number of errors and increasing user satisfaction. The ap-proach used in the development of TPNCalculator, which could be called “find it, fix it, and forget it,” identifies a specific problem and uses existing infra-structure to develop a pragmatic medical informatics solution to the problem while leaving the remaining system intact. By limiting the number of people in-volved in the process of problem identification, de-velopment, testing, and deployment, the used re-sources such as time and manpower can be drastically reduced without losing effectiveness. By shortening the time interval between the birth of an idea and its implementation, we increased the speed of innovation. Although this approach carries a greater risk for software design flaws, in our opinion, this risk can be minimized by using participant/ observers in the development and is further offset by the significant gains through early implementation and cost reduction.
CONCLUSION
Low-cost, pragmatic approaches using Internet technology in the design of medical information sys-tems might pose a viable alternative for the preven-tion of ADEs.
ACKNOWLEDGMENTS
This work was supported by the Center for Innovation and Quality Patient Care at the Johns Hopkins Hospital.
We are grateful to Harold P. Lehmann, MD, PhD, for critical review of this manuscript.
REFERENCES
1. Kohn LT, Corrigan JM, Donaldson MS, eds.To Err Is Human: Building a Safer Health System. Washington, DC: National Academy Press; 1999 2. Leape LL, Bates DW, Cullen DJ, et al. Systems analysis of adverse drug
events. ADE Prevention Study Group.JAMA. 1995;274:35– 43 3. Leape LL, Kabcenell AI, Gandhi TK, Carver P, Nolan TW, Berwick DM.
Reducing adverse drug events: lessons from a breakthrough series collaborative.Jt Comm J Qual Improv. 2000;26;321–331
4. Leape LL. Error in medicine.JAMA. 1994;272:1851–1857
5. Kaushal R, Bates DW, Landrigan C, et al. Medication errors and adverse drug events in pediatric inpatients.JAMA. 2001;285:2114 –2120 6. Horn W, Popow C, Miksch S, Kirchner L, Seyfang A. Development and
evaluation of VIE-PNN, a knowledge-based system for calculating the parenteral nutrition of newborn infants.Artif Intell Med. 2002;24:217–228 7. Hermansen MC, Kahler R, Kahler B. Data entry errors in computerized
nutritional calculations.J Pediatr. 1986;109:91–93
8. Legler JD. Physicians’ accuracy in manual computation.MD Comput. 1990;7:155–159
10. Vincer MJ, Murray JM, Yuill A, Allen AC, Evans JR, Stinson DA. Drug errors and incidents in a neonatal intensive care unit. A quality assur-ance activity.Am J Dis Child. 1989;143:737–740
11. Cohen MR, Senders J, Davis NM. Failure mode and effects analysis: a novel approach to avoiding dangerous medication errors and accidents.
Hosp Pharm. 1994;29:319 –330
12. Brennan TA, Leape LL, Laird NM, et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I.N Engl J Med. 1991;324:370 –376
13. Bates DW, Miller EB, Cullen DJ, et al. Patient risk factors for adverse drug events in hospitalized patients. ADE Prevention Study Group.
Arch Intern Med. 1999;159:2553–2560
14. McDaniel JG. Improving system quality through software evaluation.
Comput Biol Med. 2002;32:127–140
15. US Department Of Health and Human Services Food and Drug Admin-istration. General Principles of Software Validation; Final Guidance for Industry and FDA Staff. Available at: www.fda.gov/cdrh/comp/ guidance/938.html
UBIQUITOUS DETAIL PERSONS
“Drug company representatives are a major presence. They sponsor Journal Club (where trainees learn to review new data and research), they pay for many of our weekly speakers and regularly offer free dinners for the residents and faculty. They enjoy free access to our mailboxes and regularly detail our trainees in their offices, hallways and in our little kitchen.
This is not uncommon. Meredith Rosenthal at the Harvard School of Public Health reported in the New England Journal of Medicine that the industry spends roughly $15.7 billion annually marketing medications, with $44.8 million dedicated to detailing individual physicians, or roughly $5000 to $11 000 a doctor a year.
Studies indicate that most physicians meet with pharmaceutical representatives four times a month.“
Companies get too close for med school comfort.New York Times. January 20, 2004
DOI: 10.1542/peds.113.4.748
2004;113;748
Pediatrics
Christoph U. Lehmann, Kim G. Conner and Jeanne M. Cox
Preventing Provider Errors: Online Total Parenteral Nutrition Calculator
Services
Updated Information &
http://pediatrics.aappublications.org/content/113/4/748 including high resolution figures, can be found at:
References
http://pediatrics.aappublications.org/content/113/4/748#BIBL This article cites 13 articles, 0 of which you can access for free at:
Subspecialty Collections
http://www.aappublications.org/cgi/collection/nutrition_sub
Nutrition
http://www.aappublications.org/cgi/collection/neonatology_sub
Neonatology
sub
http://www.aappublications.org/cgi/collection/fetus:newborn_infant_
Fetus/Newborn Infant
following collection(s):
This article, along with others on similar topics, appears in the
Permissions & Licensing
http://www.aappublications.org/site/misc/Permissions.xhtml in its entirety can be found online at:
Information about reproducing this article in parts (figures, tables) or
Reprints
DOI: 10.1542/peds.113.4.748
2004;113;748
Pediatrics
Christoph U. Lehmann, Kim G. Conner and Jeanne M. Cox
Preventing Provider Errors: Online Total Parenteral Nutrition Calculator
http://pediatrics.aappublications.org/content/113/4/748
located on the World Wide Web at:
The online version of this article, along with updated information and services, is
by the American Academy of Pediatrics. All rights reserved. Print ISSN: 1073-0397.
