Impact of Clinical Decision Support within
Computerized Physician Order Entry
A Systematic Review
Jennifer Gillert
Candidate for Honours B.Sc. in Health Studies with Health Informatics Option
Faculty of Applied Health Science
University of Waterloo
1. INTRODUCTION
Medication errors and adverse drug events (ADEs) are considered to be common, costly, clinically important and most importantly, preventable (1). Approximately 28% of ADEs are associated with medication errors, of which 56% occur during medication prescription (2). The potential impact of computer-assisted decision-making for clinicians during the ordering process has been widely discussed as a method to reduce errors and ADEs (1,3-5). Clinical decision support (CDS) within CPOE offers physician unsolicited advice and can include a range of functionality to aid in the electronic prescription of medications.
The combination of computerized order entry and decision support has been acknowledged as promoting safe medication practice (7). The studies investigating the impact of CDS embedded within CPOE vary in setting, users, system design, implementation strategy and an unlimited number of other characteristics, and they measure a variety of different forms of outcome. The purpose of this systematic review is to examine the evidence available on the impact of clinical decision support within computerized physician order entry in an electronic medical record.
2. METHODS
2.1 Search and selection criteria
Studies were identified through 3 electronic databases: PubMed, Web of Science and CSA Illumina Databases. In addition, the references of the selected articles were reviewed to identify additional studies meeting the inclusion criteria. A total of seventeen studies (N = 17) were included in the review.
Inclusion criteria:
• Acute care, inpatient hospital
• Electronic medical record including a physician order entry with active, synchronous clinical decision support
• Studies of orders from staff physicians, fellows, residents or medical students in any clinical area
• Studies since 1996
Studies were not excluded based on study design. Quality screening of design and methods was not done due to relatively few available studies for the research question. Instead, concerns and limitations of some studies are discussed in the review.
3. RESULTS
Information on the following study characteristics were extracted from the seventeen studies: population, duration, study design, relevant types of clinical decision support, outcomes measured, and results.
“Impact” in the clinical setting can encompass many measures. In order to standardize the results, the impact outcomes of the studies were categorized by types of outcomes, presented in Table 2 with the number of studies that included at least one of the outcome types.
Each relevant measure of impact outcome of the seventeen studies is summarized in Tables 3.1 - 3.4. The results from each specific measure from the studies have been categorized by impact type.
Table 2: Summary of impact outcome classifications
Outcome Classification Type Impact Outcome
Studies with one or more of the outcome type
Guideline compliance 10
Errors 6 Practitioner
Decision support utilization 1
Adverse drug events 2
Patient Intermediate outcomes 2 Length of stay 4 Cost 5 Hospital Administrative Resource utilization 2 Qualitative Opinions 1
Table 3.1: Impact outcomes: Practitioner
Outcomes measured Specific Outcomes Ref
Compliance to corollary orders (Immediate, 24-hr, and hospital-stay)
Increased guideline compliance (p < 0.0001)
(19) Appropriate prescription Increased rate of appropriate
orders (p < 0.001)
(9) Recommended Histamine2-blocker order selection Increase in recommended order
selection (p < 0.001) Orders exceeding recommended dose Fewer orders exceeding
recommended dose (p < 0.001) Appropriate medication frequency for ondansetron Increased rate of appropriate
frequency (p < 0.001)
Heparin ordering consequent to bed rest order Increased guideline compliance (p < 0.001)
(23)
Ordering of recommended asthma treatments Increased ordering of
recommended treatments (p < 0.001)
(8)
Use of antiinfective agents Fewer doses of antiinfective agents (p < 0.001)
(15) Use of traditional sliding scale orders compared to
minimal intervention orders
Fewer traditional sliding scale orders (p < 0.0003)
(11) Hospital policy violations Fewer rule violations (p < 0.001) (20) Duration of vancomycin treatment Fewer treatment days per
physician (p = 0.05) Number of vancomycin orders Fewer treatment days per
treatment (p = 0.05)
(22)
Time for appropriate clinical response to alerts for untreated hypokalemia or hypomagnesemia, and digoxin, magnesium and potassium
Unclear effect on compliance (dependent on orders, conditions)
(16)
Guideline Complia
nce
Use of recommended early aspirin and beta-blockers
Unclear effect on compliance (dependent on orders, conditions)
(7)
Non-missed-dose medication errors Fewer non-missed dose medication errors (p < 0.0001)
(6) Nonintercepted serious medication errors Fewer nonintercepted serious
medication errors (p = 0.01) (12) Practi tioner Errors
(p = 0.003) Rate of illegible, incomplete and drug therapy
errors
Fewer total medication errors (p < 0.001)
(17) Frequency of order adjustments prompted by alert Lower order adjustment rate (p <
0.01)
(18) Medication prescribing errors Fewer medication prescribing
errors (p < 0.001) Number of potential ADEs (duplicate therapy,
inappropriate dose/interval/route, wrong drug/unit, allergy, drug interaction)
Fewer potential ADEs (p < 0.001)
(20)
Incorrect dose No difference in rate of incorrect
dose (p = 0.4) (14) CD S utiliza tion
Compliance with Acute Coronary Syndrome (ACS) order set
Unclear impact (condition) (7)
Table 3.2: Impact outcomes: Patient
Outcomes measured Specific Outcomes
Adverse drug events (non-intercepted serious medication errors)
Fewer ADEs (p = 0.0003) (6)
ADEs Adverse drug events caused by antiinfective agents Fewer ADEs (p = 0.018) (15)
Maximum serum creatinine levels No difference in intermediate outcome (p = 0.28) (19) Patien t Interme diate Outc omes
Changes in renal function Improved intermediate outcome (p < 0.001)
(9)
Table 3.3: Impact outcomes: Hospital Administrative
Outcomes measured Specific Outcomes
Length of hospital stay No difference in LOS (p = 0.94) (19) Length of hospital stay No difference in LOS (p = N/A) (8)
Length of hospital stay Reduced LOS (p = 0.009) (9)
Leng
th o
f
stay
Length of hospital stay Reduced LOS (p < 0.001) (15)
Hospital charges No difference in costs (p = 0.68) (19)
Hospital and pharmacy costs No difference in costs (p = 0.52) (9)
Inpatient charges No difference in costs (p = N/A) (8)
Cost of antiinfective treatment Reduced cost (p < 0.001) Cost of hospitalization Reduced cost (p < 0.001)
(15)
Cos
ts
Charge savings Reduced cost (p = N/A) (13)
Hospital-level vancomycin utilization No significant decrease in drug utilization (p = N/A) (22) Hospi tal Ad m inistrative Resource Utiliza ti o n
Proportion of cancelled lab orders Fewer redundant lab orders (p < 0.001)
Table 3.4: Impact outcomes: Qualitative Outcomes
Outcomes measured
Specific
Outcomes
Opinion of order sets on efficiency of order entry Positive opinion Opinion of guidelines and decision support interventions as an aid for
quality of patient care
Positive opinion Opinion of graphical representation of laboratory results on the test
ordering
Negative opinion Opinion of guidelines and decision support interventions on efficiency
of order entry No clear opinion Qualitati v e Outc omes Opinio ns (Likert Sca le)
Opinion of guidelines and decision support interventions on the provider care
No clear opinion
(21)
4.3 Limitations
Kaplan describes an informatics application as multi-faceted, including social, cultural, organizational, and cognitive aspects (25). The dynamic interaction between users of such a system and the technology itself is not constant between settings. Such a system is socio-technological in nature, dependent on not only the technology itself but also on, for example, the number of available workstations or technical support. Six of the seventeen studies included in this review were conducted at one large academic tertiary-care hospital (6,9,12,13,22,23) and the specific socio-technological characteristics of this particular hospital is thus overrepresented. The method of development, implementation and monitoring of guidelines influence the likelihood of adherence to clinical guidelines (24) and the outcomes from the particular hospital will not necessarily be generalizable to other settings.
Comparing the study outcomes from different hospitals, it is difficult to determine impact from the forms of decision support when each setting has a system with varying combinations of clinical decision support. It is difficult to assess, based on the information collected for this review, which specific forms of support are effective and which are not. There may also be an interaction effect of the forms of support when implemented together. The system design can also influence the impact. For example if a decision support system alerts clinicians of non-patient-specific information (26) or fires too frequently, the alerts may be ignored. The generalizability for any of the outcomes may not be applicable to a setting outside that in which it was conducted in, as noted by several of the authors (8,12,19,23).
In at least three of the studies simultaneous implementation and before-after outcome comparisons makes it difficult to distinguish outcomes due to the implementation of CDS within CPOE versus the impact due to the CPOE itself (8,12,20). Similarly, with decision support in the form of synchronous best-practice guidelines, it may be difficult to distinguish between the impact due to the decision support intervention and the impact due to the guideline it is based on (19).
Randomized controlled trials, although considered the gold standard for evaluation, are not always possible due to the complex nature of the implementation of a CPOE system (6). A handful of the authors (7,12,16) discuss the before-after comparison as a potential threat to validity due to the possible interference of history, where another factor occurring in the study period that may influence the outcome. One example is the simultaneous implementation of a discharge planning tool (7). Additional threats to internal validity with this study design relate to accumulation of experience with a new system when a series of measures are taken over time. On the other hand, the importance of the randomized controlled trials in informatics may not be as great when investigating outcomes with contextual importance (25), like the impact of these interventions in busy clinical workflow.
5. CONCLUSION
Studies investigating the impact of synchronous clinical decision support (CDS) within computerized physician order entry (CPOE) measure a variety of outcomes relating to the practitioner, the patient, hospital administration outcomes as well as qualitative perceptions of impact. Based on the available studies, CDS improves practitioner outcomes such as guideline compliance and errors. Limited availability
of studies and mixed study outcomes lead to an inconclusive impact on patient and hospital administrative outcomes. Clinician perceptions of the impact of these applications are also inconclusive. The impact outcomes are measured in a socio-technological setting, making it difficult to generalize one study outcome to another setting. Additional studies are required to further understand the impact of decision support within computerized physician order entry.
7. REFERENCES
(1) Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Archives of internal medicine 2003 Jun 23;163(12):1409-1416.
(2) Bates D, Cullen D, Laird N. Incidence of adverse drug events and potential adverse drug events: implications for prevention. JAMA 1995;274:29-34.
(3) Rochon P, Field T, Bates D, Lee M, Gavendo L, Erramuspe-Mainard J, et al. Computerized Physician Order Entry with Clinical Decision Support in the Long-Term Care Setting: Insights from the Baycrest Centre for Geriatric Care. J Am Geriatr Soc 2005;53:1780-1789.
(4) Rollman BL, Hanusa BH, Lowe HJ, Gilbert T, Kapoor WN, Schulberg HC. A Randomized Trial Using Computerized Decision Support to improve Treatment of Major Depression in Primary Care. J Gen Intern Med 2002;17:493-503.
(5) Palen TE, Raebel M, Lyons k, Magid DM. Evaluation of Laboratory Monitoring Alerts Within a Computerized Physician Order Entry System for Medication Orders. The American Journal of Managed Care ;12(7):389-395.
(6) Bates D, Teich JM, Lee J, Seger DL, Kuperman GJ, Ma'luf N, et al. The Impact of Computerized Physician Order Entry on Medication Error Prevention. J Am Med Inform Assoc 1999;6(4):313-321.
(7) Ozdas A, Speroff T, Waitman LR, Ozbolt J, Butler J, Miller RA. Integrating "best of care" protocols into clinicians' workflow via care provider order entry: impact on quality-of-care indicators for acute myocardial infarction. Journal of the American Medical Informatics Association : JAMIA. 2006 Mar-Apr;13(2):188-96. Epub: 2005 Dec 15.
(8) Chislom D, McAlearney A, Veneris S, Fisher D, Holtzlander M, McCOy K. The role of computerized order sets in pediatric inpatient asthma treatment. Pediatric Allergy Journal 2006;17:199-206. (9) Chertow GM, Lee J, Kuperman GJ, Burdick E, Horsky J, Seger DL, et al. Guided Medication Dosing
for Inpatients with Renal Insufficiency. JAMA 2001;286(22):2839-2844.
(10) Kuperman GJ, Bobb A, Payne TH, Avery AJ, Gandhi TK, Burns G, et al. Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc 2007;14(1):29.
(11) Achtmeyer CE, Payne TH, Anawalt BD. Computer Order Entry System Decreased Use of Sliding Scale Insulin Regimens. Methods Inf Med 2002;41:277.
(12) Bates D, Leape L, Cullen D, Laird N, Petersen LA, Teich JM, et al. Effect of Computerized Physician Order Entry on a Team Intervention on Prevention of Serious Medication Errors. JAMA
1998;280(15):1311-1316.
(13) Bates DW, Kuperman GJ, Rittenberg E, Teich JM, Fiskio J, Ma'luf N, et al. A Randomized Trial of a Computer-based Intervention to Reduce Utilization of Redundant Laboratory Tests. Am J Med 1999;106(104).
(14) Eslami S, Abu-Hanna A, de Keizer N, de Jonge E. Errors Associated with Applying Decision Support by Suggesting Default Doses for Aminoglycosides. Drug Saf 2006;29(9):803.
(15) Evans RS, Pestotnik SL, Classen DC, Clemmer TP, Weaver LK, Orme JF, et al. A computer-assister management program for antiotics and other antiinfective agents. New England Journal of Medicine 1998;338:232-238.
(16) Galanter WL, Polikaitis A, Didomenico RJ. A Trial of Automated Safety Alerts for Inpatient Digoxin Use with Computerized Physician Order Entry. J Am Med Inform Assoc 2004;11(4):270-277. (17) Igboechi CA, Ng CL, Yang CS, Buckner AN. Impact of Computerized Prescriber Order Entry on
Medication Errors at an Acute Tertiary Care Hospital. Hosp Pharm 2003;38(3):227. (18) Oppenheim MI, Vidal C, Velasco FT, Boyer AG, Cooper MR, Hayes JG, et al. Impact of a
Computerized Alert During Physician Order Entry on Medication Dosing in Patients with Renal Impairment. AMIA 2002 Annual Symposium Proceedings; 2002. p. 577.
(19) Overhage JM, Tierney WM, Zhou X, McDonald C. A Randomized Trial of "Corollary Orders" to Prevent Errors of Omission. J Am Med Inform Assoc 1997;4:364-375.
(20) Potts AL, Barr FE, Gregory DF, Wright L, Patel NR. Computerized Physician Order Entry and Medication Errors in a Pediatric Critical Care Unit. Pediatrics 2004;113(1):59.
(21) Rosenbloom S, Talbert D, Aronsky D. Clinicians' perceptions of clinical decision support integrated into computerized provider order entry. International Journal of Medical Informatics 2004;73:433-441.
(22) Shojania KG, Yokoe D, Platt R, Fisko J, Ma'luf N, Bates D. Reducing Vanomycin Use in Utilizing a Computer Guideline. J Am Med Inform Assoc 1998;5:554-562.
(23) Teich JM, Merchia PR, Schmiz JL, Kuperman GJ, Spurr CD, Bates D. Effect of Computerized Physician Order Entry on Prescribing Practrices. Arch Intern Med 2000;160:2741-2747. (24) Coiera E. Guide to health informatics. Second edition ed. London: Hodder Aarnold; 2003. (25) Kaplan B. Evaluating informatics applications - some alternative approaches: theory, social
interactionism, and call for methodological pluralism. International Journal of Medical Informatics 2001;64:39-56.
(26) Galanter WL, Didomenico RJ, Polikaitis A. A trial of automated decision support alerts for
contraindicated medications using computerized physician order entry. Journal of the American Medical Informatics Association : JAMIA. 2005 May-Jun;12(3):269-74. Epub: 2005 Jan 31.
