Electronic prescribing

Electronic prescribing

A critical appraisal of the literature

Phil Hider

New Zealand

Health Technology Assessment

Department of Public Health and General Practice

Christchurch School of Medicine

Christchurch, NZ.

HEALTH TECHNOLOGY ASSESSMENT

Department of Public Health and General Practice

Christchurch School of Medicine and Health Sciences,

Christchurch, New Zealand

Electronic prescribing

A critical appraisal of the literature

Phil Hider

2002 New Zealand Health Technology Assessment Clearing House (NZHTA) ISBN 1-877235-25-3

ACKNOWLEDGMENTS

This report was commissioned by the New Zealand Ministry of Health. Significant input into the development of the report was received from Dr Colin Feek, Chief Medical Advisor, Ministry of Health.

Staff at NZHTA completed this report. It was prepared by Dr Phil Hider (Senior Research Fellow). Dr Ray Kirk (Director) reviewed the final draft of the report. Mrs Susan Bidwell (Information Specialist) undertook the search for references and co-ordinated the retrieval of articles. Mrs Bidwell and Dr Kaaren Mathias (Research Fellow) translated non-English articles. Ms Tracy Smitheram, Miss Becky Mogridge and Mrs Joan Downey assisted with the retrieval of articles. Ms Cecilia Tolan and Mrs Ally Reid provided administrative support and assistance with document formatting.

Dr Robert Walton, Honorary Senior Clinical Lecturer, Imperial Cancer Research Fund General Practice Research Group, University of Oxford Department of Public Health and Primary Care, Institute of Health Sciences, Oxford, UK, was an external peer-reviewer and provided comments on a late draft of this report.

Sub-editing was performed by Mrs Carolyn Davies, Freelance Journalist, Christchurch.

The Canterbury Medical Library and the Institute of Health Sciences, University of Oxford assisted with the retrieval of articles.

NZHTA is a research unit of the University of Otago funded under contract to the Ministry of Health.

DISCLAIMER

NZHTA takes great care to ensure the accuracy of the information supplied within the project timeframe, but neither NZHTA nor the University of Otago can accept responsibility for any errors or omissions that may occur. The reader should always consult the original database from which each abstract is derived along with the original articles before making decisions based on a document or abstract. All responsibility for action based on any information in this report rests with the reader. NZHTA and the University of Otago along with their employees accept no liability for any loss of whatever kind, or damage, arising from the reliance in whole or part, by any person, corporate or natural, on the contents of this paper. This document should not be used as personal health advice; people seeking individual medical advice are referred to their physician. The views expressed in this report are those of NZHTA and do not necessarily represent those of the University of Otago, or the New Zealand Ministry of Health.

COPYRIGHT

This work is copyright. Apart from any use as permitted under the Copyright Act 1994 no part may be reproduced by any process without written permission from New Zealand Health Technology Assessment. Requests and inquiries concerning reproduction and rights should be directed to the Director, New Zealand Health Technology Assessment, Christchurch School of Medicine and Health Sciences, P O Box 4345, Christchurch, New Zealand.

CONTACT DETAILS

New Zealand Health Technology Assessment

The Clearing House for Health Outcomes and Health Technology Assessment Department of Public Health and General Practice

Christchurch School of Medicine and Health Sciences P O Box 4345

Christchurch New Zealand

Tel: +64 3 364 1152 Fax: +64 3 364 1152 E-mail: [email protected]

EXECUTIVE SUMMARY AND CONCLUSIONS

Background

Electronic prescribing involves the use of computers to generate prescriptions often with the assistance of computerised decision support. Prescribing involves the need to link large amounts of information and computerisation may improve performance and result in better health outcomes.

Aim

To systematically review the effectiveness of electronic prescribing to improve practitioner performance and patient-oriented outcomes.

Data sources

A comprehensive search was undertaken of a range of databases including Medline, Embase, Current Contents, Cinahl, Healthstar, Science Citation Index, International Pharmaceutical Abstracts, Cochrane Controlled Trials Register, Index New Zealand between 1990 – May 2001. A range of other library and Internet-based catalogues was also examined along with references listed in publications obtained during the review.

Study selection

Study selection criteria were based on: computerised assistance with prescribing medication limited to ordering and transcribing, any health professional and changes in surrogate outcomes, prescriber performance or patient outcomes, involving any intervention study with a comparison group and data from before and after the intervention.

Unpublished studies, conference proceedings, interventions outside of the clinical setting, those based in non-Western countries, and those lacking clear descriptions of methods and results were excluded.

Data extraction

Data was extracted by a single reviewer using data collection forms developed by the Cochrane Effective Practice and Organisation of Care Review Group (http://www.abdn.ac.uk/hsru/epoc/).

Results

Some 772 citations were identified by the search; 122 studies were obtained and considered against the inclusion criteria; 52 studies met these criteria and were formally appraised and included in this review. Three systematic reviews have assessed the effectiveness of computer support for general prescribing. Physician order entry has been evaluated in eight studies and electronic alerts after prescribing has been the subject of another four studies. Thirty-one studies have considered the use of electronic prescribing in the treatment of various conditions including: anticoagulation (14 studies), infection (6), hypertension (3), hyperlipidaemia (1), diabetes (2), asthma (2), the induction of labour (1), and anaesthesia (2). Eight studies were based in primary care.

A number of methodological problems were evident among the studies. Randomised trials have been infrequently conducted and non-randomised evaluations have been unable to limit the potential for bias. Several studies have not included an adequate sample size to provide sufficient statistical power. Relatively few evaluations have examined the effects of computerised prescribing on patient outcomes. Methodological challenges, such as difficulties with blinding study participants, exist with undertaking an evaluation of a computerised prescribing intervention.

Computerised support for general prescribing

Electronic prescribing can improve physician performance especially with respect to prescribing potentially toxic drugs with narrow therapeutic ranges where drug monitoring is commonly conducted (Level I evidence).

Relatively few studies have assessed patient outcomes. The effect of electronic prescribing on patient outcomes is not clear although there is some evidence that it can reduce the frequency of adverse drug reactions and shorten length of hospital stay (Level I evidence).

Physician Order Entry

Physician Order Entry (POE) is a computer application that enables health professionals (usually doctors) to order diagnostic and treatment services electronically. POE can effectively improve physician prescribing habits and reduce medication errors (Level III-3 evidence). However, the effect of POE on health outcomes, especially adverse drug events (ADEs), has not yet been well-established (Level III-3 evidence). POE increased the time required by doctors to order medication (Level III-3 evidence). There is inconsistent evidence about whether consultations were lengthened (Level II evidence). POE has been able to reduce patient charges even though their length of stay was not changed (Level II evidence). Sometimes doctors were not satisfied with the introduction of POE (Level II evidence). Corollary orders reduced medication errors but not patient length of stay or charges (Level II evidence). Most evaluations have been conducted in a small number of specialised hospital settings (e.g., Harvard) where unique and sophisticated computer systems allow access to clinical notes and other laboratory data as well as administrative information.

The provision of electronic alerts after prescribing

Computerised reviews of prescriptions and the electronic submission of alerts to prescribers about drug interactions have been introduced into large health care organisations with sophisticated computer systems that incorporate clinical and administrative information. Computerised checks and electronic alerts can reduce the incidence of dangerous drug interactions and the time before the drugs are changed in hospitals with large, sophisticated computer systems (Level III-3 evidence). The intervention can improve prescribing by physicians but a small proportion of doctors appear to ignore repeated warnings (Level III-3 evidence). Prescribing alerts can reduce ADEs and improve other patient health outcomes such as the risk of renal impairment (Level III-3 evidence). Current evidence suggests that alerts do not reduce length of hospital stay or inpatient costs (Level III-3 evidence). Similarly, alerts have failed to reduce the risk of exposure to drug interactions among the general population of a health maintenance organisation (Level III-3 evidence).

Primary care-based evaluations of electronic prescribing

Computers can improve documentation and administration in the primary care setting (Level I evidence). Various computerised tools such as reminders or feedback and recommendations for treatment based on guidelines can improve physician prescribing (Level I and III-1 evidence). Computerised risk charts not closely linked to the prescribing process are ineffective (Level II evidence). Evidence that computerised prescribing improves health outcomes compared to physician based treatment in primary care is very limited (Level IV evidence).

staff working in specialised outpatient clinics (Level II evidence). No study has examined patient outcomes or costs related to computerised anticoagulation therapy.

Computer support can improve the accuracy of documentation and the completeness of prescribing records for patients receiving ongoing care for chronic conditions such as hypertension (Level 1 evidence) and lipid disorders (Level III-3 evidence).

Electronic support can improve physician prescribing behaviour in relation to their management of a number of medical conditions (hypertension, infection, hyperlipidaemia, the induction of labour and anaesthesia) (Level II evidence).

Electronic prescribing systems that attempt to analyse biochemical data and provide clinical predictions or suggestions about patient management have been associated with inconsistent effects on surrogate outcomes. Beneficial effects on lipid levels (Level III-3 evidence) and blood pressure recording during surgery (Level III-1 evidence) contrast with an absence of improvement in BP recordings (Level II evidence), blood glucose readings (Level II evidence), and HbA1c levels (Level II evidence). Results from newer decision support programmes, which include Bayesian logic, have been inconsistent, although they have usually been no worse than physicians at maintaining the serum concentrations of various medications within a therapeutic range (Level III-1 evidence). Electronic prescribing may be more effective than its human counterpart in circumstances where medication must be given in special doses or at critical times, particularly when the regimen is complicated (Level II evidence).

The impact of electronic prescribing for specific medical conditions on health outcomes is unclear. Evaluations of the effect of programmes on patient outcomes have been infrequent. However, the limited evidence available suggests that electronic prescribing can reduce the frequency of ADEs related to the use of antibiotics (Level III-3 evidence). A range of other patient outcomes from electronic prescribing, although not improved were still comparable with physician-based care in the management of diabetes (frequency of hypoglycaemic events), infection (mortality), asthma (length of stay) and the induction of labour (rates of caesarean section and other maternal/foetal outcomes) (Level II evidence). Electronic tools that did not provide personalised information and specific management advice, such as drug dose and treatment recommendations, were not effective at improving health outcomes (Level II evidence).

Implications for the provision of care

Computers should be introduced when it is important that asynchronous pieces of data need to be communicated together and where the results from complex or repetitive evaluations need to be presented to health professionals rapidly. Computers can improve the administration of health care, enhance physician performance in relation to drug dosing and improve practitioners’ prescribing behaviour. Computers can improve patient management decisions and enable guidelines to be more effectively deployed but the effects of computers on health outcomes remain unclear.

Key characteristics of situations where electronic prescribing systems have worked well include: organisations where there has been significant collaboration and leadership from senior clinicians and management, the use of fast, reliable systems that are uniform throughout the organisation and interface well with their operators, easy and direct access to machines (fixed or mobile) that are available where the clinical work is undertaken, and the provision of adequate resources including staff training and information technology backup.44

Designers of computer software for use in health care settings should take more account of how other patient data can be used particularly in primary care where a substantial amount of information (e.g., medical history, tests results) is now available electronically.

Decision support should be introduced judiciously. POE should be implemented within organisations with sophisticated, computerised patient information systems. Consideration should be given to the introduction of corollary orders in conjunction with POE. Computerised alerts and warnings should be provided as backup where computerised information systems exist because electronic systems can be

bypassed or not kept up-to-date. New systems should be developed that can communicate with each other. International standards should be developed for these systems and their data. Given the rapid rate of development of computerised prescribing, the inconsistencies in results and the limited range of clinical settings in which they have been trialled, it is important that the provision of an electronic prescribing system should always be evaluated using a well-designed method that incorporates patient outcomes. This is especially true if the system has not been previously tested, if the clinical setting differs from that of previously tested sites or if specially trained staff were included in the previous evaluation.1

Future research

There is a pressing need for rigorous evaluations of:

The acceptability of electronic prescribing to health professionals that:

- determine aspects of decision support that are most helpful and acceptable to prescribers - explore the acceptability of alerts and warnings

- investigate how these warnings could be made more useful to prescribers - consider the acceptability of different types of POE to prescribers.

Economic evaluations that:

- describe costs and health outcomes associated with integrated computer systems that incorporate pharmacy, laboratory and administrative data

- define costs and health outcomes related to POE

- explore the marginal costs and effectiveness of decision support with and without POE - evaluate the marginal costs and effectiveness of decision support with and without

evidence-based guidelines

- determine whether decision support enables other professional groups to effectively and safely assume roles (prescribing, diagnosing, patient information provision etc) previously occupied by physicians.

Patient outcomes related to electronic prescribing that:

- determine the effects of all types of electronic prescribing on health outcomes

- describe the costs and health outcomes related to the provision of specialised dosing programmes for medications with a narrow safety profile for which blood concentrations can accurately, reliably and quickly be determined and that utilise a Bayesian approach to pharmacokinetics

- undertake evaluations of the safety features of electronic prescribing systems and their ability to appropriately cope with patient emergencies or detect and respond to their own electrical and mechanical problems

- ascertain the health outcomes and the cost effectiveness of computer generated

anticoagulation therapy for both the initiation and maintenance of heparin or warfarin treatment in hospital or outpatient settings.

Relative effectiveness of different types of electronic advice for different types of prescribers in a variety of settings that:

- address what decision support material should be presented simultaneously with prescribing or what should be available by an additional step

- examine electronic prescribing interventions in New Zealand - elucidate reliable indicators of the risk of an ADE

- examine the effectiveness of the provision of additional clinical information with the prescribing information that is electronically checked by the warning system

- consider the relative or additional benefits of POE relative to other interventions (such as unit dosing, bar coding and automated dispensing systems) that aim to reduce medication errors.

TABLE OF CONTENTS

ACKNOWLEDGMENTS... i

DISCLAIMER... i

COPYRIGHT... i

CONTACT DETAILS... ii

EXECUTIVE SUMMARY AND CONCLUSIONS... iii

Background ... iii

Aim ... iii

Data sources ... iii

Study selection... iii

Data extraction ... iii

Results... iii

TABLE OF CONTENTS... viii

LIST OF TABLES... xi

LIST OF ABBREVIATIONS... xii

GLOSSARY... xiii

BACKGROUND

1

Overview of computerised decision support systems...1

Decision support for dose adjustments...1

POTENTIAL ADVANTAGES AND DISADVANTAGES OF ELECTRONIC PRESCRIBING... 2

Rigorous evaluations of new systems are needed...3

General limitations of the research on electronic prescribing ...3

Medication error ...4

COMPUTERISED MONITORING CAN PREVENT OR DETECT MEDICAL ERRORS AND ADES... 5

OVERVIEW OF CURRENT TECHNOLOGY... 5

ELECTRONIC PRESCRIBING IN PRIMARY CARE AND HOSPITALS... 6

ELECTRONIC PRESCRIBING IN NEW ZEALAND... 6

Future challenges...7

METHODS

9

Intervention ...9

Subjects ...9

Outcomes...9

Study design ...9

Exclusions ...9

THE FOLLOWING ASSUMPTIONS WERE MADE FOR THE COMPLETION OF THIS REVIEW... 10

METHODS... 10

Literature search...10

Principal sources of information...10

Search terms used ...10

APPRAISAL METHODOLOGY... 11

GENERAL LIMITATIONS OF THIS REVIEW... 12

LEVELS OF EVIDENCE... 12

GENERAL ISSUES AND LIMITATIONS ASSOCIATED WITH RESEARCH DESIGNS... 13

SUMMARY OF SYSTEMATIC REVIEWS ASSESSING ELECTRONIC

SUPPORT FOR GENERAL PRESCRIBING

15

CONCLUSIONS...19

SUMMARY OF STUDIES ASSESSING THE USE OF ELECTRONIC

PRESCRIBING FOR SPECIFIC CONDITIONS

21

Introduction... 21

Individual studies that have assessed the effectiveness of computerised anticoagulation therapy ... 22

Overview... 22

Limitations of the research... 23

INFECTION...25

Overview... 25

The effectiveness of electronic prescribing of antibiotics... 26

Limitations of the evidence... 27

HYPERTENSION...27

Overview... 27

The effectiveness of electronic prescribing for hypertension... 28

Limitations of the evidence... 28

LIPIDS...28

Overview... 28

The effectiveness of electronic prescribing for hyperlipidaemia ... 28

Limitations of the evidence... 28

DIABETES...28

Overview... 28

The effectiveness of electronic prescribing for diabetes... 29

Limitations of the evidence... 29

ASTHMA...29

Overview... 29

The effectiveness of electronic prescribing for asthma management... 30

Limitations of the evidence... 30

INDUCTION OF LABOUR...30

Overview... 30

The effectiveness of electronic prescribing for the induction of labour... 30

Limitations of the evidence... 30

ANAESTHESIA...30

Overview... 30

The effectiveness of electronic prescribing for anaesthesia ... 31

Limitations of the evidence... 31

CONCLUSIONS...31

SUMMARY OF STUDIES EXAMINING ELECTRONIC PRESCRIBING IN

PRIMARY CARE

35

OVERVIEW...36

HYPERTENSION MANAGEMENT IN PRIMARY CARE...36

WARFARIN THERAPY IN PRIMARY CARE...36

EFFECTIVENESS OF ELECTRONIC PRESCRIBING IN PRIMARYCARE...37

LIMITATIONS OF THE EVIDENCE...37

CONCLUSIONS...37

REVIEW OR ALERTS AFTER PRESCRIBING

39

EFFECTIVENESS OF COMPUTERISED CHECKS ON PRESCRIBING AND ALERTS TO PHYSICIANS...40

THE LIMITATIONS OF THE EVIDENCE...40

CONCLUSIONS...40

REFERENCES

43

APPENDIX 1: LITERATURE SEARCH STRATEGY

53

Review databases ... 53

Websites ...53

Medline/Healthstar ...54

Embase...54

Current Contents...56

International Pharmaceutical Abstracts ...56

Cinahl...56

Cross database searches ...57

APPENDIX 2: STUDIES EXCLUDED FROM REVIEW

59

APPENDIX 3: EVIDENCE TABLES

61

ANTICOAGULATION... 72 INFECTION... 87 HYPERTENSION... 94 LIPIDS... 97 DIABETES... 98 ASTHMA... 100 INDUCTION OF LABOUR... 102 ANAESTHESIA... 103 PRIMARY CARE... 105

REVIEW OR ALERTS AFTER PRESCRIBING... 109

LIST OF TABLES

Table 1. Summary of potential strengths and weaknesses of paper versus electronic prescribing ..2

Table 2. Components of electronic prescribing technology ...6

Table 3. Systematic review flow chart...11

Table 4. Designation of levels of evidence ...12

Table 5. Systematic reviews assessing electronic support for prescribing...15

Table 6. Physician Order Entry...18

Table 7. Summary of systematic reviews that have assessed electronic prescribing and anticoagulation therapy...21

Table 8. Summary of studies assessing computer assistance for anticoagulation...22

Table 9. Professional groups...23

Table 10. Infection...26

Table 11. Hypertension...27

Table 12. Lipids ...28

Table 13. Diabetes ...29

Table 14. Asthma...29

Table 15. Induction of labour...30

Table 16. Anaesthesia ...31

Table 17. Primary care...35

Table 18. Hypertenision management in primary care ...36

Table 19. Warfarin therapy in primary care...36

Table 20. Reviews or alerts after prescribing...39

Table 21. Exclusions...59

Table 22. Systematic reviews: individual studies ...61

Table 23. Physician Order Entry: individual studies...64

Table 24. Anticoagulation: individual studies...72

Table 25. Infection: individual studies...87

Table 26. Hypertension: individual studies...94

Table 27. Lipids: individual studies...97

Table 28. Diabetes: individual studies ...98

Table 29. Asthma: individual studies...100

Table 30. Induction of labour: individual studies ...102

Table 31. Anaesthesia: individual studies...103

Table 32. Primary care: individual studies...105

Table 33. Review or alerts after prescribing: individual studies ...109

LIST OF ABBREVIATIONS

95% CI – 95% Confidence Interval

ADE – Adverse Drug Event

ADR – Adverse Drug Reaction

APTT – Activated Partial Thromboplastin Time

B/A – Before and after

BP – Blood Pressure

CI – Confidence Interval

CDSS – Computerised Decision Support System

CVD – Cardiovascular Disease

DOS – Disk Operating System

ED – Emergency Department

GP – General Practitioner

HbA1c – Glycated haemoglobin

HMO – Health Maintenance Organisation

IDDM – Insulin Dependant Diabetes Mellitus

INR – International Normalised Ratio

NHS – National Health Service

NIDDM – Non-Insulin Dependent Diabetes Mellitus

NZ – New Zealand

OP – Outpatient

OR – Odds Ratio

POE – Physician Order Entry

PMS – Patient Management Systems

PT – Prothrombin Time

RCT – Randomised Controlled Trial

RR – Relative Risk

SD – Standard Deviation

UK – United Kingdom

GLOSSARY

Medication error terms – based on Bates (1995)

Adverse drug event (ADE) - an injury related to use of a drug.

Medication error - any error in the process of prescribing, dispensing, or administering a drug whether there are adverse consequences or not.

Potential ADEs - errors that have potential for harm but do not result in injury. These include errors that are intercepted before injury occurs and non-intercepted potential ADEs, which are errors that by chance resulted in no injury.

Preventable ADEs - those resulting from an error or having been preventable by any means currently available.

Evidence appraisal terms – based on NHMRC (2000)

Allocation - the way subjects are assigned to the groups in a study.

Applicability - encompasses the application of results to patients and groups of patients.

Before and after study - outcomes are measured before receiving an intervention and the same outcomes are measured after.

Bias - a systematic deviation from the true value leading to an erroneous estimate of treatment effect.

Blinding - process by which subjects, observers and analysts have no knowledge as to which group treatment group subjects are assigned.

Cohort study - data are obtained from groups who have been exposed, or not exposed, to the new technology or factor of interest.

Comparative study - a study with a control group.

Concurrent controls - controls receive alternative intervention and undergo assessment concurrently with the group receiving the intervention.

Confidence interval - an interval which the population parameter is expected to lie with a given degree of certainty.

Confounding - measure of treatment effect is distorted because of differences between the treatment and control groups that are also related to outcome.

Effectiveness - the extent to which an intervention produces favourable outcomes under usual conditions.

Efficacy - the extent to which an intervention produces favourable outcomes under ideally controlled conditions.

Evidence - data obtained about the effectiveness of a new intervention derived from studies comparing it with an alternative.

Generalisability (see applicability) - extent to which a study’s results provide a basis for generalisation beyond the setting of the study and the people studied.

Heterogeneity - differences in treatment effect between studies contributing to a meta-analysis.

Historical controls - data from a previously published series or previously treated patients at an institution that are used for comparison with a group of patients exposed to an intervention.

Incidence - number of new events in a defined population over a specified time.

Intermediate outcomes - a clinical endpoint that is not the ultimate endpoint of a disease or condition but occurs late in the causal chain and represents a manifestation of the condition.

Interrupted time series - treatment effect is assessed by comparing the pattern of pre-test scores and multiple post-test scores after the introduction of an intervention in a population.

Intervention - any therapeutic procedure.

Level of evidence - a hierarchy of study evidence that indicates the degree to which bias may be eliminated in the study design.

Meta-analysis - results from several studies, identified in a systematic review are combined and summarised quantitatively.

Odds ratio - ratio of the odds of the outcome in the treatment group to the corresponding odds in the control group.

Precision - a measure of how close the estimate is to the true value.

Publication bias - bias caused by the results of a trial being more likely to be published if a statistically significant benefit is found.

P-value - the probability that the null hypothesis is incorrectly rejected.

Quality of evidence - degree to which bias has been prevented.

Randomisation - process of allocating participants to study groups by using a random mechanism.

Randomised controlled trial (RCT) - experimental comparison in which participants are allocated to study groups randomly.

Reliability - the degree of stability that exists when a measurement is repeatedly made under different conditions or by different observers.

Selection bias - systematic differences between those who are selected for study and those who are not.

Strength of evidence - includes levels of studies, their quality and the statistical precision of the results.

Systematic review - process of systematically searching, appraising and synthesising evidence from scientific studies in order to obtain a reliable overview.

Time series - a set of measurements taken over time.

Background

INTRODUCTION

Health care has lagged behind other industries in automation and computerisation. Compared with other record-based industries, such as banking and airlines, healthcare has been using outdated and inefficient methods to store, retrieve and transfer information.2 _{Computers can improve quality in} health care by providing decision support, they can provide event monitors to warn about asynchronous events and they can perform reliable quality measurement.3

Prescribing is a high volume and high expense component of health care where there is considerable variability in practice patterns. The large amounts of information used in prescribing and the need to synthesise pieces of data ensure that it is an area well suited for computerisation. Because of the increasing number of drugs, regimen complexity, changing indications and adverse effects, physician memory can no longer serve as a reliable bridge between research advances and practice.4

Electronic prescribing in its most basic form involves printing prescriptions in a legible format using standard abbreviations on suitable paper.5 _{Four trends exist that enhance on this basic form:}

information resources such as information about interactions

integration with medical records to develop the electronic medical record with clinical coding and audit functions

decision support including analysis and recall systems or evidence-based guidelines

internet connections to enable connections between doctors and pharmacists and regulators so prescriptions can be transferred electronically, better access to information, and it also serves as a conduit to updating software and databases.5

Overview of computerised decision support systems

Computerised decision support (CDS) is an important subset of electronic prescribing. CDS can be defined as computer software using a knowledge base designed to be used by a clinician involved in patient care as a direct aid to clinical decision-making.6 _{Characteristics of an individual patient are} matched to information in the knowledge base. In its most basic form this can involve an electronic link between a prescribed medication with information from a national database or formulary. In more sophisticated versions this involves the electronic presentation of patient-specific information in the form of assessments (management options or probabilities) and recommendations for the clinician.6 Essentially, CDS systems help organise data and speed its retrieval.7

Broadly, two main types of patient-specific, expert systems are available: (1) rule-based, where operations are performed subject to a set of rules obtained from a single or group of specialists in a given field and (2) probabilistic systems where evidence is weighed and then the best option calculated according to Bayes Theorem.8 _{The principle advantages of these types of CDS systems are the} reproducibility of interpretation within centres, standardisation between centres and the ease of performance assessment.

Decision support for dose adjustments

Therapeutic monitoring of drugs is recognised to be especially important for drugs with a narrow therapeutic index and for those whose activity correlates closely to the concentration in the blood.9-11 Pharmacokinetic computer programmes have been developed that can adjust and predict dosing regimens for medications where therapeutic monitoring is important.10, 11 _{Several studies have} compared physician dose adjustments with decisions made by computerised programmes to achieve or maintain therapeutic concentrations of various medications.10, 11

POTENTIAL ADVANTAGES AND DISADVANTAGES OF

ELECTRONIC PRESCRIBING

A number of potential benefits have been cited in the literature for computerised prescribing (see

Table 1). Underprescribing, overprescribing, incorrect choice of drugs and failure to recognise adverse effects are serious and potentially avoidable occurrences. Similarly, patients may not get effective treatments or they may receive ineffective ones. Advocates of electronic prescribing have suggested that it will promote appropriate, error free and cost effective prescribing.12 _{Although they recognise} that these systems are expensive they maintain that current costs from leaving errors unabated is also costly.12 _{Computers can improve the process of prescribing by reducing lost orders, removing} handwriting problems, and decreasing the time taken to fill orders. Computers can support more clinically effective decision-making by providing online reference manuals and checks on dose. Cost effective decision-making may also be enhanced by keeping prescribing in line with formularies and providing information on the relative costs of different medications. Computers can also alert doctors to interactions between different sets of data or provide reminders to assist in the care of patients. Computers can assist with audit and quality control by monitoring trends in data and relationships between different types of data (e.g., pharmacy and lab information). Physician time may be optimised by computerised prescribing, reducing the time that doctors take to complete prescriptions or negotiate with pharmacists after the script has been sent for dispensing. Finally, and most importantly, several experts have argued that electronic prescribing improves patient outcomes by reducing prescribing errors and ADEs.7

By contrast, a number of potential disadvantages of electronic prescribing have also been described in the literature. Implementing decision support computing requires large amounts of resources (human and financial) and a substantial computing infrastructure with regularly updated guidelines incorporated into the system.13

Electronic information provided at the time of prescribing can overload busy physicians with distracting or clinically unimportant information. Systems that are not quick or user friendly may generate frustration and antagonism among physicians and may be ignored or by-passed. Information technology can introduce errors. For example, if two similar medications are listed together onscreen substitution errors may occur. Prescribers may by-pass safety checks if frustrated by a large number of false positive warnings or delays in undertaking tasks.14

Table 1. Summary of potential strengths and weaknesses of paper versus electronic prescribing (developed from Trice15_{, Grandinetti}16 _{and Papshev}17₎

Paper Electronic

Familiarity Legibility

Easily modified Able to trace history – enhances audit and monitoring Cheap May save time checking prescription

May reduce medication errors Easy refills

May improve the quality of care with decision support and guidelines

Can provide decision support Strengths

Standardises prescribing

Poor legibility and completeness Unable to cope with variable dosing or stat dosing No decision support Static terminals

Provider resistance to change

Problems with incompatible systems and the lack of technology standards

Potential for commercial manipulation Weaknesses

Prescribing errors lead to ADEs and significant mortality, morbidity and costs

Rigorous evaluations of new systems are needed

Rigorous evaluations of electronic prescribing systems are needed.18 _{Systems should be evaluated like} any other health intervention using a cascade beginning with laboratory-based studies and progressing to larger field trials.19

Simply providing more information to physicians may not improve health outcomes or reduce costs. Provider behaviour is notoriously difficult to change and innovative systems may have unexpected limitations.

General limitations of the research on electronic prescribing

Although a wide range of electronic prescribing interventions now exist remarkably few have been assessed in comparison to physician-based care.7, 18 _{Many of these early systems have been described} in the literature and the performance of some have been evaluated in comparison to the judgement of expert clinicians; relatively few have been compared to actual practice in the clinical setting.20 Evaluations of the treatment recommendations provided by computerised decision support systems in comparison to the judgements of medical experts is an important measure of the validity of these systems. However, because experts may not agree, or may provide advice that is based on outdated and inaccurate knowledge, it is important that computerised decision support systems are evaluated in the clinical setting using patient outcomes. A challenge to the rigorous evaluation of these systems is that many require close interaction between clinician and computer and therefore blinded study designs are often difficult. Among those evaluations that have been conducted most have used non-randomised study designs. At least in part this reflects logistical issues associated with the introduction of large new organisational changes such as the provision of a new computer system and difficulties with attempts to randomise physicians to either use these systems or not. Significant deficiencies can occur in non-randomised studies of new computer systems. In particular, selection bias may eventuate because users of a system may be more enthusiastic, better trained or have a greater interest or aptitude in the use of computerised systems. Evaluations of new information technology interventions have often used before and after comparisons with historical controls. Unfortunately, this study design is especially prone to significant bias in relation to the presence of coexisting secular trends.18 _Changes in the clinical environment can happen during the study period and these changes may affect outcomes. New technologies or changes in hospital formularies can significantly affect prescribing practices. In addition, a Hawthorne effect may occur in the study location which may exaggerate the benefits ascribed to a particular system. Despite these concerns, evidence from these before and after studies is pertinent because the studies have the advantage of presenting information obtained in real practice settings.

Many studies that have assessed the impact of electronic prescribing systems have not included any control group. The absence of a control group prevents any accurate assessment of the relative effectiveness of an intervention even when the study has otherwise been rigorously conducted (e.g., Raschke, 199821_).

Comparative studies assessing important, well defined, patient oriented endpoints are vital in any consideration of the effectiveness of a new intervention.18 _{Unfortunately, many evaluations of} electronic prescribing have favoured intermediate outcomes (such as effects on therapeutic concentrations or medication errors) or poorly defined endpoints. Furthermore, among studies that have chosen intermediate outcomes significant variation has occurred in how the outcome was measured, not only between different studies, but even between different groups in the same study. Although medication errors occur more frequently than ADEs and are an attractive option as a more easily achievable outcome, it is important that large, adequately powered studies are undertaken using ADEs as their primary outcome because the factors that cause medication errors may differ to those that result in ADEs.14 _{Although a number of studies have assessed the effects of the interventions on} prescribing errors, few have categorised the errors according to their seriousness. Even fewer assessed the effects on actual harms. No study considered the effect of the interventions on avoiding patient deaths. It is recognised that because of the rarity of mortality related to ADEs large studies would be needed probably conducted as multi-centre trials. Multi-centre trials would also have the additional

advantage of improving the applicability of the findings to a larger range of participants and organisations.

Although large randomised controlled trials (probably conducted at multiple sites) are needed to determine the effectiveness of electronic prescribing on patient outcomes, other less expensive but still relatively methodologically rigorous study designs may still provide valuable information. Latin square designs, such as that used by Walton et al. (1997)22 _{and rigorously conducted time series} analyses (e.g., Bates23_{), offer comparatively inexpensive methods to evaluate decision support systems} that avoid some of the potential for bias and remain statistically powerful.

Although this review has identified a number of studies that have used a small sample, the key issue is the adequacy of statistical power. Statistical power needs to be considered taking into account both the study design and main outcomes. The sample recruited for the study needs to be adjusted to ensure that an adequately powered study is undertaken. Although this procedure is routinely conducted in many epidemiological or clinical areas of research, it does not seem to have been fully adopted in the methods for evaluating electronic prescribing systems. It should also be noted that some study designs inherently only require a relatively small sample (e.g., crossover studies and Latin square designs). Despite this smaller sample size, these studies may still be adequately powered and these types of evaluations can offer good value for money for those commissioning research.

The heterogeneity of the studies included in this review prevented any quantitative synthesis of results. It is notable though that most studies presented positive results. It is possible that most interventions do reduce medication errors; it is also possible that significant publication bias exists in this area of research. Additional insights would be gained from interventions or settings that have failed to improve the prescribing process.

A common limitation associated with many of the studies (either randomised or not) that were included in this review is the absence of blinding. Blinding of practitioners is difficult with studies that compare the outcomes associated with practitioners who have either been provided with a computerised system or not.18 _{This limitation may significantly influence the apparent effectiveness of the interventions.} With the general absence of any blinding in this research additional protection from contamination between the study groups is especially important.18 _{Unfortunately, few studies have made any attempt} to reduce the potential for contamination. The greatest potential for contamination arises when individual patients are randomised to study groups and the intervention is applied to practitioners. In this situation, the same clinicians treat both experimental and control group patients and they are able to useknowledge gained from the computer system in their care of control group patients. This type of contamination could lead to an underestimate of the effect of the intervention.

Finally, many studies have been conducted in a few highly specialised tertiary centres in the US. Generalising the results from studies conducted in these centres to other countries or smaller hospitals may be difficult. In addition, very few studies have been undertaken outside of the hospital setting and the effectiveness of electronic prescribing in primary care has not been adequately determined.

Medication error

The incidence rate of ADEs is between 2-7 per 100 admissions.25 _{ADEs involve substantial morbidity,} mortality and costs.25 _{Medication errors are a frequent cause of ADEs and most medication errors} occur during ordering.25 _{A significant proportion of ADEs are preventable (between 20-70%).}24 Analgesic and anti-infectives are the most common types of medication related to ADEs but generally a wide variety of drugs are associated with ADEs, suggesting that to prevent them the entire process of ordering, dispensing and administering drugs should be revised.26

(particularly the aviation and engineering industries) have been concerned with error for decades. Recently, a number of US experts applied the knowledge gained from these other industries to health care. The landmark Institute of Medicine report “To err is human: building a safer health system”30 concluded that the extent of harm from medical error is great; errors result from systems failures, not people failures; achieving acceptable levels of patient safety will require major systems changes; and a concerted national effort is needed to improve patient safety.

Estimates suggest that up to as many as 95% of ADEs could be prevented by reducing medication errors through computerised monitoring systems. POE may be able to prevent an estimated 84% of dose frequency and route errors.25 _{To be optimally effective an intervention should occur as close in} time to the event as possible and be constructive and non-judgemental.31 _{Computerised feedback is} ideal in these regards.3

COMPUTERISED MONITORING CAN PREVENT OR DETECT

MEDICAL ERRORS AND ADES

POE, decision support, computerisation of the medication administration record, bar coding, and automated dispensing all hold particular promise as agents to reduce medical error/ADEs. Computers can provide information to aid decisions, generate reminders, assist in order entry, assist in diagnosis, review new data for clinical patterns or anomalies (e.g., sensitivities to antibiotics and the therapy prescribed) and contribute to audit and quality improvement.13

Computers can prevent ADEs from occurring by providing prompts for doctors to take factors such as renal function into consideration when ordering medication. In addition, computers can also monitor clinical information and detect ADEs early so that their effects can be mitigated. However, warnings must be fast and a balance needs to be found between the number of alerts and their importance.32 POE has been associated with the largest reductions in errors from any information technology intervention. Between 55-83% of errors were eliminated in two studies of its effectiveness.23, 33 Computerisation of ordering reduces errors by ensuring that all orders are structured, legible, information can be provided to the order at the time of ordering and orders can be checked for a number of problems such as allergies or drug-drug interactions. Up to 60% of errors can be reduced without decision support.23

Monitoring the occurrence of errors/ADEs by self-reporting typically underestimates the frequency of these events by as much as 20 times.34 _{Computerised data can be used to detect signals that are} associated with an adverse event (such as the use of an antidote). 34

OVERVIEW OF CURRENT TECHNOLOGY

Personal computer-based electronic prescribing products were developed in the 1980s. Today, the latest products are handheld electronic prescribing devices. A prescription pad and sometimes a patient record are traded for a personal digital assistant or a desktop computer. Physicians can retrieve patient files and prescribe using their computer. The electronic prescribing products offer different ways of assisting physicians through the prescription writing steps. Some systems use diagnosis codes to link to therapeutic classes of medications. Others just offer doctors a list of all therapeutic categories. Another group uses smart technology to list the prescribers’ most commonly used items from the last year. In each case, a physician is offered a complete database providing medications, dosage forms, brand and generic status. The devices can serve as drug information and practice guideline resource. Once all the components have been selected the device can warn prescribers of drug interactions. New POE devices use a variety of technologies to support their functionality. Each of the 20 handheld brands that are now available use different devices, platforms, computational architecture, and means of connectivity.17 _{Wireless connectivity is the latest arrangement and attractive to physicians as it frees} them to prescribe from any geographical location. Client server architecture involves a server that carries out the requested tasks. Most software maintenance, data integration and processing occurs on

the server which is maintained by the application service provider. Software and hardware products vary and some may only print the prescription at the office while others will electronically deliver the prescription to the pharmacist either via telephone lines or through electronic data interchange (via private networks or via the Internet)17_.

In the near future speech driven wireless handheld prescribing systems will be available.35 _{A review of} three new handheld systems that offer this potential has just been published.36

Table 2. Components of electronic prescribing technology 17

Devices Platforms Means of connectivity Computational architecture Desktop/laptop personal

computer Windows CE Physical media network(Internet, private network) Client server (applicationservice provider) Handheld personal digital

assistants Palm OS Wireless network

Windows

ELECTRONIC PRESCRIBING IN PRIMARY CARE AND

HOSPITALS

Electronic prescribing has developed along largely separate paths in primary care and hospitals.17 _In hospitals sizeable information systems that include clinical, laboratory, pharmaceutical and administrative information have been developed in large US academic centres and many have been evaluated.25 _{Electronic prescribing is set to become established in hospitals in the UK, with recent} NHS information strategy proposing that all trusts should adopt it by 2005.37 By contrast, electronic prescribing in primary care has already become more widespread (at least for printing prescriptions) with a number of software packages available to practitioners. In the UK, it has been estimated that over 80% of GPs have computerised their practices (although the number with electronic prescribing capability is not clear it is considered to be a substantial percentage of this group) (http://www.rcgp.org.uk/informat/publicat/rcf0007.htm).

In Australia, an estimated 20% of GPs use a computerised prescribing package and this number is rapidly increasing as the Commonwealth government makes available financial support to adopt electronic prescribing.5 _{In Australia, three main vendors of prescribing software hold the vast majority} of the market share (Medical Director, MIMS Script and Locum).5 _{In the US, the ambulatory care} market is highly competitive with a number of software providers (See Ukens35_{) and an accent on new} handheld devices.17, 38

ELECTRONIC PRESCRIBING IN NEW ZEALAND

None of the studies examined in this review were conducted in New Zealand or Australia. A summary of New Zealand’s current use of health information management and technology in the health sector is provided by the report from the WAVE project to New Zealand’s Director General of Health in the Ministry of Health.39 _{The report presents a detailed description and analysis of the current use of} information technology in the health sector and provides a comprehensive information management and technology plan for the future.

site visits during the beginning of 2001. Providers were asked to describe the use of patient management systems (PMS) that were designed to record and store prescriptions, and clinical data. Currently, some 57% of GPs stated that they used a PMS to assist with prescriptions. However, some 89% of practitioners indicated they would be using such a system within the next three years.39 _In secondary care, nearly half (six of the 14) of the District Health Boards who responded to the survey have implemented a computerised local clinical data repository that includes prescribing information. Over the next five years, an additional five District Health Boards indicated that they would be implementing these systems in their hospitals.39

The WAVE project was concerned with improving health outcomes through the effective use of health information at the least cost to the sector.39 _{A series of eight streams were developed including:} strategy, knowledge, data architecture, electronic health records, systems infrastructure, privacy, investment and organisational design.39

Among the 91 recommendations from the project the following 10 were given top priority:39

setting up an independent organisation to lead information management/information

technology capability

collecting reliable ethnicity data

implementing the National Provider Index

fixing up the National Health Index to allow primary provider access, improve ethnicity data

gathering primary care information

fixing up pharmacy and laboratory data and providing primary care with access

cleaning up messaging standards

sorting out Health Event Summaries – with data dictionaries, electronic discharges and referrals

launching a health portal

making integrated care work by developing standards for data exchange, security and network infrastructure.

Future challenges

A number of challenges exist for the development of electronic prescribing. These have been cited by several experts in the medical literature. They can be summarised as issues with data confidentiality, the need to establish standardised evidence-based guidelines to unify a variety of systems and redefining roles for practitioners – e.g., pharmacists freed of transcription tasks etc (see Schiff for discussion of future inhibitors and strategies for progress).4

A number of tensions in implementing computer prescribing must be resolved in the future, which include:

easy access for clinicians versus confidentiality

availability of information for clinicians versus the need to prevent them from being overloaded

warnings of dangerous interactions are important but the noise of false positives is very distracting

a fail safe system is needed as well as one that is interactive and flexible

need transformation and change but also familiarity

external forces need to be aligned to indigenous institutional interests

local customisation is important but decreased variation in practice is important

pre-packaged commercial solutions versus self-built clinician led solutions

a tightly integrated system versus an interlinked mixed system

Methods

The New Zealand Ministry of Health commissioned this review. The following criteria were determined in consultation with the Ministry.

CRITERIA USED TO LOCATE AND INCLUDE STUDIES FOR

THIS REVIEW

Intervention

Any computerised assistance with prescribing medication includes computerised decision support. Prescribing is limited to ordering (i.e., selecting medication and its dose, route and frequency) and transcribing medication. It does not include dispensing or administering medication.

Subjects

Any health professionals who prescribe medication: nurses, doctors, dentists etc (primarily doctors in various specialities and levels of training).

Outcomes

1. Changes in a surrogate therapeutic outcome such as the proportion of drug concentrations in a therapeutic range. Surrogate outcomes are physiological or biochemical markers that can be easily measured and are taken as predictive of important outcomes. These endpoints serve as intermediates for patient outcomes.41

2. Changes in clinician performance related to their prescribing behaviour, including the frequency of medication errors or prescribing errors and their adherence to guidelines.

3. Changes in patient health outcomes (mortality, morbidity, costs)42 _{especially the frequency of} adverse drug events.

Study design

Any intervention study with a comparison group and data from before and after the intervention. Specifically, the following types of studies were included: systematic reviews, meta-analyses, randomised controlled trials, pseudo-randomised controlled trials, comparative studies with concurrent or historical controls, cohort studies, case-control studies, time series analyses with or without a parallel control group and before and after studies.

Exclusions

unpublished studies

studies based in non-Western countries

studies that had not clearly described their methods and results or where there was inadequate data presented or it was not possible to extract data from the published report

studies that considered nutritional/electrolyte support or maintenance

conference proceedings

studies that considered interventions outside of the clinical setting or were not prospectively related to patient care.

THE FOLLOWING ASSUMPTIONS WERE MADE FOR THE

COMPLETION OF THIS REVIEW

a significant number of studies had been undertaken that met our criteria

our criteria were valid and appropriate for the topic.

METHODS

Literature search

A systematic method of literature searching and selection was employed in the preparation of this review.

Initial searching was done in March 2001 and updated in August 2001 for information in any language. Filters for high quality study design were not employed so as to allow for a wider retrieval of literature.

Principal sources of information

Major sources of information were:

Bibliographic databases: Medline, Embase, Current Contents, Cinahl, Healthstar, Science Citation Index, International Pharmaceutical Abstracts, Cochrane Controlled Trials Register, Index New Zealand.

Review databases: Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effectiveness, NHS Economic Evaluation database, Health Technology Assessment database, Best Evidence.

Websites: Websites of major evidence-based compilations, New Zealand, Australian, British, Canadian, and United States government health departments and related agencies, and a number of major library catalogues were scanned for relevant publications, guidelines, or consensus statements. Other: reference sections of retrieved papers were scanned for relevant publications.

A complete list of sources searched for this review is given in Appendix 1: Literature search strategy.

Search terms used

Index terms from Medline/Healthstar: drug therapy-computer assisted, therapy-computer assisted, electronics-medical, computer systems, decision making-computer assisted, algorithms, artificial intelligence, pharmaceutical preparations, hospital information systems, drug therapy.

Index terms from Embase: computer assisted drug therapy, prescription, drug, pharmaceutics, pharmacy, computer program, decision making, medical decision making, electronics, algorithm. The above index terms were used as keywords in databases where they were not available and in those databases without controlled vocabulary.

references entered between December 2000 and August 2001. Complete search strategies are given in

Appendix 1: Literature search strategy. In recognition of the completeness of the searches undertaken by Walton11 _{and Hunt}1_{, selection was limited to those studies that were published since} 1989.

APPRAISAL METHODOLOGY

In total, 772 references were identified by the search. The abstracts of all of these studies were examined against the preceding criteria for inclusion in this review. From the examination of abstracts, some 122 articles were obtained and considered against the criteria. As a result of this consideration, 52 studies were found to meet all of the inclusion criteria for this review and have been appraised and included in the text.

Articles were formally appraised using the data collection tools developed by the Cochrane Effective Practice and Organisation of Care Review Group (http://www.abdn.ac.uk/hsru/epoc/).

This systematic review has not used summary scores on a quality assessment scale to represent the quality of a study. Results depend on the choice of the scale, and the interpretation of findings from these scales is difficult.43 _{The findings from previous research, and based on theoretical considerations} concealment of treatment allocation, blinding of outcome assessment and the handling of patient attrition in the analysis are key methodological considerations in the planning and conduct of randomised controlled trials.43

Summaries of appraisal results have been presented in table form and conclusions have been drawn that reflect the quality of the articles and their study design. The formal critical appraisal process systematically reviewed the methods and results of the studies included in this review.

Table 3. Systematic review flow chart

B) Citations noted by searching references and other sources n = 90

A) Abstracts located by search

n = 682

+

C) Total abstracts or citations available for consideration against selection criteria n = 772

E) Studies appearing to meet inclusion criteria that were obtained for detailed inspection n = 122

D) 772 – 122 = 650 not examined F) Rejected studies n = 66 G) Not available n = 4 H) Included studies n = 52

GENERAL LIMITATIONS OF THIS REVIEW

This study has used a structured approach to review the literature. This review has analysed studies that met the inclusion criteria and references presented in the databases cited above.

However, there are some potential limitations inherent in this process.

This review has not yet been exposed to wide peer review. In addition, the work has been largely based upon the published academic literature and has not extensively reviewed unpublished work. This review was confined to an examination of the effectiveness of the interventions and did not consider the acceptability, or any ethical, economic or legal considerations associated with these interventions. Interventions were not assessed in terms of their impact on general quality of life. The bulk of the studies included in the review were conducted outside New Zealand, therefore, it is uncertain whether their conclusions are applicable to a New Zealand population and context. That is, there may be uncertainty about whether the benefit/harm trade-off for an intervention differs between settings, or whether the presenting population is similar and the way the interventions are carried out is similar in New Zealand compared to the country in which the study was based.

While most of the articles cited in the databases were obtained, a small proportion were not available from overseas in sufficient time to be included in this review.

This review was completed within a limited period of time (approximately eight weeks spread over 10 months).

A single reviewer abstracted the study information and completed the report.

LEVELS OF EVIDENCE

Levels of evidence were ascribed to each study in relation to their study design (see Table 4). The evidence levels developed by the Australian National Health and Medical Research Council Level were used in this review.40 _{It should be noted that these evidence levels are only a broad indicator of the} quality of the research. The levels describe methods of research, which are broadly associated with particular methodological limitations. These levels are only a general guide to quality because each study may be designed and/or conducted with various limitations.

Table 4. Designation of levels of evidence

Level of

evidence Study design

I Evidence obtained from a systematic review of all relevant randomised controlled trials. II Evidence obtained from at least one properly designed randomised controlled trial.

III-1 Evidence obtained from well-designed pseudo-randomised controlled trials (alternate allocation or some other method).

III-2 Evidence obtained from comparative studies (including systematic reviews of such studies) with concurrent controls and allocation not randomised, cohort studies, case-control studies, or interrupted time series with a control group.

III-3 Evidence obtained from comparative studies with historical control, two or more single arm studies, or interrupted time series without a parallel control group.

RESEARCH DESIGNS

Systematic reviews of RCTs are generally considered to be the best design for assessing the effect of interventions because they have (or should have) identified and examined all of the randomised trial evidence about the intervention. The general strengths and weaknesses of various study designs are described in Appendix 4: Study designs.

Comparative trials, or before and after studies, are commonly employed in the assessment of electronic prescribing interventions but this study design cannot exclude other time-related factors as an alternative explanation for their results. Similarly, these studies may be subject to problems associated with placebo or Hawthorne effects.

Summary of systematic reviews

assessing electronic support for

general prescribing

A detailed description of the characteristics, results and limitations of the studies included in this section is provided in Appendix 3: Evidence tables.

OVERVIEW

Three systematic reviews1, 6, 10 _{have assessed the effect of CDSS on clinical performance and patient} health outcomes. The three reviews used similar inclusion criteria and assessment methods. In the three reviews, some 44 studies assessed the impact of CDSS in relation to prescribing.1, 6, 10

Table 5. Systematic reviews assessing electronic support for prescribing

Study Design Level

Criteria Studies Results Limitations

Johnston6 SR I Health professionals CDSS in clinical setting Clinical performance, patient outcomes Prospective trials + controls. 9/28 studies examined prescribing.

CDSS can improve physician performance at dosing toxic drugs in 3/4 studies but none found patient outcome improvement. 4/5 studies that assessed the effect of CDSS on quality of care found that physician performance was improved and 2/5 found benefit for patient outcomes.

Search limited Problems with trials Hunt1 SR I Health professionals CDSS in clinical setting Clinical performance, patient outcomes Prospective trials + controls. 20/40 new studies examined prescribing.

6/11 studies found benefit from CDSS for physician performance at drug dosing. 5/9 studies that evaluated aspects of medical care found a benefit from CDSS on physician performance.

1/2 studies that assessed patient outcomes found benefit from CDSS.

Limited search Problems with trials Walton10 SR I Health professionals CDSS in clinical setting Clinical performance, patient outcomes Prospective trials + controls, time series, before/after trials. All 15 studies examined computer systems that gave advice about drug dosage.

Computer support: significantly reduced time to therapeutic control, the frequency of toxic drug levels was significantly reduced, the frequency of adverse reactions were significantly reduced, and the length of hospital stay was significantly reduced.

Problems with trials

EFFECTIVENESS OF CDSS IN RELATION TO PRESCRIBING

The systematic reviews consistently reported that the results from most trials indicated that CDSS was effective at improving physician performance at dosing toxic drugs.1, 6, 10 _{The meta-analysis}10 concluded that CDSS was associated with a significant reduction in the mean time to therapeutic control and a reduction in the frequency of toxic levels. The reviews also consistently reported that the results from most studies suggested that CDSS was effective at improving physician performance in other aspects of prescribing.1, 6, 10 _{Relatively few studies examined patient outcomes and only a} minority reported any improvement in association with the use of CDSS.1, 6, 10 _{However, a} meta-analysis of 15 studies concerned with electronic dose adjustment concluded that CDSS was associated with a significant reduction in the frequency of adverse events and patient length of stay.10

LIMITATIONS

The systematic reviews employed similar methods and were all generally well designed and conducted.1, 6, 10 _{They all included structured searches of the literature and clear criteria for the} selection of articles and their appraisal. The reviews by Johnston6 _{and Hunt}1 _{limited their evaluation to} randomised trials and used more limited searches. All the reviews identified a number of significant limitations in the studies that were evaluated. The limitations included inadequate sample size, incomplete protection from possible contamination and the absence of any studies conducted in primary care.1, 6, 10

CONCLUSIONS

A significant number of studies (44) have considered the effectiveness of CDSS in relation to prescribing. Most studies have reported that CDSS can improve physician performance especially at prescribing potentially toxic drugs (Level 1 evidence). Relatively few studies have considered patient health outcomes and several have failed to identify any additional patient benefits compared to physician-based care associated with CDSS (Level 1 evidence). The inability of some studies to find any improvement in health outcomes may be due to their lack of statistical power. The results from a meta-analysis of studies that considered electronic dose adjustment suggest that CDSS can reduce the frequency of adverse reactions and decrease length of hospital stay (Level 1 evidence).

Physician Order Entry for general

prescribing

A detailed description of the characteristics, results and limitations of the studies included in this section is provided in Appendix 3: Evidence tables.

Computerised Physician Order Entry (POE) in which physicians write orders online has substantial potential for improving the medication ordering process.14 _{POE is a computer application that accepts} the physician’s orders for diagnostic and treatment services electronically rather than on relying on paper-based order forms or prescription pads.44 _{The computer is able to check orders against standards} for prescribing and ascertain if there are any drug allergies or interactions with