A novel system of prescribing feedback to reduce errors: A pilot study
2. Methods 1 Study design
A before and after study design was used. This study was a service development project and so ethical approval was not required. The local research and audit department approved the study.
2.2. Setting
The study took place in a district general paediatric inpatient department. There were 26 paediatric staff covering a full shift rota. Seven of these staff members were working in paediatrics as part of a rotation, but had 3 months experience within the department at baseline. All staff was sent the trust prescribing guidelines at baseline, as well as a unique study ID. A pharmacist visits the ward on a daily basis to check prescriptions and this activity was not changed during the study period.
AUTHOR COPY
M. Gordon and B. Bose-Haider / A novel system of prescribing feedback to reduce errors 209
Table 1 Technical errors assessed
Patient demographics Name Weight Date of birth Hospital number Consultant Allergy status Location
Order details Name appropriate (generics used in line with trust policy) Units correct and abbreviations appropriate
Route noted and abbreviated correctly Dose practical and measurable
Frequency correct, as required drugs maximum administrations in 24 hours noted Drug signed for and bleep number given
General Order in block capitals Legible
Tidy (no damage to chart from wear, water, etc.)
2.3. Intervention
A baseline assessment of a whole ward sample of inpatient prescription charts were reviewed against the trust prescribing policy. Each medication order was assessed in 16 areas of technical error (Table 1), with any breaches noted. These areas of error are adapted from the previously described check and correct system [11]. Additionally, the actual dose itself and relevant calculations were checked using appropriate prescribing reference texts. If they occurred, clinical errors were also recorded (previously defined as errors that are likely to cause incorrect treatment or actual harm [13]). Each order could have more than one error. Errors were not recorded if they had been corrected by the prescriber immediately, but were recorded if they had been corrected by other staff.
At the end of this assessment, a feedback poster was placed prominently within the staff areas of the department and emailed to all participants. Over a 3 month period between April and June 2011, 3 weekly re-assessments were carried out, each followed by the distribution of an up to date feedback poster. Initially this contained basic information and acted to gain attention (Fig. 1), but in subsequent audit cycles these were updated to include anonymous individual feedback using participants ID number, if patterns of error were observed within these individuals.
2.4. Outcome measures
The primary outcome measure was the rate of technical prescribing errors, defined as an incorrect, missing or unclear item in each of the 16 areas assessed (Table 1). A pro forma, based on this list, was used to screen each order during each ward assessment by MG. A pilot assessment was completed and this allowed, through author discussion, consensus on errors to be reached. During this pilot, inter-observer reliability checks were made between MG and a second paediatrician to confirm the appropriate and consistent use of the assessment tool.
AUTHOR COPY
210 M. Gordon and B. Bose-Haider / A novel system of prescribing feedback to reduce errors
Do not use brand names as this can increase risk of error at administration and dispensing. eg. do not use the term ‘atrovent’, but the generic name instead Be precise with PRN frequencies (eg. QDS 4- 6hrs, not just QDS).
Always sign the front of the chart when you have prescribed. This allows any questions to be directed at the right professional.
The Pennine Acute Hospitals
5 Front of chart not signed PRN frequency too vague Brand names used 4 3 2 1 0
Fig. 1. Example feedback poster.
AUTHOR COPY
M. Gordon and B. Bose-Haider / A novel system of prescribing feedback to reduce errors 211
Fig. 2. Error rates during the study.
The secondary outcome measures were the rates of clinical error and the patient safety attitudes of participants, measured using a modified APSQ-II survey [14]. This 37 item questionnaire has been validated in the undergraduate setting, although it has not previously been used in postgraduate trainees. It was modified to change mentions of undergraduate experience to postgraduate, but otherwise left unchanged. This was sent and returned by email and was anonymous. At completion of the 3 month study period, participants were once again asked to complete the modified APSQ-II.
2.5. Data analysis
The error rates were calculated as a percentage of all opportunities for error within each assessment. These were compared at baseline and completion using a two tailed chi-square test. Mean APSQ-II scores were compared with a wilcoxon rank signed test. Data was analysed in Statsdirect (version 2.7.8, StatsDirect Ltd, UK).
3. Results
During the assessments, we examined 74 charts containing 205 medication orders and representing 3,280 opportunities for error. Each assessment took approximately 30 minutes on the ward and 30 minutes to analyse. The percentage of trainee who prescribing contained errors showed a statistically significant drop from 75.9% to 25.9% (P= 0.007) [15]. There was a statistically significant reduction in the overall error rate (P< 0.0001) between baseline (8.8%, 69 out of 784 possibilities for error) and completion at 3 months (1.8%, 12 out of 656 possibilities for error). Table 2 presents the error rate data throughout the study and this has been summarised in Fig. 2.
There was only one clinical error during the study period (a drug allergy was not recorded, but was corrected by a pharmacist), so no analysis of this dataset was possible. This was in agreement with routine pharmacy screening of the same sample. At baseline, the mean APSQ-II score amongst participants was
AUTHOR COPY
212 M. Gordon and B. Bose-Haider / A novel system of prescribing feedback to reduce errors
Table 2
Error rate data during the study
Assessment Errors observed Total error opportunities Error rate %
Baseline 69 784 8.80
3 weeks 27 672 4.00
6 weeks 14 592 2.40
9 weeks 11 526 1.90
12 weeks 12 656 1.80
124.6 and post intervention the mean was 129.7, suggesting improved patient safety attitudes, although this result was not statistically significant.
4. Discussion
The error feedback system led to a statistically significant reduction in technical prescribing errors. A trend was seen towards improved patient safety attitudes, although this was not statistically significant. This system allows staff to have intermittent and repeated feedback on problem areas within the depart- ment, as well as allowing them to monitor their own practice. It situates error based learning within the workplace and allows the individual, as well as the team to receive educational benefit from each error that occurs, however significant. This is so difficult to achieve in a full shift based system, but key for patient safety, suggesting the utility of such a method to enhance prescribing.
This intervention was in low fidelity and extremely easy to introduce. There were no set up costs and therefore such an intervention could be implemented immediately within almost all settings. With the support of the pharmacy department, data from the routine pharmacist’s activities could be harnessed, with only additional time needed to synthesise the data. In areas with high rates of error, it could be argued that this would be an efficient method, particularly given the time savings they would make when fewer errors are encountered. Large scale multi-centre studies investigating errors have been supported by pharmacists collecting data in this way [16], suggesting this presents a viable and sustainable improvement model, particularly given the costs associated with technology based medication error reduction strategies.
Despite the promising nature of these results, this study does have a number of limitations. Given that this was a pilot project, its small sample size limits the strength of our results. Also, as this was a single centre before and after non-controlled study, this further limits the strength of our findings. Errors were measured using a process based approach. Whilst this is a well recognised method in the prescribing improvement literature and does allow statistically significant findings in small studies, it has been criticised as an approach for focussing on minor errors that are unlikely to cause harm [17], with studies focussing on harm to patients seen as preferable. The secondary outcome regarding attitudes may have indeed been limited by sample size and this may be addressed by a further large scale study. Finally, it was not possible to measure the effect of such an intervention on outcomes for patients. As this is the aim of all safety initiatives, this would be highly relevant to investigate in the future.
Future research should seek to examine the viability and effectiveness of this system if introduced in a more widespread fashion, particularly in terms of patient outcomes and cost benefit. Such a system could be used to address all stages where medication errors occur, including administration and therefore involve the multidisciplinary team.
AUTHOR COPY
M. Gordon and B. Bose-Haider / A novel system of prescribing feedback to reduce errors 213
6. Conclusions
This small pilot project has demonstrated the potential utility of an error feedback system to enhance error awareness and improve prescribing. This technique is low fidelity in design and warrants further research. Such work should use larger samples, consider multiple sites and a randomised controlled design, as well as measuring outcomes for patients and considering cost effectiveness when compared to other methods of error reduction.
Ethical approval Not sought. Competing interests None to declare. Funding None. Author contributions
Morris Gordon planned and carried out the study, analysed the data and let the write up. Bratati Bose- Haider conceived the idea, supporting planning and carrying out of the study, as well as contributing to the write up.
Acknowledgments
Thank you to Lina See for her support in collating data for presentation and presenting this work at the Royal College of Paediatrics and Child Health Annual Conference in May 2012, Glasgow, UK.
References
[1] Kohn LT, Corrigan JM, Donaldson MS. To err is human: Building a safer health system. Washington, DC: Institute of Medicine National Academies Press; 1999.
[2] Wong ICK, Wong IYI, Cranswick NE. Minimising medication errors in children. Arch Dis Child. 2009;94:161-4. [3] Ross S, Bond C, Rothnie H, Parmalee JC, Masta-Gornic V, Pohl H. What is the scale of prescribing errors committed by
junior doctors? A systematic review. Br J Clin Pharmacol. 2009;67:629-40.
[4] Ross S, Loke YK. Do educational interventions improve prescribing by medical students and junior doctors? A systematic review. Br J Clin Pharmacol. 2009;67:662-70.
[5] Lerner S, Magrane D, Friedman E. Teaching teamwork in medical education. Mt Sinai J Med. 2009;76:318-29. [6] Weiner EL, Nagel D. Human Factors in Aviation. New York: Academic Press; 1988.
[7] Gordon M, Darbyshire D, Baker P. Education to enhance non-technical skills in health care professionals: A systematic review. Submitted to ASME Scientific Meeting; 2012.
[8] Gordon M, Findley R. Educational interventions to improve handover: A systematic review. Med Educ. 2011;45(11):1081-9.
AUTHOR COPY
214 M. Gordon and B. Bose-Haider / A novel system of prescribing feedback to reduce errors
[9] Arora VM, Johnson JK, Meltzer DO, Humphrey HJ. A theoretical framework and competency-based approach to improving handoffs. Qual Saf Health Care. 2008;17;11-14.
[10] Gordon M, Chandratilake M, Baker P. Improved junior paediatric prescribing skills after a short e-learning intervention: A randomised controlled trial. Arch Dis Child. 2011;96(12):1191-4.
[11] Caldwell G. Real Time “Check And Correct” Of Drug Charts On Ward Rounds – A Process For Improving Doctors’ Habits In Inpatient Prescribing. Pharmacy Management. 2010;26:3-9.
[12] Wilson BB, Myers KM. Situated cognition in theoretical and practical context. In: Jonassen D, Land S, editors. Theoretical Foundations of Learning Environments. Mahway, NJ: Lawrence Erlbaum Associates; 2000.
[13] Ghaleb MA, Barber N, Dean Franklin B, Wong IC. What constitutes a prescribing error in paediatrics? Qual Saf Health Care. 2005;14:352-7.
[14] Carruthers S, Lawton R, Sandars J, Howe A, Perry M. Attitudes to patient safety amongst medical students and tutors: Developing a reliable and valid measure. Med Teach. 2009;31(8):e370-6.
[15] Gordon M, See L, Bose-Haider B. Error awareness to enhance paediatric prescribing amongst junior doctors: A novel evidence based educational strategy. Arch Dis Child. 2012;97:(Suppl 1):A172-4.
[16] Dornan T, Ashcroft D, Heathfield H, Lewis P, Miles J, Taylor D, Tully M, Wass V. An in-depth investigation into causes of prescribing errors by foundation trainees in relation to thier medical education: EQUIP study. 2009. Final report to the General Medical Council. University of Manchester: School of Pharmacy and Pharmaceutical Sciences and School of Medicine.
[17] Dean Franklin B, Vincent C, Schachter M, Barber N. The incidence of prescribing errors in hospital inpatients: An overview of the research methods. Drug Safety. 2005;28:891-900.
SHORT REPORT