5 Research on medication error reporting systems
5.2 Overview of studies on medication error reporting systems
Time No time restrictions. -
Study design
All methods and study designs are included.
Peer reviewed journal articles.
Review articles.
Non-scientific publications.
Articles published in professional publications or other non-scientific publications.
Other
English language.
Full-text available via University of Helsinki or Google.
Full-text is not available via University of Helsinki or Google.
5.2 Overview of studies on medication error reporting systems
The identified studies (n=33) represented a blend of approaches to examine functionality, development and implementation of MER systems in different countries and healthcare domains. The studies are summarised in Tables 6-11. The characteristics of MER systems studied are described in Appendices 2 a and b.
Based on their contents, the studies were divided into six categories: (1) studies exploring the functionality of MER systems by investigating the reported medication error
37
data (n=6); (2) studies comparing MER systems to other methods for medication error detection and learning (n=3); (3) studies describing the development and implementation of MER systems (n=9); (4) studies on the utility of MER systems (n=11); (5) studies on MER system innovations (n=3); and (6) reviews on MER systems (n=1). Because of the overlapping findings between the studies in different categories, the synthesis of the key findings of the literature review comprised studies in all the categories instead of summarizing findings within each category separately (see 5.2.3).
38
Table 6. Studies exploring the functionality of medication error reporting (MER) systems by investigating the reported medication error data (n=6). ME=medication error. Studies are presented in alphabetical order.
Reference Country &
setting
Objectives Design Materials and methods Key findings*
Armitage errors and quality of reporting. by the Human Error theory (Reason 1990).
Quantitative and qualitative analysis of a retrospective, random sample of medication error reports (n=911) from 1999-2003.
Qualitative interviews (n=40) of healthcare professionals on how the current reporting system could be improved.
The quality of reports varied greatly.
27% of reports lacked any contributory factors.
The error reports focused on individuals.
Communication difficulties, high workload and interruptions were the leading contributory factors in the interview data.
Reporters rarely received feedback on reporting.
To describe the first two years of MEs reported by the hospital staff between March 2008 and February 2010.
Descriptive statistics to describe the reported MEs (n=488).
Expert review of the reported data:
whether the reported incidents were MEs, severity ratings reported, and correct error type reported (actual error or a near miss).
Use of the Medi-Event system increased the reporting of MEs in comparison to a previously used paper based system.
Staff did not always correctly classify MEs, and their severity ratings did not frequently agree with those made by the expert raters. education, a range of reporting options, changes in report management and enhanced feedback, in order to improve
Non-equivalent group controlled clinical trial involving medical and nursing staff working in 10 intervention and 10 control units in four major cities and two regional hospitals in South Australia.
Comparison of incident reporting rates and types of reports between
baseline and study period, and control and intervention units.
Interventions: education on reporting;
reducing fear and burden of reporting (changes to the reporting process and forms), and improved feedback on reporting for the reporting staff.
A greater variety and number of incidents were reported by the
intervention units during the study, with improved reporting by doctors from a low baseline. These findings suggest it is possible to improve reporting rates and diversify the types of incidents.
However, there was considerable heterogeneity between reporting rates in different types of units.
39
incident (incl. MEs) reporting rates and change the types of incidents reported.
A range of statistical tests employed for data analysis.
To evaluate the rate, content, ease of use, reporters’ profile, and the follow-up and actions resulting from reports submitted to a Web- based electronic
Analysis of the ease of use of the system and the submitted reports (n=14 179) to a commercial Web-based reporting system at a tertiary care academic hospital for 31 months between May 2004 and November 2006.
Analysis of incidents by descriptive statistics and statistical tests.
The reporting system effectively captured incidents, actions, and follow-up on reporting (e.g., change of equipment or policy, education of staff, changes in staffing levels). It was concluded that the main strengths of an electronic reporting approach are the ability to use branching logic, the ways in which it enables analysis, and the feasibility of incorporating collection of information about action and follow-up steps in the database.
Miller et study of 581 error reports containing 1010 MEs reported between July 2001 and January 2003 at a large academic children’s institution.
Evaluation of all MEs (n=1010) in error reports (n=581) reported between July 2001 and January 2003.
Three clinician experts in patient safety independently reviewed all error reports classified by the reporter and recorded any corrections to the error type based on the information provided by the reporter in the free text field of the reporting form.
Summary data on how often the error type was altered by the clinicians.
The percentage agreement between the original classifier and the clinician was measured.
Descriptive statistics of reconciled errors.
The analysis of the accuracy of error reports showed that most of the reports were accurate.
Following expert review, 208 errors (21%) were deleted because they had been inappropriately coded as errors and 97 (10%) were added as they were not initially coded despite having occurred.
352 ME reports needed to have the subtype of error reclassified.
The overall distribution of error type categories did not change significantly with expert review.
Despite clear imperfections in the data captured, MER tools were concluded to be effective as a means of collecting reliable information on errors rapidly and in real time.
40
Ricci et al.
2004
US, paediatric cardiac intensive care unit (CICU).
To investigate the influence of two key factors, anonymity and profession (doctors versus nurses), on incident (incl. MEs) reporting by comparing two different reporting databases.
Retrospective study analysing incidents (n=211) reported via two adverse event reporting systems in a CICU.
Two adverse event reporting databases were compared: database A (DA) which is the hospital’s official reporting database (non-anonymous, reports are predominantly made by nurses), and database B (DB) (anonymous and reports are submitted by a CICU consultant who collects data from daily ward rounds).
Descriptive statistics to describe the incidents.
A total of 112 incidents were reported in DA, 143 in DB, and 44 in both, indicating that both databases gave an
unrepresentative picture of the true frequency and severity of adverse events. Underreporting was especially notable for less severe events, including near misses.
Incident reporting is heavily influenced by profession of the reporters as well as anonymity. When adverse event reporting is based predominantly on the observations of a single professional group, the data are grossly inaccurate.
*The key findings in the Table are selected from the presented studies in relation to the focus of the systematic literature search.
41
Table 7. Studies comparing medication error reporting (MER) systems to other methods for medication error detection and learning (n=3). ME=medication error. Studies are presented in alphabetical order.
Reference Country &
setting
Objectives Design Materials and methods Key findings*
Olsen et al.
To examine the value of methods for collection on the same patient cohort using incident reporting, pharmacist surveillance and patient record review as sources of information.
Data on AEs were collected on 288 patients discharged from adult acute medical and surgical units using incident reports, active surveillance of
prescription charts by pharmacists and record review at time of discharge.
Descriptive statistics to describe and compare the incidents detected by different methods.
No data source detected all AEs in the same patient group.
Record review detected 26 AEs and 40 potential adverse events (PAEs).
Incident reporting detected 11 PAEs and no AEs.
Pharmacy surveillance found 10 MEs all of which were PAEs.
There was little overlap in the nature of events detected by the three methods.
The findings suggest that incident reporting does not provide an adequate assessment of clinical AEs and that this method needs to be supplemented with other more systematic forms of data collection. and to compare the reporting mechanism review using a screening mechanism followed by structured, implicit physician review of the record.
Patient admissions (n=3146) to medical service during a four-month period were studied.
Initial screening of patients’ medical records for adverse events, followed by physician-conducted analysis of records possibly involving an adverse event.
Testing the inter-rater reliability of physician reviewers by using a 10%
sample of medical records. Confidential e-mail reporting of adverse events by medical staff and analysis of reports.
Assessment of costs for both methods
The physician reporting system identified nearly the same number of adverse events as the record review method.
The two methods identified the same patients with adverse events in only half of the cases.
The physicians reporting system detected more preventable errors than the medical record review and was also less costly. The reporting system also actively integrates physicians into quality-improvement efforts which is the advantage of the system.
42
Women’s Hospital.
and error preventability. Statistical tests to compare the two methods.
Sari et al. review of patients’ case notes and analysis of data submitted to the routine incident reporting system on the same patients.
A random sample of patient admissions (n=1006) during a three-month period was studied.
Initial screening of patients’ medical records for adverse events, followed by physician conducted analysis of records possibly involving an adverse event.
Testing the inter-rater reliability of both reviews by using a 10% sample of medical records.
Analysis of the data on incident reporting system for the sample admissions (n=1006) to see if the events had been reported.
The routine reporting system as implemented in the large
hospital missed most patient safety incidents that were identified by case note review and detected only 5% of those incidents that resulted in patient harm.
Only one third of the adverse events were detected by both methods.
Findings suggest that the routine reporting system considerably underreports the scale and severity of patient safety incidents.
Structured case note review may have a useful role in surveillance of routine incident reporting and associated quality improvement programmes.
*The key findings in the Table are selected from the presented studies in relation to the focus of the systematic literature search.
43
Table 8. Studies describing the development and implementation of medication error reporting (MER) systems (n=8). ME=medication error. Studies are presented in alphabetical order.
Reference Country &
setting
Objectives Design Materials and methods Key findings*
Beckmann system for MEs and other patient safety
Healthcare staff at three ICUs participated in the study.
Development of incident report form based on:
- staff (n=29) group interviews (n=6) to define the key areas of the reporting form;
- workplace observation by five project coordinators to observe incident occurrence, and
- literature review and experiences from previous studies by the study group.
Feedback questionnaire on incident reporting to assess staff attitudes and understanding of the incident reporting, research design and organisation.
Response rate 88% (n=116/129). The reported incidents were described by descriptive statistics.
A positive attitude and good
understanding about incident reporting was demonstrated by more than 90% of participants.
Errors in communication, technique, problem recognition and charting were the contributing factors for most incidents reported (n=128).
Incident reporting may be a suitable technique for improving patient safety in ICUs.
To explore to what extent alerts and newsletters about MEs issued from the MER system of one country can be relevant to other
Disseminated information items (n=90) issued by national MER systems (Canada, US and UK) from June 2009 until June 2012 were collected.
Items compared to CMR-NL according to the defined assessment criteria.
An e-mail survey to national MER systems (Canada, US and UK) about their characteristics.
National reporting systems can benefit from sharing alerts and newsletters between countries.
There is a broad range of MEs that the Dutch national reporting programme could learn from reporting systems in Canada, US and UK.
From the 90 items, 88% (n=79) were relevant for Dutch healthcare.
44
Descriptive statistics were used for the anlysis.
For 48% (n=43) of the items the CMR-NL had received comparable or identical errors but had not disseminated any alert or newsletter about these errors.
The CMR-NL had disseminated an alert or newsletter for 14 of the 90 items (16%).
Daniels et
Face validity and usability of FRS were measured via standardized survey instruments.
Utility of FRS was measured by the rate, typology, degree of harm, likelihood of recurrence, quality of information, and inter-rater agreement analysis of the reported adverse events.
Descriptive statistics to describe the reported adverse events.
Approx. 18% of the reports (n=103) concerned medications.
The FRS had good face validity, excellent usability, and good clinical utility.
27% of reports could not be evaluated for degree of harm due to lack of detailed information from reporter, and 34% of reports were not safety issues, indicating an inherent limitation and target for improvement in family reporting.
The application of survey and human factors methodologies to the design of an electronic system is an effective means of developing an electronic adverse event reporting system for the use of families of paediatric patients.
Elliott et
Measurement and comparison of the reported data for six months prior to the implementation of CSRS and six months after the implementation.
A structured post-implementation user satisfaction survey to frontline clinical staff and managers (response rate 33%, n=358 out of 1079).
There are benefits in moving from a paper-based reporting of errors in healthcare to an electronic Web-based system.
Several benefits were realised, e.g., increases in the number of errors reported, in errors reported within 48 hours of occurrence, in errors reported by staff other than registered nurses, in
45 services site in a rural setting.
Pre- and post-implementation key informant interviews with senior managers (n=11).
Pre- and post-implementation focus-groups to clinical staff (n=13) and managers (n=12) using the CSRS and senior managers’ interviews.
near misses reported, and improved timelines between occurrence and notification by the manager.
There was good user satisfaction with the tool e.g., regarding ease of use, accessibility, and consistency in performance.
Frontline staff and managers supported the CSRS, and identified both benefits of the system (e.g., improved
understanding on what constitutes an error and near miss) and areas for improvement (e.g., need for better feedback from reporting).
Jones et of a voluntary MER system developed
A voluntary MER system was developed and implemented.
NCRHR analysed statistically the errors (n=800) reported in 2002-2003.
Statistical tests to examine association between the CAHs and MEDMARX error data.
Statistical comparison of error reports from CAHs (limited pharmacy support to reporting) and MEDMARX hospitals with 24/7 pharmacy support.
Workshops (n=2) for the participating CAHs to facilitate learning from the reported errors (did not produce any research data).
Similar to MEDMARX, 99% of MEs reported by six Nebraska CAHs were not harmful, reported errors most often originated in the administration phase, and the most common error type was omission.
The CAHs reported smaller proportions of near misses and errors originating in the prescribing phase than in MEDMARX.
The higher likelihood of reporting errors and near misses was associated with more pharmacy support.
Limited presence of pharmacists in CAHs is a barrier to implementing double checks and learning from system failures in the medication use system.
Küng et al.
MEs (n=987) were reported by nurses (n=119) using an investigator-developed
288 (29%) MERT reports indicated that there had been a ME.
46 and to compare the number of MEs
feasibility of a novel MER system.
ME self-reporting tool (MESRT) for a one month period.
Data analysis employed descriptive statistics.
More MEs were reported with the MESRT (n=288) than with traditional CIRS (n=7).
The finding suggests it is important to use simple reporting tools which facilitate ME reporting (see the MERST description in Appendix 2b).
Nurses reported preventing 49 MEs (5%).
Overall, eight MEs (2.8%) had patient consequences.
The high reporting rate suggests that this new method may be a very effective approach to detect, report, and describe MEs.
A working group designed the new system. The number of reports before (1999–2003) and after (2004–2005) introduction were analysed. The completeness of reports was evaluated through the study of 100 randomly selected reports from each system.
Analysis was completed by statistical tests.
Perceptions on introduction were collected in semi-structured interviews to the members of working group (n=4) and one assistant and subjected to descriptive analysis.
Introducing a web-based system for reporting dispensing errors had an impact on quantity of reports and their completeness.
Information (e.g., seriousness of the error and outcomes to patient) was more comprehensively reported in the new system.
A significant difference existed to the extent to which incidents were described as well as details provided of the
medicine and the patient.
Time and patience was needed to implement the changes; users initially found the web-based system difficult to handle, taking more than six months to change this perception.
47 and evaluation of a web-based MER system.
Evaluation study. A total of 25 nursing homes tested the new system over a 4- month period, entering all errors (n=631) occurring at their facility during that time.
Participants were also asked to complete an evaluation survey of the new system (response rate 86% of the facilities, n=20). Descriptive statistics employed for the data analysis.
It was feasible to implement a large-scale web-based MER system in long-term care facilities.
Such a system can collect detailed information on the characteristics of ME.
The participants felt that the system was easy to use will help them identify areas for training and improvement, improve patient safety and reduce MEs.
Tuttle facilitate the use of an electronic of an electronic ERS, and safety culture of the study setting.
An internal ERS for safety events was implemented.
A multifaceted educational program was developed to promote safety awareness and use of the ERS. The safety events reported in 2002 (n=2843) were analysed by statistical tests and compared to events (n=1542) reported by previous paper based system.
A survey was administered to assess safety knowledge and attitudes of patient care personnel (response rate 10%, n=733 out of 7095).
Using of complementary educational efforts and the new electronic system were able to increase reporting
significantly and to improve employees’
knowledge and use of the ERS.
More work is needed to involve physicians in reporting, to improve the accuracy of submitted information, and to better prioritize, organize, and streamline event analysis.
*The key findings in the Table are selected from the presented studies in relation to the focus of the systematic literature search.
48
Table 9. Studies on utility of medication error reporting (MER) systems (n=10). ME= medication error. Studies are presented in alphabetical order.
Reference Country and setting
Objectives Design Materials and methods Key findings*
Braithwaite
A structured questionnaire to healthcare professionals using IIMS (available for healthcare professionals on a website for one month). Acquired information: demographic information of respondents;
training in and using of the system, and attitudes towards the system.
Descriptive statistics and statistical tests to analyse the responses.
The majority of respondents (n=2185) had undertaken training on using IIMS and rated it highly. Most had reported incidents and
maintained their previous reporting levels before IIMS implementation. Most attitudes regarding using the system and its security were favourable.
maintained their previous reporting levels before IIMS implementation. Most attitudes regarding using the system and its security were favourable.