We hypothesized that there was no significant difference between the accuracy of data collected by doctors/clinical officers at point-of-care compared to data collected retrospectively by nurses in the ART clinic at QECH. Based on this small study we conclude that there is no difference in the accuracy of weight, ARV regimen, pill count, and the patient’s ability to attend works or school. With respect to the presence of the side effect peripheral neuropathy, data accuracy was significantly higher (P = 0.002) when entered by the doctor/clinical officer at point-of-care than when entered retrospectively by the nurse. Based on these findings we see no deficit to data quality incurred by collecting data at the point-of-care where all the benefits of alerts and reminders, protocol guidance and decision support can be realized.
The monitoring and evaluation (M&E) community has traditionally thought about the implications of poor data quality as having some general impact on reports. However, they have not considered the impact on the individual patient. M&E efforts would benefit from thinking more broadly about the implications of poor data quality. A single incorrect piece of data in 50 pieces of data represents a 2% error rate in a report. Many would consider this to be an acceptable margin of error. However, that single incorrect piece of data when used for patient management may have serious negative consequences, in extreme cases resulting in a life threatening outcome. Consider a scenario where a child whose weight is 3.7Kg is being dosed based on a recorded weight of 7.3Kg due to a transposition error that occurred during data entry. This error may result in the child being given twice the appropriate dose, with potentially serious adverse consequences. Incorrect data may trigger rules causing alerts or reminders to fire when not warranted, or worse, not to fire when they should. As we start to use data to inform and support individual patient care we need to place greater emphasis on maximizing data quality through whatever mechanism are available to us. The use of rule-based data validation techniques delivered through a point-of-care interface is one of the best tools we have at our disposal to improve data quality.
The primary reason the Ministry of Health decided to introduce the point-of-care system into HIV care and treatment was to improve the cohort reporting capabilities for large sites (Appendix A: Sample Cohort Repot and Survival Analysis). Thus far little thought has been given beyond this to consider additional benefits. We recommend reevaluating this, placing
greater emphasis on doing more with the data at sites that have an electronic system. Recommendations falls into three broad areas: increased reporting, operational research, and enhancing decision support for the user.
• We recommend producing reports with greater frequency and with broader scope. We suggest that short reports should be produced on a weekly basis for review. These should include both aggregate-level reports for program management (reviewed by the clinic manager / district health officer), as well as patient-level reports alerting clinical staff to patients who may need follow-up or consultation. Thus far both adult and pediatric patients have been reported as a single group despite the significant differences between the two groups. At QECH we have observed attempts by the pediatric team to establish a parallel reporting system. Creating a separate set of reports for adult and pediatric patients will greatly increase the ability of the respective care teams to monitor and evaluate the performance of their departments.
• The volume of electronic patient-level data is large and growing rapidly as existing sites take on more patients and new sites come online. We recommend conducting operational research in the form of data mining and knowledge discovery using this data. Of specific interest would be the identification characteristics of the data that might help clinicians predict treatment failure, increased rate of defaulting, or ultimately death.
• Some decision support already exists in the system in terms of alerts. For example we alert a clinician to refer a patient to nutritional counseling based on a system-calculated body mass index (BMI) of less than 18.5. Based on the findings of operational research (described above) we should add to the suite of automatic alerts in the system to notify clinicians when patients exhibit potential risks, such as poor adherence, treatment failure or toxicity.
We strongly recommend changing the master card to remove reference to “ARVs give to” and have two columns for “Patient present” and “Guardian present”, if that is what we are trying to determine.
For new sites, move away from conventional master cards immediately after the system goes live. However, we do recognize the value of maintaining some paper audit trail (a paper enhanced rather than a paperless system). The latest version of the system prints a summary of
the encounter on an adhesive label to be placed in the patient’s health passport. We recommend printing a second label and affixing it to a new style card designed for the purpose.
Based on the significantly higher error rates in pill counts observed on Mondays and Tuesdays compared to the rest of the week we recommend reviewing the work load of the nursing staff and performing some form of load balancing between adult and pediatric patients to even out the workload throughout the week.