Audit and feedback (AF) is defined as the provision of clinical performance summaries to healthcare providers, teams, and organizations.9 The term “AF” is used to describe a range of interventions that vary significantly in clinical context, provider profession, duration, feed-back message design, and targeted behavior. Evidence from the most recent Cochrane review, based on 140 clinical trials of AF, shows that AF can significantly improve compliance with desired practice, but that it is unclear which approaches, under which circumstances, will work.10 Given the relatively limited insights produced by AF trials to date, AF researchers have recently called for a shift towards comparative effectiveness studies, evaluating how and when AF intervention components will work, rather than its overall effectiveness.11 Re-searchers have also recently argued that the AF research agenda should shift towards the systematic incorporation of psychological theory in the design of trials of AF, noting a lack of theory-informed AF trials and resulting evidence.12
In this chapter I first describe AF interventions and discuss a range of examples that highlight the heterogeneity of AF. Next I discuss AF evidence in general, and within low and middle-income countries in specific. Finally I review other AF research of note.
AF interventions include differing components which are used to target diverse clinical behaviors. I use the term “diverse” to indicate both qualitative differences (e.g. hetero-geneity) and quantitative differences (e.g. variability). Behavior-related diversity includes categories of routineness, disease-focus, and medical specialization. AF interventions have been used to target routine behaviors individually, such as hand hygiene, test ordering, screening, and referral that are relevant across medical domains. AF interventions also tar-get groups of related behaviors associated with the management of a particular disease, such as the management of diabetes and ischemic heart disease (Figure 4).99 Unlike routine
be-haviors and disease-focused behavior groups, AF has been used to target improvement of specialized clinical skills like ultrasonography100, surgical technique101, and diagnostic mam-mography.102 Within a single category of a targeted behavior, intervention components are heterogeneous with regard to approaches to providing feedback, professional roles of targeted providers, and influence on barriers to behavior change.
Figure 4: Prototype feedback report for diabetes care used by Ivers et al. 2010.
AF is commonly used to support CPG implementation. A review of guideline imple-mentation strategies found that 24% of guideline impleimple-mentation studies used AF alone or in combination with other implementation techniques.91 CPG implementation focuses on increasing individual adherence to best practices derived from the strongest evidence avail-able, provided in the form of guideline recommendations for specific clinical circumstances.
When AF is used to support CPG implementation, it is commonly used as part of a
multi-faceted intervention that includes other intervention components. These include educational outreach visits (also called academic detailing), financial incentives, or clinical alerts and reminders.103
AF is routinely used for healthcare quality improvement (QI). The QI process is one of the most extensively used approaches to healthcare performance improvement, emphasizing rapid iteration of changes in a clinical setting and monitoring results. QI encourages health workers to ask “What changes can I make that will improve performance?” and “How will I know if a change is resulting in improvement?” without constraining performance measures to be defined according to guideline recommendations. AF supports QI’s cyclical, data-driven monitoring and evaluation process to enable practitioners to determine if changes are effective. AF interventions may be conducted within a QI framework explicitly as part of the Plan-Do-Study-Act cycle (Figure 5), or may be implemented within an implicit quality improvement process that asks the same fundamental questions, but does not specifically use Plan-Do-Study-Act techniques. Audit and feedback conducted within a QI framework is an active process in which health workers themselves typically plan and conduct the measures to be used, data collection and analysis, and feedback delivery.103–105
Figure 5: The Plan-Do-Study-Act cycle.
The performance measures used in AF can emphasize clinical processes or clinical out-comes. Clinical processes refer to the intentions and actions of HCWs, such as prescribing a drug, referring a patient, performing an exam, using information tools, using sterile tech-nique, or ordering a test. Clinical outcomes refer to clinical end results of processes such
as a patient’s viral load, blood sugar level, or mortality within a patient population.105,106 Process-focused measures frame clinical performance in terms of the HCWs’ actions and in-tentions. When performance is based on HCWs’ own actions and intentions, the goal of achieving some level of performance is squarely within the control of the HCW. Guideline implementers typically use process-focused AF to measure performance in terms of HCW adherence to recommendations or protocols, whereas quality improvement practitioners use process measures in conjunction with outcome measures to understand the effect of process changes on clinical outcomes. HCWs receiving process-focused feedback have greater control over the performance outcome, because process measures reflect the HCWs’ intentions and actions. In contrast to process-focused measures, outcome-focused measures frame clinical performance in terms of the clinical end results experienced within the patient population.
When performance is based on the health outcomes of a patient population, HCWs have less control over the performance outcomes, because many factors can worsen patient outcomes despite the actions and intentions of HCWs.
The data sources used in conducting AF differ in terms of temporality, medium, primary use, and creator. Temporality refers to retrospective data collection, such as in a medical chart review, or prospective data collection, which occurs during the clinical encounter. The medium of the data source can be paper-based records or electronic records. The primary use of data analyzed for AF can be as records for patient medical charts, laboratory, pharmacy, treatment registers, public health reporting, or in the case of prospective data collection, the primary use may be for audit itself. Finally the creator of the data may be a health worker, a supervisor, or some other administrative staff, each of whom may have varying perceptions and goals within the clinical processes that occur in the workplace. Each of these dimensions may influence the fitness-for-use of the data for AF and thereby impact the effect of feedback on performance.
Feedback features refer to the presentational attributes that convey performance infor-mation within a feedback report. Feedback features can vary in terms of aggregation level, confidentiality, social comparison, velocity, correct solution information, frequency, and de-livery format. Aggregation level refers to the provision of feedback about the performance of an individual or a group. Confidentiality is the provision of feedback about an
individ-ual performance to only the individindivid-ual who performed tasks, or to others who may or may not have shared responsibility for the performance. Social comparison, also called normative feedback, refers to the inclusion of performance information about one’s peers compared with group or individual performance information. Benchmarking is a kind of social comparison in which one’s performance is compared with the highest performers within a population ranging from a local to a national level. Velocity refers to the inclusion of data showing performance changes over time. Correct solution information informs the feedback recipient about what can be done to improve performance. Frequency refers to the number of feed-back reports that are provided within a specified time period, and can range from bi-weekly to annually. Delivery format refers to both the medium through which the feedback is pre-sented and the means by which the information is conveyed. Delivery formats include verbal, written, computer-based, tabular or graphical display, group or individual presentation, and customizability.107,108 Figure 4 shows a prototype feedback report that uses benchmarking, displaying an individual’s performance in comparison with the top 10% of peer performance, presented in both graphical and tabular form. This prototype does not include velocity feed-back or correct solution information, but includes both process measures such as “A1C test in 6M” and outcome measures such as “A1C <= 7.0˙’’.
The nature of the task or behavior that an AF intervention addresses in process-focused feedback may also significantly influence the effect of feedback on performance. Tasks may require team coordination, or may be performed independently by an individual. The fre-quency with which the task is performed can impact the appropriate reporting frefre-quency.
Finally, some tasks require dichotomous measures, indicating whether or not the task was performed correctly, while other tasks require continuous measures, indicating the total num-ber of performances, depending on the nature of the process and the goal.