Guideline implementation

Guideline implementation is the process of facilitating the integration of guideline-based knowledge into routine practice in a clinical setting. Early guideline dissemination efforts rested on the assumption that the primary barrier to the uptake of a new evidence-based practice was lack of knowledge, and that once a guideline was made available, providers would incorporate the new knowledge into routine practice. However, recognition that dis-semination of guideline documents alone is largely ineffective has increased the importance of CPG implementation relative to the development and dissemination of CPGs.^87,88 CPG implementation targets multiple influences of knowledge acquisition and behavior change in a clinical environment, typically using a combination of interventions. Multifaceted guide-line implementation can include clinical reminders, educational outreach and tailoring for specific health care worker roles, AF, practice facilitators, dissemination of educational

ma-terials, and other approaches. However, no evidence demonstrates a single intervention or combination of interventions to have greater efficacy that any other intervention, therefore the determinants of guideline implementation success or failure are not well understood.

A 2008 meta-review by Francke et al. analyzed 12 systematic reviews of factors influ-encing guideline implementation.⁸⁹ The meta-review concluded that the evidence base for determinants of success or failure is thin, and that research directly comparing combinations of implementation strategies is needed. The authors found that 10 of the systematic reviews were of low quality, having extensive or major flaws according to the Quality Assessment Checklist for Reviews.⁹⁰ Two of the reviews were of higher quality, having minimal or minor flaws. Most of the reviews excluded non-English language publications and were constrained to a single medical domain. All of the common findings in the meta-review were drawn from reviews with extensive methodological flaws, lessening the significance of all findings.

A common finding of the systematic reviews of guideline implementation indicated that once implementation activities concluded, guideline adherence returned to pre-implementation baseline levels. Francke et al. identified factors influencing guideline implementation and grouped into five categories of attributes of the implementation. The categories were at-tributes of: the guideline, the implementation strategy, the professionals, the patients, and the environment. Regarding attributes of the guideline, a single common barrier to guideline implementation was the complexity of the guideline. Attributes of strategies that may improve implementation are a) multi-faceted implementation approaches over single-intervention implementation approaches, and b) strategies that involve active participation and are more closely integrated into the clinical workflow, such as point-of-care reminders.

With regard to attributes of professionals, awareness of and disagreement with the guide-lines, and amount of experience in the workplace were identified as significant influences.

Attributes of patients influencing implementation included patient’s resistance to guideline recommendations and patients having co-morbidities, which may decrease health profession-als’ adherence to guidelines. Attributes of the environment that decreased the likelihood of individual guideline adherence were lack of resources in the clinical setting and negative attitudes among peers or superiors. Again, these findings were common only to the lower-quality reviews, making their claims less significant, and leading the authors to conclude

that little can be decisively claimed about factors influencing success or failure of guideline implementation.⁸⁹

The highest-quality systematic review identified by the authors of the Francke et al.’s meta-review is a 2004 review by Grimshaw et al. This review included 235 studies of guide-line implementation that contained 309 comparisons of implementation intervention effects.

Beyond comparing implementation effects, the authors also compared the overall cost of the guideline development and implementation efforts with the cost savings or benefit of the intervention where cost data was available. Like Francke’s meta-review, this system-atic review’s primary conclusion was that a weak evidence base prevented the authors from identifying preferential guideline implementation intervention strategies or attributes. The low quality of evidence was a result of common methodological and reporting weaknesses including a lack of reported details, contextual factors, and rationale for the intervention, and potential methodological errors such as missing sample size calculations, unit of analysis errors, and, for interrupted time series designs, having intervals that were too frequent or infrequent to adequately account for potential bias. The studies included had a large number of different combinations of multifaceted intervention comparisons, preventing the authors from conducting a meta-regression analysis with adequate statistical power. Regarding the cost of implementations, economic data were reported in less than 30% of studies, and a majority of the studies reporting costs reported only the cost of treatment, leaving only four studies with adequate economic data to permit cost-benefit analysis.

However, the study provided insight into the most commonly evaluated interventions and their relative effectiveness, despite the poor quality of most studies. Measuring the number of comparisons available in the literature, the authors found that the most commonly evaluated single interventions were clinical reminders (38 comparisons), dissemination of educational materials (18 comparisons), AF (12 comparisons), and multi-faceted interventions that in-cluded educational outreach (23 comparisons). The authors found that cluster-randomized evaluations of guideline implementation yielded small to moderate improvements in guide-line adherence with the following median absolute improvement in adherence for singleintervention comparisons: clinical reminders 14%, dissemination of educational materials -8.1%, AF - 7.0%, and multi-faceted interventions including educational outreach - 6.0%. One

important note is that the studies included in the review are from 1998 or earlier, excluding nearly the past decade and a half of implementation research.⁹¹

Like high-income countries, low-income countries do not have a strong evidence base regarding barriers and facilitators of SCPG implementation, with even less evidence avail-able to inform the effectiveness of interventions. A 2005 review by Siddiqi et al. identified common approaches to guideline implementation in low-income countries. The review in-cluded 44 publications about guideline implementation research in low-income countries. Of these, no systematic reviews were identified, but eight randomized controlled trials were included. The type of outcomes measured by the studies were either adherence to the guide-line or patient outcomes. The authors concluded that the heterogeneity of the studies and methodological problems prevented them from discerning the effectiveness of different ap-proaches to guideline implementation. However, the authors identified the following common approaches to guideline implementation: AF, local consensus development, education and training, educational outreach, educational materials, local opinion leaders, mass media, marketing, reminders, patient mediated interventions, and combined interventions. Other common approaches in low-resource settings are the use of job aids (pictorial or graphical handouts showing clinical algorithms and treatment recommendations), local facilitators, and supervision within multi-faceted implementation strategies.^92–94

Although evidence supporting the effectiveness of guideline implementation strategies is thin, studies of higher methodological quality using rigorous methods and providing a study rationale rooted in theoretical models of implementation science have recently appeared.

For example, a 2011 study by English et al. evaluated guideline implementation in Kenya for pediatric care using a cluster-randomized trial design informed by models of behavior change from psychological theory.⁴⁸ The study compared two multifaceted implementation approaches using a cluster-randomized trial with intervention and control arms in eight dis-trict hospitals. The intervention group included dissemination of evidence-based guidelines, one week of training, job aids, local facilitators to support implementation, external supervi-sion, six-monthly survey with written feedback, and face-to-face group feedback. The group of hospitals referred to as a control group also participated in a guideline implementation intervention of lower intensity that included dissemination of evidence-based guidelines, one

and a half days of training, job aids, and a six-monthly survey with written feedback. The study by English et al. measured changes in 18 performance indicators that included guide-line adherence measures for prescribing and clinical assessment, structural changes reflecting availability of resources, and aggregate scores for the quality of care provided. The au-thors observed significant improvement in 12 of the 18 measures for the intervention group, relative to the performance of the control group, but noted significant variability in perfor-mance across the participating hospitals. Evidence for the effectiveness of the intervention is strengthened by the fact that the participating hospitals experienced high staff turnover dur-ing the intervention period, such that 18 months after the initial traindur-ing was provided, an average of only 8% of the staff who received guideline implementation training remained.⁹²

A major component of the English et al. study was a qualitative evaluation of health worker’s perspectives of the guideline implementation activities and barriers to guideline implementation.^95,96 The authors interviewed 29 health workers and used thematic analysis to identify the following ten themes as barriers to guideline implementation:

1. Incomplete training coverage resulting in inadequate knowledge and skills 2. Inadequacy in standard setting and leadership

3. Lack of recognition and appreciation 4. Poor communication and teamwork

5. Organizational constraints and limited resources 6. Counterproductive health worker norms

7. Absence of perceived benefits linked to adoption of new practices 8. Difficulties accepting change

9. Lack of motivation

10. Conflicting attitudes and beliefs

The identification of these themes called the authors’ attention to differences in barriers to guideline implementation in high-income countries. The aspects of barriers in low-income countries identified that are not typically found in high-income countries were variability in the acceptance of guidelines across different health worker roles, a lack of demand for evidence behind the new guideline recommendations, a clear impact of resource constraints

on the ability of the health workers to adhere to the guideline, and a desire for payment related to the implementation that fostered poor expectations when none was given. These themes and differences were echoed by another 2009 study investigating reasons for health worker non-adherence to pediatric disease management guidelines in Tanzania, which fur-thermore identified disagreement with the guideline as a major barrier to implementation.⁹⁷ The findings of both of these qualitative studies highlight the complexity of the interaction between environment, social norms, workplace culture and individual personalities, and their influence on learning and behavior change in the clinical setting. English et al. characterize the understanding of best practices for implementation in African settings as being at the

“blank sheet” stage, further emphasizing the need for qualitative research methods.

English et al.’s qualitative study and cluster-randomized trial represent a cutting-edge approach to implementation research in low-resource settings. The authors followed up on this work by conducting a cost-effectiveness study of the implementation.⁹⁸ They found that intervention resulted in a 25% increase in the estimated quality of care in intervention hospi-tals, at a cost of approximately $50 per child admission, compared to a cost of approximately

$31 per child admission in control hospitals. Their analysis, which used incremental cost-effectiveness ratios to assess cost per percentage point improvement in the quality of care, found the intervention to be cost-effective relative to other interventions to improve child health in low-income countries.

The authors recognize that a multi-faceted implementation is a complex task that occurs at multiple levels and is shaped by stakeholders at multiple levels of the health system. As such, the understanding of the success or failure of the implementation needs to be negotiated by all stakeholders, and can not be limited to the “mean effect size” observed in the clinical trial. Furthermore the authors use a conceptual framework that relates the study design to industrial/organizational psychology theory and other theoretical constructs that permit their results to be more broadly interpreted and generalized. The authors mapped their findings to the CFIR framework to further demonstrate the relevance of their findings to shared knowledge in the field of implementation science. Finally, English et al.’s approach to guideline implementation is explicitly is designed to treat health workers respectfully, taking a participatory and re-educative approach that involves local problem solving and

partnership in changing clinical practice.⁴⁸