Chapter 1
Implementation Challenges
& Opportunities for Evidence
Based Practices
Anne D. Strode, MSW
Diane Norell, MSW, OTR/L
Dennis G. Dyck, PhD
.Washington State University Spokane
7
Implementation Challenges and
Opportunities for Evidence Based
Practices
In spite of significant evidence supporting evidence-based practices (EBPs), few service providers currently offer them. The reasons for this are varied and complex. In this chapter, the authors describe some of the challenges and opportunities of implementing evidence based practices for persons with mental illness. The researchers discuss the importance of collaborative relationships, funding, training and supervision and the role of research. Appendix 1 contains a summary of the authors’ experiences with three evidence-based practices in Washington State: Multi-family Group (MFG), Multi-systemic Therapy (MST) and Program for Assertive Community Treatment (PACT). Evaluation research directed toward these themes provides valuable information for implementing EBPs.
Collaborative Relationships
Evidence-based practices do not occur in a vacuum. Rather, front-line mental health practitioners work in a large and complex system of care. Many of the resources and their allocation are influenced by the larger systemic context. The implementation of best practices invariably is influenced by administrative, fiscal and political factors. Looking into the dynamics of collaboration, within and among involved organizations, sheds light on the challenges of implementing evidence-based practices.
Gaining administrative support to implement EBPs may be a challenge , but is essential for success. In part, the reason is attitudinal. Administrators need accurate and credible information about the costs, benefits and risks entailed in adoption. Support is critical whether the EBP is introduced with research grant funding or natural (local) funding. If administrative support is lacking, clinicians attend
workshops, return with enthusiasm and high expectations, but then become frustrated and discouraged due to obstacle s inherent within the organization.
The degree of collaboration and methods of working together should be clear to all involved both within and among involved organizations. Organizational changes may be necessary to promote new programs. Frequently, implementing an EBP requires administrators to rework priorities, such as: staffing, training, budgeting, and timelines. For a new practice to succeed, providers may need to alter their methods to fulfill their new job requirements.
Successful implementation of best practices requires collaborative relationships at multiple levels of the service delivery system. Collaborative relationships are viewed on a continuum of willingness to accept risk and commitment ranging from cooperation and coordination to collaboration. Informal cooperation has the lowest risk. Participants usually share information, while authority and resources are separate. Coordination is slightly more formal and involves shared information and harmony of actions to reach a mutual goal. Resources may be shared, but authority is separate. Risk is slightly higher with coordination than cooperation. The highest risk level exists with collaboration, where there is a formal relationship surrounding a common mission. Methods of communication are defined, and authority is determined by the members of the group even though power may be unequal (Winer & Ray, 2000).
An EBP is more likely to be implemented and to succeed if the range of these collaborative relationships is appropriate and supportive. Seaburn et al., (1996) states that healthy collaborations depend upon several factors, including:
• time, mutual respect, good manners
• flexible hierarchies
• clear and explicit common purposes
• comprehensive paradigms that support change
• timely and mutually agreed upon communication
• a business relationship
Less complex programs require the involvement of fewer organizations and people, thus they are more likely to succeed. If a best practice involves more than one organization, a more formal collaborative relationship may be needed. Shared mission statements, funding and defined authority may be necessary to carry out the program. The business relationship might best be maintained through a written formal agreement. Frequently, the relationships between organizations are governed by laws and policies. While laws, policies and procedures may dictate formal rela tionships, they do not minimize the importance of informal working relationships.
To provide adequate services to persons with mental illness, one must frequently cross organizational boundaries and operate under different service paradigms. When staff members from different systems support the best practice, they can find ways to achieve their common goals while
maintaining their professional paradigms. In fact, they may find that working in both paradigms helps them achieve their goals more effectively. When a person supports a particular best practice, they tend to promote it. If there is limited support, roadblocks tend to be created either actively or passively. If healthy collaborative relationships exist around the best practice, then the practice will be easier to implement. If these bonds are not in place, then they need to be created.
Agencies and communities with a history of collaboration are more likely to have success with a best practice. Developing a coalition is much more difficult and time consuming than starting with an organized group. One must identify who needs to be involved and develop mechanisms to engage and maintain their involvement. Building a foundation of trust and a shared vision are essential to success. Yet this is not enough. It takes time, effort, commitment, vision and, more often than not, a champion or leader to keep the vision alive.
In order to increase the likelihood of implementation and dissemination of EBPs, it is necessary to communicate at least four informational elements about it. First, what itis; second, what resources are needed to implement it; third, what it will cost and; finally, how it will benefit the host agency
(savings). Experience indicates that workbooks and training workshops, while necessary, are not sufficient to communicate the type of information needed to influence stakeholders and policy makers. Not only does the information need to be credible, but the source needs to be trusted and respected, as well. While there is no substitute for good science, personal relationships are critical. It is noteworthy that, while researchers establish their scientific credibility through establishing a ‘track record’ in peer reviewed journals and related academic venues (conferences), they often have less credibility in practice settings or with policy makers. Researchers and innovators of EBPs need to engage in an ongoing dialog with the end users, and acknowledge that the process and outcomes of implementing an EBP are complex and require a true partnership with bi-directional communication. Another challenge is communicating the advantages of a cost effective EBP to all the stakeholders. In part, this is a problem of mental health funding and organization. To illustrate, we have recently reported inpatient service cost offsets of family psycho-education (Dyck et al., 2002b). The most
9
Division, with reduction in first year state hospital admissions, and the Regional Support Network (RSN), with reduction in the first year of community hospital admissions. The provider agency, Spokane Mental Health, did not experience a cost saving in reduction of outpatient service hours until three years later. On the contrary, in the first year, outpatient service hours were actually higher among consumers who were provided the EBP. This outcome illustrates how in a mult iple-level system not all stakeholders will share equally in the benefits/savings of an EBP. They may actually cost more to implement, with the savings not realized for some time. Failure to communicate accurately and clearly about this reality damages long-term credibility among providers and policy makers.
Finding a best practice that fits both the needs of persons with mental illness and fits within the organization and cultural domains is important. Once the practice is chosen, one must either have the needed relationships in place or develop them through the life of the program. Relationships take time to develop, continually change and have to be nurtured. Often, time and effort to establish these relationships is not a considered cost when promoting the implementation of an EBP.
Funding
For persons with severe and persistent mental illness, meeting the person’s basic needs is complicated and difficult. Often, persons with mental illness are involved with a number of service delivery systems. Each one has its own eligibility requirements, policies and procedures and may or may not have a different funding stream. Anyone trying to arrange services for persons with mental illness often must work with and across many organizational boundaries in order to understand the person’s environment and to secure the needed services. It is not surprising that people with the most needs often fail to be provided with adequate services.
Often special projects, featuring evidence-based practices, are created to serve special populations, such as persons with serious mental illness, or youth with co-occurring disorders. Most funding for evidence based practices has come from federal sources. The Substance Abuse and Mental Health Services Administration (SAMHSA) has been a reliable source of funding for mental health and substance abuse projects. Obtaining these funds can be a very competitive endeavor. SAMSHA has recognized that project sustainability is dependent upon community collaboration. They have awarded one-year planning grants to communities to give them an opportunity to establish necessary linkages to create natural funding streams.
SAMHSA administrators recognize that once federal funding ends it is difficult to maintain the program with natural funding. Too often, the grant ends before the necessary collaborative relations exist and before positive outcomes are realized. This funding problem also impacts the providers because they are paid only for certain services and, in turn, clinical staff are restricted to providing only what is considered an allowable service. The emphasis on what is allowable for funding (which services count) and the bottom line discourages innovation and implementation of EBPs because of the front end resource investment and on-going costs.
In addition to inadequate funding for public mental health, there is the additional systemic problem of competition for scarce funding due to organizational and systemic barriers. Scarcity typically leads to competition over rank (precedence) and territory (turf) rather than collaboration and the willingness to take the risks associated with adopting innovations. One example is the current funding patterns of serving persons with co-occurring mental illness and substance abuse disorders. Approximately 50 percent of persons with serious mental illness have an associated drug/alcohol disorder. In an ideal world, it would make sense for funding systems to pool their resources to support the implementation and dissemination of integrated treatment programs for persons with co-occurring disorders. This practice is not the norm. This is not to say that partnerships and cooperation do not exist, simply that they are not encouraged by prevailing funding systems. Indeed the prevailing system encourages
competition rather than cooperation. To the extent that an EBP requires system coordination, cooperation, and perhaps even collaboration, there may be more financial disincentives than incentives to the implementation and dissemination of such EBPs.
Training and Supervision
Specific training and on-going supervision are key requirements for implementing a best practice. Research has shown that programs administered with fidelity are more likely to yield positive outcomes.
Clinicians who are effective practitioners of an evidence-based practice generally express an openness to change and embrace new learning. They tend to emulate the values and practice ideology associated with the new practice model. They appreciate how the intervention positively affects treatment outcomes and assists them in other areas of their work.
Clinicians who are more difficult to convince tend to be those who are not eager to go through the rigors of adopting a new practice. Some clinicians need time to experience how the model can be helpful to them and persons with mental illness. As these clinicians became more comfortable with the model and apply it successfully, they became enthusiastic advocates to their colleagues.
Some clinicians are constrained by large caseloads and heavy work demands. Other clinicians are not convinced that the philosophical underpinnings of the best practice model are in keeping with their perspective or experience. Some silently object to what they consider to be an ideal but naive approach to working with complex issues.
Once convinced to participate in an evidence based practice, clinicians tend to find that the new practice model is initially time intensive , requiring them to work outside the regular workday. They may need to avail themselves to the work in a more flexible way.
Because any new practice model requires learning new skills, supervision is necessary. Supervision helps clinicians move the concepts of a best practice model from the abstract to the practice setting. A person who has developed an expertise with the model and has skills in clinical supervision makes the best supervisor. It is imperative that the supervisor is a person with skill and passion for the work and who can help motivate and direct others effectively. Clinicians need to be receptive to constructive feedback and directive supervision.
Supervision aids in ensuring fidelity to the model that is critical for successful outcomes. Clinicians need to recognize the benefits of fidelity to the practice model and be willing to adhere to the model as prescribed.
The role of research and training grants in achieving practice change
Recently, federal initiatives have provided the impetus for implanting best practices and for measuring practice outcomes. For example, the Mental Health Statistical Improvement Project, ORYX, and 16 states have led to the creation and measurement of performance indicators in a number of states, including Washington. Similarly, the Joint Legislative Audit Review Committee (JLARC, 2000) mandated a state-wide consumer outcomes system in Washington State.
For many years, mental health researchers have used special projects as a vehicle to demonstrate the efficacy, effectiveness, and cost effectiveness of novel interventions. Research is the necessary step for the establishment of an EBP and until recently it was naively assumed that an intervention demonstrated to be efficacious and effective within clinical trials would be easily transmitted to the field. Unfortunately, this has not been the case. Recent literature and experience has underscored the
11
use a given intervention. As a consequence, the National Institute of Mental Health (NIMH) is now actively soliciting research proposals that address how information about mental health interventions is created, packaged, transmitted, and interpreted among a variety of stakeholder groups. NIMH is seeking research on implementation strategies that address the level to which mental health interventions can fit within and benefit real-world mental health service systems.
The National Institute of Health (NIH) makes a distinction between “dissemination” and
“implementation.” “Dissemination” is the targeted distribution of information to a specific audience. The intent is to spread knowledge, in this case about mental illness and the associated evidence-based interventions. “Implementation” is the use of strategies to introduce or adapt evidence-based mental health interventions within specific settings (NIMH; PA-02-131; 2002).
According to NIH, there are a number of barriers to dissemination. For example, information on effective mental health interventions may or may not reach many different stakeholders. Successful dissemination of various interventions (including information about underutilized interventions) may occur quite differently depending upon whether the audience consists of consumers, caregivers, practitioners, policymakers, employers or administrators. A number of the EBPs for adults (e.g. family psycho-education and PACT) have had limited consumer input or buy-in. Evidence for consumer-valued outcomes such as ‘recovery’ are limited in the research. Likewise, there has been inadequate attention paid to qualitative evaluations to communicate the impact of interventions on various stakeholders and particularly persons with mental illness. NIH now recognizes the need to package specific approaches and to convey the evidence-based information to all relevant audiences in order to achieve success.
Development of a knowledge base about “how” interventions are transported to real-world practice settings is needed. Mental health researchers often have assumed that interventions can be transferred into any service setting without modification and that a uni-directional flow of information (e.g., publishing a guideline) is enough to achieve practice change. Rarely is this the case. Research plans must include collaboration with stakeholders from multiple mental health and social service settings, policy makers, persons with mental illness and their families/social networks. Process evaluations need to supplement quantifiable outcomes. Process evaluations typically focus on the ‘barriers’ and ‘opportunities’ to implementation and how those can influence the success of the implementation. Additionally, research must provide cost effectiveness information across time. While many EBPs have evidence of long-term cost effectiveness, the short term, start up costs of implementation typically were not provided, or were supported initially by research grant funding (NIMH; PA-02-131; 2002).
While research is a tool for establishing an EBP, one of its major limitations is facilitating the
difficult transition to natural funding (sustainability). As noted above, most research regarding mental health treatments has paid inadequate attention to the recovery model outcome measures. In addition to including recovery measures and qualitative evaluations, there is a need to develop culturally appropriate measures and methods. Furthermore, researchers need to assume a more proactive role in disseminating research findings to policy makers, agency directors and consumers in useable formats for each reader. For example, policy makers would benefit from a brief one-page summary that covers the major findings and key issues. Others need more detailed information to aid in further research and implementation. Regardless, more collaboration is needed to design research projects that meet the needs of the communities they serve and to improve practice.
References
Dyck, D.G., Hendryx, M.S., Short, R.A., Voss, W.D., & McFarlane, W.R. (2002b). Service use among patients with schizophrenia in psycho-educational multiple -family group treatment.
Psychiatric Services,53 (6), 749-754.
Joint Legislative Audit Committee.Mental health system performance audit. December, 2000. State of Washington.
NIMH: http://grants.nih.gov/grants/guide/pa-files/PA-02-131.html
Seaburn, D.B., Lorenz, A.D., Gunn, W.B. Jr., Gawinski, B.A., & Mauksch, L.B. (1996). Models of Collaboration, New York: Basic Books.
Winer, M., & Ray, K. (1994). Collaboration Handbook, Creating, Sustaining and Enjoying the Journey, St. Paul: Amherst H. Wilder Foundation.