Alberta Health Services Addiction and Mental Health Primary and Community Care

(1)

(2)

(3)

Acknowledgements

Without our partnered organizations’ good will and support, this evaluation would not have been possible. Additionally, the significant contributions of many Alberta Health Services (AHS) In Roads staff whose willingness to participate was invaluable to the completion of the evaluation. Finally, a heartfelt thank you two individuals who sorted through volumes of information to provide such a thorough and comprehensive evaluative report:

Kate Woodman, PhD AHS Evaluator

Nicole Van Kuppeveld, MBA Manager, DTFP—In Roads

We are grateful for the opportunity to deliver the In Roads program in Alberta and we look forward to continuing the program in 2013/14.

(4)

LIST OF TABLES

Table 3.1 In Roads Evaluation Issues & Questions ... 18

Table 3.2 In Roads Indicators & Data Sources for the First Phase of In Roads ... 20

Table 3.3 Indicators & Data Sources for Capacity Building with Community Agencies .. 21

Table 3.4 Indicators & Data Sources for Service Delivery to Clients ... 22

Table 3.5 Outcomes & Indicators of the In Roads Project ... 22

Table 3.6 Summary of Data Collection Methods ... 25

Table 3.7 Topics Covered by the Agency & Service Provider Needs Assessments ... 27

Table 3.8 Semi-structured Interview Key Areas ... 28

Table 4.1 Outputs, Indicators & Data Sources for Phase 1 In Roads Program ... 33

Table 4.2 In Roads Partner Agency Program Categories ... 34

Table 4.3 Outputs, Indicators & Data Sources for Phase 2 In Roads Program ... 42

Table 4.4 Knowledge & Skills Include in the In Roads Project ... 43

Table 4.5 Number of Activities, Participants & Hours Spent on Capacity Building ... 45

Table 4.6 Calgary – Number of activities, participants & hours spent ... 45

Table 4.7 Edmonton – Number of activities, participants & hours spent ... 46

Table 4.8 Central – Number of activities, participants & hours spent ... 46

Table 4.9 Knowledge exchange activities with non-partnered agencies ... 56

Table 4.10 Outputs, Indicators & Data Sources for Phase 3 In Roads ... 58

Table 5.1 Pre-post paired sample t-test of participants - substance abuse ... 79

Table 5.2 Pre-post paired sample t-test of participants - counseling practices ... 82

Table 5.3 Pre-post paired sample t-test of participants in addiction screen learning .... 84

Table 5.4 Pre-post paired sample t-test of participants knowledge in intervention ... 87

Table 5.5 Pre-post paired sample t-test of participants confident in mental health ... 90

Table 5.6 Self report on increased capacity - summary of data by highest responses .. 99

(7)

LIST OF FIGURES

Figure 1 In Roads Logic Model ... 5

Figure 2 . In Roads Governance Structure ... 15

Figure 3 . Job classifications of In Road’s Mentees ... 35

Figure 4 . Mentee Educational Background ... 36

Figure 5 . Would the Following Learning Areas be of Interest to your Staff? ... 37

Figure 6 . Mentee Self-Report on Previous Training ... 38

Figure 7 . Mentee Interest in the In Roads Learning Areas ... 39

Figure 8 . Pre-mentorship Rating of Knowledge, Skill & Knowledge ... 40

Figure 9 . Service Provider Responses on Mentorship & Resources ... 48

Figure 10 Attendee Feedback on All Presentation Provided by In Roads ... 50

Figure 11 Percent of Sample Response: Participation in Presentation/workshop ... 51

Figure 12 Participant Feedback on Putting Unity in Community Conference ... 54

Figure 13 Percent of Sample Response: Action Towards Collaboration in Community ... 55

Figure 14 Client Gender ... 59

Figure 15 Client Age ... 59

Figure 16 Employment or Education status of Clients ... 60

Figure 17 Housing Status of Clients ... 60

Figure 18 Percent of Sample Response: Was client administered a screener? ... 61

Figure 19 Why screening was not administered? ... 62

Figure 20 Percent of Sample Responses: Were any brief interventions used? ... 62

Figure 21 In Roads interventions used during client interactions ... 63

Figure 22 Percent of Sample Response: Was client referred for additional services ... 63

Figure 23 Number & type of referrals given to clients at partnered agencies ... 64

Figure 24 Agency Representative Survey Responses ... 76

Figure 25 Participant perspective on collaboration & networking ... 78

Figure 26 Pre-post comparison of participants knowledge, skills & confidence ... 79

Figure 27 Number of participants reporting decreased ... 80

Figure 28 Do you plan on incorporating some of the knowledge & skills ... 80

Figure 29 Pre-post comparison of participants knowledge, skills & confidence ... 82

Figure 30 Number of participants reporting in counseling practices ... 83

(8)

Figure 32 Pre-post comparison in addiction screening ... 85

Figure 33 Number of participants reporting in addiction screening learning area ... 85

Figure 34 Incorporating knowledge & skills from learning area ... 86

Figure 35 Pre-post comparison of participants in intervention techniques ... 87

Figure 36 Number of participants reporting decreased in intervention techniques ... 88

Figure 37 Incorporating knowledge & skills from learning area ... 88

Figure 38 Pre-post comparison of participants knowledge in mental health learning ... 90

Figure 39 Participants reporting decreased or improved knowledge ... 91

Figure 40 Incorporating knowledge & skills from learning area to practice ... 91

Figure 41 How likely are you to work with clients at your organization ... 93

Figure 42 Participant perspectives on impact of In Roads program on client ... 94

Figure 43 Client feedback responses ... 96

Figure 44 Percent of Sample Response: Would you like more help ... 97

(9)

Executive summary

Overview of the In Roads Project

In 2009, Alberta Health Services (AHS) received an $8.05 million grant from health Canada’s Drug Treatment Funding Program (DTFP) to strengthen the overall addiction service system in Alberta. In partnership with Health Canada’s DTFP program, AHS Addiction and Mental Health created the In Roads project. The project was intended to improve Alberta’s addiction service system by

coordinating and collaborating with community agencies that serve high-risk youth and young adults. The aim of In Roads was to assist community agencies in developing capacity to incorporate

evidence-based addiction practices into their service provision to high-risk youth and young adults (ages 12-14 years) with substance abuse issues.

Through knowledge exchange activities, education and a mentorship model that paired addiction counselors with community providers, In Roads increased the capacity of community agencies. The addiction counsellor (mentor) and the community provider (mentee) worked collaboratively to determine how to implement evidence-informed practices such as addiction screening and early intervention into day-to-day interactions with clients. Over a period of 32 months, In Roads partnered with 48 agencies in three centres in Alberta: Calgary, Edmonton, Red Deer.

Final Evaluation

The purpose of the final project evaluation was to assess whether In Roads achieved the stated Health Canada and Alberta Health Services projected outcomes. Evaluation results confirm the project was implemented as planned; it delivered the activities and outputs identified at project inception; it was successful in terms of the identified target population. Not only does it meet immediate outcomes, it also shows trends towards contributing to longer-term objectives.

The Final Evaluation elaborates on the four key conclusions garnered from In Roads evaluation findings:

a) collaboration is essential and improves service; b) the mentorship model is effective;

c) clients indicated they received key services from the mentees; and,

(10)

The evaluation process used multiple lines of evidence and multiple methods (including pre/post mentee assessment). Change in sector capacity was assessed.

The evaluation process was divided into three key phases that occurred in tandem depending upon the readiness and current competency levels of the stakeholder organization, its staff, and the time at which the agency joined the In Roads program.

Phase One: Partnership with Community Agencies Phase Two: Capacity Building with Service Providers Phase Three: Service Delivery to Clients

The evaluation data includes aggregates on the activities and outputs delivered, from metrics gathered across three cohorts of agency intake:

Cohort 1: August 2010 Cohort 2: May 2011

Cohort 3: May 2012

Some Key Findings

 service providers indicate increased confidence in assisting the target population (youth and young adults with substance abuse and addictions concerns)

 community service providers have increased understanding of substance use and addiction  utilization of evidence-informed practice includes improvement at time of intake; new

capacities to accept clients and commit to working with them; new abilities to implement brief intervention; and increased empathy for clients

 adoption of a Harm Reduction Model; clients are reportedly staying in programs longer and referrals to AHS Addiction and Mental Health Services are more accurate and appropriate  all agency leads indicated observing implementation of skills and learning from In Roads;

service providers affirmed their ongoing commitment to the utilization of evidence-informed approaches and techniques

 service providers report improved capacity to engage parents and care providers as well as increasingly calling AHS to find resources for their clients

(11)

Conclusion

In Roads has achieved its objective of linkages and knowledge exchange between agencies, service sectors and jurisdictions. Stakeholders report nearly unanimous agreement of the value of In Roads and subsequent improvement in service delivery. Conclusions of the Evaluation demonstrate improved competency in delivering services and affirmation that the learning achieved through In Roads is—and will continue to be—used to inform practice change.

(12)

SECTION 1: INTRODUCTION

Background

In partnership with the Health Canada Drug Treatment Funding Program (DTFP), Alberta Health Services (AHS) created the In Roads project. The initiative was intended to strengthen the overall addiction service system through innovative coordination and collaboration with community agencies that serve at-risk youth and young adults (12 – 24 years old).1 The aim of the In Roads project was to:

 Partner with community agencies;

 Assist them in developing capacity to incorporate evidence-based addiction practices into their service provision;

 Increase collaboration across the addiction treatment continuum and  Increase awareness of available resources.

Throughout the project cycle In Roads partnered with 48 community agencies located in Calgary, Edmonton, and Red Deer. The initiative was implemented from August 2010 – March 2013.

The In Roads project increased community agency capacity by providing knowledge exchange and skill development through direct mentorship relationships between an experienced addictions counselor (mentor) and the community service providers (mentee). The two worked collaboratively to determine how to implement basic evidence-based practices, such as addiction screening, into the mentee’s day-to-day interactions with youth. The In Roads Knowledge Exchange training model involved the development of customized learning programs, designed to address the identified learning needs, previous training, and clients served by each mentee.

The In Roads project integrated and supported the DTFP’s key investment areas of enhancement of linkages; the exchange of knowledge between people, organization service sectors and

jurisdictions; and, increasing the implementation of evidence-informed practice information.

1_{Note that throughout this report, “youth and young adults” stands for this target population of at-risk youth and young}

(13)

Purpose of the Evaluation

This final (and summative) evaluation is designed to measure outcomes achieved in providing addiction-related services by the community agencies and their staff to the youth and young adults they serve. Utilizing multiple lines of evidence, and multiple methods (including pre/post mentee assessment), change in sector capacity is assessed.

The approach and direction taken for the In Road’s project evaluation was developed in

conjunction with Health Canada (HC) to ensure alignment with the national DTFP outcomes. The purpose of this final project evaluation is to assess whether the In Roads initiative has achieved the stated HC and AHS outcomes. Evaluation data in support of this final report includes

aggregates on the activities and outputs delivered, from metrics gathered across three cohorts of agency intake (n=48):2

 Cohort 1: August 2010 (n=19 agencies)  Cohort 2: May 2011 (n=10 agencies)  Cohort 3: May 2012 (n=19 agencies)

Evaluation Implementation

To support project development and evidence-based decision making, the evaluation was

implemented by an internal AHS evaluator, supported by a team of data input and analysis staff. This team was overseen by the In Roads Provincial Program Manager. Over the course of the project, two evaluators worked with the team; one managed the formative report (delivered January 2012) and the second one completed the final (summative) report (May 15, 2013).

The In Roads evaluation plan was developed collaboratively with Health Canada to ensure alignment with national DTFP program evaluation goals. In addition to direction from HC, and the provincial team noted above, provincial and zone program coordinators and addiction counselor supervisors also participated in the evaluation process. Finally, In Roads partnered community agencies and their clients provided data for the evaluation.

2_{Note that the first wave of formative data (from Cohort 1, only) was reported on in the In Roads January 2012 Interim}

(14)

Evaluation Participation

The In Roads project evaluation utilized multiple lines of evidence from key stakeholders:

 AHS In Roads team staff (Community Facilitators, Mentor Supervisors, and Addiction Counselor Mentors)

 Partner agency representatives/ leads  Partner agency service providers (mentees)  Agency clients

These stakeholders participated in multi-method data collection, responding to surveys, semi-structured group interviews, and providing administrative data (e.g., quarterly reports, meeting minutes).

SECTION 2: PROJECT DESCRIPTION

Project Goal

As agreed to at project inception, the In Roads initiative is unique in that it contains aspects of both the treatment systems and treatment services targets of the DTFP investment areas. The overall

goal of the In Roads project is to increase access to evidence-based, early intervention treatment services for youth and young adults who are at-risk or experiencing substance abuse problems.

The initiative achieves this goal through wider system-level changes and practice changes at the individual, service provider level.

Best practices for assisting youth at-risk of, or with, an addiction problem were created by Health Canada (2001, 2008) and suggest that to be effective, services for youth at-risk need to be accessible "where they are” both physically and developmentally. The Alberta In Roads project was designed understanding that the first point of contact for many high risk youth is community organizations (e.g., emergency shelters, youth drop-in centers) rather than the formalized

substance use treatment system. In addition, many high risk clients may not enter (or remain in) the formalized addiction treatment system due to their readiness to change, the extent of their addiction or various other barriers. The In Roads project is based on the premise that, for these hard to reach clients, a community agency may be a less intimidating initial engagement and

(15)

service delivery setting than a more formalized treatment system. The initiative theorizes that at-risk youth are already attending community agencies to meet a variety of needs (e.g. food, clothing, shelter, employment assistance) and have established relationships with the agencies’ service providers. However, while these community service providers have established

relationships with the youth, they may lack evidence-based knowledge, and the skills required to detect problematic substance use, discuss and intervene appropriately and match clients to additional treatment services.

The In Roads project was designed to address this gap by facilitating knowledge exchange and skill development within and among these community agencies. The goal was not to enable community services providers to become experts in substance abuse, but to build capacity in community agencies by helping them develop basic evidence-based knowledge and skills regarding substance use and addiction. They in turn are then better able to understand and engage in conversations about substance use, recognize and screen for substance use and mental health issues, provide some early interventions and refer to additional services where necessary (In Roads Project Manual, 2013).

Project Objectives

The In Roads project was undertaken in three phases (outlined below); each phase aligned with and contributed to achieving the following three objectives:

 Engage stakeholder organizations and service providers who work with high risk youth and young adults in Edmonton, Red Deer and Calgary;

 Enhance the substance abuse-related competencies of service providers who are in contact with at-risk youth and young adults; and,

 Community service providers provide screening, referrals and brief early interventions to the youth and young adults that frequent their agencies.

The three phases were delivered through a two streamed approach to assisting community agencies develop their capacity to address substance abuse and addiction challenges with their clients (see Figure 1 Logic Model).

(16)

Figure 1 IN ROADS LOGIC MODEL In c r e a s e a c c e s s t o e v id e n c e - in f o r m e d , e a r ly in t e r v e n t io n t r e a t m e n t s e r v ic e s f o r y o u t h a n d y o u n g a d u lt s 1 2 - 2 4 a t r is k o f o r e x p e r ie n c in g s u b s t a n c e a b u s e p r o b le m s G O A L O B J E C T IT V E S E n g a g e s ta k e h o ld e r o r g a n iz a tio n s a n d s e r v ic e p r o v id e r s w h o w o r k w ith h ig h r is k y o u th a n d y o u n g a d u lts in E d m o n to n , R e d D e e r a n d C a lg a r y E n h a n c e th e s u b s ta n c e a b u s e - r e la te d c o m p e te n c ie s o f s e r v ic e p r o v id e r s w h o a r e in c o n ta c t w ith a t - r is k y o u th a n d y o u n g a d u lts . P h a s e 1 C a p a c it y t o D e liv e r S e r v ic e P h a s e 2 A t - r is k y o u th a n d y o u n g a d u lts a r e s c r e e n e d , r e c e iv e e a r ly in te r v e n tio n a n d r e fe r r e d a s a p p r o p r ia te fo r s u b s ta n c e a b u s e p r o b le m s . P h a s e 3 S e r v ic e D e liv e r y A C T IV IT IE S Id e n tify s ta k e h o ld e r o r g a n iz a tio n s th a t p r o v id e s e r v ic e s to a t - r is k y o u th a n d y o u n g a d u lts 1 2 to 2 4 Id e n tify s e rv ic e p ro v id e r s w ith in th e o r g a n iz a tio n fo r tr a in in g /m e n to r s h ip C o lla b o r a te w ith o rg a n iz a tio n s to d e te r m in e

o r g a n iz a tio n /s ta ff tr a in in g n e e d s U n d e rta k e k n o w le d g e s h a rin g a n d e x c h a n g e a c tiv itie s M e n to r in g /tr a in in g o f o rg a n iz a tio n ’s s e r v ic e p r o v id e r s D e v e lo p a n d c o n d u c t p re s e n ta tio n s a n d w o r k s h o p s D e v e lo p re s o u rc e m a te ria ls O rg a n iz a tio n s e r v ic e p ro v id e r s p r o v id e e a rly in te r v e n tio n a d d ic tio n s e r v ic e s to y o u th in c lu d in g s c r e e n in g a n d b r ie f in te r v e n tio n s O rg a n iz a tio n s e r v ic e p ro v id e r s p r o v id e re fe r ra ls to a d d itio n a l tr e a tm e n t o r s e r v ic e s T A R G E T P O P U L A T IO N S S ta k e h o ld e r o r g a n iz a tio n s a n d th e ir s e r v ic e p r o v id e r s th a t a r e in c o n ta c t w ith a t - r is k y o u th a n d y o u n g a d u lts S e r v ic e p r o v id e r s a n d fr o n t lin e w o r k e r s w h o a r e in c o n ta c t w ith a t - r is k y o u th a n d y o u n g a d u lts A t - r is k y o u th a n d y o u n g a d u lts 1 2 - 2 4 O U T P U T S P a rtn e r s h ip s b e tw e e n A H S a n d s ta k e h o ld e r o r g a n iz a tio n s e s ta b lis h e d S e rv ic e p ro v id e rs fo r m e n to rs h ip /tra in in g id e n tifie d O r g a n iz a tio n /s ta ff tr a in in g n e e d s id e n tifie d M e n to r s h ip re la tio n s h ip s e s ta b lis h e d T ra in in g p r o v id e d P r e s e n ta tio n s a n d w o rk s h o p s c o n d u c te d R e s o u r c e m a te r ia ls d e v e lo p e d S c r e e n in g , b r ie f in te rv e n tio n a n d a d d itio n a l tr e a tm e n t o r s e r v ic e r e fe r r a ls a r e c o n d u c te d

E n h a n c e d C o lla b o r a tio n b e tw e e n A H S a n d s ta k e h o ld e r o r g a n iz a tio n s

Y o u th a n d y o u n g a d u lts h a v e in c r e a s e d k n o w le d g e a b o u t s u b s ta n c e a b u s e is s u e s / e ffe c ts a n d tr e a tm e n t o p tio n s a v a ila b le In c r e a s e d s e r v ic e p r o v id e r s ’ k n o w le d g e a n d s k ills o n s u b s ta n c e a b u s e s c r e e n in g , e a r ly in te r v e n tio n a n d r e fe r r a l. In c r e a s e d a w a r e n e s s o f tr e a tm e n t o p tio n s fo r y o u th a n d y o u n g a d u lts a t - r is k o f o r e x p e r ie n c in g s u b s ta n c e a b u s e p r o b le m s . S H O R T - T E R M O U T C O M E S E n h a n c e d c a p a c ity to d e liv e r e v id e n c e - in fo r m e d e a r ly in te r v e n tio n s e r v ic e s fo r a t -r is k y o u th a n d y o u n g a d u lts . In c r e a s e d a c c e s s a n d u s e o f e v id e n c e - in fo r m e d p r a c tic e in fo r m a tio n . In c r e a s e d e n g a g e m e n t in e a r ly in te r v e n tio n a c tiv itie s w ith in p a r tn e r o r g a n iz a tio n s

IN T E R M E D IA T E O U T C O M E S

In c r e a s e d a c c e s s to s u s ta in a b le , e v id e n c e - in fo r m e d e a r ly in te r v e n tio n tr e a tm e n t s e r v ic e s b y a t -r is k y o u th a n d y o u n g a d u lts

(17)

The first stream was the primary modality in which the initiative helps community service providers gain substance use capacities and it was where the majority of resources and activities were devoted.

The first stream imparted substance use knowledge and skills through a direct mentorship

relationship. In this approach, and based on a needs assessment of relevant knowledge and skills, an AHS addiction counselor mentored a community service provider on how to implement

evidence-based practices into their everyday work with clients. In other words, the two service providers worked collaboratively to address the substance use issues typically found in the environment where the agency service provider worked with youth. To avoid disruption of the regular services provided at the agency, the In Roads project provided partnered agencies with funding to cover-off for the staff that was participating in the phase I portion of the initiative.

The In Roads project took a needs-based learning approach, where the transfer of knowledge and skills required to work with the target population was tailored to the agency’s service need, client demographics, as well as the individual service provider’s previous learning and training

experiences. In addition to adapting the individual learning, the AHS mentor worked with the community service providers to facilitate the incorporation of knowledge and skills into their everyday work with clients, largely through one-to-one engagement.

The information was delivered to the agency and service providers through direct mentorship training, information sharing, job shadowing, presentations or workshops, and tours and

observations of AHS formalized treatment settings such as detoxification or residential treatment services. According to project developers the strength of this learning approach was that it allowed for a community service provider to participate fully in learning evidence-based knowledge and skills. Further, it provided them time to gain practice in using and incorporating these skills in their regular work environment (In Roads Project Manual, 2013).3

The second stream of the In Roads project was developed in recognition that to address system level change, it was important to collaborate with a wide variety of community agencies and programs (In Roads Project Manual, 2013). However, it was recognized that some community

3_{Note that this training design is congruent with adult education principles, such as problem-based and collaborative,}

(18)

agencies would be not able to participate in the in-depth mentorship and capacity building involved in the first stream. The In Roads project developers wanted to ensure that a wide variety of

community agencies would have the opportunity to benefit from the evidence-based information provided through the project initiative. As a result, the second stream of the In Roads initiative provided non-partnered agencies with knowledge exchange opportunities such as presentations or workshops on topics of interest to their organization and staff (In Roads Project Manual, 2013).

In sum, the objectives of the project were to collaborate with community agencies to determine how to build evidence-based capacity, using a needs-based learning approach, to promote a more coordinated and accessible system of addiction services. The initiative was based on the premise that:

 Strengthened and collaborative partnerships and increased capacity among community agencies to provide preliminary substance use information and services contributes to,  Youth and young adults at-risk having greater access to addiction information and basic

services through the community agencies that they frequent; resulting in,

 A strengthened addiction system that provides evidence-based services to at-risk youth and young adults (In Roads Project Manual, 2013).

Target Population

At times, clients encounter difficulties navigating complicated and diverse systems to receive addiction treatment. This decreases the likelihood of follow-through and engagement in a treatment process. To that end, the In Roads project targeted youth and young adults at-risk of substance abuse (12 – 24 years of age), who were living in what has been coined the “drug corridor” of Edmonton, Red Deer, and Calgary.4

Activities

The In Roads project was undertaken in three phases:

1. The first phase was the implementation of the In Roads project within partnered community agencies;

4

(19)

2. The second phase involved enhancing capacity to deliver services using a needs-based learning approach; and,

3. The third phase described the activities and outcomes involved in the delivery of services to the client.

These phases occurred concurrently depending upon the readiness and current competency levels of the stakeholder organizations and their staff members and the time/ cohort at which the agency joined the In Roads project.

Phase One: Partnership with Community Agencies

Engaging stakeholders was the first objective of the In Roads project. Based on the results outlined in the initial situational analysis5, the target populations for this objective were community organizations and their service providers within the cities of Edmonton, Red Deer and Calgary, who were already in contact with at-risk youth and young adults aged 12 to 24.

First, partnerships between community organizations and AHS were established and service providers (mentees) were identified and agreed to participate. Second, an organization-level needs assessment and individual needs assessment was conducted to determine the learning needs of agency staff. AHS addiction counselor mentors and program coordinators used the information gathered during the needs assessment to tailor the mentorship and knowledge

exchange activities specific to the organization’s and service provider’s needs. The collaboratively determined learning priorities grounded the training and mentorship activities that occurred in Phase Two.

Phase Two: Capacity building with service providers

The objective of the second phase was to increase capacity to deliver services to youth and young adults by enhancing the substance abuse-related competencies of the service providers identified during Phase One. The target populations associated with this objective were community agency service providers.

Knowledge exchange activities were key to building mentee competencies and supporting the ongoing sharing of evidence-informed knowledge. The activities identified for this phase included:

5

(20)

 Providing mentorship and facilitating learning;

 Designing and implementing presentations or workshops; and  Developing resource materials to support the mentorship.

All service providers engaged in learning sessions/activities to increase their knowledge of

substance use and the resources available across the continuum of treatment services in Alberta. Mentees further developed knowledge and skills in substance use, abuse and dependency, basic addiction counseling practices, addiction screening, mental health and early intervention

techniques.

AHS addiction counselor mentors provided community participants with support, feedback, problem-solving and guidance. Further, the mentors shared learning resources so that mentees could incorporate the evidence-based practices into their day-to-day work with clients.

Training, workshops and presentations introduced participants to new and evidence-informed information or practices in a specific area of interest or identified need. While learning was ongoing as part of an established mentorship relationship, workshops and presentations were time-limited (i.e., a half day to two days). Part of this experience included learning how to administer screening tools and participating in workshops on specific topics (e.g. motivational interviewing, harm reduction). It should be noted that both partnered and non-partnered agency staff were invited to participate in these learning events.

Description of the In Roads Learning Areas

Knowledge exchange provided to In Roads participants was based on selected topics from the CCSA Technical Competencies as well as materials used by AHS Addiction and Mental Health. The stated goal of the In Roads project was not to make participants into addictions experts, but rather to assist

participants in gaining evidence-based knowledge and skills to support their interactions with the youth and young adults that frequent their agencies. In Roads used the foundational and basic levels identified in selected CCSA key competencies and built curriculum to facilitate agency service providers in those learning areas. The first five learning areas are based on CCSA

technical competencies, while the topic of referral and knowledge of addiction treatment options and resources was added to as a core component of In Roads by program developers. The In Roads program’s learning areas are as follows:

 Understanding substance use, abuse and dependency  Addiction counseling practices

(21)

Another activity of this phase included creating resource materials (e.g., curriculum and standard operating procedures) to support learning and incorporate the knowledge and skills into practice within the agency. These resources were intended to increase the service providers’ range of knowledge and skills, and to function as an ongoing resource allowing participants to share with other agency staff members who did not directly participate in the In Roads initiative. The curriculum designed in the framework of the In Roads project was vetted by the project’s Clinical Council. This Council was convened by the project team and composed of key project

stakeholders (e.g. project coordinator, community facilitators; AHS Addiction Services and Mental Health supervisors; and, family counsellors) to regulate the standard of the material with Canadian Centre on Substance Abuse (CCSA) competencies and its evidence-informed base. The final product was a 32 module curriculum that will be published and made available to AHS staff.

Phase Three: Service delivery to clients

The third and final phase of the In Roads project corresponded to the service delivery component of the initiative. The target populations associated with this objective were at-risk youth and young adults. The objective linked to this phase was that the target population was provided information regarding substance use, abuse and dependency, addiction screening (if appropriate) and offered support, early interventions and/or referral for addiction services.

The primary activities involved in service delivery include:  Substance abuse screening;

 Using early intervention techniques; and,  Providing referrals to treatment or services.

It is important to note that although these activities were offered by the partner organizations, it is recognized that not all youth frequenting the agency would require or consent to participating in screening, brief intervention or follow through on recommended referrals. According to the In Roads initiative developers, teaching the mentees to provide early intervention addiction services included capacity building in knowing what services and techniques were effective for an individual client. For example, not all youth and young adults screened would require brief interventions or referral to specialized services. The In Roads project was designed to assist mentees in presenting customized options to youth that best meet the client’s needs as well as/or including their

(22)

Outputs

Throughout the implementation of the project, as this report shows, the following outputs were both targeted and achieved:

Phase 1:

 Partnerships between AHS and stakeholder organizations established;  Service providers for mentorship/knowledge exchange identified; and,  Organization/staff learning needs identified.

Phase 2:

 Mentorship relationships established;  Knowledge exchange provided;

 Presentations and workshops conducted; and,  Curriculum developed.

Phase 3:

 Screening, brief intervention and additional treatment or service referrals are provided to clients.

Expected Outcomes Across the Three Phases (Including Links to DTFP Program

Outcomes)

Phases one and two objectives and activities focused on collaborating and building capacity within the partner organization. Both phases contributed to the primary short-term outcome of enhanced collaboration between AHS and partner organizations. This increased collaboration contributed to the secondary short-term outcomes of developed service provider knowledge and skills in

substance abuse, screening and early intervention. In addition, it contributed to increased service provider awareness of treatment options for youth and young adults. Together these short-term outcomes facilitated the intermediate outcomes of enhanced capacity to deliver evidence-informed early intervention services and increased access, use of evidence-informed practice information, and increased engagement in early intervention activities within the partner agencies.

(23)

Phase three outcomes were identified as the following:

Immediate outcomes: By training mentees to offer early intervention addiction services, the goal was that the youth and young adults they engaged with would have increased knowledge about their substance use, the effects of their use and possible abuse, and the treatment options available to them.

Intermediate outcome: It is anticipated that if at-risk youth and young adults have increased awareness of the early intervention services available in the organizations they frequent, they will be more likely to participate in activities that address their substance abuse problems with

community service providers or in the specialized treatment system, when needed.

Long-term outcome: All three phases contribute to the long-term outcome of increased access to evidence-informed early intervention services by youth and young adults.

In Roads linkages to DTFP outcomes

The In Roads project’s goals and outcomes were aligned with the larger, national DTFP outcomes. As depicted by the shaded areas on the logic model (see Figure 2. Logic Model) the initiative supports DTFP investment areas and has direct links to DTFP immediate, intermediate and long-term outcomes.

A primary DTFP investment area was the enhancement of linkages and the exchange of knowledge between people, organization service sectors and jurisdictions. Attached to this

investment area was the immediate outcome of enhanced collaboration on responses to treatment system issues among jurisdictions and stakeholders. The In Roads project partnered with youth and young adult-serving community organizations to address gaps in the substance use treatment continuum (from community services to specialized treatment services). This collaboration assisted in strengthening the linkage between community organizations and Alberta Health Services,

Addiction and Mental Health. It allowed AHS and community organizations to address collaboratively the treatment service issues directly related to individual agencies, the larger community of service providers and the province of Alberta. In addition to strengthening the link between AHS and partnered agencies, the In Roads project created collaboration and networking

(24)

opportunities among those community agencies participating in the program, as well as with non-partnered community agencies.

Another DTFP investment area closely aligned with the In Roads initiative is that of enhancing the implementation of evidence-informed screening and early intervention information. Within the framework of the In Roads initiative, an evidence-based training curriculum was developed. Guided by the Clinical Council that was convened for this work, the In Roads Program Manual and Curriculum (32 modules) was created and vetted to ensure the document was aligned with CCSA standards. In addition, the project team included consulting psychologists (one for each zone) who met regularly with the mentors (AHS addiction counselors) to facilitate their core training role in the In Roads project.

To implement the learning modules, the In Roads project paired AHS addiction counselors with mentees. The mentors provided knowledge and skills on implementing evidence-based screening and early intervention information into the mentees day-to-day work with youth and young adults. It was anticipated that support for, and mentoring of, agency service providers would enhance their capacity to provide early intervention services to the youth and young adults they serve.

The DTFP and In Roads project shared the long-term goal of increasing access to sustainable, evidence-informed early intervention treatment services for at-risk youth and young adults. The project foregrounded collaboration with community agencies and service providers. It assisted them in developing their capacity to address the substance use issues of the youth and young adults seeking supports. In Roads anticipated that as a result, those youth and young adults will have greater access to the continuum of addiction services offered throughout the community.

Operating Context

In Roads began as a project proposal created (2008) by the Alberta Alcohol and Drug Abuse Commission (AADAC) and submitted to Health Canada’s Drug Treatment Funding Program (DTFP). AADAC was the organization charged with implementing substance use prevention, treatment and information programs in Alberta. In May of 2008, Alberta’s healthcare system was restructured and healthcare-related organizations, including AADAC, were merged into one – Alberta Health Services (AHS). As a result of this merger, the former AADAC is now part of the

(25)

larger AHS organization and has been further merged with mental health services, now known as Alberta Health Services, Addiction and Mental Health. Alberta’s DTFP project, In Roads, falls under the purview of AHS Addiction and Mental Health.

The In Roads project is a community-based program and thus most of the activities included in the program are undertaken within partnered community agencies. As a result, the operating context of the In Roads program varies based on the partnered agency’s context, culture and clientele.

Project Management/ Governance Structure and Administration

The In Roads project was administered by the Community and Specialized initiatives unit within the provincial arm of Alberta Health Services (AHS) Addiction and Mental Health, Primary and Community Care. The project was implemented in three (3) operational zones (Edmonton Zone, Central Zone [Red Deer] and Calgary Zone) and delivery of the project was managed by the addiction and mental health leadership in each of the respective zones.

Collaborative by design, decision-making valued consensus building across all entities involved with the initiative. This value flowed down to the individual mentee, who collaborated with their agency lead and the mentor in the design of a curriculum that would meet her/his individual needs.

The In Roads project also engaged a Clinical Council that provided input and direction to the knowledge exchange/mentorship component of the project, by supporting and advising on the curriculum, and vetting the document for alignment with CCSA standards. The Council convened by the Provincial Co-ordinator on an as needed basis to provide clinical expertise to the project and included AHS Addiction and Mental Health representatives from across the province. In addition, Consulting Psychologists were retained to offer information and support to the mentors, also on an as needed basis.

(26)

Figure 2, below, presents the governance structure of the In Roads project. Note that the Zone diagram offered is replicated three times, for each of Edmonton, Central, and Calgary.

(27)

Project Stakeholder Relationships

There are various stakeholders who are directly or indirectly involved in developing, supporting and/or delivering the In Roads project. The following are descriptions of the various stakeholders involved in health services in Alberta and the In Roads project.

Health Canada’s Drug Treatment Funding Program (DTFP): Health Canada provides funding to the In Roads project. HC has overall responsibility to ensure the DTFP partners (e.g.

provincial/territorial governments and other key stakeholders) establish common objectives, priorities, and outcomes that are in alignment with the DTFP initiatives (HC, 2008).

Alberta Health (AH): is the branch of the Alberta government that works with Alberta Health Services to set the strategic direction for health services, including addiction and mental health.

Alberta Health Services (AHS): is the organization responsible for providing health care to all Albertans. Addiction and Mental Health services are part of this larger provincial health system. As such, AHS Addiction and Mental Health provide the support required for the delivery of the In Roads initiative, such as overall project coordination, project operation, project monitoring, as well as project reporting (financial, progress and performance).

AHS, Primary & Community Care – Addiction & Mental Health: is responsible for the delivery, overall coordination and implementation of the program, evaluation and reporting. This includes planning all program specific responsibilities that align with project goals and objectives. The In Roads team is composed of AHS staff responsible for implementing the provincial direction for the In Roads project. This team includes the program manager, provincial coordinator and the

evaluator.

In Roads Clinical Council: a group of clinicians from AHS Addiction and Mental Health responsible for providing clinical guidance to the In Roads project, through the creation and vetting of the project curriculum.

 Community Facilitators: these team members are experienced AHS staff members responsible for building relationships with community agencies and conducting local-level administrative duties (e.g., contract management). Community Facilitators also provide

(28)

knowledge exchange opportunities to community agencies who are not formally partnered with the In Roads project. There are three Community Facilitators, each responsible for one operational zone (Edmonton, Calgary or Red Deer).

 AHS Mentor Supervisors: these addiction counselor supervisors are responsible for supporting AHS Addiction Counselor Mentor staff in implementing the In Roads project in community agencies. The In Roads program has one locally-based addiction counselor supervisor for each operational zone (Edmonton, Calgary or Red Deer (Central)).

 AHS Addiction Counselor Mentors: these team members are trained addiction counselors employed by Alberta Health Services to act as mentors to community agency staff

members. There are four mentors in the Calgary zone, four mentors in the Edmonton zone and three mentors in Red Deer (Central zone).

 Consulting Psychologists: registered psychologists are contracted by In Roads to support AHS Mentors in developing their clinical and mentorship skills. There are three consulting psychologists supporting the In Roads program (one per zone).

In Roads Partnered Community Agencies: Community agencies (e.g. youth emergency shelters, drop-in centers) who are in a formal partnership with the AHS In Roads project and who are

funded to allow staff to engage in the project and are provided cover-off for these service providers to participate in the In Roads mentorship initiative as well as knowledge exchange presentations or events. The partnered community agencies are composed of:

 Agency Representatives: These individuals act for the community agency that In Roads is partnered with regarding contracting, staffing, scheduling, invoicing, reporting and any other agency-level implementation or decision making.

 Participants (Mentees): Community agency staff members participating in the In Roads project.

Non-Partnered Community Agencies: These community agencies are not formally partnered with the In Roads project, but benefit from the initiative by participating in knowledge exchange

(29)

SECTION 3: EVALUATION SCOPE AND METHODS

Evaluation Scope

The approach, direction and scope of this evaluation were developed collaboratively with Health Canada.6 This final evaluation focuses primarily on the contribution achieved towards stated project outcomes (immediate and intermediate), including both the DTFP and AHS identified outcomes. It should be noted that this final evaluation report is an analysis of data collected throughout the In Roads project cycle (August 2010 – March 2013).

The In Roads project addressed substance abuse through larger, system level change as well as practice changes achieved by individual providers. Many indicators were measured on both an agency (as a whole) and an individual (agency staff member) participant levels in order to capture impacts in each of these domains.

Table 3.1 lists the key evaluation questions that the In Roads project has reported on (see the Interim Report, January 2012) and reports on in this final document to Health Canada. The following sections lists outputs, outcomes, indicators and data sources for the three phases of the In Roads project.

Evaluation Issues, Questions and Indicators

Table 3.1 In Roads Evaluation Issues and Questions

Reporting Status

Interim Evaluation

Issues Evaluation Questions

Interim Report See: In Roads Program Interim Evaluation (January 2012) Implementation process challenges and areas for improvement

 Was the project implemented as intended?  Were the expected activities undertaken and

outputs delivered?

 What course corrections were made, why and what is the impact on the project’s ability to reach its expected outcomes?

 What challenges were encountered and how were they addressed?

 What worked particularly well?  What could have been improved?

Evaluate progress  To what extent was progress made toward the

6_{The evaluation results are presented in terms of the progress made on the project performance indicators that were}

(30)

Reporting Status

Interim Evaluation

Issues Evaluation Questions

towards outcomes project’s expected outcomes? Final Evaluation Report

See: In Roads Final Report (May 15, 2013)

Mandatory summative evaluation questions

 To what extent was progress made toward the project’s expected outcomes (immediate and intermediate)?

 Were there any unintended impacts or outcomes?  What are the key lessons from this project?  What are the implications for future work on

systems and/or service improvement?  What are the project’s key contributions to

treatment systems and/or service improvement?  What are the project’s key contributions to the

DTFP outcomes?

 To what extent has the project provided good value for DTFP dollars invested?

 What recommendations flow from this project?

In Roads project activities, outputs and indicators (Interim Report – January 2012)

In order to answer the question “were the activities undertaken and outputs delivered?”

performance indicators were developed for each phase of the program. Tables 3.2 through 3.4 detail the outputs, indicators and data sources used in the January 2012 report. Relevant factors are reported on in this final evaluation report, in aggregate form (total project results).7

Phase 1

The first phase of the In Roads project was focused on engagement and partnership with

community agencies that serve at-risk youth and young adults. The primary objective of this phase attended to relationship building and working collaboratively to determine the direction for the capacity building phase. Table 3.2 shows the indicators used to assess whether the activities and outputs of the first phase of the project had been completed.

7

(31)

Table 3.2 In Roads Indicators and Data Sources for the First Phase of In Roads

Output and Indicator Data Source

Partnership between AHS and community organizations established  Number and type of organizations entering into

partnership

 Partnered agency documentation Service providers for mentorship/training are identified

 Number and job classification of service providers  Partnered agency documentation  Service provider needs assessment Organization/staff substance abuse training needs identified

 Organization learning priorities set

 Staff learning needs determined  Agency level needs assessment  Agency Service Provider needs assessment

Phase 2

The second phase of the project focused on the delivery of the mentorship stream within partnered agencies. Under the direction of the In Roads provincial project coordinator and Clinical Council, AHS addiction counselor mentors, supervisors, and Zone program coordinators developed resources and curriculum to support the In Roads project. In this phase, In Roads addiction counselors worked with community agency service providers to promote the development of knowledge and skills in the In Roads learning areas. Table 3.3 shows the indicators for phase 2 of the project. Relevant factors are reported on in this final evaluation report, in aggregate form (total project results).8

8

(32)

Table 3.3 Indicators and Data Sources for Capacity Building with Community agencies

Mentorship relationships established and training provided  Number and description of mentorship relationships formed  Type/nature of training provided

 Organization representative and service provider feedback on mentorship training

 Partnered agency documentation  In Roads Project Manual document

review  ASIST data

 Agency representative Online Survey

 Agency service provider Post Mentorship Survey

 Agency representative Follow-up Interview

 Agency service provider Follow-up Interview

Presentation or workshops developed and conducted

 Type/nature of presentations or workshops developed as part of the project

 Number of presentations or workshops provided9

 Number of participants attending presentation session or workshops

 Participant feedback on workshops or presentations

 Presentation documentation  ASIST data

 Participant Feedback Surveys10

Resource materials developed to support the program

 Type/nature of resources developed to support the initiative

 Agency service provider feedback on resources  Resource document review  Agency Service Provider Post Mentorship Survey

Phase 3

The third phase of the project concerned the partnered agencies’ service providers (mentees) delivery of services to their clients. This phase addressed practice changes attributed to the evidence-based knowledge and skills learned in this initiative (see Table 3.4 for a list of phase 3 indicators). The project developers expected, at the point of the Interim Report, that the third phase would be largely dependent on agency readiness to adopt evidence-based practices. It was assumed, in order to capture the effects of these capacity building efforts that it would require several participants from the same agency to participate in the initiative before the evidence-based practices would become part of agency culture (In Roads Project Manual, 2013). Therefore, the data from this phase, as captured in the Interim Report, was considered to be preliminary. Relevant factors are reported on in this final evaluation report, in aggregate form (total project results).11

9

Presentations to both partnered and non-partnered agencies are included in this report.

10

Feedback Surveys are only available for a select number of presentations.

11

(33)

Table 3.4 Indicators and Data Sources for Service Delivery to Clients

Youth and young adults are screened, provided brief intervention and referral  Number of screenings and brief interventions provided at the

community agency

 Number and type of referrals

 Stakeholder perception of the extent to which screening and brief intervention practices are meeting the needs of clients

 Client service tracking data  Agency representative Follow-up

Interview

 Agency service provider Follow-up Interview

Outcomes (Final Evaluation Report [May 2013])

The key evaluation question requested from Health Canada was to assess what progress the In Roads project made toward identified project outcomes. The outcomes, indicators and data sources of the initiative are detailed in Table 3.5, below. Those outcomes that are linked with Health Canada’s DTFP outcomes are highlighted.

Table 3.5 Outcomes and Indicators of the In Roads Project

Outcome and Indicator Data Source

Short term outcomes:

Health Canada outcome: Enhanced collaboration on responses to DTFP treatment systems issues within and among jurisdiction and stakeholders.

In Roads project outcome: Enhanced collaboration among AHS and stakeholder organizations  Nature, scope and depth of collaborative activities between

AHS and partnered agencies on treatment systems issues  Stakeholders’ perception of the extent to which

AHS-community agency partnerships are collaborative

 Stakeholder perceptions on what worked well and what were the challenges with collaboration to improve capacity to deliver service to the target population

 Partnered Agency documentation  ASIST data

 Agency Final Survey

 Agency Semi-Structured Interview  AHS Staff Semi-Structured

Interview

 Service Provider Semi-Structured Interview

In Roads project outcome: Increased community service provider knowledge and skills on substance abuse screening, early intervention and referral

 Change in service provider knowledge, skills and confidence in providing screening, referral and early intervention to youth and young adults

 Agency representative perception of change in staff members’ knowledge and skills in screening, brief intervention and referral as a result of participation in the project

 Comparison of needs assessment to Post Mentorship Follow-up Survey

Interview

 Change in service provider knowledge, skills and confidence in providing screening, referral and early intervention to youth and young adults

 Agency representative perception of change in staff members’ knowledge and skills in screening, brief

(34)

Outcome and Indicator Data Source intervention and referral as a result of participation in the

project

 AHS Staff Semi-Structured Interview

In Roads project outcome: Increased awareness of treatment options for youth and young adults  Organization service providers have knowledge of treatment

options available in Alberta

Interview

In Roads project outcome: Youth and young adults have increased knowledge about substance abuse issues and effects and the treatment options available

 Organization service provider perceptions on the extent to which the target population have increased knowledge of substance abuse issues and effects and the treatment options available

 Change in target population’s awareness/knowledge of substance use and its effects

 Change in targeted population’s awareness/knowledge of treatment options available

 Service Provider Post Mentorship Survey

Interview

 Client Services Recording  Client Feedback Survey Intermediate outcomes:

Health Canada outcome: Increased access to evidence-informed practice information

In Roads project outcome: Increased access and use of evidence informed practice information  Extent to which agencies and service providers report

increased availability, access and use of evidence informed practice information

 Perceptions of organization staff that training and information provided fits with their agency needs and target population  Extent to which service providers report increased

availability, access and use of evidence informed practice information

 Service provider Post Mentorship Survey

Interview

Health Canada outcome: Enhanced P/T capacity to deliver evidence-informed early intervention treatment programs & services to at-risk youth and young adults in high-needs areas

In Roads program outcome: Enhanced capacity to deliver evidence-informed early intervention services for at-risk youth and young adults

 Type/nature and changes in the capacity to deliver services

 Description of new substance use services delivered by agencies as a result of participation in the program

 Service provider Post Mentorship Survey

 Agency Semi-Structured Interview

(35)

Outcome and Indicator Data Source In Roads program outcome: Increased engagement in early intervention activities within partner organizations

 Agency and service provider perceptions on their client’s change in their participation in early intervention activities (e.g., participation in screenings and brief interventions) within the agency

 Target population feedback on services provided

 Agency Semi-Structured Interview

 Service Provider Semi-Structured Interview  Client Services Recording  Client Feedback Survey Long term outcomes: (not included in the final evaluation report)

Health Canada outcome: Increased availability/access to sustainable, evidence-informed early intervention treatment programs and services for at-risk youth and young adults in high-needs areas In Roads program outcome: Increased access to sustainable, evidence informed early intervention treatment services by at-risk youth and young adults

 Change in service delivered by organization as a result of participating in the In Roads program

 Number of screening, brief interventions, and referrals provided at the organization

 Demographic characteristics of clients screened and provided brief intervention or referral

 Description of agency sustainability plans (e.g., Train-the-Trainer activities)

Because the In Roads project operated in a variety of contexts, several mechanisms and techniques were used to capture program metrics, as well as In Roads knowledge exchange processes.12 To provide a comprehensive picture of In Roads implementation and progress towards outcomes, a combination of quantitative and qualitative methods were used to report on the key evaluation questions. Multiple lines of evidence were gathered from key project

stakeholders (see data source descriptions, below) in order to ensure robust results. Table 3.6 provides an overview of the data collection methods and sources.

12

(36)

Table 3.6 Summary of Data Collection Methods Collection Method Partnered agency representatives Agency service providers (Mentees) AHS staff (Mentors, Zone teams) Non- partnered agencies Clients Final Evaluation Report (May 2013)

Document review X

Agency pre and final

survey X Service Provider pre/post survey X Semi-structured interviews X X X Client Services Recording X Client Feedback Survey X ASIST data X Presentation Feedback Survey X X

Description of data collection mechanisms

Document review

To gather information relevant to this outcome evaluation, several key documents were reviewed.  Quarterly reports: each agency was required to submit a brief update on successes and

challenges of the In Roads project (n=65 [2012/2013]);

 Quarterly reports: each clinical supervisor was required to submit a brief update on the successes and challenges of the In Roads project (n=3); and,

 In Roads progress reports: submitted to Health Canada (n=2).

Agency and Service Provider Surveys

The In Roads project was designed to be tailored to the agency and individual needs of the service providers that participated in the initiative. The purpose of the In Roads needs assessment was to answer the question, “What do the agencies and service providers need in terms of substance use training and how can the In Roads project assist in meeting those needs?” To answer this

question, a semi-structured interview/ questionnaire was conducted at both the agency and individual service provider level. The goal was to determine what knowledge and skills may have already been part of the agency’s in-house training program and what types of previous

(37)

The agency-level needs assessment explored addiction and substance use training provided by the agency, current policies or procedures for addressing substance use with clients, the agency’s addiction screening practices and asked agency representatives to rate their level of priority for mentoring in each of the In Roads learning areas.

The individual service provider (mentee) needs assessment was similar to the agency-level needs assessment, but included questions that ask participants to rate numerically their current level of knowledge, skill and confidence in the learning areas covered by the In Roads project.

The needs assessment serves two purposes. First, it supported the initiative by targeting the areas of capacity building need (avoiding duplication of previously trained areas) and ensured the

information provided was appropriate for the agency’s baseline capacity. Second, the needs assessment served to provide a baseline indication of the community service provider’s capacity to deliver evidence-based services pre- In Roads program.

As a result of gathering this baseline data, change in capacity could be determined through completion of the post-program comparison with the service providers. However, for the agency representatives, a final survey format was used. Table 3.7 lists the topics contained within both the agency and individual needs assessments. Topics highlighted by an asterisk form the baseline of agency and service provider capacity prior to the In Roads project.

Alberta Health Services Addiction and Mental Health Primary and Community Care

Acknowledgements

TABLE OF CONTENTS

LIST OF TABLES

LIST OF FIGURES

Executive summary

Overview of the In Roads Project

Final Evaluation

Some Key Findings

Conclusion

SECTION 1: INTRODUCTION

Background

Purpose of the Evaluation

Evaluation Implementation

Evaluation Participation

SECTION 2: PROJECT DESCRIPTION

Project Goal

Project Objectives

Target Population

Activities

Outputs

Expected Outcomes Across the Three Phases (Including Links to DTFP Program

Outcomes)

Operating Context

Project Management/ Governance Structure and Administration

Project Stakeholder Relationships

SECTION 3: EVALUATION SCOPE AND METHODS

Evaluation Scope

Evaluation Issues, Questions and Indicators