SIGNIFICANTLY IMPROVING YOUTH MENTAL HEALTH

1O%

2O%

3O%

4O%

5O%

6O%

7O%

8O%

9O%

1OO%

PROOF POSITIVE

K i d s H e l p P h o n e ’s P h o n e a n d L i v e C h a t C o u n s e l l i n g Ev a l u a t i o n s

SIGNIFICANTLY IMPROVING YOUTH MENTAL HEALTH

2O%

3O%

4O%

5O%

6O%

7O%

8O%

9O%

1OO%

Executive Summary:

Key Learnings

4

I

ntroduction: Assessing

the Impact of Kids Help

Phone’s Services

8

Phone Counselling Evaluation

12

Methodology 13 Sample Size 13 Results 14 Conclusion 27

Live Chat Counselling

Evaluation 28

Methodology 3O Sample Size 3O Results 3O Conclusion 41 Next Steps for Live Chat 41

Kids Help Phone’s Service

Evaluations: Conclusion

42

Acknowledgements & Credits 45

Kids Help Phone is a Canadian and world leader, known for our

expertise in providing vital, innovative counselling services to

children and youth.

Our service is a lifeline available around-the-clock to the

6.5 million young people in Canada, instantly accessible from

urban, rural and remote communities, online and by phone, in

English and in French. We are well-known and respected as the

only national service providing youth across Canada with free

professional counselling on an anonymous and confidential

basis. Our award-winning websites are considered a model

of child-focused interactive design, and offer counselling,

information, and tools to encourage resilience and self-care.

Since 1989, we have supported the mental health and

well-being of millions of youth in Canada and become an essential

part of the continuum of mental health care for young people.

A community-based national charity, Kids Help Phone receives

no guaranteed ongoing financial support from any large funder

and relies on community and corporate support to keep its

essential service available.

Executive Summary: Key Learnings

In 2O11 and 2O12, Kids Help Phone conducted rigorous evaluations of its phone and Live Chat pilot

services to better understand their impact on young people, with a longer-term goal of ensuring

and improving upon service quality. We are pleased to report that our service evaluations revealed

a number of very significant key learnings about the impact of our service, summarized below.

Kids Help Phone’s professional

counselling is effective

Kids Help Phone counselling

is associated with statistically

significant reductions in distress

Kids Help Phone

counselling is associated

with statistically significant

increases in clarity

Kids Help Phone’s phone and Live Chat services are associated with statistically significant increases in young people’s clarity about how to cope with their issues or concerns. For the phone service, 73% of callers stated that they had a plan about how to respond to or deal with their problem after speaking with a Kids Help Phone professional counsellor, and 68% of callers said their counsellors helped them become more aware of their personal strengths and abilities. Seventy-one percent of young people who used Live Chat to connect with a counsellor also reported experiencing an increase in their clarity about how to move forward as a result of the counselling they received.

1_{StatisticsCanada.(2006a).Aboriginal identity population by age groups, median age and sex, 2006 counts for both sexes, for Canada, provinces and territories –}

20% sample data[Datatable].Retrievedfromhttp://www12.statcan.ca/census-recensement/2006/dppd/hlt/97558/pages/page.cfm?Lang=E&Geo=PR&Code =01&Table=1&Data=Count&Sex=1&Age=1&StartRec=1&Sort=2&Display=Page

Kids Help Phone counselling

is associated with statistically

significant increases in

confidence

Kids Help Phone’s counselling boosts young people’s confidence significantly. Sixty-six percent of youth who chatted online with a Kids Help Phone professional counsellor reported a statistically significant improvement in their feelings of confidence that they could cope with or change their situations (averaging an increase of 1.09 points on a seven-point scale).

Kids Help Phone is reaching

some of the most under-served

and potentially vulnerable

young people in Canada

The phone counselling service evaluation found that 10% of callers self-identified as Aboriginal, which represents more than two times the proportion of young people who are Aboriginal in the Canadian population.1 _{Further, 8% of} callers self-identified as Black, which is more than double the representation of young people who are Black in the Canadian population. In addition, 4% of young people calling self-identified as transgender and 16% self-identified as gay,

lesbian, bisexual, or questioning. This shows that our counselling service is being used by youth who may need and benefit from it the most, in high numbers.

Kids Help Phone is an

important part of the

continuum of mental

health services in Canada

Kids Help Phone is clearly filling an important role in the continuum of mental health care in Canada. At 41%, a considerable number of Kids Help Phone callers stated they were currently seeing a mental health professional or had seen one in the past, while 11% stated they were on a waiting list for counselling or therapy – indicating that Kids Help Phone fills a need for mental health care while kids are waiting to access other supports or between appointments. Of those who were currently accessing formal mental health supports, the number one reason they were calling Kids Help Phone was to receive support with a mental or emotional health struggle – this was the case in both the phone and Live Chat evaluations. Among Live Chat clients particularly, there are high proportions of young people dealing with serious mental health concerns such as suicide, eating disorders, or self-harm. These findings help us better understand how Kids Help Phone’s counselling service complements and helps to address gaps in the formal youth mental health sector.

Young people who reach out

to Kids Help Phone via one

counselling channel would

not necessarily reach out in

other ways

In each of the evaluations, young people explained the ways in which phone and Live Chat counselling address their different preferences and concerns. Respondents were very clear about their preference for reaching out via one particular counselling channel or another. For example, many young people who telephoned Kids Help Phone stated that they prefer to talk about their problems rather than writing about them (34%), while many Live Chat users stated that they would rather write than speak about their problems (42%). Seventy-five percent of respondents in the Live Chat evaluation indicated they felt too nervous or uncomfortable to call the phone service, and 62% said they do not have enough privacy to speak on the phone.

Kids Help Phone is meeting

young people’s expectations

Kids Help Phone is often the

first point of intervention for

young people

Forty-three percent of callers stated that they had not spoken to anyone about their problem before calling Kids Help Phone. In addition, in analyzing young people’s answers to open-ended questions included in the phone and Live Chat evaluations, we found that a number of young people contacted Kids Help Phone for reasons such as receiving an opinion about the severity of their concerns, exploring their options, and determining their next course of action based on what they themselves assessed was right for their situation. For example, when asked why they called Kids Help Phone instead of talking to someone else, a respondent to the 2012 phone evaluation stated that they “…wanted advice on a situation. Other people said to call the police. I wasn’t ready to take that step.”

Young people want confidential

and anonymous services

Young people using all of our counselling channels stated that a primary reason they chose Kids Help Phone was that our service is anonymous and confidential. In fact, 75% of young people who participated in the Live Chat evaluation and 21% of those who participated in the phone evaluation stated that they chose to contact Kids Help Phone rather than anyone else because anonymity is guaranteed. In addition, 9% of respondents to the phone evaluation listed confidentiality as one of their reasons for contacting Kids Help Phone rather than anyone else.

Young people often come

back to Kids Help Phone for

counselling many times

While Kids Help Phone provides single-session counselling, the service evaluations helped to highlight that young people often return to receive counselling for many different challenges over the course of their lives. While 39% of respondents in both evaluations had never contacted Kids Help Phone before, it was interesting that 50% of callers and 45% of Live Chat users stated that they had used Kids Help Phone’s service between two and ten times before.

Live Chat is more accessible for

young people who lack privacy

Boys are less likely to reach out

to Kids Help Phone than girls

Live Chat has tremendous

potential to help those with

serious mental health concerns

Through this research, Kids Help Phone has experienced how service evaluations can yield new learnings that are unexpected and help the organization to really understand “where kids are at” to fulfill its mission and vision. We are committed to continuing to build on these learnings, to provide an effective professional counselling service that benefits young people all across Canada, improving their mental health and well-being.

We trust that this report will confirm the

confidence that our many loyal partners, donors and other supporters have in Kids Help Phone and that together we can continue to meet the quickly-evolving mental health and well-being needs of young people in Canada.

Over the years I have worked with Kids Help

Phone they have consistently demonstrated

what an important service they provide to

young people across Canada. These strong

evaluation results reinforce what I already

knew: that Kids Help Phone provides a critical

service as part of Canada’s mental health

service continuum. As well, Kids Help Phone

technologies through Live Chat and their Mobile

App especially show their commitment to truly

being available for young people, however,

whenever and wherever the youth of Canada

need them.”

“

Introduction: Assessing the Impact

of Kids Help Phone’s Service

Every day, Kids Help Phone’s professional counsellors hear from young people who are experiencing the loneliness of depression; the paralyzing fear of anxiety; the pressure to succeed, compete, or conform; or any number of other issues. And far too often, they hear from kids who have lost all hope and are struggling against self-harm or thoughts of suicide. Kids Help Phone is there when other services and supports are not – because young people are on waiting lists for professional mental health support or in-between appointments, because they are not ready or able to reach out for help in other ways, or because no other help is available to them in their communities. Since 1989, we have been Canada’s only national counselling service supporting these young people – and their mental health and well-being. Over the years, we have provided our service to millions of children and youth in Canada, with young people reaching out to a professional counsellor an average of 5,000 times each week as of 2012. Our service is available 24/7, 365 days a year, and our counsellors speak with young people from urban, rural and remote communities, from every walk of life, and in both official languages.

In recent years, through the generosity of thousands of donors and supporters in Canada, we have been able to respond to the increased interconnectedness between young people and technology by adding several online modalities to the service we provide. These include our “Ask Us Online” web-based message board service, which provides an individualized response from a counsellor to a young person’s posted question; our award-winning Kids and Teens websites, which provide engaging, clinically reliable information on over 55 topics of interest to young people; youth-focused interactive games and tools; an online community where young people can learn from each others’ questions and concerns; and most recently our Always There app, which allows young people to connect to clinically-supported information and professional counsellors directly from any mobile device. Kids Help Phone believes in providing the highest possible quality of service to young people, and therefore is committed to evaluating and better understanding the impact our services have on the children, teens, and young adults who rely on us. Because of Kids Help Phone’s full commitment to anonymity and confidentiality, we have, up until now, only

and demand and the reasons young people reach out for support. In addition, we have relied on thank you letters, thousands of comments on our websites, and client satisfaction surveys on kidshelpphone.ca to demonstrate service impact. These measures have always been important in telling the story of how Kids Help Phone’s professional counsellors make a difference in the lives of young people in Canada. However, we have continued to receive questions from our many supporters, such as: How does our service – where kids are able to communicate directly with a professional counsellor – contribute to meaningful change in their lives? What is the impact of single-session chat, web, or phone counselling? Why do youth seek help from Kids Help Phone as opposed to from other resources? How important are emerging technologies, such as our Live Chat pilot service, in helping youth in distress? How does Kids Help Phone help youth from under-served or potentially vulnerable populations?

To answer these and other questions, Kids Help Phone committed to evaluating its phone and Live Chat pilot services to better understand their impact on young people, with a longer-term goal of ensuring the delivery of an ongoing, consistent, and outstanding quality of service. We are pleased that the service evaluation results outlined in this report help to answer these and other important questions, and help to explain how and why our professional counselling is an essential support for young people and a key component of the mental health service sector in communities across Canada.

The Importance of Evaluations

Kids Help Phone believes that evaluation is an essential component of effective service delivery, since having a rigorous process for evaluation is, first and foremost, a sign of respect for and dedication to the users of that service. A commitment to evaluation indicates that client needs are of the utmost priority, and that systems are in place to ensure that those needs are being met. Second, the use of systematic evaluations demonstrates a commitment to maximizing the impact of every donor’s gift.

Conducting an evaluation allows us to:

• Learn more about clients and their needs, which can then inform and support professional and program development

• Determine where the service meets or exceeds benchmarks, and also where development or additional resourcing is needed

• Illuminate whether the service is effecting positive change among its client base, and what that change looks like, thereby providing a foundation for service enhancements and for new programs in future

• Share with our donors and supporters a more detailed analysis of their impact on the mental health and well-being of young people in Canada

The need for evaluation of what we think works for children at risk is more

important now than ever as we reconsider where to invest public dollars…

These evaluations help prove that alternative ways of delivering

community-based mental health care not only work, but that these programs are

experienced as excellent care by young people themselves.”

“

Three core components drive Kids help Phone’s service: delivering professional counselling to young people, providing them with clinically reliable and age-appropriate information, and offering them information about local programs and services where they can get more help in their communities. Our three direct professional counselling modalities (phone, web, and Live Chat) each carry somewhat different possibilities and constraints (e.g., emotional nuance can be experienced differently depending on the modality) and can elicit varying reactions and perceptions from young people (e.g. believing certain modalities are more anonymous or having a preference for verbal communication over typing). What all of the direct counselling services offered by Kids Help Phone have in common is that they provide on-demand, single session, professional counselling. By working consistently within this particular framework, our goal is to provide effective counselling that meets the needs of the service user, regardless of the counselling modality they use.

At the start of the evaluation process, we began mapping out the short, intermediate, and long-term outcomes that we projected to occur for most of our clients after receiving professional counselling from Kids Help Phone (see diagram at right). As our counselling service reaches a wide age range (five to 20 years old), we knew that it supports young people at a variety of different developmental stages and coming to us with an incredibly diverse range of issues (ranging from friendship and puberty to questions about moving out on your own or clinical-level mental health concerns).

Grounded in Kids Help Phone’s clinical framework, which uses client-centred, strengths-focused principles as well as Solution-Focused Brief Therapy and Narrative Therapy, we anticipated that in the short term, service users would experience positive changes in their feelings and attitudes, followed by changes in awareness and knowledge. As we understand change to

Understanding our Service and Projected

Evaluation Outcomes

Skills Development

Our

Mission

Changes

in Behaviours

& Practices

Changes in Awareness & Knowledge

Changes in Feelings & Attitudes

Intermediate Outcomes

As an on-demand, single session

counselling service, most of our client

outcomes were expected to be

11 KIDS HELP PHONE | PROOF POSITIVE

66%

of youth who chatted online with a Kids Help Phone

professional counsellor reported a statistically significant

improvement in their feelings of confidence that they

could cope with or change their situations.

Phone Counselling Evaluation

Since 1989, Kids Help Phone has provided telephone counselling to countless young people from every corner of Canada dealing with every conceivable challenge. Over the years, we have used informal feedback received from young people, parents, and educators to make changes to our service. But as with most other helplines internationally, we had not conducted a formal and rigorous evaluation of our phone counselling service to determine what difference the service makes to young people – and how to make it better.

Consequently, in 2011, we developed and tested an outcome-focused evaluation framework for our telephone counselling service. This evaluation project was partially supported by a Planning Evaluation Grant from The Ontario Centre of Excellence for Child and Youth Mental Health and included staff from our Counselling, Knowledge Mobilization, and Donor Relations departments.

The aim of this evaluation was to establish preliminary client demographic, user

satisfaction, and outcome benchmarks beyond those we already had from existing service data (e.g., gender of callers, issues they discussed), which was limited by the anonymous and confidential nature of our service. As a result, the evaluation focused on the following five questions to help guide data collection:

• Who accesses the telephone counselling service? • Are callers satisfied with the service they receive? • Do telephone counselling clients experience

positive changes in feelings and attitudes (specifically reductions in distress and isolation)? • Do telephone counselling clients experience

positive changes in awareness and knowledge (specifically a growth in awareness of their personal strengths and abilities)?

• Do telephone counselling clients experience positive skills development (specifically in their problem-solving abilities)?

Methodology

The evaluation’s methodology involved counsellors inviting callers at the end of their counselling sessions to answer a survey that was administered by a research assistant3 located in a different room. If a caller agreed to be transferred to participate in the survey, the research assistant took them through an informed consent process, including advising them that their responses would be entirely anonymous and confidential and that they could skip any question or end the survey whenever they wished. If consent was then granted, the research assistant administered the 45-question survey. This methodology was chosen for two primary reasons: to protect each caller’s anonymity and confidentiality, which is essential to the appeal and relevance of our counselling service, and to address the fact that the telephone counselling service is a single-session service that does not allow us to contact former clients, and therefore requires that we ask for their feedback right away.

Sample Size

During the six-week data collection period, counsellors invited 615 clients who fit a

previously designed set of criteria to participate in the evaluation.4 _{Three hundred and eighteen} of the callers who were invited to participate in the evaluation declined with reasons including not feeling up to it, feeling that their situation was too serious or dangerous and therefore not being in a position to participate, or no longer having enough privacy to talk. In total, 246 callers met the inclusion criteria and agreed to participate in the survey. This means that the survey response rate was 40%. This is a remarkably high response rate, larger in fact than any other published child evaluation research study of which we are aware.

Phone Counselling Evaluation

“They listen even if you don’t have

a lot to say, they listen to you.”

“It’s like really fast; you can just call and talk to someone

and you don’t have to worry that they’ll judge you.

Results

Who Uses the Service?

Demographics

Given the anonymous and confidential nature of the telephone counselling service, Kids Help Phone counsellors are unable to regularly collect comprehensive demographic data about callers. Information gathered through this evaluation, therefore, helps us better understand which young people in Canada use our telephone counselling service. GeographicLocation

The evaluation provided a clear demonstration of the reach of the phone service. To best understand its geographic reach, we compared the percentage of young people who called us from each province with Statistics Canada data on the percentage of young people ages five to 20 residing in each province. By doing this, we learned that young people from Ontario, the Atlantic provinces, and Canada’s territories are somewhat over-represented among our service users, while we have room to reach more young people in British Columbia, Saskatchewan, and Quebec.

Kids Help Phone is there for all young people, including some

of the most under-served and potentially vulnerable in Canada.

11%

3.4%

52%

5%

11%

9%

2%

2%

Distribition of young people:

Kids Help Phone users compared

to general population

5_{2011 General Child Helpline Data.http://www.childhelplineinternational.org/resources/data/connecting-to-children/connecting-to-children-2011-data/} 6_{Taylor,C.&Peter,T.,withMcMinn,T.L.,Elliott,T.,Beldom,S.,Ferry,A.,Gross,Z.,Paquin,S.,&Schachter,K.(2011).Every class in every school: The first national climate}

survey on homophobia, biphobia, and transphobia in Canadian schools. Final report.Toronto,ON:EgaleCanadaHumanRightsTrust.

Gender

Seventy-three percent of callers participating in the survey self-identified as female, while 23% self-identified as male. This was expected, as traditionally Kids Help Phone has received a much higher proportion of calls from females than males. In addition, health services in general attract more female than male clients – and this is also true for most child helplines around the world.5 _{So this weighting} of female versus male callers is not a distinguishing characteristic of Kids Help Phone’s service, but it does show the importance of finding ways to encourage boys and young men to reach out to Kids Help Phone when they need help.

There were also a noteworthy number of young people who identified as transgender (4%) when asked, “What gender do you identify with?” This is an encouraging sign that the phone service is used by young people on different parts of the gender continuum.

SexualIdentity

An unexpected finding for Kids Help Phone was that 16% of respondents self-identified as gay, lesbian, bisexual, or questioning when asked “What sexual orientation do you best identify with?” While there are no comparable statistics for LGBQ identification among the Canadian population, we assume that this group is over-represented among the young people calling Kids Help Phone compared to their representation in the Canadian population.

We know that young people from these communities often face systemic barriers as a result of homophobia and heterosexism, which create social environments that place them at greater risk of a number of challenges including feelings of loneliness, guilt and shame; experiences of harassment and abuse; a lack of acceptance from significant others; and mental and emotional health concerns including depression and suicide.6 _{It is reassuring to know that young people} from LGBTQ communities are reaching out to Kids Help Phone to get the support they need and deserve to cope with these, and other, challenges.

4%

of callers identified

as transgender.

16%

of callers identified as

gay, lesbian, bisexual,

or questioning.

73%

23%

4%

Young people

who are Aboriginal

and between the

ages of five and

2O represent about

5% of the Canadian

population and

of Kids Help

Phone’s callers.

Racial,Ethnic,andCulturalAffiliation

In an effort to better understand the racial, ethnic, and cultural affiliations of young people who call Kids Help Phone, research assistants asked an open-ended question: “People are often described as belonging to particular racial, ethnic or cultural groups. For example, Filipino, Jamaican, English or Inuit. To which ethnic or cultural groups do you see yourself belonging?” Young people could then self-identify in any way they liked, and they could identify with as many groups as they liked.

What we found was a demonstration of the diversity of Kids Help Phone’s callers. In fact, while the largest numbers of callers identified as White, European or Caucasian (34%) or Canadian (31%), the next three largest categories were:

• Aboriginal (young people self-identifying as “First Nations,” “Aboriginal,” or “Métis”) – 10% of callers • Young people self-identifying as Black or of African

or Caribbean descent – 8% of callers • South Asian – 5% of callers

Overall, Kids Help Phone was heartened to find that young people from historically under-served and potentially vulnerable groups were calling our professional counsellors in large numbers, as it means we are fulfilling our mission of truly “being there” for young people who need us the most.

Reasons for Using the

Phone Service

Kids Help Phone provides support to young people at very different developmental stages (for example, five year olds compared to 20 year olds), and for an incredibly diverse range of issues, from relationship challenges with family or friends and bullying to issues such as self-injury, substance abuse, violence and thoughts of suicide. For many years we have collected data on the issues young people raise with our counsellors. This data, which comes from our counsellors’ perspectives, has consistently demonstrated that the four primary areas that young people contact Kids Help Phone about are mental health, including suicide and addictions (22% of contacts), struggles in peer relationships (21%), struggles in family relationships (15%), and violence and abuse (13%).7

The phone service evaluation offered a unique opportunity to allow young people to explain the reason or reasons they were calling to speak to a Kids Help Phone counsellor. As such, callers were asked, “What problem or situation did you call us about today?” after which the research assistant read out a list of issues ranging from violence and abuse and friend/peer or family relationships to dating and mental or emotional health. Of the 239 respondents who answered this question, 266 reasons for calling were given, with family relationships receiving the most mentions (25%), followed by friend/peer relationships (22%) and dating/relationships (15%).

“Stress. I felt like I wanted

to run away and then I

talked to a counsellor and

he settled me down. I’m not

going to run away now.”

7_{2012data.Withintheselargecategoriesaresub-categorieswithmorespecificissues.Forexample,ayoungpersoncontactingusinrelationto“familyissues”mighthave} beencategorizedasspeakingaboutoneorseveralsub-issuesincludingcustody,anabsentparent,loss,divorce,etc.

Kids Help Phone is there

for all young people,

including some of the most

under-served and potentially

vulnerable in Canada.

The issue data for callers during the evaluation differed somewhat from our annual data. This is most likely due to the fact that youth were asked to evaluate for themselves why they chose to contact Kids Help Phone (as opposed

to how we collect our data annually, which involves having a counsellor indicate the reason on a survey completed during the counselling contact). Another reason contributing to the difference may have been the fact that some

callers, especially the most distressed, were not asked to participate in the phone evaluation when they did not meet the inclusion criteria. Thus, data from the most distressed callers is not included in these results.

Reason for calling (n= 239, multiple choice)

Becoming independent Information about laws or rights Sexual orientation or gender Suicide Self-injury Physical or sexual health Substance use or addictions Violence or abuse School Mental or emotional health Bullying Dating / relationships Friend / peer relationships Family relationships

1

2

2

3

3

4

4

4

6

9

12

15

22

25

Frequency of Service Use

While Kids Help Phone provides single-session counselling, the phone service evaluation helped us better understand callers’ relationship with the telephone service and indicated that many young people use our phone counselling service more than once. In fact, while 39% of the young people who participated in the evaluation were first-time callers to Kids Help Phone, 61% reported that they had called previously. The majority of repeat users (45%) reported having called between two and five times before, while 16% had reached out to Kids Help Phone more than five times previously.

How many times have you contacted

Kids Help Phone? (n=239)

Kids Help Phone was a unique organization

for The Centre to work with because of their

service model which includes a promise to

kids of total anonymity and confidentiality.

We were challenged to support the Kids Help

Phone team in creating a robust evaluation

that protects the anonymity and confidentiality

of callers to the helpline while also enabling

research on the effectiveness of the counselling

service. The Centre is proud of the role we

played in Kids Help Phone’s evaluation process

and very pleased to see the commitment the

organization has made to evaluating its service.

The results are very encouraging and point

towards the effectiveness of their service.”

Ian Manion, Ph.D., C.Psych.

ExecutiveDirector OntarioCentreofExcellence forChildandYouthMentalHealth

“

5%

3%

8%

39%

45%

Use of Other Supports

Our phone evaluation demonstrated the unique importance of Kids Help Phone both as a stand-alone, 24/7/365 professional counselling service for youth, and as an essential part of the continuum of mental health care as an adjunct to other mental health supports at the local level. Phone counselling at Kids Help Phone is often the first place young people turn to for help, and in many cases, it is the only source of counselling young people will, or can, access. In fact, when asked who they had spoken to prior to calling Kids Help Phone, 43% of callers stated that they had not spoken to anyone else about their problem or concern before they called.

When asked why they chose to call Kids Help Phone rather than speaking with someone else about their concern, 21% of callers stated that they did not think there was anyone else they could talk to. In addition, 21% of callers stated that they called Kids Help Phone because of the service’s promise of anonymity.

The evaluation also revealed that Kids Help Phone is a critical adjunct to other mental health services and helps young people to connect with local resources in their communities. Forty-one percent of phone clients stated that they were currently seeing or had seen a professional counsellor or therapist before (a remarkably high proportion), and 11% said they were currently on a waiting list for local mental health services.

Impact

As a part of the phone evaluation, Kids Help Phone set out to determine how contact with our professional counsellors is having an impact on young people. We wanted to better understand if the service is simply providing a caring adult to listen to young people’s concerns, or if professional counselling sessions are making a significant change in young people’s mental health and well-being on a clinical level.

Distress and Isolation

To help us understand if young people’s distress levels were reduced between the start and end of their calls, we asked callers a pair of questions. First we asked them to reflect on how upset they felt about their problem or situation before working with a counsellor. The subsequent question asked them to rate how upset they felt about this same situation or problem after they had spoken with a counsellor. Two hundred and thirty-three respondents answered both of these questions. The chart on p.23 reveals that more than half of phone clients scored

themselves as a one or two on our seven-point scale (where seven was “no distress” and one was “very high distress”). This means that before speaking with Kids Help Phone, many of these young people were feeling as distressed as it is possible for them to feel.

The average distress rating before speaking with a Kids Help Phone counsellor was 2.84 on the seven-point scale. The average distress rating after

speaking with a counsellor was 5.35. Eighty-seven percent of callers reported a large reduction in distress, averaging 2.5 points on the seven-point scale. This is a statistically significant (p<.001) reduction in distress, meaning that we can be very confident that the reduction in distress among callers was not due to chance.

We also asked callers about the extent to which they agreed or disagreed with six statements about the relationship they had with their Kids Help Phone counsellors. The vast majority of callers reported feeling respected, listened to, supported and understood; they felt as if they had a good connection to their counsellors and found their counselling calls helpful, indicating reduced levels of distress and isolation.

11%

of callers were on a

waiting list for mental

health services in their

community.

“I had a couple instances of self-harm,

sporadically for the past year. Every time

I got really sad, I would feel like doing it,

it felt like a little voice in the back of

my head saying, “You can feel better, this

will help you feel better,” even though I

knew it wasn’t good for me. It was really

late at night, and none of my family knew

about what had happened before, and

so I didn’t know what else to do, and I

called them and it was honestly the best

decision I’ve ever made. The counsellor

was extremely helpful and it helped having

someone who didn’t know anything about it

to talk to. It helped level my head out a

little bit. Kids Help Phone is an absolutely

amazing, amazing service. So, thank you

for all that you’ve done, and I’m just,

I’m just… Thank you.”

“Wasn’t alone and that it’s

normal for me to feel anxiety

about what’s going on and

that things will get better”

According to the evidence,

Kids Help Phone is an excellent

example of meeting kids where

they’re at.”

“Felt like I was listened

to and understood.

Made me feel better.”

8O 6O 4O 2O O

Very high distress No distress Very high distress No distress

Before

phone counselling,

level of distress

After

level of distress

phone counselling,

1

2

3

4

5

6

7

1

2

3

4

5

6

7

“

41%

of callers were currently

seeing a therapist or had

done so in the past.

- client / evaluation participant

Awareness of Personal Strengths

We wanted to determine if callers felt meaningfully involved in their counselling call and if, as a result, they were more aware of their own strengths (abilities and capacities) by the end of the call. We thought of the first question as a prerequisite for the second, since meaningful involvement in a call could be expected to help callers become more aware of their own strengths. Eighty-six percent of callers reported feeling meaningfully involved in the conversation with their counsellor and 68% said that their counsellor helped them become more aware of their personal strengths.

“My counsellor was able to

pin-point my strengths and

how I could use them in

my situation.”

“Yeah, they told me I was

very brave to call and I had

to speak up for myself more.”

96

95

93

89

88

8O

I felt respected by my counsellor

I felt listened to by my counsellor

Talking with a Kids Help Phone counsellor was helpful

I felt supported by my counsellor

I felt understood by my counsellor

I felt like I had a good connection with my counsellor

Reducing Distress and Isolation

O 2O 4O 6O 8O 1OO

% who chose “agree” or “strongly agree” (n= 235)

“He said to calm down and stop worrying so much and

that was helpful because I feel like I need to do that;

I need to be patient and live in the moment more and

Problem-Solving Skills

We wanted to assess the extent to which young people develop problem-solving skills as a result of speaking with a professional counsellor. As demonstrated in the chart below, 84% of callers stated that they spoke with their Kids Help Phone counsellor about options or strategies for dealing with their problems, 73% said they had a plan for what to do following their call, and 68% stated that their counsellors helped them become more aware of their personal strengths and abilities.

Referrals

In the past, Kids Help Phone created, and it now maintains, Canada’s largest database of community-based resources for children, teens, and young adults, containing more than 46,000 local programs and services across Canada (such as shelters, food banks, mental health services, and child protection agencies). Our phone evaluation showed that 48% of callers were provided with a referral for further support in their community: 22% were referred to a local adult in their lives (parent, coach, teacher, etc…), 17% were referred to a local agency, and 9% of callers were referred to both an adult and an agency. While not all callers receive a referral, we know that following up on a referral can be an important and actionable next step for a client. Because of this, we were pleased to find that between 70 and 76% of those who were provided with a referral stated they planned to follow up on it.

“Being able to talk to you guys

was helpful and the counsellor

saying if I can’t find shelter,

just call back… I thought that

was helpful.”

“The one thing he said

that stood out was to

write a note. It stuck out

for me. You hear about

it in movies and stuff,

but it was cool that he

said to actually do it.”

Increase in problem-solving skills among callers to Kids Help Phone

68%

73%

84%

Are more aware of

their own strengths

Have a plan to deal

with their situation

Talked about options or strategies

for dealing with their situation

- client / evaluation participant

Client Satisfaction

The evaluation proved that the phone counselling service has a positive impact on young people in other ways, but we also wanted to understand whether they themselves were satisfied with their counselling experience.

First, we wanted to identify what kids and youth

to have unrealistic expectations of counselling in general, or, more specifically, of our service, (e.g. for a counsellor to simply “fix” their problems). Our research assistants began by asking callers, “What did you hope would happen as a result of talking with a Kids Help Phone counsellor today?” and then read out a list of possible hopes. As the chart above demonstrates, 90% of callers said they hoped they would have a chance to talk out their issues, while 89% stated they wanted to feel better, feel listened

Responses to the follow-up question about whether or not their Kids Help Phone counsellor provided the help they had hoped for were overwhelmingly positive, with 91% of callers indicating this was the case. Finally, to further assess caller satisfaction, young people were asked, “Would you call Kids Help Phone again if you needed help?” Ninety-six percent of callers stated they would call the service again.

91%

96%

of callers said their Kids

Help Phone counsellor

provided the help they

had hoped for.

of callers said they

would call Kids Help

Phone again.

What did you hope would happen as a result of talking with a Kids

Help Phone counsellor today? Did you hope that: (multiple choice)

28

65

79

81

89

89

9O

Get connected with a service in your community The counsellor would tell you what to do Feel less alone Get information that would help you Figure out what to do next Feel listened to and like someone understands Feel better Have a chance to talk out your issue

66

Conclusion

Our phone counselling evaluation revealed many important things about the young people who reach out to this service for help, including what they value about it and how it makes a difference in their lives.

Kids Help Phone’s phone

counselling is effective

Based on our clinical framework and our single-session, on-demand service model, we anticipated that the youth who called our counselling service would experience changes in feelings and attitudes, followed by changes in awareness and knowledge. To determine if our hypotheses were correct, we chose to evaluate changes in levels of distress and isolation, changes in levels of awareness around personal strengths and abilities, as well as any increases in problem-solving skills.

The results of the evaluation were clear: Kids Help Phone’s phone counselling service is effective. Respondents reported statistically significant decreases in levels of distress and isolation, increases in awareness of their own personal strengths and abilities, and increases in problem-solving skills. They also expressed high levels of satisfaction with the service, with 91% of respondents reporting that their counsellors helped them in the ways that they hoped they would.

Kids Help Phone is reaching

kids who may need help most

We also discovered that Kids Help Phone is reaching some of the most historically under-served and potentially vulnerable youth in Canada, including young people who are Aboriginal and/or LGBTQ, and young people who are dealing with mental health challenges (41% of callers said they have accessed, or were currently accessing, other mental health services or supports, and 61% had received phone counselling from Kids Help Phone before).

Given these results, we can say confidently that we are providing an essential and effective service to children and youth in Canada, and comprise an important part of the mental health continuum of care.

Kids Help Phone is constantly

growing and improving itself,

becoming an inspiration and a

model for child helplines around

the world. The two studies

produced by Kids Help Phone

have shown the importance of

conducting regular evaluations

on the services offered by child

helplines and the impact that they

have on the lives of children.”

ExecutiveDirector ChildHelplineInternational

O 2O 4O 6O 8O 1OO

“

“He listened to my problems,

didn’t interrupt me, gave me

really good advice, told me

not to run away, told me to

call child support or go to my

friend’s house.”

Live Chat Counselling Evaluation

“My teacher said that I

should since I’m to shy to

talk to in person”

-client/evaluationparticipant

“I can’t talk on the phone,

and I have no trusted adults

or relatives.”

-client/evaluationparticipant

Over the years, young people have become increasingly insistent that they would like to access Kids Help Phone’s professional counselling over the Internet in “real time.” In 2010, we began researching the potential to add a new platform for them to access our counselling in a way that met this need. During this research phase, we considered instant messaging, email, texting, chat rooms and even social networking sites as possible platforms. Many criteria were used to weigh each option including:

What young people want: A 2010 survey of Kids Help Phone website clients found that 71% of respondents wanted Kids Help Phone to provide some form of instant-message/chat counselling.

Current and emerging practices in the international helpline community: Other child helplines around the world8 _{were increasingly} offering counselling through an instant messaging/ chat service medium.

Expected clinical utility: We looked at the extent to which a new service medium could support quality counselling.

Anonymity and confidentiality: We wanted to ensure that any new service modality would align with the organization’s service promise of anonymity and confidentiality.

Logistics of service delivery: We needed to make sure we understood what wait times and hours of service we could reasonably deliver with each option.

Live Chat Counselling Evaluation

“ i dont know how to express

my self on the phone”

-client/evaluationparticipant

“She helped me come up

with a strategy to deal

with my problem.”

-client/evaluationparticipant

Kids Help Phone is to be commended for its efforts to ensure the

continuous improvement of the relevance and accessibility of

its services. To that end, these evaluations provide them with a

better understanding of the diversity of the population they work

with. We know that access to mental health services is particularly

complicated for certain groups, such as ethno-culturally diverse

youth, due to language, stigma and lack of awareness and availability

of appropriate resources. The reflections on the use of different

counselling media will be of interest to anyone who conducts

telephone or online counselling with young people.”

After significant research, and with the early support of enabling donors, Kids Help Phone was able to pilot an instant message/Live Chat service, recognizing it had the potential for significant therapeutic value. We chose the service to address young people’s new preferred means of communication, allowing them to exchange one-on-one messages in real time with a counsellor from either a computer or smartphone. We also felt that a chat service could be developed in a manner that would not compromise a service user’s anonymity and confidentiality.

Before fully launching Live Chat, Kids Help Phone decided to pilot and evaluate the service to ensure it complemented our existing services and, most importantly, that it made a significant difference to young people’s mental health and well-being. The evaluation of the Live Chat pilot included both process and outcome components. In particular, we focused on client demographic and issue data, along with assessing client expectations of the new service and changes in clinical mental-health indicators. We evaluated against three expected outcomes: young people experiencing reduced distress, increased clarity about what to do next, and increased confidence that they could cope with or change their situation.

Mónica Ruiz-Casares, Ph.D. AssistantProfessor/ProfesseureAdjointe DivisionofSocialandTransculturalPsychiatry McGillUniversityCenterforResearchonChildrenandFamilies McGillUniversityTransculturalResearchandInterventionTeam

“

Methodology

Data was collected through a pre/post-test research design. This means that Live Chat clients were presented with a series of questions before chatting with a counsellor (pre-test) and given another questionnaire after ending the chat (post-test). Counsellors also attached the survey invitation link in the text of their final chat counselling reply.

Kids Help Phone’s research team, in

consultation with external evaluation experts, developed the survey in-house. Young people also reviewed the survey tool in multiple formal and informal usability tests. Our final survey contained nine pre-chat questions and 15 post-chat questions, for a total of 24 items. The first four basic information questions were mandatory (gender, age, province and community type) and the remaining 20 were optional. A series of scaling questions were used to help quantify change in the areas of distress, clarity about next steps, and confidence in ability to navigate a concern – areas in which our clinical approach is positioned to effect positive change.

Sample Size

Over the two-month evaluation period, the service was open between the hours of 6 and 11 pm EST, Thursday through Monday nights

chat counselling. Of these, 347 young people answered both the pre- and post-test survey (including 70 French-language respondents, 20% of our sample). Our sample represented 45% of all chat counselling contacts provided in June and July of 2012, which is a very strong sample size for a study of this nature.

Results

Who Uses the Service?

Demographics

While the demographic data collected during the Live Chat pilot is interesting, it is important to note that there was no marketing of the Live Chat service to youth across Canada during the pilot period (a decision made intentionally, given the service’s limited hours of availability and our tight resource restrictions on the number of counsellors who could be dedicated to this modality). It is also important to note that during the Live Chat service delivery time, our 24/7-phone service remained open.

Due to the absence of marketing, the demographic results cannot be considered to represent the diversity of young people in Canada who would, potentially, be interested in chatting with a Kids Help Phone counsellor. In particular, we anticipate that certain populations of youth

53%

of young people said

they had contacted

Kids Help Phone

because they could

not talk to anyone

likely to use Live Chat to access counselling, once additional funding of the service makes it possible for it to be expanded and promoted widely.

GeographicLocation

Fifty percent of youth who used the Live Chat service (i.e., “chatters”) over the course of the evaluation were from Ontario – compared with their 40% weighting within Ontario’s share of the population of Canadian youth. This was followed by young people from Quebec (22%), Alberta (9%), and British Columbia (6%), which compares to 22% (Quebec), 11% (Alberta), and 13% (British Columbia) in terms of these provinces’ proportions of Canadian youth, respectively. While the majority of respondents stated they lived in a city or large town (67%), 32% lived in rural areas or small towns. Gender

Eighty-nine percent of young people in the sample were female and 11% were male. Only one of the Live Chat survey respondents self-identified as transgendered.

As stated earlier, Kids Help Phone has not yet done any marketing or outreach to increase awareness of Live Chat among particular populations. In general, boys tend to access formal help services much less often than girls. Research shows that this is, in part, related to boys believing that showing feelings is embarrassing or that displaying emotions could lead to vulnerability and a lack of control. There is, however, some research that demonstrates that electronic communication is an effective way of decreasing the stigma boys attach to help-seeking behaviours and reducing their potential risk of embarrassment and rejection by peers, since they are

less likely to be seen accessing a support service.9 Through Kids Help Phone’s promise of anonymity and confidentiality, and by combining the electronic counselling option with a targeted marketing campaign directed at boys, we hope to increase the proportion of young men accessing our counselling service. In fact, reaching more boys is a strategic priority for the organization.

Age

The bulk of our respondents (40%) said they were 14 or 15 years old, followed by 17 year olds (13%), and 13 year olds (12%).

Reasons for Using Live Chat

Before beginning the pilot Live Chat counselling service, Kids Help Phone staff reviewed existing literature and had in-depth conversations and in-person visits with staff at other child helplines around the world where instant message counselling was offered. Through this research, we predicted that Live Chat was going to attract a large proportion of young people with serious mental health challenges, including those dealing with suicidal thoughts, self-harm and depression. To determine if our predictions were correct and to gain insight into why young people contacted Kids Help Phone via Live Chat, we asked chatters

“What issue or issues did you contact us about today?” This question, which was on the post-survey, included a list of concerns. Young people could choose as many reasons as they felt applied from the list. Most respondents indicated that they brought

“Faster and more

private than calling”

“because i needed to let

it out and try and get

more info”

“My parents suggested it,

so I can have more ideas

about what to do.”

“I come here when I feel

I can’t talk to my mom

and in this case she is

not ok with my sexual

orientation”

-clients/evaluationparticipants

9_{HutchinsonandStJohn(2012).Creatingconnections:Strategiestoimproveadolescentboys’accessandengagementwithschool-basedhealthservices.} ContemporaryNurse:Vol.40,AdvancesinContemporaryComplexHealthCare:NursingInterventions,pp.258-268.

more than one issue to Kids Help Phone, which makes sense given the complexity of young people’s lives. In all, the 317 respondents who answered this question selected 723 issues. In all, 41% of young people using Live Chat identified mental health as one of their reasons for accessing support, followed by family relationships (35%), and friend/peer relationships (32%). Nineteen percent of youth stated that they were

contacting the service to discuss self-harm, while 18% indicated that thoughts of suicide were one of their reasons for contacting

could feel comfortable reaching out for help in the face of feeling quite vulnerable.

It is interesting to note that the serious nature of many Live Chat counselling sessions presents unique challenges given the slower pace of text-based communication and its limited potential for revealing a client’s emotional cues. This can make it difficult for counsellors to stay connected to witnessing-related counselling work (e.g., staying with empathetic listening rather than moving to solution-finding work). The serious nature of the

“its easier to talk

on chat than on

the phone”

“I needed quick help,

as my family was

still fighting.”

Reason for chatting (multiple answers accepted)

Substance use or addictions Information about laws or rights Sexual orientation or gender Becoming independent Bullying Physical or sexual health School Violence or abuse Suicide Self-injury Dating / relationships Friend / peer relationships Family relationships Mental or emotional health

8

1O

1O

19

26

27

43

48

58

6O

73

1OO

11O

131

O 2O 4O 6O 8O 1OO 12O 14O

of Live Chat clients said

one of their reasons

for contacting Kids

Help Phone related to a

mental health concern.

“This service is excellent and

it is a lot safer than calling

because there when you do

this you wont get caught but

you might when you call.”

“kids help phone for me is

a place to just chat with

someone about your problem

with someone trusty i love kids

help phone and my counsler”

Why Live Chat?

As Live Chat was a pilot service, we wanted to understand why young people chose it instead of calling our helpline. This was particularly interesting because Live Chat has limited hours and young people can experience long wait times before being connected to a counsellor, indicating that those who are making the effort to use the service are particularly motivated to do so. When we asked them why they chose chat instead of using the phone service, 75% of respondents indicated they felt too nervous or uncomfortable to call the phone service, and 62% said they did not have enough privacy to speak on the phone. In addition, 42% said they would rather write

than speak about their problems. This information reinforces the critical importance of offering online counselling to youth to reach those who would not reach out for counselling any other way.

Use of Other Supports

As with the phone evaluation, this evaluation demonstrated Kids Help Phone’s unique role as a stand-alone mental health service and a critical part of the mental health continuum of care. Live Chat provides counselling and support to young people in the evening when other services are closed and when kids may not know where else to turn or have any other option available to

How many times have you contacted

Kids Help Phone? (n=311)

1O%

8%

8%

39%

35%

Frequency of Service Use

As with our phone evaluation, the largest proportion of clients in the Live Chat pilot evaluation said it was the first time they had contacted Kids Help Phone (39%). This makes sense, given that our mission is to provide single-session counselling. We know, however, that many young people use the counselling service more than once to work through a single challenge, or as they grow up to discuss different issues. In fact, 61% said they had contacted Kids Help Phone before – 35% had contacted us between two and five times previously, 10% between six and ten times, 8% between 11 and 20 times, and 8% had contacted us more than 20 times in the past.

them. In fact, when asked why they chose to chat with Kids Help Phone rather than talk to someone or visit a service in their community, 34% of respondents said they wanted help right away and 18% said they were not aware of any local supports where they could access help. Fifty-three percent said that they contacted Kids Help Phone because they could not talk to anyone around them and 16% said they chose Live Chat because they wanted to practice talking about their issue before raising their concerns with someone they knew.

However, by far, the number one reason young people chose to chat with Kids Help Phone rather

Kids Help Phone. It is interesting to note that all of Kids Help Phone’s counselling services are anonymous and confidential, but this promise appears to be particularly valued by users of Live Chat, who may feel the text-based nature of this modality somehow offers a better guarantee of this promise.

While Kids Help Phone provides single-session counselling to young people at any hour of the day or night, we recognize that many kids will benefit from being connected to ongoing services within their communities. For this reason, we were not surprised to find that 24% of Live Chat clients said they had received a referral to a local service

75%

of young people chose

to contact Kids Help

Phone on Live Chat

because of our promise

of anonymity.

“I can’t really talk to

anyone because they

don’t know the problem

and I’m scared about

how their gunna react”

Impact

Kids Help Phone chose to evaluate the impact of Live Chat during its pilot stage to ensure it was an effective method of providing young people with counselling. It was particularly important to evaluate Live Chat, since there is a dearth of research on the use of this new modality for short-term therapeutic benefit in a helpline setting.

For the evaluation, we focused on measuring the impact of Live Chat counselling in three important clinical areas: young people’s feelings of distress, their clarity about what to do about their situation or concern, and their confidence that they could cope with or change their situation. We anticipated that Live Chat would be most likely to change feelings and attitudes (decreasing distress levels) followed by developing skills (increasing clarity and confidence).

Distress

To help us understand if young people’s distress levels were reduced from the start of a chat with a counsellor to the end, we asked a seven-point scaling question (“on a scale of one to seven, how do you feel right now?”) before a young person chatted with their counsellor, and then we asked the same question after they finished. We were then able to compare young people’s answers on these two questions to determine if chatting with one of Kids Help Phone’s professional counsellors helped reduce their levels of distress.

Kids Help Phone is reaching out to the many children who remain hidden from the mental health system. It’s proven

to be a remarkably effective approach that is using technology in positive ways to benefit our children. This is the

future of intervention, especially for children who are socially withdrawn, anxious, or simply lacking the certainty

that they will be believed when they tell adults what they’ve experienced and are feeling. The anonymity of the

contact makes Kids Help Phone an ideal intervention to prevent more serious mental health problems, stop a child’s

exposure to violence, or simply reassure them that their thoughts, feelings and behaviours are normal.”

“The counsellor was really nice,

and I’m glad I had someone to

talk to. It made things seem

a little less overwhelming and

intimidating.”

“She helped me from committing

suicide. KHP is amazing.”

“It’s hard to talk face-to-face,

running the risk of crying... etc.”

“

As the chart below demonstrates, young people’s distress levels before their counselling sessions were extraordinarily high, with 18% choosing a distress level of one, the most distressed a person can be, and 23% choosing a distress level of two. These very high distress levels make sense when we consider the very difficult issues (including suicide, self-harm,