Policy Implementation, Feedback and Evaluation

Of particular interest, especially from a policy analysis perspective, are the concepts of policy implementation, feedback and future evaluation. The evidence in our studies is not conclusive enough to suggest that all school boards in Ontario have implemented a concussion policy. While it's true that they all have an electronically accessible policy on their boards' websites, it is not clear whether or not they have actually been implemented and are in fact, active and working. Unfortunately, very little research exists on the topic of concussion policy implementation with most U.S. studies focusing on evaluating the RTP aspect of the laws [163]. They center heavily on the impact and outcome of the policies and don't devote much time on the aspects of concussion education, training and prevention. The CDC even refers to them as RTP Laws; designed to reduce the impact of concussion, not prevent them [174]. Contrarily, the school laws in Ontario do not just focus on RTP but involve multiple other components, at least on paper. Evidence from the physical education teacher's responses from Study 3 suggest that while some boards have implemented a clear, multi-component concussion policy at their schools, others didn't go farther than an information binder, and/or some pamphlets.

It is quite possible that the lack of efficient implementation of the concussion policies at the school level have affected the observed difference in suspected vs. diagnosed concussions at

105

the CHIRPP hospitals. The boards that have a viable concussion policy in place, have trained (or are in the process of training) their staff and have introduced concussion awareness seminars, presentations and training sessions. These measures have contributed to an improvement in concussion diagnosis by allowing the supervising staff (Health and Physical Education teachers/coaches) to better differentiate amongst concussions thus improving overall identification of the injury. This could also partially explain the spike in diagnosed concussions witnessed at the school level (Chapter 3, Figure 3), especially taking into account some of the responses given by Health and Physical Education teachers about their recent training and/or participation in concussion awareness/information seminars. However, in a couple of cases the teachers have reported that they did not have a concussion training session at their school and were instead given an info pamphlet to "familiarize" themselves with new concussion information. Others said they only had taken a mandatory CPR - First-Aid course that had a concussion component included in the lessons. The primary researcher hypothesizes based on some of the responses that the teachers were not asked for input or weren't consulted in any other way about their board's policy and in some cases, were even unsure of the exact month that the policy was implemented.

This could all be tied to a major flaw in the PPM that most likely also extends to the school board policies. Particularly, the lack of feedback. There is limited opportunity for policy feedback included in the PPM itself [141]. There is support for the boards on a web portal that was established by the ministry, along with a ServiceOntario Infoline contact number, but it is not clear how one can leave feedback or comment on the issue at hand [175]. Despite the fact that it is NOT law, the PPM positions itself as one, indicating that all school boards must report to the Ministry of Education as per paragraph 27.1 of subsection 8(1) of the Education Act [141].

106

The boards have to, in theory, report the implementation of the policy and respond to any board inquiries about meeting the PPM-outlined concussion policy expectations. The actual "Implementation" section of the PPM is 2 sentences long; one highlighting the PPM deadline (Jan. 30, 2015) and the second stating: "School boards should ensure that a process is in place to

support ongoing implementation and compliance with the board policy at the school level"

[141]. There is no mention of what or how this implementation is going to look like, nor how it will be evaluated, measured or even used. Does having one general concussion lecture and placing concussion posters all over the school count as policy implementation? And if not, where do you draw the line, and most importantly how do you know that it is working or having any effect at all? It is important to understand, especially after taking into account the multiple failed policy attempts discussed in Chapter 1.7, that feedback is essential when creating public policy of any kind and that it can serve to both improve the attitudes about an issue and can enhance or contribute to future policy making, in our case, improving the future injury outlook and helping create better and more effective province- or nation-wide concussion prevention and management policies.

A useful approach to implement in this case could be the policy feedback effect first described by Andrea Campbell in 2011 [176]. The approach stipulates that the policy-specific design influences the attitudes and political behaviors of both target populations and other members of the public and policy-generated attitudes and behaviors can feed back into the political system, shaping the political environment and the possibilities for future policy making [176]. But the PPM and the way it is worded leaves little room for possible feedback and instead outlines a vague course of action to, hopefully, reduce the number of concussion in the school system. It is important to note however, that there are tools on the Ministry website that provide

107

reliable, evidence-based information on preventing, identifying, and managing concussions to parents, children and youth, educators, coaches, athletes, and health care providers [141]. The tools are there to be used, it is just not clear whether they are being implemented in an effective way. This could be described as the so-called "partial solution" whereby the response to address a problem undermines the issue in general and then the focus or interest shifts to other, possibly more serious policy needs [176]. Public opinion, and in our concussion policy discussion case, the opinion of the teachers and to some extent, students and parents need to be somehow incorporated into the concussion policy to create an environment that is conducive to further policy developments in the desired direction (i.e. concussion prevention). These first school board concussion policies should be considered no more than pilot projects, designed to pave the way for future, larger-scale laws and policies.