sports coach UK has developed an M&E tool kit consisting of 14 tools that you may find useful at various stages of development within your coach education programmes. Each tool should be used individually with guidance and support from your CEA.
M&E 1: Monitoring and Evaluation Strategic Approach
This document provides an opportunity to outline the approach that your sport will take to the M&E of its UKCC coach education programmes for the period specified.
The document provides an opportunity to describe the approach being adopted (or to be adopted) by your sport to the M&E of the design and delivery of its
UKCC-endorsed coach education award programmes. It forms part of a set (a ‘toolbox’) of recommended procedures. Sports may select from the recommended procedures to suit their needs and circumstances, and this document provides the description and rationale for the M&E strategy adopted.
M&E 2: Programme Design Checklist
The aim of this tool is to provide an opportunity to evaluate some aspects of the design of UKCC-endorsed coach education programmes. Its purpose is to act as a vehicle to focus attention on elements of good practice and to demonstrate improvements in the design of
programmes. Given the potential range of questions that could be posed, the tool is somewhat selective and pragmatic.
This tool can be used to stimulate your thinking about the design of your coach education qualification programmes. It will also act as a position statement on existing programmes and can be used as a reference point or baseline for evaluating both current and amended programmes.
M&E 3: Programme Descriptor
An important part of the M&E of UKCC-endorsed programmes is whether courses are being delivered as intended. In order to make this judgement, it is necessary to have a programme template against which comparisons can be made. M&E 3 is a template that enables coach education managers to specify how courses should be delivered.
M&E 4: Analysis of Programme Barriers
This tool is based on the fact we need to understand the factors that influence whether or not your coach education is likely to be effective. We can begin with the assumption that a number of factors prevent your coach education programmes being delivered in the most effective way. It is important to identify these, and how you intend to reduce their impact.
M&E 5: Programme Data
The purpose of the tool is to summarise the scope and scale of UKCC-endorsed programme throughput. It will perhaps be taken for granted that there should be a record of coach education throughput, and this is a common feature of annual reports. This basic monitoring and the analysis of data that is possible depend on sound monitoring procedures and the functionality of the database or
recording systems.
M&E 6: Coach Learner Feedback
This is a very important form of feedback, partly for the information it provides on the delivery of the course, and partly to give the learners a sense of ownership. It may be appropriate to have a form of reward or sanction to encourage completion.
M&E 7: Course Tutor Feedback
Feedback from the course tutor(s) is essential; partly on issues of programme fidelity (being delivered as you had intended), but also as a record of the course delivery for cross-referencing to coach progression/achievement issues and for comparison to other forms of feedback.
M&E 8: Course Observation Template
The purpose of the observation tool is to provide an ‘external’ perspective on the delivery of the course. The context is programme fidelity (is it being delivered as intended?), but there is an emphasis on the quality of the course. The observations are centred on the quality of the learner experience.
Although the observer role is not the same as the verifier role, the same individual may carry out both roles. However, it would not be helpful to think of the observer as an ‘external inspector’; the purpose is to assure your sport that the courses are likely to achieve what they set out to, and to improve them where necessary. It would be helpful if observers were familiar with the coach education context. This implies some experience, perhaps as tutors/assessors or teachers/deliverers in another context. There should also be some familiarity and orientation to the content of the course.
M&E 9: Evaluation of Pre-course Data
This M&E tool is focused on analysing information gathered about learners. An assumption is made that some form of pre-course questionnaire or other form of data collection will be completed by learners who apply for a qualification programme. Our purpose here is not to create the questionnaire, although we do offer some guidance on the information that we recommend should be collected. The objective is to identify how this information should be used, and the M&E issues that should be addressed.
The collation of information about individuals should be part of the sport’s tracking of coaches, and permits some cross-referencing of data against future coach education and coaching practice. Much of the data may subsequently be used for reporting purposes.
In line with the approach taken to M&E, we offer the flexibility for you to decide on the information to be gathered, how it should be gathered, and which elements of analysis you will prioritise.
M&E 10: Coach Competence Monitoring Tool
It is necessary to have a monitoring tool that contributes to an evaluation of the effectiveness of UKCC-endorsed coach education programmes. (Is it having the intended effect on those who complete the qualification successfully?) This monitoring tool has been devised to assess changes to a coach’s expertise as a result of having undergone a specific period of coach education. It provides a profile of coaching behaviours that can be used at various points in the education process to produce a self-reported measure of coaching ‘functions’ or competences that provide a valuable pre- and post-course evaluation of expertise.
M&E 11: Outcomes Statement
It was recognised that a number of monitoring and evaluation tasks require a relatively concise and
easy-to-communicate statement of what might be expected of a coach who successfully completes a specific UKCC-endorsed award.
This competency framework should put into sport-specific language what you would expect someone who has successfully completed each level of UKCC-endorsed qualifications to be able to do.
The purpose of the tool is to:
• act as a catalyst to stimulate further reflection about your coach education programme
• provide a simple mechanism for communication • provide an opportunity to make coach competences
sport- and context-specific.
M&E 12: Analysis of Assessments
The assessment associated with UKCC-endorsed coach education delivery is an important source of evidence about the effectiveness of programme delivery.
This evaluation tool invites the coach education manager or evaluation team to carry out a secondary analysis of the assessment output, and to ask questions about the appropriateness of the assessment for making judgements about learners’ enhanced status (which really means achieving learning outcomes).
M&E 12 is not so much a ‘form’ but more of an ‘evaluation agenda’. It makes the assumption that the assessment data and products/outputs are available for analysis and interpretation.
M&E 13: Case Studies
Many M&E tools demonstrate trends, and for this reason, M&E 13 will attempt to:
• employ reasonably rigorous or standardised methods • collect data from a sufficiently comprehensive range of
sources or individuals to permit some generalisation; however, there is a need to delve deeper on occasions and seek to understand ‘why’ questions, or provide a more detailed illustration of issues.
M&E 14: Report Template
A commitment to a reporting schedule encourages M&E activity and provides some structure to the procedures being undertaken. This is an essential and key part of the M&E process. We think that it is very important there is a tangible output to which the M&E process can be directed. This ensures a degree of accountability and facilitates the identification of consequent action.
All M&E tools are available by contacting [email protected]
The process of endorsement sets out the criteria required by a coach education programme in order to obtain UKCC endorsement.
A governing body of sport’s coach education programme comprises a variety of components at a number of levels. Each level offers progression for a coach in a particular sport and reflects best practice and current educational and sport needs.
Components of coach education programmes at each level are:
• quality assurance – a qualification awarded by an organisation fulfilling an awarding function, which is responsible for ensuring the quality of the qualification over time
• learning programme – a programme of learning to guide those with responsibility for the delivery and assessment of the qualification, or a programme of training for the personnel responsible for delivering, assessing and quality assuring the competence of coaches
• learning resources – including technical information on coaching the sport, available in appropriate media, to ensure coaches have access to support materials • coach developer workforce – a programme of training
for the personnel responsible for delivering, assessing and quality assuring the competence of coaches
• continuous improvement – including a programme of CPD for coaches.
Governing bodies of sport will take lead responsibility for demonstrating their achievement of the UKCC