In conclusion this research study came from the initial position of how an action research approach could develop an impact evaluation tool to support managers in developing their bands 1-4 staff. It was carried out in line with a project identified by the WBEF network to enable both qualitative and quantitative evidence to be gathered to meet the demands for information requested by stakeholders. It also offers the opportunity for managers who have introduced new roles, evaluate the holistic effectiveness and influence of that role in their own service. Considering evidence from research articles reviewed regarding the evaluation of APs, it was found that the findings were predominantly qualitative in nature. This confirmed the hypothesis of the action research group that there was a lack of quantifiable data in relation to the cost benefits of the AP role and its direct effects on patient
throughput. Information received from an initial market research questionnaire also provided an indication of what priorities stakeholders had when considering the
development of new roles in their service. There was very little evidence when considering the literature review that any tangible tool had been developed. The action research team concluded that the development of a tool that would holistically evaluate the impact of the AP role, would be a useful resource for stakeholder to use in their service. Furthermore, it was anticipated with permission, their information could be incorporated into information sheets and shared with others.
The research was carried out in phases and anticipated future phases identified. Each phase was evaluated by the action research team to assist in the design of the final tool. The initial market research questionnaire did return results however these were limited and the population of the survey could have been more wide-reaching. It did provide a baseline to work from and assisted in identifying what needed to be incorporated into the impact evaluation tool.
The draft tool was piloted with a service manager and the results and methodology analysed. It was evident from the pilot that there were limitations to the tool. The pilot indicated that the tool needed redesigning, reformatting and be more explicit if it was to produce the data deemed most useful to stakeholders and potential future stakeholders. The results from the pilot had mostly captured qualitative data and not a mix as expected. Some of the data gathered was unusable as the service
manager had used the internet as a source of evidence which meant it did not relate to her service. Time restraints was also highlighted as an issue and whether it would be realistic to expect busy managers to give the attention the tool needed. An interview with the WBEF who carried out the follow up interview with the service manager, also raised issues around motivation of the participants and highlighted that the service area completing the impact evaluation tool, must have an intrinsic interest in the process and results. Comments on terminology were also considered as was the necessity to conduct an interview with the participant. The action
research group also considered further adaptation to the impact evaluation tool. The redesigned tool was then scrutinised by a small expert focus group with vast
experience of supporting APs before it was finalised. The issue of ensuring high motivation of the participants was addressed by the creation of a flyer which highlighted the expected time scale for completion and the benefits of conducting the impact evaluation itself. The flyer will ensure that only managers who wished to participate in the process would come forward in the first instance. For logistical reasons the impact evaluation tool has to date only been piloted with one
stakeholder. It is now necessary to work with other organisations, using the newly designed tool to compare results. The finalised tool has been developed and is now ready to be used not only to evaluate the AP role but any new role that has been developed.
The Aims of the project were twofold:
1. To provide managers with evidence based resources that can inform
their decision making when contemplating the training and development of non-registered staff into the role of AP.
2. To develop a tool that will evaluate the impact of the AP role within a
service area.
The impact evaluation tool will add a different dimension to the information available and so add to the resources that stakeholders will have access to. As the impact evaluation tool requires participants to consider what evidence they have to support their claims, it will offer real life data that can then be used to consider future
decision making. Equally, stakeholders will be able to objectively evaluate the effectiveness of new roles they have introduced themselves, including potential cost efficiencies, effects on patients and impact on key performance indicators by using the tool. Although it is acknowledged that the tool has limitations and may not have wide appeal, it is concluded the aims of the research study have been achieved. The true potential of the finalised impact evaluation tool will only become evident with future use.
Recommendations
The researcher makes the following recommendations in the future use of the finalised impact evaluation tool: -
1. That the WBEF network identifies suitable areas that might be appropriate and are motivated to carry out an impact evaluation of new roles they have introduced into service. These should include both NHS and non NHS organisations and cover a variety of service setting, to establish the flexibility of the tool and its adaptability.
2. The members of the WBEF network are trained in the use of the impact evaluation tool and the purpose of the follow up semi-structured interview. 3. Stakeholders who currently have AP positions are encouraged to fully
4. The tool is used to evaluate new roles other than that of the AP. 5. The evidence gathered from carrying out the impact evaluation are
converted into case studies that highlight both qualitative and quantitative data which can be shared to a wider audience. The evidence can be used to evaluate whether the initiative has had a direct impact on services and so can be used to justify future investment into the roles.
6. The impact evaluation tool is used as a method of showcasing excellence and sharing good practice.
7. Data collected around the education or training programme is fed back to the education/training provider to influence the curriculum content
8. An abridged version of the impact evaluation tool is developed as an on line resource that stakeholders can fill out themselves in the future.
9. The action research continues to monitor the use and usefulness of the impact evaluation tool and reviews its effectiveness at regular intervals. 10. The impact evaluation tool evolves to continue to capture the evidence that
is useful to all stakeholders both current and potential. Final thoughts
The aim of this dissertation was ‘An action research project to develop a tool to evaluate the impact of the AP role in practice and inform service development within NHS and non NHS organisations. To this extent the aim has been recognised. The action research model has provided an effective way to investigate the research question and ultimately produced an impact evaluation tool that can be used by current service areas and potential service areas. It encourages services to focus on the evidence base and share good practice with other practitioners, whilst offering stakeholders an additional resource to evaluate their service.
Bibliography
Accredited qualifications. (2012). Qualifications and Credit Framework. [Online] Available from: http://www.accreditedqualifications.org.uk/qualifications-and-credit- framework-qcf.html. [Last accessed 30th April 2016].
Allen, K. McAleavy, J.M. and Wright, S. (2012). An evaluation of the role of the Assistant Practitioner in critical care. British Association of Critical Care Nurses. 18 (1), pp.14-23.
Bamberger, M. (2012). Introduction to mixed methods of impact evaluation. [Online] Available from: https://www.interaction.org/sites/default/files/Mixed%20Methods %20in%20Impact%20Evaluation%20(English).pdf. [Last accessed 14th March 2016].
Basit, T. N., Eardley, A. Borup, R., Shah, A. and Hughes, A. (2015). Higher education institutions and work-based learning in the UK: employer engagement within a tripartite relationship. Higher Education. 70 (6), pp.1003-1015.
Bell, J. (2006). Doing your research project. A guide for first-time researchers in education, health and social care. 4th ed. Berkshire: Open University Press.
Benson, L. and Smith, L (2006) Delivering The Workforce. Evaluation of the
introduction of Assistant Practitioners in seven sites in Greater Manchester second report May 2006.Manchester: Centre for Public Policy and Management University of Manchester.
Bonbright, D. (2012). Use of impact evaluation results. [Online] Available from: https://www.interaction.org/sites/default/files/Use%20of%20Impact%20Evaluation %20Results%20-%20ENGLISH.pdf. [Last accessed 14th March 2016].
Boud, D., Solomon, N. and Symes, C. (2001). New Practices for New Times. In: Boud, D. and Solomon, N.(eds.) (2003) Work-based learning: a new higher education? Buckinghamshire: Open University Press. 3-17.
Bowling, A. (2009). Research methods in health: Investigating health and health services. Maidenhead: Open University Press.
Brown, L., Hedgecock, L., Simm, C. and Swift, J. (2011). Advanced Paramedics deliver on the front line. Health Service Journal. 25 (None available). pp. 24 - 26.
Bungay, H., Jackson, J. and Lord, S. (2015). Exploring assistant practitioners’ views of their role and training. Nursing standards. 30 (30), pp.46-52.
Burke Johnson, R. and Onwuegbuzie, J. (2004) Mixed method research: A research paradigm whose time has come. Educational researcher, [Online] 33 (7), pp. 14-26. Available from: http://www.jstor.org/stable/3700093. [Last accessed 6th February 2016]
Cavendish, C. (2013). The Cavendish Review an independent review into Healthcare Assistants and Support Workers in the NHS and social care settings.
https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/ 236212/Cavendish_Review.pdf .[ Last accessed 5th April 2016].
Chapman, A. (n.d.). Pest market analysis tool [Online] Available from:
http://www.businessballs.com/pestanalysisfreetemplate.htm. Last accessed 30th November 2015.
Chivite-Matthews, N. and Thornton, P. (2011). Guidance on evaluating the impact of interventions on business. [Online] Available from:
https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/ 212318/11-1085-guidance-evaluating-interventions-on-business.pdf. [Last accessed 3rd April 2016].
CIPD. (2015). PEST analysis: Resource summary. [Online] Available from:
http://www.cipd.co.uk/hr-resources/factsheets/pestle-analysis.aspx . [Last accessed 15th March 2016].
Clifford, C. (1997). Qualitative research methodology in nursing and health care. London: Churchill Livingstone.
Cohen, L., Manion, L., and Morrison, K. (2013). Research Methods in Education. 7th ed. Oxon: Routledge.
Creswell, J.W. (2007). Qualitative inquiry and research design: Choosing among five approaches. 2nd ed. London: Sage.
Department of Health. (2010). Equity and excellence: Liberating the NHS.
[Online] Available from:
https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/ 213823/dh_117794.pdf. [Last accessed 11th March 2015].
Department of Health. (2013). Patients first and foremost the initial government response to the report of the Mid Staffordshire NHS Foundation Trust public inquiry.
[Online] Available from:
https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/ 170701/Patients_First_and_Foremost.pdf. [Last accessed 18th April 2016].
Duffy, B. (2006). The analysis of documentary evidence. In: Bell, J.(ed.) Doing your research project: A guide for first-time researchers in education, health and social sciences. 4th ed. Berkshire: Open University Press. 123-133.
Francis, R. (2013) Report of the Mid Staffordshire NHS Foundation Trust Public Inquiry: Executive Summary. London: The Stationery office
Garbarino, S. and Holland, J. (2009). Quantitative and qualitative methods in impact evaluation and measuring results. [Online] Available from:
http://www.gsdrc.org/docs/open/eirs4.pdf. [Last accessed 11th March 2016]. Gray, D. E. (2009). Doing research in the real world. London: Sage.
Health Education England. (2014). Widening participation, it matters: Our strategy and initial action plan. [Online] Available from:
https://www.hee.nhs.uk/sites/default/files/documents/WES_Widening-Participation- Strategy_Booklet.pdf. [Last accessed 2nd April 2016].
Health Education England. (2016). Building capacity to care and capability to treat a new team member for health and social care in England. [Online] Available from: https://hee.nhs.uk/sites/default/files/documents/Response%20to%20Nursing %20Associate%20consultation%2026%20May%202016.pdf. [Last accessed 5th July 2016].
Health research authority. (n.d.). Determine whether your study is research. [Online] Available from:
http://www.hra.nhs.uk/research-community/before-you-apply/determine-whether- your-study-is-research/. [Last accessed November 2015].
Health research authority. (n.d.). Is my study research? [Online] Available from: http://www.hra-decisiontools.org.uk/research/. [Last accessed 15th November 2015].
Heron, J. (1996). Co-operative inquiry: Research into the human condition. London: Sage
Hickson, M. (2008). Research handbook for health care professionals. Singapore: Blackwell Publishing.
Howat, C. and Lawrie, M. (2015). Sector insights: skills and performance challenges in the health and social care sector. [Online] Available from:
https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/ 430164/
Executive_Summary_Skills_and_performance_challenges_in_health_and_social_c are.pdf. [Last accessed 6th April2016].
Kessler, I., Heron, P., Dopson, S., Magee, H, Swain, D. and Askem, J. (2010). The Nature and consequences of support workers in a hospital setting.
[Online] Available from: http://www.netscc.ac.uk/hsdr/files/project/SDO_FR_08- 1619-155_V01.pdf. [Last accessed 5th January 2016].
Killgannon, H. and Mullens, C. (2008). What are assistant practitioners? British Journal of Healthcare Assistants 2 (10), pp.513-515.
Kirkpatrick, J. and Kayser Kirkpatrick, W. (2009). The Kirkpatrick four level: A fresh look after 50 years. [Online] Available from:
http://www.kirkpatrickpartners.com/Portals/0/Resources/Kirkpatrick%20Four %20Levels%20white%20paper.pdf. [Last accessed 16th May 2016].
Kirkpatrick, D.L. and Kirkpatrick, J.D. (2012). Evaluating training programs: the four levels. 3rd ed. San Francisco: Berrett-Koehler.
Kolb, D.A. (1984). Experiential learning: Experience as a source of learning and development. Jersey: Prentice Hall.
Leach, C. and Wilton, E. (2008). Evaluation of Assistant/Associate Practitioner roles across NHS South Central. [Online] Available from:
http://www.workforce.southcentral.nhs.uk/pdf/NESC_Evaluation_Report_Final_2009 0212.pdf. [Last accessed 9th April 2016].
Mackey H. and Nancarrow, S. (2005) The introduction and evaluation of an occupational therapy assistant practitioner. Australian Occupational Therapy Journal;52: (4), pp. 293-301.
Miller, L., Williams, J., Marvell, R., and Tassinari, A. (2015). Assistant Practitioners in the NHS in England. [Online] Available from:
http://www.skillsforhealth.org.uk/index.php?
option=com_mtree&task=att_download&link_id=175&cf_id=24. [Last accessed 11th March 2016].
Miller, L. (2011) The Role of Assistant Practitioners in the NHS. Factors affecting evolution and development of the role. Bristol: Skills for Health
Miller, L. (2013) Assistant Practitioners in the NHS: drivers, deployment, development. Bristol: Skills for Health
Miller, L., Williams, J. and Edwards, H. (2014). Assistant Practitioner roles in the Welsh Health Sector enhancing the potential for future development. [Online] Available from: https://www.myhealthskills.com/uploads/articles/files/Assistant %20Practitioners%20in%20Wales%202014(1)-1393511549.pdf. [Last accessed 12th April 2016].
Mortimer, D. (2016). Building capacity to care and capability to treat – a new team member for health and social care: Consultation - NHS Employers response. Leeds: NHS Employers
National Institute for Health Care and excellence. (2014). Safe staffing for nursing in adult inpatient wards in acute hospitals. [Online] Available from:
https://www.nice.org.uk/guidance/sg1/resources/safe-staffing-for-nursing-in-adult- inpatient-wards-in-acute-hospitals-61918998469. [Last accessed 13th April 2016]. NHS Education for Scotland. (2010). Healthcare Support Workers. [Online] Available from:
http://www.hcswtoolkit.nes.scot.nhs.uk/media/3752/hcsw_literaturereview.pdf. [Last accessed 1st April 2016].
NHS Employers. (2012). Evaluating an assistant practitioner project. [Online] Available from:
http://www.nhsemployers.org/your-workforce/retain-and-improve/standards-and- assurance/developing-your-support-workforce/assistant-practitioners/evaluating-an- assistant-practitioner-project#1. [Last accessed 11 May 2016].
NHS employers. (2012). Evaluating an assistant practitioner project. [Online] Available from:
http://www.nhsemployers.org/your-workforce/retain-and-improve/standards-and- assurance/developing-your-support-workforce/assistant-practitioners/evaluating-an- assistant-practitioner-project. [Last accessed 31st January 2016].
NHS employers. (2015). Assistant practitioners. [Online] Available from:
http://www.nhsemployers.org/your-workforce/retain-and-improve/standards-and- assurance/developing-your-support-workforce/assistant-practitioners. [Last accessed 31st January 2016].
NHS employers. (2015a). Talent for care and widening participation resources.
[Online] Available from: http://www.nhsemployers.org/news/2015/09/update-on- talent-for-care-and-widening-participation. [Last accessed 4th July 2016]. NHS employers. (2016). Our response to the nursing associate role consultation. [Online] Available from:
http://www.nhsemployers.org/news/2016/03/our-response-to-the-nursing-associate- role-consultation. [Last accessed 5th May 2016].
NHS England. (2014). Five year forward view. [Online] Available from: https://www.england.nhs.uk/wp-content/uploads/2014/10/5yfv-web.pdf. [Last accessed 30th March 2016].
NHS Wirral Research & Development Team. (2011). Fact sheet 6: How do I evaluate my project or service? [Online] Available from:
http://info.wirral.nhs.uk/document_uploads/evidence-factsheets/6Howevaluateprojec tservice.pdf. [Last accessed 31st January 2016].
Organisation for Economic Co-operation and Development. (n.d.). Outline of Principles of impact evaluation: Part 1 key concepts. [Online] Available from: http://www.oecd.org/dac/evaluation/dcdndep/37671602.pdf. [Last accessed 11th March 2016].
Perrin, B. (2012). Use of impact evaluation results. [Online] Available from: https://www.interaction.org/sites/default/files/Use%20of%20Impact%20Evaluation %20Results%20-%20ENGLISH.pdf. [Last accessed 14th March 2016].
Philips, S. (2012). Work-based learning in health and social care. British Journal of Nursing. 21 (5), pp.918-922.
Phillips, J.J. (2003). Return on investment in training and performance programs (Improving human performance). 2nd ed. London and New York: Routledge, Taylor and Francis group.
Polit, D.F. and Beck, C.T. (2006). Essentials of Nursing research: Methods, appraisal and utilization. 6th ed. Philadelphia: Lippincott, Williams and Wilkins. Powell, M., Brown, T. and Smith, J. (2016). Skill mix: Using the assistant practitioner to drive efficiency. Practice Nursing. 27 (1), pp.40-43.
Raelin, J. A. (2008). Work-Based Learning. [Online]. Jossey-Bass. Available from:<http://www.myilibrary.com?ID=121745> [Accessed 29 April 2016].
Richard, W. (2002) Work-based Learning in Health: Evaluating the experience of learners, community agencies and teachers, Teaching in Higher Education, 7 (1), 47-63
Rogers, P. J. (2012) Introduction to impact evaluation. [Online] Available from: https://www.interaction.org/document/introduction-impact-evaluation. [Last accessed 11th March 2016].
Royal College of Nursing. (2009) Policy Unit, Policy Briefing 06/2009. The Assistant Practitioner Role. A Policy Discussion Paper, Policy Briefing 06/2009. London: RCN Policy Unit.
Royal college of Nursing. (2010). Assistant practitioner scoping project. [Online] Available from:
https://www2.rcn.org.uk/__data/assets/pdf_file/0003/379155/003880.pdf. [Last accessed 12th May 2016].
Royal College of Nursing. (2016). Royal College of Nursing response to Health Education England’s consultation: Building capacity to care and capability to treat – a new team member for health and social care. [Online] Available from:
https://www.rcn.org.uk/professional-development/publications/pub-005567. [Last accessed 5th April 2016].
Seagraves, L., Osborne, M., Neal, P., Dockrell, R., Hartshorn, C., and Boyd,
A. (1996) Learning in Smaller Companies: Final Report: Stirling. Educational Policy
and Development University of Stirling.
Shaw, A. (2012). Scope of Practice of Assistant Practitioners. [Online] Available from: http://www.sor.org/learning/document-library/scope-practice-assistant- practitioners. [Last accessed 14th April 2016].
Sheehan, J. (1986). Aspects of research methodologies. Nursing education today. 6 (5), pp.193-203.
Skills for Health (2009). Nationally Transferable Skills. Bristol: Skills for Health. Skills for Health (2009a). Core Standards for Assistant Practitioners. Bristol: Skills for Health
Skills for Health. (2010). Key Elements of the Career Framework. [Online] Available from: http://www.skillsforhealth.org.uk/index.php?
option=com_mtree&task=att_download&link_id=163&cf_id=24. [Last accessed 20th April 2016].
Skills for Health. (2015) The Healthcare Support Workforce A case for ongoing development and investment.Working paper 1 [Online] Available from:
http://www.skillsforhealth.org.uk/index.php?
option=com_mtree&task=att_download&link_id=179&cf_id=24. [Last accessed 5th April 2016].
Skills for health. (2015a) New apprenticeship standards for healthcare support workers and assistant practitioners – Approved by BIS. [Online] Available from: http://www.skillsforhealth.org.uk/news/latest-news/item/209-new-apprenticeship- standards-for-healthcare-support-workers-and-assistant-practitioners-approved-by- bis. [Last accessed 5th July 16].
Skills for Health. (2016) How can we act now to create a high-quality support workforce in the UK's health sector? Pt 1/2. British Journal of Health Care Assistants. [Online] 10 (3). pp. 44-47. Available from:
http://www.magonlinelibrary.com/doi/10.12968/bjha.2016.10.3.134 [Last accessed 5th April 2016].
Skills for Health. (2016a). How we can act now to create a high quality support