Theory of change

The evaluation also tested the programme’s ‘Theory of Change’ (Judge and Bauld 2001).

This makes explicit the links between programme goals and the different contexts and ways in which the project works. It provides a framework for mapping subsequent

outcomes and outlining how these fit with the overall objectives of the Eatwell and Livewell project.

Figure 5.1 – Theory of Change for Eatwell and Livewell

Approach to gathering evidence

Evidence for the evaluation derived primarily from interviews with stakeholders (4 in total), referrers (2), volunteers (3), service users (3) and carers (2) as well as analysis of routinely collected monitoring data.

Qualitative data collection

Stakeholders: 4 semi-structured interviews were conducted with key stakeholders at two different points in time (March 2016 and then December 2016) to capture their views and experiences of involvement. An interview schedule was developed in line with the

objectives for the evaluation and broadly covered the following key areas: the project background, the approach adopted, the changes that had taken place as a result of the project, the impact of the project on service users, and any aspects of learning during the project delivery (see appendix 7.1 for the interview schedule).

Service Users/Carer: 3 interviews were conducted with service users and 2 interviews were conducted with a carer. The delivery partner advised the research team

of suitable participants to invite. Service users and their carers were given the opportunity to self-select to participate within the evaluation by the

• Project aims Eatwell and Livewell -aiming to work with older people and their carers in relation to improving nutrition

• Mechanism for change - community support: assessment, tailored advice, monitoring and the establishment of volunteer relationships

• Intermediate outcomes

• New referral pathways

• Improved nutrition

• Improved quality of life (health-related and more broadly) for service users and carers

• Long term Outcomes

• Added value to mainstream provision

• The establishment of a more effective use of health resources

• Improvements in the lives of volunteers

project worker. The semi-structured interview schedule was designed in line with the objectives of the evaluation (see appendix 7.2 for the schedule).

Volunteers: 3 volunteers were interviewed from the Doncaster site in March 2016 to capture their experiences of the project and its impact upon them. The schedule was once again designed in line with the objectives of the evaluation (see appendix 7.3).

Data analysis

The verbatim transcripts from all of the interviews, along with the accompanying notes, were analysed using Framework Analysis. Framework Analysis develops a hierarchical thematic framework to classify and organise data according to key themes, concepts and emergent categories. The framework is the analytic tool that identifies key themes as a matrix where patterns and connections emerge across the data (Ritchie et al., 2003). The matrix was constructed using the aims of the evaluation. Themes were agreed by

members of the research team. The monitoring data was analysed using descriptive statistics and is presented in charts to illustrate key points.

Desk-based data

The aim of the desk-based analysis was to provide a rigorous synthesis of monitoring data collected by the delivery partners. The primary data sources were demographic data collected from participants at both delivery sites, quality of life data from the Doncaster site and 20 case studies from the Calderdale/Kirklees site. There were only 19 clients, but the case studies included the perspective of a carer. The desk-based analysis was also intended to ascertain if there had been any changes in health service usage in terms of reduced uptake of services and therefore decreased demand and associated costs. This was not possible as access to health service user data was not provided.

Research ethics

The evaluation was given ethical approval through Leeds Beckett University ethics procedures. The following practices were adhered to ensure ethical rigour:

 Informed consent – written or verbal consent was obtained from all participants in the interviews

 Confidentiality and anonymity – no personal identifying information has been used in the reporting the data

 Secure information management – security was maintained through password protected university systems

Limitations of the Evaluation

The evaluation has sought to identify and bring together a range of perspectives in order to highlight what has worked and what might be done differently. Nevertheless, in all evaluations there are limitations to what can be achieved, with the limitations in this instance listed below;

 The evaluation team were reliant on regional staff and delivery site partners to access clients and in-house data relating to the project. Staff changes, reductions in staff hours at the regional level and delivery issues limited the time and attention that was afforded to the evaluation. The evaluation team were also conscious of overburdening staff with requests for information when delivery needed to be done.

 The evaluation team were not always cognisant of the range of activities that were being undertaken within the delivery sites despite repeated contacts seeking

updates and further information during the lifetime of the evaluation. In addition, the evaluation team were not able to access one delivery site until after the end of the project limiting the scope of the evaluation, with contact details withheld at the regional level due to staffing-related issues.

 Both delivery partners only worked with a small number of participants hence there is a limited data set. The data would have been richer had the evaluation team been able to talk to more service users as well as being able to compare client experiences across both sites. However, sensitivity was needed in terms of where some of the older people were at on their journeys and their willingness/availability to discuss their experiences.

 The evaluation team were supplied with in-house monitoring data but this too had limitations. For example, the delivery teams were expected to use tools to measure changes in service user’s self-related quality of life. There is a partial data set related to this from one site however, the collection of this data was suspended by the worker when service users became distressed and upset when asked to report upon this.

 The evaluation team were not supplied with NHS numbers for all clients from either delivery partner. Whilst we had NHS numbers from some of the clients in

Doncaster, data sharing and information governance requirements were not

clarified therefore we were unable to access health service usage data and cannot comment upon any potential reductions in health service usage.

 There was no follow-up after discharge from Eatwell and Livewell therefore it is not possible to judge whether any positive changes reported during the intervention were maintained.

 The lack of quantitative data at follow-up means it is not possible to comment on the potential social value or return on investment of this project. For future work,

consistent data should be collected at baseline and follow-up on outcomes such as quality of life, functioning, general health or wellbeing, as well as health

service use and costs, so that the potential social value of the intervention can be demonstrated.

6. References

Artaza-Artabe, I., Saez-Lopez, P., Sanchez-Hernandez, N., Fernandez-Gutierrez, N. &

Malafarina, V. (2016) ‘The relationship between nutrition and frailty: effects of protein intake, nutritional supplements, vitamin d and exercise on muscle metabolism in the elderly. A systematic review’ Maturitas 93, pp.88-99.

BAPEN (2003) ‘Malnutrition Universal Screening Tool’ available at http://www.bapen.org.uk/pdfs/must/must_full.pdf.

Beck, A.M., Beerman, T., Kjaer, S. & Rasmussen, H.H. (2003) ‘Ability of different screening tools to predict positive effect on nutritional intervention among the elderly in primary health care’ Nutrition 29, pp. 993-999.

Bokhorst-de van Schueren et al (2014) ‘Nutrition screening tools: does one size fit all? A systematic review of screening tools for the hospital setting’ Clinical Nutrition 33, pp. 39-58.

Brotherton, A., Simmonds, N. & Stroud, M. (2010) Malnutrition Matters. Meeting Quality Standards in Nutritional Care. A Toolkit for Commissioners and Providers in England Bapen.

Craven, D.L., Pelly, F.E., Lovell, G.P., Ferguson, M. & Isenring, E. (2016) ‘Malnutrition screening of older adults in the community setting: practices reported by Australian Dieticians’ Nutrition and Dietetics 73, pp. 383-388.

Dinsdale. P. (2006) ‘Malnutrition. The real eating problem’ Nursing Older People 18, 3, pp.

8-11.

Hernandez-Galiot, A. & Goni Dra, I. (2017) ‘Quality of life and risk of malnutrition in a home-dwelling population over 75 years old’ Nutrition 35, pp. 81-86.

Judge, K. & Bauld, L. (2001) Strong theory, flexible methods: evaluating complex community-based initiatives. Critical Public Health 11, pp.19-38.

Marshall, S., Agarwal, E., Young, A. & Isenring, E. (2017) ‘Role of domiciliary and family carers in individualised nutrition support for older adults living in the community’ Maturitas 98, pp. 20-29.

Pirlich, M. & Lochs, H. (2001) ‘Nutrition in the elderly’ Best Practice and Research Clinical Gastroenterology 15, 6, pp. 869-884.

Poulia, K.A., Yannakoulia, M., Karageorgou, D., Gamaletsou, M., Panagiotakos, D.B., Sipsas, N. & Zampelas, A. (2012) ‘Evaluation of the efficacy if six nutritional screening tools to predict malnutrition in the elderly’ Clinical Nutrition 31, pp. 378-385.

Ritchie, J., Spencer, L. & O’Connor, W. (2003) ‘Carrying out qualitative analysis’ in Ritchie, J. & Lewis, J. (Eds) (2003) Qualitative research practice: A Guide for Social Scientist Students and Researchers. Pp. 219-262. London: Sage.

Wilson, L. (2010) Personalisation, nutrition and the role of community meals ILC-UK Wilson, L. (2013). ‘A review and summary of the impact of malnutrition in older people and the reported costs and benefits of interventions’ Malnutrition Task Force.

7. Appendices