What Is Evaluation?

Outcome evaluation should not be confused with other project activities that also critically examine aspects of the programme, such as assessment, monitoring, reassessment or quality assurance. These activities, however, may serve a role in evaluation efforts and need discussion in their own right.

6.2.1 Assessment

Most professionals would agree that in order to develop a care plan, a structured assessment is necessary. Typically, its purpose is to identify the individual’s difficulties or problem areas, but it should also include identification of strengths and supports to that person. While programmes often aspire to undertaking a holistic view of the individual, the individuals administering them often resort to assessing those specific areas and factors that they themselves value most or are most competent at assessing.

150

Equally they may only assess the factors that are most amenable to change through their given profession at intervention. Standardising the assessment procedures and measures is of key importance in enabling comparisons across projects and/or health boards. Each project must similarly determine the characteristics of the older people that they are serving and their level of need. Without standardised assessments, project evaluation would be akin to comparing apples to oranges. For example, projects that targeted older people who were at much greater risk for hospitalisation or had higher levels of unmet needs might be unfairly evaluated against those projects that targeted a less needy group.

In the UK, the recent policy document A National Service Framework for Older People

(Department of Health, London, 2001) recommends a ‘single assessment process’. This document underlines the importance of having a standardised assessment process, which is of a high standard, and is tailored to their needs. A ‘fuller’ assessment (which assesses a number of domains to identify any unmet needs) may be required for some older people. This fuller assessment may identify an area of concern that requires a more detailed or specialist assessment by a qualified professional. Thus, although there is a single assessment process, there may be more than one tier of assessment. The single assessment process is designed to decrease duplication and provide a more seamless service.

6.2.2 Monitoring

Once a care plan is initiated, procedures must be put in place to ensure that the care plan is appropriately carried out. Monitoring is critical to the role of a Care Manager so that any changes in an older person’s status can be responded to quickly and

appropriately. The frequency with which monitoring occurs may largely depend on the scope or intensity of that person’s needs and the types of services provided to them. For example, monitoring may occur much more frequently after a discharge from hospital to ensure that all the community care services are in place and that the transition is smooth. Alternatively, once all key aspects of the care plan are functioning and the person’s situation becomes more stable, intensive monitoring is less necessary and may be much less frequent. Unlike assessment, which has as its primary focus the individual and their situation, monitoring needs to encompass not only the individual’s current status, but also the procedures and processes that make up that individual’s care plan. For example, if communication breaks down between services, a good monitoring system should be able to detect this difficulty before it unduly impacts on the individual’s quality of care. Because monitoring includes activities relevant to the process of care, its undertaking can also inform the process components of any evaluation efforts.

151

6.2.3 Reassessment

In theory, a good monitoring system should identify any new unmet needs or difficulties of an individual. However, the benefits and importance of a formal reassessment should not be underestimated. Reassessments can be scheduled to occur after a specified period of time, or initiated after some event which may indicate a change in an individual’s status or needs. Because of its regularity, or because it may initiate a change in the care plan, the reassessment period is often a good time to collect data for outcomes research.

6.2.4 Quality Assurance

Also referred to as quality control, quality assurance involves activities that occur after project evaluation. Once it has been established (through evaluation) that a project is achieving its goals and desired effects, the next task is to ensure that it continues to do so. Programmes that are initially successful can easily lose their effectiveness over a period of time because of the interdependent nature of the factors making up the solution to a complex problem. Factors such as changes in key personnel, changes in the characteristics of the older people being served and competing demands on

resources such as time and money to name a few, can have a substantial impact on the effectiveness of any programme. Quality assurance effectively means compliance to standards. In order to have any meaningful quality assurance, two components are necessary. First, the project must be operationally defined – what is Case Management and what is not? Second, the standards need to be developed – what exactly

constitutes a successful Care Management programme? Setting these programme standards can also be considered establishing ‘best practice’ (see below for further elaboration). Until the programme is defined and the standards set, it is not possible to assess the quality of its service (Applebaum, 1991). Combining the tasks of evaluation and quality assurance results in a comprehensive approach to ensuring high quality care.