Authors: Derick T. Wade Affiliations:
From the Department of Neurological Disability, Oxford Centre for
Enablement, Oxford, UK.
FIM™ is a trademark of the Uniform Data System for Medical
Rehabilitation, a division of UB Foundation Activities, Inc. Correspondence:
All correspondence should be addressed to Derick T. Wade, Department of Neurological Disability, Oxford Centre for Enablement, Windmill Road, Oxford OX3 7LD, UK.
0894-9115/03/8210(Suppl)-0S26/0
American Journal of Physical Medicine & Rehabilitation
Copyright © 2003 by Lippincott Williams & Wilkins
DOI: 10.1097/01.PHM.0000086996.89383.A1
Outcome Measures for Clinical
Rehabilitation Trials
Impairment, Function, Quality of Life, or
Value?
ABSTRACT
Wade DT: Outcome measures for clinical rehabilitation trials: Impair-ment, function, quality of life, or value? Am J Phys Med Rehabil
2003;82(Suppl):S26 –S31.
Choosing outcome measures in rehabilitation research depends on the standard research skills of clear thinking, attention to detail, and mini-mizing the amount of data collected. In rehabilitation, outcome is more difficult to measure because (1) usually several outcomes are relevant, (2) relevant outcomes are affected by multiple factors in addition to treatment, and (3) even good measures rarely reflect the specific inter-est of any individual patient or member of the rehabilitation team, lead-ing to some dissent. Measurement of general quality of life is not pos-sible because there is little agreement as to the nature of the construct; moreover, measurement of relevant aspects of quality of life would probably give similar results. Cost in terms of resources can be esti-mated, but there is no validated or even widely accepted method of relating this to benefit in a fair, open, and rational way. Outcome is best measured at the level of behavior (activities), with other measures being used to aid interpretation.
Key Words: Rehabilitation, Rehabilitation Research, Outcome Measures
R
esearch should be undertaken to answer a question. The question asked will determine two aspects of the research: the most appropriate design and the most appropriate measure or measures to use. The logic of the design deter-mines the logical conclusions that can be validly drawn from the results. The measures used, along with sample size, determine the precision of theesti-Analysis
mates made. This article discusses measures that might be used in reha-bilitation research.
The article assumes that all read-ers are familiar with the basic frame-work put forward within the World Health Organization (WHO), both in its original (1980) International Classification of Impairments, Dis-abilities, and Handicapsversion1and the revised International Classifica-tion of FuncClassifica-tioning2 (ICF; see also Wade and Halligan3). The WHO ICF
terminology will be used in this article. This article is primarily concerned with evaluative research investigating the effectiveness of specific interventions in well designed studies, usually ran-domized controlled studies or single-case study designs. The principles apply in other situations.
Outcome: What Is It?. Although the term “outcome” is simple and its meaning may seem obvious, it is worth some consideration in the con-text of rehabilitation research. Out-come refers to the expected or looked-for change in some measure or state. In other words, a patient will enter a study or rehabilitation pro-gram in one state and may change as a result of the intervention. The new state constitutes the outcome. In this sense, clinical trials are primarily concerned with changes in those as-pects of quality of life (QOL) or
activ-ities that are due to or caused by the treatment more than QOL or activi-ties as a whole.
In rehabilitation, there are often many interventions given over time, sometimes months, and so the rele-vant outcome is not always obvious. For example, for a patient who has had a stroke and has moved into a rehabilitation unit in a hospital, the outcomes shown in Table 1 might constitute relevant outcomes, de-pending on who wants to know and why. Other outcomes that might be relevant include stress on family, ability to work or drive, and ability to communicate using the telephone.
It is important not to refer to the outcome but to be specific and refer to the outcome of a specified inter-vention measured in a specific way, reflecting the interests of specific groups/stakeholders, which might in-clude researchers or funding agen-cies, although patients should re-main the primary focus. It is also better to use the phrase “measure of outcome” rather that “outcome mea-sure.” This emphasizes that one is focusing on outcome. Of equal im-portance, it avoids the sloppy think-ing associated with such questions as “what outcome measures should I use at baseline?” The term outcome measure has become synonymous with measure in many situations.
What Is Different in Rehabilitation Research?.Clinical trials in rehabili-tation are no different in principle from any other clinical trials, but they differ in practice in several ways. First the outcomes of interest are dif-ferent, and indeed, there are often multiple outcomes of interest. Reha-bilitation usually explicitly aims to alter activities (i.e., behavior) or par-ticipation (i.e., role functioning in the community), whereas in other medical domains the primary aim may legitimately be to alter body function or structure (also known as pathology, disease, and impairment in the earlier WHO terminology). Thus, rehabilitation research uses primary measures that come from different domains and, more impor-tantly, will usually study more than one major outcome.
Next, rehabilitation studies usu-ally involve multifocal and sometime ill-defined interventions rather than the single well defined interventions typical of drug research trials. Core rehabilitative interventions—those in physical therapy, occupational therapy, speech-language pathology, and rehabilitation psychology, for in-stance—involve close interaction be-tween the therapist and the patient. These therapies work through the ac-tion or behavior, and the thoughts, of the patient, rather than being the
rel-TABLE 1.
Outcome varies depending on the intervention being evaluated
Intervention Outcome Comment
One 30-min session with an occupational therapist
Being able to put on socks without assistance
This relates to one single-patient-therapist interaction
Course of therapy in the physiotherapy department
Being able to walk about ward safely
This relates to an interaction between the patient and one department,
accumulated over time Inpatient treatment in stroke unit Scoring 18/20 or more on the
Barthel activities of daily living index at discharge
This relates to overall effect of the program but will not reflect much of any specific speech therapy input Full inpatient and outpatient program
with follow-on community rehabilitation
Able to live in own first-floor apartment alone, without depending on family, doing shopping, and cooking meals
This depends also on the network of rehabilitation services in the
community and the precise location of the apartment
atively mechanical and predictable ef-fects of drugs. This complexity may lead to some internal debates within the rehabilitation or research team about an appropriate outcome. For example, most stroke-unit trials used a simple measure of activities of daily living (such as the Barthel), which will not reflect much of the input from a speech therapist.
Moreover, rehabilitation studies will usually require time both to un-dertake the intervention and for the intervention’s effect to become ap-parent, whereas medical interven-tions are often brief and have rapid effects. This not only increases the cost of rehabilitation studies but also makes interpretation of outcome more difficult because many factors may have an influence on the ob-served outcome.
Finally, rehabilitation is often context specific, whereas medical in-terventions are usually independent of context.4The term context is used within the WHO ICF to refer to three aspects of a patient’s situation: the physical environment, the social en-vironment, and the patient’s own personality, including beliefs and at-titudes and experiences and expecta-tions. For example the rehabilitation of a patient left with reduced mobility after stroke will depend on, inter alia, the structure of their home (e.g., stairs, internal steps, ease of getting out), their wishes (e.g., they might be quite content to use a wheelchair), the other people in the house (who may or may not be willing and able to assist with toilet transfers), and the local rules concerning support from the State or other organizations. In other words, social, background, per-sonal motivations, and physical fac-tors in the environment will affect rehabilitation outcome and these contextual factors make it less easy to generalize the results of rehabilita-tion trials.
Nonetheless, any research design aims to minimize bias and to reduce error variance. Randomization is
used in place of elective choice to control for selection biases and to ensure that experimental and control groups are comparable (within calcu-lable, probabilistic limits). The effects of expectation and reporting bias are minimized by blinding as many par-ticipants and staff as possible at any points at which they may effect the trial. The third objective of a research trial is to maximize the chance of detecting change or the difference between treatment groups, and this requires choosing an appropriate outcome measure.
What Domains?.One can argue that rehabilitation trials could, and per-haps should, collect data on all four domains mentioned in the title and, indeed, potentially in other domains also. This article will now consider each of the domains from the WHO ICF, and wider constructs of QOL, and explain why data might be needed from each domain.
Pathology. This information is needed to set the other data items in context and to help in interpretation and generalization. Pathology (dis-ease, diagnosis) will determine the expected natural history, will inform the choice of prognostic variables needed to interpret findings and en-sure control of severity, and will al-low others to draw appropriate con-clusions for their practice. It will not be a measure of outcome.
Body Structure and Function. Data from this level may have several uses. Impairments (also called impair-ment, often assessed as symptoms and signs) are usually the best mark-ers of prognosis (case mix) and help in interpreting other data. In addi-tion, change at the level of impair-ment might be used to show that the intervention had its primary effect (an indication of process), assuming that it is expected to alter impair-ment. Lastly, there will be occasions when reducing impairment is itself
the primary goal. For example, stud-ies investigating control of pain or reduction of spasms might correctly have pain intensity or spasm fre-quency as primary measures of outcome.
Activities.In most rehabilitation tri-als, the main aim is to reduce limita-tion on activities (disability, behav-ior), and the primary outcome is therefore usually at the level of activ-ities. The activities may be within or outside the home and could include the activities needed to take part in work. (Being an employee and having a job are outcomes at the level of participation, but being able to weld or use a word processor are activities needed to work.) The main difficulty is to choose a measure that will de-tect the change expected. Many mea-sures may either include too many items that are not of relevance or too few to cover the range of possible outcomes.
Participation. Most clinical rehabili-tation aims to maximize social par-ticipation. Unfortunately, measures of participation are currently less de-veloped than measures of more basic activities. Moreover, many factors well outside the control of the reha-bilitation team may affect participa-tion. Consequently, it is rarely the focus of rehabilitation research trials, and it is rarely measured.
Personal Context. Some interven-tions, for example, those focused on self-efficacy, are intended to alter per-sonal factors (beliefs, expectations, background, or preinjury factors). Consequently, one might measure some aspect of this both as a measure of initial state and case mix and also to show that the intervention has had some direct effect (i.e., as a measure of process). Psychological studies might focus on self-efficacy as an out-come, but from the viewpoint of the rehabilitation process as a whole, one would want to directly assess effects
on activities. The known link be-tween self-efficacy and activity per-formance is not sufficiently strong to allow its use as a surrogate marker. Similarly, patients’ knowledge of spe-cialized health self-care practices may be an important process mea-sure, but actual performance of these activities is even more important, and (for example) reducing the rate of complications might be the primary study endpoint.
Physical Context. Some studies will investigate the effects, benefits, risks, and costs of providing suitable items of equipment such as wheelchairs, ankle-foot orthoses, adapted cloth-ing, and so on. Measures of their pro-vision and use might be needed, and the effects of equipment provision on community participation may also need to be measured. Equally physi-cal context data (e.g., stairs vs.
ramps) might be needed to interpret data on activities. Major environmen-tal factors are noted in the ICF.
However, there may be studies in which measures of equipment might constitute an important outcome measure. For example, a trial that investigated different ways of assess-ing and providassess-ing equipment might include the speed of provision of equipment and the provision of ap-propriate equipment as relevant proximal outcome measures. Con-ceivably, a study might investigate whether an intervention improves patients’ independence to the level that the amount of personal support needed or used is reduced.
Social Context. Potentially, studies might investigate the effects of alter-ing the attitudes, knowledge, skills, or expectations of family members (for example), and then it would be appro-priate to measure this change as an indication of process. However this has only been studied occasionally.5
Value and QOL.Value may be defined as the relationship between the
re-sources used and the (change in) out-come achieved. Ultimately, data on value are needed to justify the use of resources and to set priorities. How-ever, it is subject to much debate and is not an easy measurement to make. Before value is considered, the intervention has to be shown to have an effect. (Of course, the specific trial may not have had the statistical power to detect a relevant change or difference, or it may have studied the wrong outcome or at least not have studied all important outcomes that might have been affected.) Once an effect has been shown, then the diffi-culties begin. In essence, there are two questions: what resources are to be included in calculating the cost, and how is the benefit to be valued? It is also important to consider whether the study or trial had adequate statis-tical power to measure the costs; it may well be that studies that can de-tect differences at the level of activi-ties are inadequately powered to cal-culate differences at the level of costs. Resources that might be used in any calculation of cost include:
Direct financial costs of the interven-tion (drugs, equipment, travel) Costs of therapists’ time
Overhead costs of the organization providing the intervention Time, effort, and other resources given
by family members and the patient Side-effects of the intervention Aspects of the benefit that can be
considered include:
Reduction in actual care costs over a lifetime, including both medi-cal care and routine attendant and nursing costs
Reduced other medical costs Personal evaluation of the increased
independence or benefit achieved Family evaluation of the benefit For some patient groups, increased
productive activity and paid work Johnston and others have written on approaches to the study of value,
cost-benefits, and cost-effectiveness analysis in rehabilitation.6(For a more thorough set of measurement terms and definitions, see Johnston et al.,
Measurement Standards for Interdisci-plinary Rehabilitation.)
The primary reason for calculating a value for the benefit is to allow com-parison between different interventions so that apparently rational choices can be made. If one restricts benefits to direct financial benefits, then some-times the benefits of intervention are so great that there is no argument; the intervention is worthwhile. More often, it is necessary to attribute some (finan-cial) value to the benefit. This can only be done successfully if all health stud-ies use the same outcome measure, and QOL seems to offer one solution. But does it?
The term “quality of life” is de-ceptively simple, but it is telling that the WHO has not included QOL within the new ICF; this was probably because there was insufficient agree-ment on the construct, which is largely subjective. One way of con-ceptualizing QOL is to consider that it is the person’s (patient’s) own eval-uation of all of his or her life, includ-ing the patient’s perception of his or her pathology, impairments, activi-ties, and participation interpreted in the light of the patient’s own con-text.7,8 If this is accepted, then the only way to establish a QOL (which would apply equally to people who were outside the healthcare system) is to ask the person directly. This is frequently done.
In considering more formal mea-sures that supposedly measure QOL, there are two major findings of im-portance. First, there are over 1200 existing measures that reputedly measure QOL, and they vary greatly in their content.9This illustrates dra-matically the lack of any agreed phe-nomenon that is QOL. Moreover, there is now a trend toward develop-ing disease-specific QOL measures that further devalues the concept of QOL in general; if QOL means
any-thing, it must be independent of any specific disease.
Second, there is reasonable evi-dence that people in a stable clinical condition generally re-normalize their self-assessed QOL.10
Conse-quently, there may be little point in using QOL to compare different health interventions because most patients will achieve an approxi-mately similar outcome in terms of QOL, whether an intervention is ef-fective and despite any differential in effectiveness. At the same time, QOL has been little used as an outcome measure, so it is conceivable that cer-tain rehabilitation milieu or ap-proaches substantially increase it.
One development within the field of QOL that specifically also consid-ers value is to weight the quality in economic terms, identifying in some way the strength of (economic) pref-erences that individuals have for specified health states. Such mea-sures have begun to be developed,11
but they have not been used in reha-bilitation trials and have rarely been used in medical care trials, which are concerned with the impact of inter-ventions on the patient’s experience of health or illness (a major compo-nent of QOL). Their validity and ap-plicability will need further develop-ment and experience.
Choosing a Measure.When consider-ing outcome measures in rehabilita-tion, including research but also more generally, it is wise to consider a series of questions. First, why am I measuring outcome? Who wants me to measure outcome? In research, this is probably obvious. You are likely to want to know for yourself whether an intervention works. How-ever, even in this setting it is worth thinking of other relevant interested parties such as the patients, the fund-ing agency, the journal editor and peer reviewers, and so on. In addi-tion, the person actually doing the work needs to know the answer be-cause he or she will only be
moti-vated to acquire the data if the data are considered important.
Next, you should specify as closely as possible what effects you expect. This may seem obvious, but it is sur-prising how many projects either fail to do this or ignore the results of this step. In rehabilitation research, it is not uncommon to expect several effects. Each may need a measure.
You then need to discover what measure or measures can capture the specific effect or effects expected. The major considerations when choosing a measure are shown in Table 2, and a few points will be emphasized here. It is vital to choose a measure that as closely as possible matches the range of the expected outcome. If you an-ticipate changing the ability to dress, do not use the FIM™ as a whole because only a small part of the FIM covers dressing. Conversely if you are trying to improve mobility after stroke, covering the whole range, simply using gait speed is unwise be-cause a proportion of people will not walk and yet you need to measure
their outcome. One important prac-tical consideration is how the data will actually be collected. Who will undertake the measures, how, where, etc.? This may strongly influence your choice of measure because a complete set of even relatively crude data are much more valuable than an incomplete set of much more de-tailed data.
You must also consider what ad-ditional data you and others will sen-sibly require to interpret the outcome. This must be restricted to major items such as known markers of prognosis. Avoid collecting excessive data.
Conclusion. Generally, primary out-come measures in rehabilitation re-search will be at the level of activities. Impairment-based measures are use-ful both to measure prognosis (case mix) and to track the (process) effects of interventions. In large or confir-matory trials, some information on costs should be collected, but at-tempts to measure value through us-ing indicators of QOL as a whole
TABLE 2.
Questions to ask when choosing a measure
Clinical Question Psychometric Term or Question
Will the data collected using this measure answer the clinical question?
Is it valid for the purpose? Does the measure cover the whole
range of possible activities or impairments without including items that are not relevant?
Content validity and avoidance of floor or ceiling effects.
Do I know how much variability there will be in the data, and is that satisfactory?
What is its reliability when used as I will use it?
Is the measure going to deliver data that are able to discriminate or detect the change or difference that I am interested in or expect to find?
What is its sensitivity, and is that adequate?
Will the measure be correctly used by staff and accepted by the patients so that I get a full data set under the circumstances? Is it sufficiently short and simple?
Is the measure feasible?
Will anyone reading the results of my study understand the data given by this measure?
How communicable is the result given by this measure?
seem unlikely to succeed, either in logic or in practice. By using simple measures that assess relatively large or definite changes in activities that are quite evident to patients, family, and therapists, one takes a great step toward ensuring that the outcomes are of value and toward ensuring that the trial will be feasible.
ACKNOWLEDGMENTS
I thank Dr. Mark Johnston for his helpful comments on an earlier draft, the Kessler Medical Rehabilita-tion Research and EducaRehabilita-tional Cor-poration for their support, and the very many people over the years who, by asking me for help or by disagree-ing with me, have helped me to de-velop and clarify my thoughts (as-suming that they are clear!).REFERENCES
1. International Classification of Impair-ments, Disabilities, and Handicaps. Ge-neva, World Health Organization, 1980 2. International Classification of Func-tioning, Disability, and Health. Geneva, Switzerland, World Health Organization, 2001. Available at: http://www3.who.int/ icf/icftemplate.cfm. Accessed April 3, 2003
3. Wade DT, Halligan PW: New wine in old bottles: The WHO ICF as an explana-tory model of human behaviour.Clin Re-habil2003;17:349 –54
4. Haley SM, Costner WJ, Binda-Sund-berg K: Measuring physical disablement: The contextual challenge. Phys Ther
1994;74:443–51
5. Buchbinder R, Jolley D, Wyatt M: Pop-ulation based intervention to change back pain beliefs and disability: Three part evaluation.BMJ2001;322:1516 –20 6. Johnston MV: Cost-benefit
methodol-ogies in rehabilitation, in Fuhrer M (ed):
Rehabilitation Outcomes: Analysis and Measurement. Baltimore, Brookes, 1986, pp 99 –114
7. Carr AJ, Gibson B, Robinson PG: Mea-suring quality of life: Is quality of life determined by expectations or experi-ence?.BMJ2001;322:1240 –3
8. Carr AJ, Higginson IJ: Are quality of life measures patient centred?BMJ2001; 322:1357– 60
9. Garratt A, Schmidt L, Mackintosh A, et al: Quality of life measurement: Biblio-graphic study of patient assessed outcome measures.BMJ2002;324:1417–22 10. Albrecht GL, Devlieger PJ: The dis-ability paradox: High quality of life against all odds. Soc Sci Med 1999;48: 977– 88
11. Brazier J, Deverill M: A checklist for judging preference-based measures of health related quality of life: Learning from psychometrics.Health Econ1999;8: 41–51
