Main components of ‘payback’ models

The main components and characteristics of the ‘payback’ framework are discussed below with reference to identified ‘payback’ models67;90-92;94;98;100

. A brief summary of these characteristics is given in Table 3.1.

3.2.2.A. Purpose

With regards to their purpose, some ‘payback’-based models have an explicit focus on setting priorities for primary, evidence-generating research (typically clinical trials)91;92;97;100

while other67;90;94;95;98

can be potentially useful for funding decisions on both primary and secondary, evidence-synthesising research. Thus, depending on the variation used, the framework appears capable of informing funding decisions for either primary or secondary research.

3.2.2.B. Specification of possible research outcomes

The first important step in ‘payback’ models is the specification of possible research results. These represent hypothetical but plausible outcomes from research, typically expressed in terms of different measures of effectiveness. An important assumption is that a possible result represents the true effectiveness (or cost-effectiveness) of the treatments in question, which holds true irrespective of whether a study has been carried out and revealed it or not. Different terms are used for such results, including ‘Delta results’⁹⁰ and ‘exemplar outcomes’⁹². In terms of considerations to be taken into account when specifying possible research results, most of the identified studies agree that these

Table 3.1: Characteristics of ‘payback’ models

Study

Aims Methods Results and conclusions

Prioritisation of

Measure of final results Decision criterion

Weinstein(1983)

Yes Yes Cost per year of life saved Prioritise research

programme with lowest cost

Study

Aims Methods Results and conclusions

Prioritisation of

Measure of final results Decision criterion

Drummond et al.

In three of the identified ‘payback’ models^90;98;100, it is assumed that research will produce two kinds of results:

a. ‘Positive’, where research reveals that the treatment of interest is effective and cost-effective, and

b. ‘Negative’, where the treatment is shown to be inferior in terms of its effectiveness and cost-effectiveness.

The studies by Townsend and Buxton⁹² and Townsend et al.⁶⁷ allow for the additional possibility of ‘inconclusive’ outcomes^67;92, while Detsky⁹¹ represents possible results in the form of a distribution. Arguably, the latter resembles more closely the type of results that are likely to be observed in a clinical study.

3.2.2.C. Change in the use of the assessed treatment

Another important point in the ‘payback’ methodology relates to the impact of the results on clinical practice. Here, it is assumed that observing a specific research outcome will trigger a change in clinical practice. Trial results showing the treatment not to be effective are expected to result in a gradual decline in its use, while ‘positive’ research results (i.e.

treatment is effective or cost-effective) are anticipated to lead to greater use of the treatment in clinical practice.

All the identified studies attempt to establish a link between the results and subsequent use of the treatment of interest. In general, clinical practice is assumed to change on the basis of a treatment’s hypothesised effectiveness. The contemporary view that cost-effectiveness is

a key consideration in policy change is explicitly accounted for in the PATHS model⁶⁷. Eddy⁹⁰ stresses that change in policy will depend on a number of factors, including the results of other assessments, existing policies that might affect the uptake of the assessed technologies, geographic regions and time periods. In the absence of further research (i.e. if the proposed research study is not undertaken), it is typically assumed that clinical practice will not change^90;92;100.

The concept of change in use is particularly important in ‘payback’, because it allows translating possible research results into potential benefits experienced by the population67;91;98;100

. As obtaining hard evidence on the possible change in clinical practice is problematic, estimates for this change usually comes in the form of expert opinion⁶⁷.

3.2.2.D. Estimation of costs and benefits

All the ‘payback’ models attempted to quantify the costs and benefits following change in uptake of a treatment. Information for this is typically derived by combining existing evidence on costs of the treatment—typically obtained from the literature—with the hypothesised research results^67;92;100.

Costs and benefits are first estimated for an individual patient and they are subsequently extrapolated to the population that is expected to benefit from a decision informed by additional evidence over a number of years. Costs are typically measured from a provider’s perspective. An exception is the Diabetic Retinopathy Study, where Drummond et al.¹⁰⁰ explored a variety of alternative perspectives, including those of government, health care

measures67;90;98;100

and natural units such as premature death avoided^91;94, while in one study benefits were expressed as QALYs⁹².

3.2.2.E. Weighted results and decision rules

Typically, the streams of costs and benefits associated with each possible research result are weighted by the likelihood of the result being observed. Such likelihoods weights were accounted for in all the identified models. Eddy⁹⁰ determined the probability of observing a

‘positive’ or ‘negative’ result to be 0.50; similarly, Davies et al.⁹⁸ specified the probability of

‘positive’ results at 0.67 on the premise that approximately two out of three treatments assessed in clinical trials show ‘positive’ findings. In two studies, the authors^91;102 followed a more sophisticated approach, where a probability distribution was assigned to the possible difference in effectiveness between assessed treatments. Likelihood weights have a sizeable effect on final ‘payback’ results and thus, it is important that chosen values are plausible and justified.

Different approaches have been followed in assigning likelihood weights to possible results.

In early ‘payback’ models, each specified research result carries the same likelihood of occurrence90;94;98;100

. A more sophisticated approach involving different combination of weights is employed by Townsend and colleagues^67;92. In these studies, three possible research results (‘negative’, ‘positive’ and ‘inconclusive’) merge in ‘optimistic’, ‘neutral’ and

‘pessimistic’ combinations.

The area where ‘payback’ models present the greatest diversity is the interpretation of the generated results and the determination of rules for action. In general, models aim to be

consistent with decision rules used in economic evaluation and attempt, whenever possible, to calculate ratios of costs per expected benefits, with the difference that, in this context, the comparison is between a state with and without research67;92;98;100

.

It is often not clear how such ratios should be interpreted. In cases where ratios indicate that an assessment would result in cost savings and increased effectiveness, an obvious option would be to advocate carrying out research. Nonetheless, a clear rule is needed for the common situation where research is associated with additional benefits and higher costs. In such a case, Townsend and Buxton⁹² compared the estimated ICER of the Hormone Replacement Treatment (HRT) trial against cost per QALY values of commonly-used interventions such as renal transplant or breast cancer screening, and pointed out that the HRT trial appears to represent value for money. As Davies et al.⁹⁸ point out, interpretation of results and decision rules are topics where further research would be particularly useful.