The National Institute for Health and Care Excellence’s threshold range
The National Institute for Health and Care Excellence has been reluctant to specify a single cost-effectiveness threshold used in its decision-making.6It has also consistently emphasised that factors other than CEA are taken into consideration by the various advisory committees.3,5,6,38–40Therefore, it has preferred to indicate the range within which its threshold value lies (i.e. £20,000–30,000 per QALY gained).3,5Alongside this, it has provided an indication of the role other factors play in determining which point of threshold range is relevant. The latest guide3suggests that an ICER<£20,000 is likely to lead to recommendation unless the evidence is considered highly uncertain; an ICER between £20,000 and £30,000 will lead to
recommendation if the committee is also happy with the levels of uncertainty in the evidence and/or the QALY does not capture all aspects of benefit; and an ICER>£30,000 would be recommended only if issues related to levels of evidential uncertainty and a failure to capture all benefits in the QALY are particularly compelling.
In 2009, NICE issued further supplementary guidance relating to the appraisal of interventions for patients with short life expectancy (LE), although this can be considered to relate more to the measure of benefit than factors to be considered outside of cost-effectiveness.41In 2012 NICE issued a draft update of its methods guide which added that, if a new technology has an ICER>£20,000 per QALY, the committee’s deliberations would also consider‘aspects that relate to non-health objectives of the NHS’ (e.g. wider social considerations and/or costs that fall outside the NHS budget).42
DOI: 10.3310/hta19140 HEALTH TECHNOLOGY ASSESSMENT 2015 VOL. 19 NO. 14
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Claxtonet al.under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.
Although NICE has carefully argued the case for why its decisions are not driven entirely by a comparison of the ICER with its threshold range, it has not provided any empirical evidence for why the threshold range takes the value it does. Indeed it has been widely argued than an empirical basis for these values should be generated.4,43–47For example, the House of Commons Health Select Committee in 2008 argued:
The affordability of NICE guidance and the threshold it uses to decide whether a treatment is
cost-effective is of serious concern. The threshold is not based on empirical research and is not directly related to the budget, it seems to be higher than the threshold used by [primary care trusts] PCTs for treatments not assessed by NICE. Some witnesses, including patient organisations and pharmaceutical companies, thought that NICE should be more generous in the cost per QALY threshold it uses, and should approve more products. On the other hand, some PCTs struggle to implement NICE guidance at the current threshold and other witnesses argued that a lower threshold should be used. We recommend that the threshold used by NICE in its full assessments be reviewed; further research comparing thresholds used by PCTs and those used by NICE should be undertaken . . .
p. 64 The basis for empirical work
Although there is acceptance of the need for empirical work on the NICE cost-effectiveness threshold, a set of issues exists regarding the starting point for such analysis. One aspect of this is the view that the nature of the services that are displaced in response to additional costs being imposed by NICE guidance, and hence the magnitude of the health forgone for other patients, will depend on the productivity of the NHS and its overall (inflation-adjusted) budget, both of which have increased since NICE initially defined its threshold range.48,49In principle an increase in the (real) NHS budget would allow it to introduce interventions which were previously not cost-effective which might be expected to increase the threshold if these interventions were the marginal ones displaced in response to the budget impacts of NICE recommendations. However, any increase in the NHS budget may be allocated to non-discretionary expenditure. This would include, for example, expenditure relating to national initiatives such as new contracts for consultants and activities to meet waiting list targets as well as, of course, the implementation of NICE guidance. The non-discretionary nature of such expenditure means that these types of activities cannot easily be disinvested from given a need to release resources to fund NICE
guidance. Therefore, if an increase in the NHS budget is largely devoted to these types of non-discretionary expenditure, there will be a limited impact on the threshold.
Gains in productivity may come through doing worthwhile activities more cost-effectively, including for those marginal interventions displaced by NICE recommendations, suggesting a reduction in the threshold. Alternatively, productivity gains might come through discontinuing activities which are not worth doing (i.e. that produce no health improvement), freeing resources for additional cost-effective interventions which may be the marginal services displaced by NICE guidance–this can have the result of increasing the threshold.
The net effect of these changes on the threshold could not be determined a priori and would depend on how any additional (real) budgets were allocated and how the gains in productivity where achieved. This does emphasise the fact that the threshold may change over time in response to these and other broader developments, and this would have to be considered as part of any regular updating of the empirical analysis of the threshold.
A second issue to be considered relates to how decisions are taken locally about any displacement following NICE guidance. The principles of CEA suggest that such displacement should relate to interventions which are the least cost-effective of those currently covered by the budget.14The basis for how local commissioners and providers make their disinvestment decisions is not clear, however, and there have been calls for greater transparency and guidance in this area.48It would be entirely unrealistic to assume that displacement only takes place in those existing services which are the least cost-effective. The reality is that numerous criteria are likely to be used by commissioners in implementing disinvestment,
and that significant variation will exist between local decision-makers.8Such criteria might include, for example, equity concerns about a particular disadvantaged group locally or capacity constraints regarding particular services. Therefore NICE needs to know what is likely to happen on average across the NHS given the reality of local decisions. If local decision-making changes over time–for example, if local commissioners become more focussed on displacing services which are the least cost-effective, in terms of population health–this may affect the estimate of the threshold.
Studying displacement locally
A reasonable conclusion from a consideration of these issues is, therefore, that local decisions about disinvestment are likely to be an important determinant of the NICE threshold.50–55Applebyet al.56 sought to assess whether or not it was possible to study local decisions about service investment and disinvestment to infer the cost-effectiveness thresholds being used (implicitly) locally and to draw conclusions about the appropriate level of the NICE threshold. They identified six PCTs and undertook structured interviews with each of the directors of public health. They also administrated questionnaires to an opportunistic sample of finance directors from NHS trusts. On this basis they developed a list of new services as well as those that had been deferred or discontinued. An attempt was made to estimate the implicit local ICER relating to these decisions by using any cost-effectiveness evidence used to inform the decisions together with relevant evidence on cost-effectiveness from the published literature. The study found it quite straightforward to identify specific services that had been introduced,
discontinued or deferred, but concluded that these decisions were typically based on clinical and other non-economic factors. A number of‘decisions at the margin’were identified but none of these were based on CEA. Instead, the basis for changes in services was a‘business case’, or overall cost impact. It was possible to impute cost-effectiveness for most of the services affected, but the study concluded that, even with a larger sample of commissioners and providers, it would be very difficult to estimate an
implied cost-effectiveness threshold locally. This would be because, first, most PCT decisions were service reconfigurations including demand management and waiting list initiatives. By their nature, teasing out the incremental cost and health effects, potentially across numerous types of patients, would be an enormous challenge. Second, there would be difficulty in identifying all local decisions as many options for investment, deferment or discontinuation are rejected before they are made more explicit in documentation. A third problem would be the finding that a range of criteria is used to make local decisions, with relatively little concern for cost-effectiveness, making a local threshold estimated in this way hard to interpret. A final challenge would be that it would be very difficult to establish a causal link between a change in local NHS budget and specific local investment and disinvestment decisions. The Applebyet al.56study highlights the problems that exist in deriving a cost-effectiveness threshold from a bespoke study of specific local resource allocation decisions.
What evidence is needed?
Given the challenges of studying local decisions as a means of establishing the NICE threshold, and keeping in mind NICE’s remit, it is possible to suggest a series of important characteristics that estimation methods should have from the perspective of principle and practice:
l They should reflect the effect of NICE guidance on theaverageof the displacement decisions taken across the NHS, with less consideration on which types of patients and interventions are affected and why the decisions are taken. NICE cannot be expected to reflect what is likely to be marked variation between local commissioners and providers in how they react to an effective reduction in their budget as a result of positive guidance. Given NICE’s remit, it is the expected health effects (in terms of length and QoL) of the average displacement within the current NHS (given existing budgets, productivity and the quality of local decisions) that is relevant to the estimate of the threshold.
l The methods used should not be a‘once and for all’effort but should facilitate regular updates to reflect changes in the broader NHS context such as changes in the overall real budget and productivity. This requires the use of data sources that are currently routinely available, are expected to become so
DOI: 10.3310/hta19140 HEALTH TECHNOLOGY ASSESSMENT 2015 VOL. 19 NO. 14
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Claxtonet al.under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.
in the future or could be made available at reasonable cost. It may be possible to glean some idea of how the threshold may change in the future by studying how it has changed in the past, which would require routine data sources to extend back over a period of time. Periodic updating using explicit scientific methods would encourage accountability through scrutiny of estimates by relevant stakeholders. It would also provide more predictability in likely changes to the threshold for the investment decisions of technology manufacturers.
l The nature of the displacement of existing services (and hence the magnitude of the health forgone) will depend on the scale of the budget impact coming through NICE guidance. Therefore, the methods used to estimate the threshold should ideally be able to reflect this budget impact.
l The methods should recognise the inevitable uncertainty relating to the evidence currently available for threshold estimation and translate this into an expression of the uncertainty in the estimate of the threshold. As well as providing information with which NICE can determine the appropriate implications for its choice of a threshold value, this consideration of uncertainty can help to prioritise further research or the collection of routine data.