The following sections outline the key assumptions and data sources used to populate the model structure outlined above. In addition to data drawn from the IPD meta-analysis, modelled assumptions and inputs were primarily informed by three literature reviews. These included a review of observational studies to help characterise the prognosis of autistic children and adults; a utilities review to identify studies reporting on the HRQoL of autistic children and adults; and a resource review to identify resource and cost data relating to the care needs of autistic children. Details of all three of these reviews, including searches undertaken, inclusion criteria and findings, can be found in Appendices 14–16.
Furthermore, additional searches of the literature were also undertaken to identify any other input parameters not covered by these three reviews. Inputs were also informed by discussions with UK ABA practitioners, clinical academic experts, autistic adults and parents of autistic children, who provided feedback on the model assumptions, including current provision of services for autistic
TABLE 9 Description of independence levels
Independence level Description
Completely independent Achieving a high level of independence; has some friends/acquaintances; maintained reciprocal relationships; employed at some level (worked on voluntary/supported basis), now or in the past
Mostly independent Achieving a good level of independence, but may require some support in daily living; has some friends/acquaintances; maintained reciprocal relationships; likely to be employed at some level (worked on voluntary/supported basis), now or in the past Some independence Some degree of independence, but requires support and supervision; does not need
specialist residential provision; no close friends but some acquaintances; reciprocal relationships with reduced sharing; may have been employed now or in the past Mostly dependent Specialist residential provision/high level of support; no friends outside of residence;
no enduring relationships; never in paid employment
Completely dependent Needs high-level hospital care; no friends; no autonomy; never in paid employment
ECONOMIC ANALYSIS
NIHR Journals Librarywww.journalslibrary.nihr.ac.uk
children, the delivery of ABA-based therapies in the UK and resource use. Information was also obtained from social care teams in several local authorities (York, Leeds and Newcastle) to ensure that we accurately reflected the current provision of services for autistic children and adults.
Treatment effect
In the TAU and eclectic arm of the model, changes in cognitive ability and adaptive behaviour scores are modelled using autism-specific natural history data to predict changes over time. To model the treatment effect, in the early intensive ABA-based arm, cognitive ability and adaptive behaviour scores are modelled by applying the treatment effect derived from the IPD meta-analysis. Outcome scores in the early intensive ABA-based interventions arm of the model are therefore the sum of the score predicted from the natural history data plus the treatment effect.
Natural history
To model changes in cognitive ability and adaptive behaviour scores over time, data were extrapolated from four studies95,96,107,110identified in the systematic review with extended follow-up. These data were
used to estimate change in baseline scores on usual care over time and were extrapolated assuming a simple linear trend. The studies identified in the systematic review of effectiveness studies was used in favour of data reported in observational studies, because few children with intellectual disability were recruited to the observational studies reporting changes in both cognitive ability and adaptive behaviour scores. See Appendix 15 for details of the review of observational evidence. The data from the effectiveness studies were therefore considered more consistent with the population modelled, although both sources reported similar patterns of change. To explore the impact of this assumption on model outcomes, sensitivity analyses were conducted using alternative rates of change in cognitive ability and adaptive behaviour scores. The mean changes in cognitive ability and adaptive behaviour scores used in the base case are presented in Table 10.
Treatment effect
The treatment effect was modelled in two phases: (1) a short-term phase, covering the first 2 years, and (2) a long-term phase, covering 2 years and onwards. This distinction is made to reflect the fact that most of the studies included in the IPD meta-analysis had a follow-up of ≤ 2 years.
Short-term treatment effect
In the first 2 years of the model (up to cycle 24), the treatment effect applied was drawn from the IPD meta-analysis and was based on analyses conducted at 1 and 2 years (Table 11). As discussed in Chapter 6, UK-based studies, the average treatment effect observed in the UK studies was somewhat
TABLE 10 Mean change in cognitive ability and adaptive behaviour scores per year
Outcome measure Mean SE
Adaptive behaviour –0.28 1.24 Cognitive ability –2.69 1.27
TABLE 11 Short-term treatment effect
Outcome measure
Base case (all studies with IPD) UK scenario
Year 1 Year 2 Year 1 Year 2
MD SE MD SE MD SE MD SE
Adaptive behaviour 2.92 2.46 7.00 2.58 1.70 2.36 2.78 2.42 Cognitive ability 9.16 2.44 14.13 2.54 6.27 4.04 9.39 4.32
smaller than that observed when combining all studies, and although these differences are not statistically significant, it is plausible that, because of either the differences in setting or the precise interventions compared, the benefits of early intensive ABA-based interventions may differ in the UK from those observed elsewhere in the world. A scenario analysis was therefore conducted using treatment effectiveness data drawn only from the UK studies.
Long-term treatment effect
Evidence on the longer-term effects of early intensive ABA-based interventions on cognitive ability and adaptive behaviour scores, and in particular the degree to which any early benefits are sustained, is very limited, with few studies reporting outcomes post end of therapy; see Chapter 6, Individual participant data meta-analysis of early intensive ABA versus treatment as usual or eclectic interventions. Given the lack of evidence on long-term effects, it was not felt that following the norm of developing a single base-case analysis would be reasonable. Two scenarios were therefore explored:
1. an optimistic scenario, which assumed that the treatment effect persists throughout the time horizon of the model
2. a pessimistic scenario, which assumed that the treatment effect dissipates over time, such that at 7 years no treatment effect remains.
In both scenarios, these increases or decreases in the size of the treatment effect are modelled as a linear trend, as evidence from intermediate time points is unavailable. Furthermore, no changes in the size of the treatment effect are assumed after 7 years. This time limit reflects the maximum follow-up of Magiati et al.,96which was the study with the longest follow-up, and attempts to acknowledge that
any further changes in the treatment effect are unknown. Pragmatically, it also prevents the magnitude of the treatment effect trending to extreme values (e.g. negative scores).
Mapping childhood outcomes to levels of independence used in the adult scenario analysis
Evidence linking adult levels of independence (see Appendix 15) to childhood cognitive ability and adaptive behaviour is somewhat limited. Although a number of studies report strong correlations between adult cognitive ability, adaptive behaviour scores and adult levels of independence,15,131–134none reports results
in a way that would allow contemporaneous predictions about adult outcomes (e.g. an appropriate regression analysis). However, one study, Farley et al.,134does report mean adaptive behaviour scores
for several of the independence levels described in Scenario analysis exploring adult outcomes (see also Appendix 18 for further details).
To generate an algorithm with which to predict independence levels, the summary data reported in Farley et al.134were used to simulate IPD; details of methods used to do this are reported in Appendix 18.
An ordinal logistic regression was then implemented to generate a regression model that could be used to predict adult outcomes. The coefficients generated from this analysis are reported in Table 12, along with the appropriate cut-off points.
TABLE 12 Estimated coefficients based on simulated sample
Parameter Coefficient SE
Adaptive behaviour 0.07 0.01 Cut-off point 1 (very poor) 0.98 0.29 Cut-off point 2 (poor) 2.74 0.51 Cut-off point 3 (fair) 4.51 0.77 Cut-off point 4 (good) 6.55 1.05
ECONOMIC ANALYSIS
NIHR Journals Librarywww.journalslibrary.nihr.ac.uk
Education
Few studies identified in the IPD meta-analysis reported educational placement and there is significant potential that this outcome was subject to significant bias, as predicted effects were implausibly large (see Model development). We therefore modelled educational outcomes indirectly by using regression models to predict educational placement using cognitive ability and adaptive behaviour scores.
Two regression models were generated, one to predict primary educational placement and a second to predict secondary educational placement. The proportion of children attending each of the three types of school described in Model structure is therefore determined at two points in the model, once in cycle 3 (age 4.5 years) and again in cycle 10 (age 11.5 years), with the probabilities of attending each type of educational placement estimated using two separate data sets.
The first regression model, used to predict primary educational placement, was based on pooled data from two of the studies identified in IPD meta-analysis: Smith et al.103and Magiati et al.31Secondary placement
was based on data from Barret et al.,135a costing study identified in the resource review. As data from
Barret et al.135were not reported in a form that allowed cognitive ability and adaptive behaviour to be
linked with educational placement, we requested IPD, which were kindly provided by the authors to allow appropriate reanalysis. Both analyses used ordinal logistic analysis, with appropriate adjustments for repeated measures. Regression coefficients from these analyses are reported in Table 13.
To acknowledge the potential impact of utilising an indirect approach to estimating the impact of early intensive ABA-based interventions on educational placement, a scenario analysis used the direct evidence available in the three effectiveness studies26,89,103reporting subsequent educational placement. In this
scenario analysis, children are assumed to attend the same type of education throughout childhood (see Appendix 18 for the proportions used).
Adverse effects
Adverse effects of ABA-based interventions were not reported in the studies included in the IPD meta-analysis and available information from other sources is limited, and largely anecdotal.30,31,136
It was therefore not possible to include adverse effects in the economic model.
Mortality
A number of epidemiological studies have shown that autistic people experience reduced life expectancy compared with the general population.137–142
To account for this reduction in life expectancy, mortality rates were estimated by applying a relative risk drawn from Hirvikoski et al.138to general population mortality rates adjusted for age and sex
from UK life tables.143This study was selected because it is the largest study identified in the literature
and included 27,122 autistic people and a further 2,672,185 controls. For males without intellectual disability (IQ ≥ 70), a relative risk of 1.69 was applied and a relative risk of 7.71 was applied for those with intellectual disability. For females without intellectual disability (IQ ≥ 70), a relative risk of 2.47 was applied and a relative risk of 4.75 for those with intellectual disability.
TABLE 13 Education type regression coefficients
Parameter
Primary school Secondary school
Coefficient SE Coefficient SE
Adaptive behaviour –0.08 0.02 –0.04 0.02 Cognitive ability –0.02 0.01 –0.06 0.02 Cut-off point 1 (mainstream with support) –8.75 0.02 –9.37 1.58 Cut-off point 2 (specialist schooling) –6.10 0.86 –6.60 1.39
Note
Mortality was not linked to intervention effectiveness. Given the observed effects of early intensive ABA-based interventions on cognitive ability scores and the observed differences in mortality rates in autistic people with and without intellectual disability, it is possible that early intensive ABA-based interventions may influence mortality. However, as agreed with our Advisory Group, extending any impact of early intensive ABA-based interventions to mortality was considered too strong an assumption, given the lack of evidence, and therefore no such benefits were included within the model.
Modelled population
In line with the IPD meta-analysis, the modelled population was assumed to consist of preschool children with a diagnosis of autism. Baseline characteristics modelled included age and sex, as well as the following outcome measures: autism symptom severity (ADOS), cognitive ability (IQ) and adaptive behaviour (VABS). Baseline characteristics were drawn from a pooled analysis of the studies comparing early intensive ABA-based interventions with TAU or eclectic interventions.84–86,89–94,97,104The exception to
this is age, which was rounded down from 38 months to 36 months to ensure that the time horizon is a constant 15.5 years (this has a negligible impact on the predictions of the model). Note that because the model assumes that children will receive early intensive ABA-based interventions for 2 years, the starting age implies that some children will receive early intensive ABA-based interventions in place of regular schooling. Children in the UK are mandated to attend school by their fifth birthday144but typically
start in the September after their fourth birthday. This assumption was validated by the authors of the effectiveness studies and experts within our Advisory Group, who agreed that at least a proportion of children would receive early intensive ABA-based interventions in a school setting. In a scenario analysis in which effectiveness data were drawn from only UK studies, baseline characteristics were (when possible) drawn from the UK studies only. Starting characteristics used in the base-case analysis and in the UK studies only scenario are summarised in Table 14.
Subgroup analysis
The results of the subgroup analysis presented in Chapter 6 show little evidence of differential effects in the groups investigated. The economic model therefore does not explore subgroups any further.
Intervention and comparators
Reflecting the main comparison in the IPD meta-analysis, the model compares early intensive ABA-based interventions with TAU or eclectic interventions.
TABLE 14 Baseline characteristics of children in the model
Characteristic Base case (10 studies84–86,89–97,104,109,110) UK scenario (three studies89–92,96,109,110) Mean SE Mean SE Proportion male (%) 87.57 1.47 91.59 2.70 Proportion with intellectual
disability (IQ < 70) (%)
59.43 1.08 82.95 3.32
Age (months) 36 0 36 0
Adaptive behaviour (VABS) 63.19 0.43 57.60 0.55 Cognitive ability (IQ) 59.43 1.08 54.66 1.50 Autism symptom severity (ADOS)a 6.98 0.18 6.98 0.18
a Based on four studies reporting scores.
ECONOMIC ANALYSIS
NIHR Journals Librarywww.journalslibrary.nihr.ac.uk
Early intensive applied behaviour analysis-based interventions
Modelled early intensive ABA-based interventions reflected our understanding of current UK provision following advice from the authors of the effectiveness studies and several UK practitioners. Data were also sourced from a UK study by Griffith et al.,36which described provision of ABA-based therapy in
14 UK ABA schools. In the base-case analysis it was therefore assumed that children will receive 30 hours of early intensive ABA-based interventions per week for a period of 24 months. Intensity was based on a consultation exercise with study authors and UK practitioners, who, although not in complete consensus, suggested that 30 hours would represent a typical programme. This also aligns with the average of 27 hours per week reported by the effectiveness studies and data reported in Griffith et al.36Duration
of therapy was conservatively assumed to be 24 months on the basis that this aligns with the maximum follow-up period for the bulk of the available effectiveness evidence. However, consultation with study authors and data reported in Griffith et al.36suggest that it is common for children to continue receiving
intensive ABA-based therapy in a school setting and the number of previous economic evaluations assumed a longer duration of therapy. A scenario analysis was therefore run assuming that children receive an intervention for a period of 36 months.
Comparator interventions
Comparator interventions included in the IPD meta-analysis were very diverse and the wider literature reveals few details about service provision in the UK. Advice from expert advisors suggested significant variability in current provision and highlighted that it usually consists of a mixture of activities delivered in a nursery or home setting. These services may also be supplemented by additional input from specialist services, such as speech and language therapists, educational psychologists, occupational therapists and mental health workers. To help provide further details on TAU and eclectic interventions, service providers in York and Newcastle local authorities kindly provided insight into provision in these areas, suggesting that TEACCH and PECS are used regularly, and that approximately one-third of children attend specialist nurseries, with the remainder supported in local authority mainstream nurseries. The model defined TAU and eclectic interventions based on York and Newcastle local authority provision, with costs also based on funding provided in these localities.
Health-related quality-of-life data used in cost-effectiveness analysis
As described in Model development, a systematic review was undertaken to identify appropriate utility data with which to populate the model. The studies identified in this review led to the adoption of a statistical algorithm developed in Payakachat et al.127to predict HRQoL benefits.
Application of the Payakachat and colleagues algorithm
Quality of life was estimated at each time point in the model based on the algorithms presented in the Payakachat et al.127study, using adaptive behaviour, age, baseline cognitive ability (IQ scores) and
baseline autism symptom severity (ADOS scores). Quality-of-life scores therefore changed in accordance with changes in cognitive ability and adaptive behaviour scores, and with increasing age over time. Table 15 gives the coefficients applied in the model.
TABLE 15 Regression model used to predict quality of life in children
Parameter Scenario analysis Coefficient SE Constant −0.2438 0.2015 Age 0.0119 0.0186 Age79 0.0003 0.0010 ADOS score 0.0079 0.0078 IQ (log) 0.0304 0.0478
Quality of life for autistic adults (scenario analysis only)
Health-related quality of life in adulthood is also based on the Payakachat et al.127algorithm, with two
adjustments made to reflect the fact that the algorithm is based on a paediatric population. The age parameter is held constant at 18.5 years and age-related decrements are applied to account for the natural effects of ageing on HRQoL. These decrements were calculated based on Szende et al.,145which
described the health utilities of healthy populations of adults for different age groups. Decrements were applied as multiplier to the mean utility of the cohort from the age of 25 years onwards (see Appendix 18).
Quality of life for parents and carers
A number of studies reported lower (health-related) quality of life in parents and carers of autistic children compared either with parents of typically developing children or with population norms.120–125
This literature has shown that child behavioural difficulties and lack of social support are particularly important drivers of quality of life in parents of children with autism and related conditions. The evidence in the IPD meta-analysis provides only very limited evidence that early intensive ABA-based interventions have any comparative advantage over other early interventions on problem behaviours, with few studies recording appropriate outcomes. Furthermore, the quality-of-life literature does not show any consistent relationship between other clinical outcomes, such as child cognitive ability,123,125
adaptive behaviour121and autism symptom severity.121,123–125It is therefore not clear what, if any,
effect early intensive ABA-based interventions has on parent and carer quality of life. The economic analysis therefore does not include any analysis of any possible impact of early intensive ABA-based interventions, either positive or negative, on parent or carer quality of life, although it is acknowledged that these potentially exist.
Resource use
Relevant costs and resources were identified in a systematic review of the literature and appropriate routine sources, such as Personal Social Services Research Unit (PSSRU).146The studies identified in
this review were used to assign appropriate costs associated with the delivery and care of autistic children and adults. Information on the precise description of resources required for each individual therapy was partially based on data derived from the IPD review, and augmented with further