We faced a number of challenges in the design and execution of this study. We have chosen to highlight four of these here because they illustrate some of the difficulties inherent in addressing the research
recommendations on physical activity and the environment published by NICE in 2008,17which formed
part of the background to the inception of the study. The first challenge was that of combining different disciplinary perspectives on the assessment of behaviour change in the context of the evaluation of a transport infrastructure project. The fields of physical activity research and transport research have tended to assess travel behaviour using different instruments and summary measures, and we aimed to establish some common methodological ground between these different approaches. This was important in order to be able to assess the effect of the intervention using a suite of complementary metrics to serve the needs of different groups of users of the findings, while also understanding the social and physical contexts of the target behaviours. The second challenge was that of implementing rapid and robust baseline measurement in the somewhat unpredictable setting of a natural experimental study of an intervention that was outside the control of the research team. This required us to make trade-offs between the competing merits of different approaches and technologies for capturing travel and physical activity behaviour change. The third challenge was that of finding a way of defining exposure to the intervention that would enable us to make controlled comparisons and thereby strengthen the basis for causal estimation of the effect of the intervention. This required us to deal rigorously and transparently with the emerging realisation that a simple parallel-group design was not suitable for the evaluation of the busway. The fourth challenge was that of designing and adapting an analytical strategy that was able to cope with the delayed implementation of the intervention. This required us to use different types of data and approaches to analysis to piece together a combination of evidence for causal estimation and evidence for causal explanation along an extended putative causal pathway linking environmental change with behaviour change and population health impacts. In the following sections, we briefly discuss our responses to these four challenges in terms of their implications for the strengths and limitations of the study and for future research, most which have been described elsewhere in the report but are
summarised here for the sake of completeness. Strengths of the study
Together with our funders, we demonstrated a flexible approach to the unpredictable realities of natural experimental research in two important ways: first by implementing rapid baseline data collection in what we believed to be a brief window of opportunity before the originally scheduled opening of the busway, and later by extending the study and endeavouring to make the best possible use of the available data for causal inference and understanding. We measured physical activity in general, and travel to and from work
in particular, using a combination of simpler and more detailed, self-reported and objective measures. This enabled us to assess the main outcomes of the evaluation using a comparatively simple self-reported
instrument that was readily administered to the entire study cohort in the core questionnaire at baseline–
and could be used in future studies– while later adding more detailed measurement data to validate
our primary outcome measure and investigate more detailed patterns and relationships. By combining disciplinary perspectives on the measurement of travel to and from work, we were able to collect
disaggregated data at the level of the trip and, to some extent, the stages of each trip. This enabled us to identify more complex patterns of behaviour such as the use of different combinations of modes of
transport or different‘intensities’ of car commuting, and, therefore, to investigate their relationships with
potential targets of interventions such as the provision and cost of parking at work. We also demonstrated a variety of ways of using qualitative data, separately and in combination with quantitative data, to understand travel behaviour and behaviour change in commuters. For example, we used mixed-method approaches to interrogate initially counterintuitive quantitative data with insights from qualitative data; and we used a variety of qualitative methods, including vignettes constructed from the results of quantitative analyses, to investigate the effects of the busway. Our detailed exploration of the target behaviours and
their relationship with perceived and objective attributes of the environment– particularly in relation to the
route to work– generated valuable original observational evidence in its own right as well as highlighting
the spatial distribution of the target behaviours with respect to the busway, which had an important bearing on our ultimate analytical strategy for the evaluation. We built on the methods of analysis developed in this work to create a general graded measure of intervention exposure with demonstrable face validity that could be adapted to the causal analysis of different outcomes, and an example of how similar principles could be applied in future studies. In our ultimate estimation of the effects of exposure to the intervention, we systematically accounted for a set of confounders representing plausible alternative explanations, thereby providing a considerable increase in rigour over most previous evaluation studies in the field.17,18,132Finally, the study provides a rich observational data set that constitutes a valuable platform
for further analyses, particularly using the intermediate years of the core questionnaire data and the objective measurement data, about which potential collaborators can find out more at our data sharing portal at http://epi-meta.medschl.cam.ac.uk.
Limitations of the study
Our rapid and rolling approach to the recruitment of participants in multiple workplaces meant that we had no defined sampling frame such as a population register, and, therefore, could not compute a
population response rate as such. As is often the case with studies of this kind, our cohort sample included a predominance of women and people educated to degree level or above; although this was partly offset by the more heterogeneous intercept survey sample, it limited the statistical power of the study to detect associations in men and to examine socioeconomic gradients in the effects of the intervention. Our cohort also suffered considerable attrition over time; although this was not surprising, particularly in a city such as Cambridge with a high turnover of professionals, together with the need to extend the study this resulted in a loss of statistical power for comparisons between the first and last years of the cohort study. Although we validated our primary outcome measure, it was far from perfect, and self-reported outcome data are inevitably susceptible to potential bias as a result of intentional or unintentional misreporting. The
multiplicity of ways of summarising travel behaviour change– and the distributions of these outcome
variables, which rendered them unsuitable for linear modelling– meant that our evaluative findings were
expressed in terms of the likelihood of various categorical outcomes, rather than in the neater terms of a
simple‘effect size’ such as an increase in the number of minutes spent cycling per week. We ultimately
defined exposure to the intervention solely in terms of where participants lived. Using more detailed individual measures, for example including both the location of the workplace and the route followed between home and work, might have captured exposure more fully and produced stronger associations with behaviour change, but it might also have involved considerably more work for little additional causal
understanding.14,131,141Our analysis of mediators of the effects of the intervention was limited, both in
terms of the types of mediators considered and in terms of how those were measured. It is possible that if the questionnaire items used to ascertain the cognitive constructs had been framed differently, the analysis would have suggested stronger support for their role as mediators. Finally, owing to circumstances beyond
our control we were unable to assess the maintenance of travel behaviour change after the intervention over a second year of follow-up as originally planned; and because we did not observe an effect of the busway on overall physical activity after 1 year, we did not attempt to model the direct impacts of the busway on the other health outcomes of interest, but instead investigated their longitudinal associations
with active commuting by way of contributing a different piece of the‘evidence jigsaw’.