Solutions to selection bias without using experimental data

So, if the best quality of study design is unavailable due to the difficulties of randomisation, what can be done to improve the reliability of longitudinal studies with observational data?

Fixed-effects help to control for selection bias fitting a dummy variable for every individual within the model. Modelling neighbourhoods in this way removes selection bias, but when working with very large datasets and hundreds of thousands of geographical areas, it also creates other problems (e.g. loss of model power through using up degrees of freedom) (Allison, 2005).

Sibling studies (i.e. surveying twins to control for selection bias) might be able to control for some unobserved bias as both siblings would have grown up in the same conditions (e.g. parenting). However, data of this kind is not available for studies of ethnic inequalities in social mobility at the neighbourhood scale in England (Aaronson, 1997).

Instrumental variables is a technique used in a model that involves fitting a variable which is significantly associated with the ‘treatment’ (e.g. neighbourhood deprivation), but not the dependent variable. This condition allows the instrumental variable to remove all selection bias

from the treatment effect, making the study design quasi-random. However, instrumental variables are difficult to find and even those which work in models may be difficult to justify theoretically (Cutler et al., 2008b). Difference models may also help, which involve measuring differences between two time periods, removing unobserved time-invariant characteristics (Galster et al., 2008). Solutions to selection bias for analyses in this thesis will be discussed further in the Methodology chapter.

Some recent studies have made use of ‘natural experiments’ (Bolster et al., 2007, Oreopoulos, 2003, van Ham and Manley, 2010). These studies made innovative use of existing longitudinal data that included smaller populations in social housing who, at least in theory, had reduced or no choice over the neighbourhood in which they were allocated. The general methodological approach was to test for influences of neighbourhood characteristics separately for individuals in private-tenure (who can choose their neighbourhood which creates selection bias) and compare the results with another model for individuals in social housing (with little or no choice of neighbourhood).

Oreopoulos (2003) used longitudinal data in Canada to analyse earnings, employment and welfare claims. Oreopoulos reported significantly positive associations with living in a more affluent neighbourhood. However, this was only for those in private-tenure, with no significant “neighbourhood effects” found for those in social housing. For Oreopoulos, the logical conclusion was that the influences of neighbourhood characteristics on the private-tenure study participants was not real – it was selection bias (Oreopoulos, 2003).

In the UK, Bolster et al (2007) investigated the effect of neighbourhood disadvantage on earnings over the space of 10 years in the British Household Panel Survey. No evidence was found of a negative influence of neighbourhood disadvantage on earnings. Though counter-intuitively, positive association was found for couples and for home owners. No significant association at all

was found for renters, though it is important to note that Bolster et al did not attempt to explain these differences in a similar way to Oreopoulos.

Most recently, Van Ham and Manley (2009) used longitudinal data in Scotland to explore transitions from unemployment to employment (and employment to unemployment). They were interested in the extent that housing tenure mix and socioeconomic deprivation of the neighbourhoods in which individuals lived in 1991 would influence changes in employment status by 2001. They found that deprivation was a more important predictor of labour market outcomes; tenure mix was not significantly associated with change in employment status. Further, calculating separate models for individuals in private-tenure and social housing demonstrated only significant association with deprivation for the former group. In other words, Van Ham and Manley found that neighbourhoods only appeared to influence the group that could relatively freely choose where to live and, like Oreopoulos (2003), they concluded that these associations were the result of selection bias, but not a real effect (van Ham and Manley, 2010).

The wider conclusion for this thesis on how to cope without MTO-style experimental data is that

quasi-experimental settings may offer a readily-accessible alternative in some of the largest longitudinal datasets available. A particular advantage of longitudinal data is that I am able to model and compare the theorised impact of selection bias upon results, though a disadvantage is that the findings may sometimes only be generalisable to the selected group (e.g. individuals in social housing). The viability of such an approach, however, does rely upon the degree to which individuals in social housing are able to choose their neighbourhood, which was not a problem for the Scotland-based study but may be in other settings (van Ham and Manley, 2010). In other places, social-housing may involve an element of choice, which would reduce the feasibility of the quasi-experimental study design.

In this section of the literature review, it has become clear that many studies have demonstrated impressive effort in careful design, innovative thinking and statistical application in the use of both observational and experimental data resources. But there remains considerable uncertainty over whether we can a) measure ‘neighbourhood’ and b) identify independent causal effects of neighbourhood characteristics (if they exist).

Longitudinal design is a solution as it helps us to observe cause before effect. In many longitudinal studies of observational data, neighbourhood characteristics have been shown to have association with social mobility. But even longitudinal studies are limited by the problem of selection bias in observational data. This problem was solved to an extent by MTO, but this solution was not perfect (MTO could not force people to remain in their neighbourhoods, so only those who chose to be part of MTO were included. This means MTO is a selected population which influences all potential outcomes, including the likelihood that people would selectively move out of their ‘treatment’ neighbourhoods before any effect could occur). Innovative use of existing longitudinal observational data has provided interesting avenues for taking research forward, producing alternative findings and extending the debate on whether some neighbourhood characteristics really matter. These ideas are still in developmental stages and require further testing. But there are other major unresolved issues that continue to cast a cloud over all research on neighbourhood effects.

From this literature review, it is clear that there are some problems that can be avoided (e.g. avoiding reverse causality by using longitudinal data). But there are a lot of other problems with measuring and identifying potential influences of neighbourhood characteristics which have not been solved. My research, as others, is limited in the respect of finding solutions to these problems and will use the most appropriate data and method available to investigate the research questions: longitudinal data, making adjustment for selection bias through examining quasi- experimental population groups if possible.

In knowledge of the evidence for neighbourhood influences on social mobility, and in mind of the challenges that face studies of this type, I examine the literature that looks specifically at ethnic inequalities of social mobility and potential influences of neighbourhood characteristics.

2.5

Review of the studies that investigate the relationship between