Bayesian methods

Bayesian variable selection

We fit standard regression models to the data. We additionally consider Bayesian variable selection, which is useful in this context because it accounts for the pos- sibility of non-linearity and interactions. This framework provides a more rigorous test of the importance of a covariate because a larger number of possible alternative explanations are considered, including interaction effects that are sometimes key (e.g. in Gelmanet al., 2007) and yet are often overlooked.

The models Mγ for seatbelt use that we consider are a particular form of the

multinomial-Dirichlet Bayesian variable selection introduced in Section 2.4.1. The models describe the distribution of seatbelt use Y in terms of a collection of po- tential predictors C1, . . . , Cp−1 and well-being X, which we refer to collectively as

the covariates C1, . . . , Cp for simplicity. Recall that the models are defined by an

indicator variable γ = (γ1, . . . , γp), so that {Ci: γi = 1} is the subset of covariates

included in model Mγ. We let the number of covariates pγ = ITγ included in the

containing all qγ = Qpi=1riγi combinations of values of the covariates included in

the model. Note that this is equivalent to defining C as the sample space of the included covariates. To control complexity in this setting, we simplify the data by reducing the levels of some variables with many categories, as shown in Tables 3.1 and 3.2 (on pages 59 and 71), and binarise the response, enabling a simple contrast between those who always wear seatbelts with those who do not. For each of the nindividuals, letyi be the indicator of whether individual ialways uses a seatbelt,

and ci be the p-dimensional vector of covariates that incorporates an indicator of

well-being. We use a binomial model for the responses, with parameterθj depen-

dent on the configuration C_jγ ∈ C of the covariates. Thus the joint probability for vector of responsesy depends onnj, the number of observed individuals who have

covariatesC_jγ, and mj, the number of these individuals who use a seatbelt.

The posterior distribution over models Mγ, given the data, provides a measure of

the fit of each model that incorporates a preference for simpler models of lower dimension. The posterior, up to proportionality, is given by the product of the model priorπ(Mγ), and, using the standard assumption of independent beta(α, β)

parameter priors (Cooper and Herskovits, 1992), the closed-form marginal likelihood

p(y|c1, . . . ,cp, Mγ) = qγ Y j=1 Γ(mj+α)Γ(nj−mj+β)Γ(α+β) Γ(nj +α+β)Γ(α)Γ(β) ,

wherec1, . . . ,cp are the vectors of observations of the covariates.

As usual and following previous authors (Heckermanet al., 1995), we set the hyper- parametersα=β = (2qγ)−1 for eachθj. We choose a flat priorπ(Mγ)∝1, but the

large sample results in insensitivity to this choice. Penalised likelihood approaches offer an alternative to the Bayesian approach taken here: indeed, here we find that a BIC-based analysis (withpγ≤5, for computational reasons) in this setting selected

C1 C2 . . . Cp−1

Cp =X Y

Figure 3.2: The form of the model used in model selection for joint confounding by multiple factors. A graphical representation of family of models for considering the influence of conjectured explanatory variable X on response Y with potential observed confounders C1, . . . , Cp−1. A model selection approach is used to explore

evidence in favour of a direct link fromX toY in light of subsets of{C1, . . . , Cp−1}

which may jointly explain bothX and Y.

Joint confounding

An alternative to regression approaches, which model risk-taking behaviour con- ditional on the observed covariates and life-satisfaction, is additionally to model life-satisfaction conditional on the observed covariates (Robins et al., 1992; Senn

et al., 2007). This approach has the advantage of explicitly modelling the unbal-

anced distribution of subjective well-being among individuals, for which we must account to compare meaningfully how seatbelt-use varies with life-satisfaction. We can restore balance by identifying covariates that explain both subjective well-being and seatbelt use, and examining the effect of life-satisfaction within particular values of these covariates.

We take a model selection approach to discovering such covariates (Robins and Greenland, 1986) that is similar to Bayesian variable selection, but as shown in Figure 3.2 we now mirror dependences between covariates Ci and seatbelt use (Y)

with corresponding direct dependences, fori≤p−1, betweenCiand subjective well-

being (X). This approach can be thought of as exploring different stratifications for a model of the effect ofXonY. Any residual relationship after stratification between subjective well-being and seatbelt use represents the controlled effect (Rosenbaum, 2002). The approach taken here can also be regarded as a special case of structural

inference in Bayesian networks (Heckerman et al., 1995; Madigan and York, 1995; Mukherjee and Speed, 2008).

Each model M_γ,δ is defined by a set of confounders (a subset of the covariates C1, . . . , Cp−1 defined by γ, excluding subjective well-being X, and with pγ ≤ 9

for computational tractability) and an indicator variable δ for whether the direct dependence betweenX andY is present. We redefineCto be the set containing all combinations of values of the confounders alone (i.e. excluding subjective well-being) inMγ,δ, and, with qγ0 =qγrδp, denote byD ={D1γ, . . . , D

γ q0

γ}, the corresponding set

including subjective well-being. We denote the number of observed individuals with confounding variablesC_jγ∈ C by wj, and number of these individuals who are ‘very

satisfied’ by vj. Similarly defining nl to be number of observed individuals with

covariates Dγ_l ∈ D and the number of these who always use a seatbelt by ml, we

have the following marginal likelihood for seatbelt use y, subjective well-being x, and confoundersc1, . . . ,cp−1. p(y,x|c1, . . . ,cp−1, Mγ,δ) = q0 Y l=1 Γ(ml+α)Γ(nl−ml+β)Γ(α+β) Γ(nl+α+β)Γ(α)Γ(β) × q Y j=1 Γ(vj +α)Γ(wj−vj+β)Γ(α+β) Γ(wj+α+β)Γ(α)Γ(β)

We again choose beta priors for α, β, with α = β = (2qγ)−1 for X, and α = β =

(2q_γ0)−1 for Y. Note that the result of adding extra dependencies is simply an additional term in the marginal likelihood, and so the computation time is identical to variable selection.

Proportion of individuals In cre asi ng SW B Very satisfied Satisfied Dissatisfied Very Dissatisfied 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

Always Nearly always Sometimes Seldom Never

Seatbelt use:

Sample size n=3295

n=13524 n=152021 n=144514

Figure 3.3: Frequency of seatbelt use cross-tabulated by subjective well-being (SWB). Each category contains at least 101 individuals. Pearson’s chi-squared statistic is 3242 (p-valuep <2.2×10−16).