3.1 Introduction
3.2.2 Generalized resample model averaging
Rather than obtaining a binary estimate of each γj, we instead seek to estimate its
samples from the population. We start by drawing K random weights w1, . . . ,wK
wherewk = (w1k, . . . , wnk) withwik iid
∼Weighting(·) to be dscussed later. Each weighted resample comprises dataD(k)={y,A,D,w
k}. For each weighted resamplek, we apply
a model selection procedure to produce ˆγ(D(k)) = ˆγ(k)
, the m-length binary vector of estimated loci inclusions based on the kth weighted resample. Applying this to K weighted resamples gives the m×K matrix Γ= [ˆγ(1),γˆ(2),· · · ,γˆ(K)]. The proportion of times that the jth predictor is included in the resample based models is given by
\ RMIPj = 1 K K X k=1 ˆ γ(D(k)) j = 1 K K X k=1 ˆ γj(k) = 1 K K X k=1 Γjk, (3.2)
which we refer to as its resample model inclusion probability (RMIP).
Generalized RMA weights
We propose to model wi,k iid
∼ Weighting(·) where Weighting(·) = U(0,1). By doing so, we obtain continuous observation weights that ensure that each observation carries some weight in the model fitting, ensuring that the dimension of the data remains constant on each resample. Under this setting,E(1Tw
k) = 12n, indicating that on average, each
resample uses half of the available data. One may consider the interpretation ofwi,k as
the proportion of observation iused in resample k.
To connect the weights, wk, of the generalized RMA to the subsampling based
RMA framework of Valdar et al. (2012), we consider wk iid
∼ Subsampling(φ), where Subsampling(φ) is a function that returns a n-vector of 0’s and 1’s withφn 1’s. Here, we consider φ = 1/2 so that the amount of data used on each resample is consistent with our proposed weighting method.
Our generalization of the discrete subsampling weights to continuous uniform weights is similar in spirit to how bootstrap weights were generalized to continuous weights in the Bayesian bootstrap (Rubin, 1981). Specifically, one can represent the bootstrap
in the generalized RMA setting by using wk iid
∼ Bootstrap(·), where Bootstrap(·) is a draw from a multinomial distribution withngroups, each equally probable. Under the bootstrap weights,wi,k ∈ {0,1, . . . , n}with1Twk =n. As some weights will be zero on
a given resample, the bootstrap has the same potential issue the data’s dimension may be reduced when excluding subjects that presented under subsampling. The Bayesian bootstrap, a generalization the bootstrap, proposed to modelwk
iid
∼Dirichlet(1), where Dirichlet(1) is a uniform dirichlet distribution. The Bayesian bootstrap obtains similar results to the bootstrap, but would avoids the issues arising from weights of 0 in the bootstrap.
Selection within a resample using the group-LASSO
The group-LASSO estimates β for subsample k via the minimization
ˆ βgrp(λ;D(k)) = argmin βa,βd ( −`(βa,βd;D(k)) +λ m X j=1 q βa,j2+βd,j2 ) , (3.3)
whereβˆgrpT = [βaT,βdT],λ is a penalty parameter, and `(βa,βd;D(k)) is the weighted
log-likelihood of βa, βd, and data D(k) given by,
`(βa,βd;D(k)) =
n
X
i=1
wiklog(f(βa, βd;yi,xi))
wheref(βa, βd;yi,xi) is the Gaussian likelihood. We note that the Gaussian log likeli-
hood may be replaced by the sum of squares for quantitative phenotypes, but we have presented the material in the likelihood form to show the generality of the procedure. Also, we note that when a group consists of a single predictor the group LASSO penalty simplifies to the LASSO penalty, i.e. pβ2 =|β|. Although our model can account for
various types of effects, as the group LASSO estimate requires that each member of the group has either a zero or a nonzero coefficient, we are unable to distinguish which
loci have additive or dominant effects. However, this is not an issue as our main goal is to identify true loci, not their underling effect types. To arrive a single estimate of
γ, as required for model averaging, we must devise a suitable criterion for choosing the penalty λ. We propose to identify a value λ(k) specific to weighted resamplek (ie,
local) by permutation selection.
Discovery-based selection of λ(k): permutation selection
We continue to use the permutation selection criterion described in Chapter 2. Specifi- cally, given a weighted resamplek, we estimate for a given permutation of the response,
π(y(k)), the smallest penalty,λ
0(π, k), required to zero out all predictors. We note that
observation weights (and randomization of penalty, see next section) are fixed under permutations. For each of S permutationsπ1, . . . ,πS, the permutation selection λ for
weighted resamplek is defined to be
ˆ
λ(k)= median(λ0(π1, k), λ0(π2, k), . . . , λ0(πS, k)). (3.4)
Model selection among highly correlated SNPs: the randomized group- LASSO
A new generalization of the LASSO, the randomized LASSO, was presented in Mein- shausen and B¨uhlmann (2010). Whereas the LASSO penalizes the absolute value of the coefficients proportional to the penalty λ, the randomized LASSO changes the penalty of each coefficient to a random value in [λ, λ/α]. The weights of the penalty are reminiscent of the adaptive LASSO (Zou, 2006), but with the perturbation being random rather than based on previous estimates. Applying the randomized LASSO many times (e.g. when used in stability selection or RMA) and considering variables which are often chosen can be rather powerful.
We have found that the randomized LASSO can be quite useful in the RMA frame- work, especially among highly correlated data. The LASSO can produce unstable
estimates when in the presence of highly correlated data, and minor changes in the data may cause the LASSO to switch included predictors. Although the resampling of RMA allows us to see how the LASSO performs over a number of slightly different data realizations, these realizations are conditioned upon the observed sampling (i.e. within sample variability). When considering highly correlated SNPs, their may only be a few subjects for which the SNP values differ. When considering such SNPs, it may not be clear which SNP is clearly the true signal, i.e. an independent second sample may switch the preferred SNP. The additional perturbations of the randomized LASSO may allow for the preferred SNP by the LASSO to switch within the observed resampling. The use of such randomized penalty comes with a tuning parameter which may need calibration specific to the data.
We follow the motivation of Meinshausen and B¨uhlmann (2010) and propose a new generalization of the group LASSO, the randomized group LASSO. Applied to a subsample k, the randomized group LASSO estimatesβ as
ˆ βgrp(λ;D(k)) = argmin βa,βd ( −`(βa,βd;D(k)) +λ m X j=1 1 Uj q βa,j2+βd,j2 ) , (3.5)
whereUj ∼(α,1) is a weighting parameter with α∈[0,1]. Uj randomly down-weights
some predictors relative to others. We note that the the randomized group LASSO can be easily implemented with any group LASSO software by scaling the predictors within each group by the generated weight Uj. The performance of the randomized
group LASSO is dependent on the value of α used. We calibrate the randomization parameter based on simulations, see Results.
When using a randomized selection procedure, one must consider more closely the number of resamples K to run. With the additional variability in the estimation of
in the RMIP. Meinshausen and B¨uhlmann (2010) proposed to generate Ui’s as either
α or 1 with equal probability for the randomized LASSO. We have found that this method of generatingUi’s works well with the randomized group LASSO, and hat the
choice of α is robust with respect to the model choice. Our results suggest that 250 resamples is sufficient when using α = 0.7. This is a 2.5 fold increase to the number of resamples sufficient for RMIPs to converge when using the standard LASSO, i.e. no randomization (Valdar et al., 2012).
3.2.3
Competing methods
Our RMA generalization of LLARRMA calculates a score (an RMIP) for each SNP in the identified hit region. We compare the ability of those scores to discriminate true signals from background SNPs with the SNP scores calculated by two alternatives: the traditional GWAS approach of single locus regression, and LLARRMA (Valdar et al., 2012).
Single locus regression
We perform single locus regression with linear regression as used in, for example, PLINK (Purcell et al., 2007). For each SNP, we fit a single locus dominance model version of Eq 3.1 and score its −log10P (“logP”), where P is the p-value from a likelihood ratio test against an intercept-only model. We also compare with with the additive only single locus regression model.
LLARRMA
We compare the generalized RMA procedure with the original additive only LLAR- RMA procedure. We also use the standard LLARRMA procedure with only some of the generalizations proposed as intermediate steps to our full proposed method, see Terminology used in the paper for considered variations.