Consider vectorsxi = (xi1,xi2, . . . ,xip) >
∈ p, observed fori = 1, . . . ,nindividuals.
The factor regression model definesxias a regression onpv observed covariates denoted byvi ∈ pv, andq low-dimensional latent variables denoted zi ∈ q, also known as latent coordinates or factors. The standard factor regression model is
xi =θvi+Mzi+ei, (3.1) whereθ ∈ p×pv is the matrix of regression coefficients, andM,zi andei are as in the
standard FA model (2.1) .
Equation (3.1) regresses the observed dataX on known covariates and on a latent factor structure. In particular, it allows additive batch effects to be accounted for by in- corporating the variables recording the batches intovi. However, in practice one often observes more complex batch effects; for example in bioinformatics it is common to ob- serve multiplicative effects on the variance (Johnson et al., 2007). We will later describe an example of this, shown in Figure 30. Such artefacts cannot be captured by (3.1) given thatΣis assumed constant across all individuals.
To address this issue we extend (3.1) by allowingΣto depend oni. Suppose the data were obtained inpbbatches, e.g. from different days, laboratories or instrumental calibra- tions, withnl individuals in batchl, forl = 1, . . . ,pb, such thatn1+n2+· · ·+npb =n. Letbi be the indicator vector of lengthpb defined asbil := 1if individualiis in batchl,
bil :=0otherwise.
We incorporate batch effects by adding a mean and variance adjustment. We let xi =θvi+Mzi+βbi+ei, (3.2) whereθ, vi, M and zi are as (3.1), β ∈ p×pb captures additive batch effects and the
variance ofei captures multiplicative batch effects. We denote byτjl,j = 1, . . . ,p and
l =1, . . . ,pb as thejthidiosyncratic precision element in batchl. Then, givenbil =1, the
errors are independently distributed aseij ∼ N(0,τjl−1). Further, denote byT thep×pb matrix that hasτjl as its(j,l)element.
To help interpret the practical implications of the model, suppose that one has or- thonormal factor loadingsM>M =
I. Then (3.2) implies
zi =M>(xi− (θvi+βbi+ei)) (3.3) and thus,E(zi | xi,vi,bi,M,θ,β) = M>xi −M>θvi −M>βbi. That is, the mean of the latent coordinates is the projectionM>
xi plus a translation given by the batch effect ad- justment and (potentially) the observed covariates. An interesting observation is that their covariance Cov(zi | xi,vi,bi,M,θ,β,T )=M>Tb−1
i M depends on the multiplicative
batch-dependent noise. As an example, Figure 30(b) show the two first factors of an ovar- ian dataset pre-processed by ComBat. Relative to the unadjusted Figure 30(a), ComBat removes systematic differences in mean and variance accross the 2 batches, however the latent coordinates exhibit distinct covariances. To obtain suitably-adjusted low-dimension coordinates one should estimateT jointly with(M,θ,β).
Model (3.2) can be represented in matrix notation as
X =Vθ>+ZM>+Bβ>+E, (3.4)
whereE ∈n×p is the matrix of errors.
As mentioned in Section 2.2, the latent factor model is non-identifiable up to orthog- onal transformations, of the formM∗> =A>M>andZ∗=ZA, whereAis any orthogonal q×qmatrix. Through this chapter we follow the same strategy as in the previous chapter, inducing sparse solutions via local and non-local penalties.
3.3
Prior formulation
To complete Model (3.2) we set priors for the loadingsM, precisionsτjl, and regression parameters(θ,β). Through our proposed default prior formulation we assume that the columns inX have been centred to zero mean and unit variance. For the idiosyncratic precisionsτjl we set
xi bi vi zi M (θj,βj) τjl µ(θ,β) ψ(θ,β) η ξ n pb p
(a) Flat prior
xi bi vi zi (θj,βj) τjl mjk γ·k ζk aζ bζ µ(θ,β) ψ(θ,β) n q λ0 λ1 η ξ pb p (b) Spike-and-slab prior Figure 21. Directed acyclic graph (DAG) for Bayesian factor regression with Batch Effect correction for different prior formulation: (a) Flat or non-sparse loading matrix. (b) Spike-and-slab or sparse loading matrix.
independently acrossj=1, . . . ,pandl =1, . . . ,pb. By default in our examples we set the
fairly informative valuesη=ξ =1, leading to diffuse though proper priors.
For the regression parameters we set
(θj,βj) ∼N(0,ψI), j =1, . . . ,p (3.6)
whereψis a user-defined prior dispersion that in our examples by default we set toψ =1.
The choice ofψ = 1assigns the same marginal prior variances to elements in(θj,βj)as the unit information prior often adopted as a default for linear regression (Schwarz, 1978). We remark that this prior does not encourage sparsity in the regression parameters (θ,β), which we view as reasonable provided the number of variablespv and batchespb are moderate. For largepv orpb, a direct extension of our prior on the loadingsM could be adopted.
As shown in the Chapter 2, sparsity in the loadings matrixM plays a crucial role to ease interpretation. In this chapter we study the same five prior formulations:
• Flat: Equation (2.10);
• Normal-spike-and-slab (Normal-SS): Equation (2.21); • Laplace-spike-and-slab (Laplace-SS): Equation (2.31); • Normal-spike-and-MOM-slab (MOM-SS): Equation (2.36);
We finish the prior specification with a hierarchical prior over the latent indicator
γ = {γjk,j = 1, . . . ,p,k = 1, . . . ,q}as in (2.20) (Chapter 2) for all the spike-and-slab priors.
Figure 21 provides the DAG for the Model 3.2 for Flat and spike-and-slab priors (SS) for the loadings.