Initialization and first run of the EM algorithm

3.4. Computational details 55

can be achieved in case the density to be approximated is available. Therefore, it serves as a starting point on how to come up with initial values in case of a sample of observations. A priori, it is however not clear how to translate the property to a finite sample setting.

Initializing data In a finite sample setting, we do not have the underlying density function at our disposal and initialize the parameters making use of an initializing data matrixyof dimensionn×dwhich containsxijif thejth element of observation i is uncensored, lij in the case of right censoring, uij in the case of left censoring, and (lij +uij)/2 in case of interval censoring. Hence, we use popular simple imputation techniques (see e.g. Leung et al., 1997) to deal with the censoring in the initial step. If thejth element of observationiis missing or right censored at 0, we setyij equal to missing.

Shapes For any given initial common scale θ(0), instead of using an infinite set of positive integer shape parameters in each dimension (cfr. Property 1), we restrict this to a maximum numberM of shape parameters in each dimension. We select these shape parameters in a sensible way by usingM quantiles ranging from the minimum to the maximum in each dimension in order to make a data-driven decision on the locations of the shape parameters. Denoting thep-percent quantile of the initializing data in dimensionj by Q(p;yj), and taking into account that the expected value of a univariate Erlang distribution with shaper and scale θ equalsrθ, the set of positive integer shapes in dimensionj is chosen as

{r1,j, . . . , rMj,j}= Q(p;y_j) θ(0) p= 0, 1 M −1, 2 M −1, . . . ,1 . (3.17) where d e denotes upwards rounding, due to the fact that the shapes have to be positive integers. Consequently, several shapes might coincide which results in Mj ₆M shape parameters in dimensionj. The initial shape set is then con- structed as the Cartesian product of thedsets of positive integer shape parameters in each dimension:

R={r1,1, . . . , rM1,1} × · · · × {r1,d, . . . , rMd,d}. (3.18)

Weights The shape parameters in each dimension, multiplied with the com- mon scale parameterθ(0)_{, form a grid that covers the sample range. As an em-}

r = (rm₁,1, . . . , rmd,d) in R, with 1 6 mj 6 Mj for all j = 1, . . . , d, are ini-

tialized by the relative frequency of data points in the d-dimensional rectangle (rm₁−1,1θ(0), rm1,1θ

(0)_{]× · · · ×(rm}

d−1,dθ

(0)_{, rm}

d,dθ

(0)_{] defined by the grid:}

α(0)_r=(r m1,1,...,rmd,d)=n −1 n X i=1 d Y j=1 Irmj−1,jθ (0)_{< y} ij 6rmj,jθ (0)_{, (3.19)}

withr0,j = 0 for notational convenience and the indicator equal to 1/Mj in case yij is missing. If this hyperrectangle does not contain any data points, the initial weight corresponding to the multivariate Erlang in the mixture with that shape vector will be set equal to zero. Consequently, the weight will remain zero at each subsequent iteration of the EM algorithm (see formulas (3.12) and (3.14)). Therefore, these shape vectors can immediately be removed from the setR. At initialization, the truncation is only taken into account to transform the initial values forα into the initial values forβvia (3.7).

The maximal number of shape vectors is limited to Md _{at the initial step.} However, due to the fact thatMj ₆M and many shape parameter vectors will receive an initial weight equal to zero, the actual number of shape vectors at the initial step will be lower.

Common scale The initial value of the common scaleθis the most influential for the performance of the initial multivariate Erlang mixture, as is the case in the univariate setting (Verbelen et al., 2015). A value which is too large will result in a multivariate mixture which is too flat (‘underfit’); a value which is too small will lead to a mixture which is too peaky (‘overfit’). A priori, it is not evident how one can make an insightful decision onθ. Similar to Verbelen et al. (2015), we therefore introduce an additional tuning parameter: an integer spread factor s. We propose to initialize the common scale as

θ(0) =minj(maxi(yij))

s . (3.20)

Due to the use of marginal quantiles in (3.17) , the range of the shape parameters varies according to the sample ranges in each dimension j = 1, . . . , d with a maximum shape parameter equal to

rMj,j = _max i(yij) θ(0) = _max i(yij) minj(maxi(yij))s . (3.21)

3.4. Computational details 57

Hence, the spread factor s determines the maximum shape parameter in the dimension with the smallest maximum. The fact that the common scale parameter is equal across all dimensions is compensated by the different choice of the shape parameters in each dimension based on marginal quantiles. This ensures that the initialization works well when the ranges in each dimension are different and also gives reasonable initial approximations in case the data are skewed.

Apply EM algorithm Given an initial choice for the setRof shape parameter vectors, the initial common scale estimate θ(0) _{and the initial weights β}(0)

=

{βr(0)|r∈ R}, we find the maximum likelihood estimators for (β, θ) corresponding to this initial multivariate mixtures of Erlangs, denoted by MMEinit, via the EM algorithm as explained in section 3.3.3. An overview of the initialization and the EM algorithm written in pseudo code is given in Algorithm 1.

Algorithm 1EM algorithm for a multivariate Erlang mixture.

{Initial step}

ChooseM ands Compute:

θ as in (3.20)

shape parameters in each dimension as in (3.17) shape set Ras in (3.18)

mixture weights αas in (3.19)

R ← {r∈ R |αr6= 0}

Transform weightsα toβas in (3.7)

{EM algorithm}

whilelog-likelihood (3.6) improvesdo

{E-step}

Compute: posterior probabilities (3.12) conditional expectations (3.13)

{M-step}

Update: weightsβas in (3.14)

scaleθ by numerically solving (3.15) end while

Transform weightsβ toαusing (3.16) return MMEinit= (R,α,β, θ)