Computation via GVEM

To update A matrix, we use coordinate descent algorithm developed by Friedman et al. (2010). The general estimation procedure as follows; The regularization parameter is chosen as follows: For each λ we first obtain the estimate ( ˆAλ, ˆBλ, ˆCλ) via (III.6). Then we obtain

from a zero structure, ˆQλ matrix from ˆAλ. We fit a MIRT model with ˆQλ as zero structure

for A using GVEM algorithm without penalty. Hence, the final estimate ˆA satisfies Q( ˆA) = ˆ

Qλ. We compute the information criteria using estimates from a MIRT model without

penalization. The regularization parameter is chosen to be the one admitting the smallest information criterion values.

For each item j, there are one difficulty parameter bj and K discrimination parameters αj.

to the following updating rule. See appendix for detailed derivation of the updating rule. Define a function S to be a soft threshold operator such that S(δ, λ) = sign(δ)(|δ| − λ)+

we update model parameters bj and cj following (III.7) and (III.8), respectively. For

M3PL, we penalize αj’s with adaptive lasso penalty as well. Then ˆαjk is updated according

to the (III.9). See Appendix for the detailed derivation.

b(t+1)_j = PN i=1(1 − Yij + sijYij) h 1 2 − Yij + 2η(ξi,j)α > jµi i + µb σ2 b 2PN i=1(1 − Yij + sijYij)η(ξi,j) + _σ12 b , (III.7) c(t+1)_j = PN i=1Yij(1 − sij) + α − 1 N + α + β − 2 . (III.8) and α(t+1)_jk = " _N X i=1 (1 − Yij+ sijYij) 2η(ξi,j)[Σi+ (µi)(µi)>]k,k !#−1 × S N X i=1 (1 − Yij + sijYij)  (Yij − 1

2)µi,k + 2bjη(ξi,j)µi,k − 2η(ξi,j)

X l6=k αjl[Σi+ (µi)(µi)>]l,k  , λ ! (III.9)

where λ is the sparsity parameter of choice.

The detailed algorithm of the regularized estimation of the test structure via Adaptive Lasso penalization is illustrated below in Algorithm 3.

Algorithm 3 Regularization with Adaptive Lasso Penalization

1: Set a range of λ. Choose γ > 0.

2: Initialize model parameters A₀, B₀, Σ₀ such that the identifiability condition holds. 3: Run confirmatory factor analysis (CFA) to obtain ˆA_w := [ ˆα_jk]_{J ×K}

4: for each λ starting from smallest do

5: Set the adaptive penalization weights, λ

| ˆAw(j,k)|γ for j = 1, . . . , J and k = 1, . . . , K. 6: Update A according to (III.9) using the adaptive weight as a sparsity parameter.

Update B, C according to (III.7), (III.8). Update Σθ as in regular GVEM estimation.

7: Estimate AICˆ ?, BICˆ ?, and GICˆ ? with recent updates. 8: Set ˆAλ, ˆBλ as the initial values for next step.

9: end for

Remark III.1. Here, we present the Algorithm 4 that summarizes the detailed regularized

estimation procedure with Lasso regularization. It is to illustrate the slight difference between Lasso and Adaptive Lasso penalization approaches. After regularization, we re-estimate pa- rameters in the Algorithm 4 step 7 to correct for the biased estimation of Lasso. Although this re-estimation might help correct the biasedness, it does not guarantee the consistent es- timation after all. In addition, Lasso estimation requires more computation time due to this additional step. However, this difference in the estimation procedure is minor and we can easily check that the Adaptive Lasso is a simple extension of Lasso. Hence, it is easy to implement the adaptive lasso penalization using the existing numerical algorithms for Lasso.

Algorithm 4 Regularization with Lasso Penalization

1: Set a range of λ.

2: Initialize model parameters A₀, B₀, Σ₀ such that the identifiability condition holds.

3: for each λ starting from smallest do

4: Update A, B, C according to (III.9), (III.7), (III.8). Update Σ_θ as in regular GVEM estimation.

5: Re-estimate A, B, and Σ_θ according to confirmatory factor analysis with most recent updates (i.e. ˆAλ, ˆBλ) as initial values.

6: With re-estimated A, B, and Σ_θ, estimate AICˆ ?, BICˆ ?, and GICˆ ?. 7: Set ˆAλ, ˆBλ as the initial values for next step.

8: end for

9: Find λ∗ that minimizes the information criteria. Calculate the correct estimation rate of ˆAλ∗.

Remark III.2. In addition to our choice of Adaptive Lasso for Pλ(A) in Eqn (III.6), there

are generally other methods of penalizations. For instance, Fan and Li (2001) showed that the Lasso penalization problem is suboptimal to their proposed method called smoothly clipped ab- solute deviation (SCAD) penalty as Lasso produces biased estimates for the large coefficients. They showed that the SCAD penalization enjoys asymptotic normality and oracle properties with proper choice of regularization parameters. Due to its solid theoretical properties, SCAD has been widely applied in variable selection problems (T. Wang, Xu, & Zhu, 2012; Liu, Yao, & Li, 2016; Breheny & Huang, 2011). Additionally, Minimax Concave Penalty (MCP) has

been presented as a fast, continuous and nearly unbiased method of penalization and hence claimed to be a good alternative to Lasso (C. H. Zhang, 2010). Truncated lasso is also another popular penalization method; however penalty function for these methods are non- convex and it makes local solutions to be nonunique in general. The nonconvex optimization problem is computationally challenging to solve as well. On the other hand, Adaptive Lasso is a convex problem and it is also computationally efficient, which makes it a good candidate for regularization problem under complex MIRT models. Hence, we chose Adaptive Lasso for solving our regularized problem.