Lawley (1942) introduced an EFA model in which both the common factors and the factor loadings are treated as fixed unknown quantities. To fit the EFA model with fixed common factors, Lawley (1942) proposed to maximize the log-likelihood of the data (see also Young, 1941):
71
£ i = - - [log(27r) + lo g (d e t(^ 2)) + trace(Z - F A T) ^ ~ 2(Z - F A T)T] . (4.1)
Instead of maximizing (4.1), one might try to minimize the function
£2 = — £1 + — log(27r) ,
n 2
= 1 [log(det($2)) + trace(Z - F A T) ^ ~ 2(Z - F A T)T] . (4.2)
Anderson and Rubin (1956) showed th a t the fixed EFA model cannot be fitted to the data by the standard maximum likelihood approach as the corresponding log-likelihood loss function (4.2)' to be minimized is unbounded below. Hence, maximum-likelihood estimators do not exist for the fixed EFA model.
Attem pts to find estimators for loadings and factor scores based on the likelihood have persisted (W hittle, 1952; Joreskog, 1962), based partly on the conjecture th a t the loadings for the fixed EFA model would resemble those of the random EFA model (Basilevsky, 1994). McDonald (1979) circumvented the difficulty noted by Anderson
F itting the fixed EFA model 41
and Rubin (1956) in the original treatm ent of the fixed EFA model by Lawley (1942). He proposed to minimize the logarithm of the ratio of the likelihood under the hypothe sized model to the likelihood under the alternative hypothesis th a t the error covariance m atrix is any positive definite matrix:
£ 3 = 1 [log(det(diag(ETE ))) - log(det(ETE))] , (4.3)
where E = Z — F A T. McDonald (1979) showed th a t (4.3) is bounded below by zero, a bound which is attained only if E TE is diagonal. Thus, minimizing (4.3) yields maximum-likelihood-ratio estimators (see also Etezadi-Amoli and McDonald, 1983). Moreover, McDonald (1979) proved th at the likelihood-based estimators of the factor loadings and uniquenesses are the same as in the random EFA model, while estimators of the common factor scores are the same as the arbitrary solutions given by G uttm an (1955) discussed in the previous Chapter.
McDonald (1979) also studied LS fitting of the fixed EFA model. Consider the following objective function to be minimized:
^mcd(F ,A ,® ) = I I ( Z - F At)t( Z - F A t ) - ^ 2| | | . (4.4)
Unlike the log-likelihood loss function (4.2), the LS loss function (4.4) is bounded below (Golub and Van Loan, 1996, p. 605). McDonald (1979) showed th a t the param eter estimates found by minimizing (4.4) can be compared to the standard EFA least squares estimates (with random common factors) obtained by minimizing (e.g. Joreskog, 1977):
F itting the fixed EFA model 42
Indeed, the gradients of .TTs^A, \F) with respect to the unknowns A and \I/ are (for convenience the objective function (4.5) is multiplied by .25):
= - ( Z TZ - A At - ^ 2)A ,
= — [diag(ZTZ — A A t ) — .
McDonald (1979) found th a t the gradients of .Fmcd(A, *&, F) with respect to the un knowns A, \I/ and F can be w ritten as (for convenience the objective function (4.4) is multiplied by .25):
^ v mcd = _ [ ( Z - F A t )t ( Z - F A t ) - ^ 2] ( Z - F A t )t F , V f cD = —diag((Z — F A t )t (Z — F A T) — 1F 2)1F ,
V McD = _ ( Z - F A T) [ ( Z - F A T)T( Z - F A T) - ^ 2]A .
The values of the gradients are then calculated at F = F ^ from (3.27):
V McD = _ ( z TZ — A At — 4>2)(Z - F GAT)TF G = Opxt , = -[diag(ZTZ - A A T) - W 2] ^ = V^s ,
v mcd = _ ( Z - F gAt)(ZtZ - A A t -' ®' 2)A = ( Z - FgAt)V^s .
While calculating the gradients of Tm cd{.A, \If,F) at F = F ^ one simply makes use
of the features F J F g = I* and Zt F g = A. Of course, any other common factors F satisfying these conditions and F TU = Okxp would produce the same results.
Thus, McDonald (1979) established th a t the LS approach for fitting the fixed EFA model gives a minimum of the loss function as well as estimators of the factor loadings and uniquenesses which are the same as the corresponding ones in the random EFA model. The estimators of the common factor scores are the same as those given by the expressions of G uttm an (1955).
Fitting the fixed EFA model 43
A LS procedure for finding the m atrix of common factor scores F is also outlined in Horst (1965). He wrote: “Having given some arbitrary factor loading matrix, whether centroid, multiple group, or principal axis, we may wish to determine th a t factor score m atrix which, when post-multiplied by the transpose of the factor loading matrix, yields a product which is the least squares approximation to the data m atrix. This means th a t the sums of squares of elements of the residual m atrix will be a minimum.” (Horst, 1965, p. 471). Following this strategy, the suggested LS factor score m atrix is sought to minimize
^ „ (F ) = ||Z —F A t |||. , (4.6)
which is simply given by
Ftf = ZA (AtA )“ 1 (4.7)
for an arbitrary factor loading m atrix A (Horst, 1965, p. 479).
Horst (1965) also proposed a rank reduction algorithm for factoring a d ata m atrix Z. For some starting approximation Ao of the factor loadings, let Lq be the k x k lower triangular m atrix obtained from the Cholesky decomposition (e.g., Golub and Van Loan, 1996) L0Lq of A g Z TZA0. Then, the successive approximation A i of the factor loadings is found as (Horst, 1965, p. 274):
= ZtZA0(Lq )-1 .
It follows from
ZTZ - AjAj" = ZTZ - ZTZA0(Aj'ZTZAo)“:lA jZ TZ ,
th a t the successive approximation A] is always a rank reducing m atrix for Z Z. After convergence of the algorithm, the final A found is the m atrix of factor loadings. The
Fitting the fixed EFA model 44
common factor scores are obtained as F = ZAdiag(ATZTZ A )“ _1/2, i.e. F is an oblique m atrix with diag(FTF) = I*.
Both algorithms in Horst (1965) find a pair {A, F}. No care is taken to obtain unique factor scores U or the uniquenesses tP. In this sense, the proposed procedures resemble PC A rather more than EFA.