Simple processes, like the power method, require, in principle, an infinite number of expensive matrix-vector products to converge to an eigenvector. The method of minimized iterations, proposed by Lanczos in 1950, expands each eigenvector in a converged series with at most n terms. For eigenvectors belonging to extreme eigenvalues the convergence is usually very quick. However, Lanczos’
method was first used only as a process to tridiagonalize a symmetric matrix.
To compete in accuracy with the Givens and Householder method, the Lanczos process has to be supplemented with the explicit orthogonalization of the
Lanczos vectors which, in exact arithmetic, would be orthogonal automatically.
One advantage is that the only way the matrix
A enters the Lanczos algorithm is through a subprogram which computes a matrix-vector product.In the following we present a Krylov-Lanczos method in exact arithmetic.
The use of finite precision arithmetic provokes significant departures from the exact version, especially the loss of orthogonality among the Lanczos vectors. Left to itself, a simple Lanczos program will run forever, finding more and more copies of the outer eigenvalues for each new inner eigenvalue it discovers. So it has to be stopped after a number of steps, then restarted with a new starting vector [8.24].
One simplification with respect to the Arnoldi vectors is that each Lanczos vector is made orthogonal to the previous two Lanczos vectors, and this makes the present Lanczos vector (theoretically) orthogonal to all prior vectors.
The following algorithm presents the Lanczos method (in exact arithmetic) with emphasis on the physical meaning of Lanczos vectors when the starting vector is load dependent [8.25]. While closely related to the powerful and popular Lanczos eigensolvers, the procedure discussed here does not employ an eigenvector subspace and avoids the computational expense of reorthogonalization.
The starting vector
r 1 can be the static deflection of the structure due to the load distribution vector
f , given by
k
r 1 f . (8.75) This vector is mass normalized to form the first Lanczos vector} 1 { }
{ 1
1
1 r
v (8.76)
where the normalizing factor is
1
11 r T m r
. (8.77)
The second Lanczos vector is obtained by first solving for the static deflection
r 2 of the structure subjected to inertia loading due to the first vector deflection
k r 2 m v 1. (8.78) Then, the Gram-Schmidt orthogonalization is used to remove the starting vector component
v of this iterate 1
r 2 r 21
v 1, (8.79) where
1
21 v T m r
(8.80)
is the amplitude of
v along 1
r 2.Finally, the vector
r 2 is mass normalized to form the second Lanczos vector} 1 { }
{ 2
2
2 r
v , (8.81)
where the normalizing factor is
2
22 r T m r
. (8.82)
The general Lanczos vector,
v , j j3,4,..., is obtained by the following steps:First, solve the equation
k
r j1
m
v j (8.83) for the static deflection
r j1.Use the Gram-Schmidt procedure to remove both the
v component j and the
v j1 component of this iterate
r j1
r j1 j
v jj
v j1 (8.84) where
1 Tj j
j v m r
(8.85)
and j
v Tj1
m
r j1 (8.86)which can be shown to be just the preceding normalizing factor.
Finally, mass normalize the vector
r j1 to form the
j1
st Lanczos vector} 1 { }
{ 1
1
1
j
j
j r
v , (8.87)
where the normalizing factor is
1
11
Tj j
j r m r
. (8.88)
Let
Vm contain the first m Lanczos vectors as columns
Vm
v 1, v 2,...,
v m
, (8.89) and let the corresponding tridiagonal matrix containing the coefficients i and i be It can be shown that the differential equation of motion
m
x
k
x f , (8.93) where
f is the column vector of external forcing, can be reduced using a Ritz-type coordinate transformation
x
Vm
xm , (8.94)to the following form
Tm
xm
Im
xm
1 0 0
T. (8.95)Note that the transformed mass matrix is the tridiagonal matrix of orthogonalization coefficients, while the transformed stiffness matrix is the identity matrix. Also note that the only nonzero forcing term acts on the first Lanczos coordinate. The remaining coordinates are only coupled through the off-diagonal terms of the transformed mass matrix in equation (8.95).
There are several improvements of the basic Lanczos method such as the shifted Lanczos, the block Lanczos, the two-sided Lanczos, and the implicitly restarted Lanczos algorithms [8.12], [8.23]. Their presentation is beyond the aim of this lecture course.
8.7. Software
Numerous codes for solving eigenproblems can be found in the software repository Netlib on the Internet at http://www.netlib.org/. They are also available by anonymous ftp.
Reliable high quality software for linear algebra was first published in the book edited by Wilkinson and Reinsch [8.26] as Algol 60 subroutines. In the early 1970s most of these were transcripted in FORTRAN and included in the software package EISPACK [8.27] and later in NAG, IMSL and MATLAB packages. The Jacobi method was not included in EISPACK and at that time the Lanczos and Arnoldi methods were not even considered as candidates.
EISPACK was superseded in 1995 by LAPACK [8.6]. The authors of LAPACK developed new routines and restructured the EISPACK software to achieve much greater efficiency, where possible, on modern high-performance computers. This was accomplished by writing routines that call all three levels of the BLAS (Basic Linear Algebra Subprograms) [8.28].
In 1970 there were few robust and well-understood iterative methods available, and mainly for this reason, these methods were not included in the packages constructed then. Since 1998 MATLAB has had iterative methods for eigenproblems available and the eigs.m function [8.16] was available in source code.
Improvements made to the Arnoldi method, in particular the implicit restart technique [8.17], lead to the ARPACK software [8.21] which seems to be the default choice for large sparse eigenproblems. ARPACK makes extensive use of BLAS and LAPACK.
MATLAB files that implement the Lanczos method for finding eigenvalues of a symmetric matrix, written by J. Demmel, are available on the Internet for the applications in his book [8.29]. The LANSEL eigenpackage based on the Lanczos algorithm with selective orthogonalization is presented in [8.30].
The subspace iteration code SRRIT [8.31] computes an orthonormal basis for the invariant subspace corresponding to the eigenvalues of largest modulus.
The code LOPSI [8.32] uses a subspace iteration combined with a lopsided oblique projection to compute the eigenvalues of largest modulus together with the corresponding eigenvectors.
The JDQZ and JDQR algorithms, described in the paper [8.33], have MATLAB 5.1 and FORTRAN 77 implementations available on the Internet. They are based on the Jacobi-Davidson method [8.34]. This is an iterative subspace method incorporating an effective restart strategy for computing one or more eigenvalues and eigenvectors of an eigenproblem. The MATLAB implementation is based on algorithms presented in [8.23].
A black-box implementation of the inverse free preconditioned Krylov subspace method [8.35] is the eigifp.m MATLAB program that computes a few (algebraically) smallest or largest eigenvalues of large symmetric matrices.
irbleigs.m [8.36] is a MATLAB program for computing a few eigenvalues and associated eigenvectors of a sparse Hermitian matrix of large order. This program implements a restarted block-Lanczos method with judiciously chosen acceleration polynomials. ahbeigs.m is for non-symmetric matrices.
A unified overview of theory, algorithms, and practical software for engineering eigenvalue problems is presented in the book [8.23]. Numerical recipes and “black box” methods are given for Hermitian and non-Hermitian eigenvalue problems, generalized Hermitian and non-Hermitian eigenvalue problems and nonlinear eigenvalue problems.
References
8.1 Van der Vorst, H.A., Computational Methods for Large Eigenvalue Problems, Springer, Berlin, 1988.
8.2 Golub, G.H. and Van der Vorst, H.A., Eigenvalue computation in the 20th century, Journal of Computational and Applied Mathematics, vol.123, no.1-2, pp 35-65, Nov 2000.
8.3 Bathe, K.-J. and Wilson, E.L., Numerical Methods in Finite Element Analysis, Prentice-Hall, Englewood Cliffs, New Jersey, 1976.
8.4 Moler, C., Numerical Computing with MATLAB, SIAM, Philadelphia, PA, 2004.
8.5 *** Using MATLAB (Version 5), The MathWorks Inc., Dec. 1996.
8.6 Anderson, E., Bai, Z., Bischof, C., Demmel, J., Dongarra, J., Du Croz, J., Greenbaum, A., Hammarling, S., McKenney, A., Ostrouchov, S. and Sorensen, D., LAPACK Users’ Guide, SIAM, Philadelphia, 2nd ed, 1995.
8.7 Kohnke, C. (ed.), ANSYS Engineering Analysis System, Theoretical Manual, Rev 4.3, Swanson Analysis Systems Inc., Houston, PA, 1987.
8.8 Wilkinson, J.H., The Algebraic Eigenvalue Problem, Clarendon Press, Oxford, 1965.
8.9 Parlett, B.N., The Symmetric Eigenvalue Problem, Prentice-Hall, Englewood Cliffs, N.J., 1980.
8.10 Garbow, B.S., Boyle, J.M., Dongarra, J.J. and Moler, C.B., Matrix eigensystem routines – EISPACK Guide extension, in Lecture Notes in Computer Science, Springer, N.Y., 1977.
8.11 Meirovitch, L., Elements of Vibration Analysis, 2nd ed., McGraw Hill Book Comp., New York, 1986.
8.12 Stewart, G.W., Matrix Algorithms II: Eigensystems, SIAM, Philadelphia, 2001.
8.13 Radeş, M., Application of Lanczos and Schur vectors in structural dynamics, Shock and Vibration Journal, vol.15, no.3-4, pp 459-466, June 2008.
8.14 Radeş, M. and Ewins, D.J., Some applications of Arnoldi vectors in modal testing, Proceedings of IMAC 21 Conference on Structural Dynamics, Kissimmee, Florida, Paper S11P06, 2003.
8.15 Hill, D.R. and Moler, C.B., Experiments in Computational Matrix Algebra, Random House, N.Y., 1988.
8.16 Radke, R.J., A Matlab Implementation of the Implicitly Restarted Arnoldi Method for Solving Large-Scale Eigenvalue Problems, M.A. Thesis, Rice University, Houston, Texas, 1996.
8.17 Sorensen, D.C., Implicit application of polynomial filters in a k-step Arnoldi method, SIAM Journal of Matrix Analysis and Applications, vol.13, pp 357-385, 1992.
8.18 Arnoldi, W.E., The principle of minimized iterations in the solution of the matrix-eigenvalue problem, Quarterly of Applied Mathematics, vol.9, pp 17-29, 1951.
8.19 Lanczos, C., An iteration method for the solution of the eigenvalue problem of linear differential and integral operators, Journal of Research of the National Bureau of Standards, vol.45, no.4, pp 255-282, 1950.
8.20 Saad, Y., Variations on Arnoldi’s method for computing eigenelements of large unsymmetric matrices, SIAM Journal of Numerical Analysis, vol.17, pp 687-706, 1980.
8.21 Lehoucq, R.B., Sorensen, D.C. and Yang, C., ARPACK Users’ Guide:
Solution of Large Scale Eigenvalue Problems by Implicitly Restarted Arnoldi Methods, SIAM, Philadelphia, 1998.
8.22 Daniel, J., Gragg, W.B., Kaufman, L. and Stewart, G.W., Reorthogonalization and stable algorithms for updating the Gram-Schmidt QR factorization, Mathematics of Computation, vol.30, pp 772-795, 1976.
8.23 Bai, Z., Demmel, J., Dongara, J., Ruhe, A. and Van der Vorst, H., Templates for the Solution of Algebraic Eigenvalue Problems: A Practical Guide, SIAM, Philadelphia, PA, 2000.
8.24 Parlett, B.N. and Scott, D.S., The Lanczos algorithm with selective orthogonalization, Mathematics of Computation, vol.33, no.145, pp 217-238, 1979.
8.25 Craig, R.R. Jr., Krylov-Lanczos methods, in ‘Encyclopedia of Vibration’ (S.
Braun, D. Ewins, S.S. Rao, eds.), Academic Press, London, pp 691-698, 2002.
8.26 Wilkinson, J.H. and Reinsch, C. (eds.), Handbook for Automatic Computation, vol.2, Linear Algebra, Springer, Heidelberg, 1971.
8.27 Smith, T., Boyle, J.M., Dongarra, J.J., Garbow, B.S., Ikebe, Y., Klema, V.C.
and Moler, C.B., EISPACK Guide, 2nd ed., Springer, Berlin, 1976.
8.28 Lawson, C.L., Hanson, R.J., Kincaid, D.R. and Krogh, F.T., Basic linear algebra subprograms for FORTRAN usage, ACM Transactions on Mathematical Software, vol.5, no.3, pp 308-323, 1979.
8.29 Demmel, J., Applied Numerical Linear Algebra, U. C. Berkeley, 1993.
8.30 Hughes, T.J.R., The Finite Element Method. Linear Static and Dynamic Finite Element Analysis, Prentice-Hall, Englewood Cliffs, N.J., 1987.
8.31 Bai, Z. and Stewart, G.W., SRRIT – A FORTRAN subroutine to calculate the dominant invariant subspace of a nonsymmetric matrix. Technical Report 2908, Department of Computer Science, University of Maryland, 1992.
8.32 Stewart, W.J. and Jennings, A., ALGORITHM 570: LOPSI a simultaneous iteration method for real matrices, ACM Transactions of Mathematical Software, vol.7, no.2, pp 230-232, 1981.
8.33 Fokkema, D.R., Sleijpen, G.L.G. and Van der Vorst, H.A., Jacobi-Davidson style QR and QZ algorithms for the reduction of matrix pencils, SIAM Journal on Scientific Computing, vol.20, no.1, pp 94-125, 1998.
8.34 Sleijpen, G.L.G. and Van der Vorst, H.A., The Jacobi-Davidson iteration method for linear eigenvalue problems, SIAM Journal on Matrix Analysis and Applications, vol.17, pp 401-425, 1996.
8.35 Golub, G. and Ye, Q., An inverse free preconditioned Krylov subspace method for symmetric generalized eigenvalue problems, SIAM Journal on Scientific Computing, vol.24, pp 312-334, 2002.
8.36 Baglama, J., Calvetti, D. and Reichel, L., irbleigs: A MATLAB program for computing a few eigenpairs of a large sparse Hermitian matrix, ACM Transactions of Mathematical Software, vol.29, no.5, pp 337-348, 2003.