In this chapter we have studied a generalization of the well known GMRES algorithm for multiple right- hand side scenario, the block GMRES. We have developed a theoretical basis related to the subspaces being spanned by BGMRES in the presence of a variable preconditioner. The result found in Theorem 2.4.3is new to the best of our knowledge even in the single right-hand side case. Some other new properties have been demonstrated (e.g. Proposition 2.5.3which is a consequence ofTheorem 2.4.3).
We also determined a generalization of a set of concepts which are common in the single right-hand side scenario, but not globally formalized for the multiple right hands side scenario (as Definition 2.6.1,
Definition 2.8.1 and Definition 2.9.1). Among those, we remark the definition of partial stagnation (cf.
Definition 2.9.1) which is not trivially deduced from the single right-hand side case. The importance of these definitions will come clear as we advance in a more complex scenario, inChapter 3.
Chapter 3
Deflation
3.1
Introduction
In the previous chapter we have studied Block GMRES (BGMRES) due to Vital [134] for solving the problem AX = B, with A ∈ Cn×nnonsingular, B, X ∈ Cn×pwhere n p and rank (B) = p. We extended concepts common in the single right-hand side scenario for the block scenario, as partial convergence (when a linear combination of approximate solutions is found rather than the approximate solution of the entire block system) and the partial breakdown (which is basically a linear combination of happy breakdowns). BGMRES has ever since been improved based on the assumption that a subspace of the correction subspace can be discarded, a process called deflation [64, 78, 103]. It is recognized that to be effective in terms of computational operations, block iterative methods must incorporate a deflation strategy [64], most notably when a partial convergence is detected.
We briefly summarize now the most common deflation techniques available in the literature. In [23,64,
78,96] it is proposed the initial deflation. It consists in performing a block size reduction of the (scaled) initial residual R0 relying on its (near) rank deficiency. BFGMRESD [23,78,96] computes
R0= QT (QR decomposition)
T D0= U+Σ+W+H+ U−Σ−W−H+ (singular value decomposition) (3.1.1) where all the singular values larger or equal than a threshold εd lie in Σ
+ and those smaller lie in Σ−, and D0 is the nonsingular scaling matrix. This gives raise to the low rank approximation of R0 as Cn×k1 3 ˜R0 = QU+Σ+W+H. The algorithm then proceeds with one cycle of nonsingular BGMRES, minimizing ˜R0− A ˜X rather than R0− AX. The cost per iteration is thus reduced since every Vj has k1 columns instead of p. At the end of the said cycle, some manipulations are performed in order to retrieve the approximate solution for the original problem. A truncated variant called BFGMREST [23, 78, 96] allows the user to set a maximum number of columns allowed in ˜R0 thus truncating the block initial residual even if the singular values of the residual are not smaller than εd. This strategy aims at reducing the memory requirements of the method when many right-hand sides are considered at once. Furthermore, both BFGMRESD and BFGMREST are proposed for variable preconditioner scenario.
The BlMResDefl [64] uses a very similar technique, but relying on a rank-revealing QR decomposition to obtain the decomposition
R0= h V1 V1∆ i " Λ0 Λ∆ 0 # ΠHc
(cf. [64, (12.1)]), where Πcis a permutation matrix responsible for reordering the columns of R0such that the elements in the diagonal of [(Λ0)T (Λ∆0)T]T are given in nonincreasing order. BlMResDefl then sets the new initial residual to V1Λ0and proceeds executing one cycle of BGMRES algorithm, as BFGMRESD.
The BlMResDefl [64] proposes not only initial deflation techniques, but also the so called “Arnoldi deflation”, already discussed inRemark 2.5.1. It consists of determining which columns of S inline 5 of
Algorithm 2.5.2are linear dependent to ensure that Vj+1is not rank deficient. In case of linear dependency, these columns are removed from Vj+1(characterizing thus the deflation), and the required manipulations are performed over Hj. This deflation is reported in [64] to never become active in practical numerical experiments.
In [103] a deflation technique similar to Arnoldi deflation is proposed for BGMRES-W, therein called inexact (Arnoldi) breakdown. Supposing that no deflation was performed, BGMRES-W deflates whenever S is near rank deficient, basing this choice on the singular values of Hj+1,j and a threshold εd. The deflation incurs some modifications in the block Arnoldi iteration, and the subsequent deflation relies on a submatrix of Hj instead of Hj+1,j. We refer the reader to [103] for more details. Nevertheless, as the authors observe, BGMRES-W tends to deflate only at the end of the convergence history, and this observation is confirmed in [76].
In the same publication [103], BGMRES-R is proposed as an alternative to BGMRES-W relying on the singular values of the (scaled) residual Rj every iteration j. Similarly to BFGMRESD, it computes a decomposition as (3.1.1) for Rj, and using a submatrix of U+ to choose which columns of Vj are going to be carried out for the next block Arnoldi iteration, postponing the remaining ones. The postponed columns are used in the block Arnoldi algorithm for orthogonalization purposes only. A FOM variant of BGMRES-W and BGMRES-R can be found in [102], and further numerical experiments in [76].
Besides the aforementioned methods, strategies based on rank-revealing QR-factorizations [21] or sin- gular value decomposition [60] have been notably proposed both in the Hermitian [89, 104] and non- Hermitian cases [2,9, 32, 54, 81, 92] for block Lanczos methods. They have been shown to be effective with respect to standard block Krylov subspace methods, but since we are focusing on iterative methods showing a minimal residual property for non-Hermitian problems, we do not focus on the study of these methods.
Variable preconditioning is often required when solving large linear systems. This is notably the case when inexact solutions of the preconditioning system using, e.g., nonlinear smoothers in multigrid [96] or approximate interior solvers in domain decomposition methods [127, Section 4.3] are considered. The combination of block methods performing deflation at each iteration and variable preconditioning has been rarely addressed in the literature, although the combination of initial deflation with variable preconditioning has been already explored in [96, 22, 23]. Thus the main purpose of this chapter is to derive a class of flexible minimal block residual methods for non-Hermitian problems that incorporate deflation at each iteration.
This chapter is organized as follows. In Section 3.2 we propose a generalization of the concepts established inChapter 2 aiming at a method able to judiciously choose which (block) Krylov directions are interesting for expanding the correction subspace Zjevery iteration j, to then present and re-interpret a block iterative procedure firstly introduced in [103] which is able to build an orthonormal basis for the chosen directions only (the “deflated block Arnoldi”). InSection 3.3a general framework for deflated block Krylov subspace methods is presented. We show that the resulting method (named “deflated minimal block residual” method or DMBR for short) always minimizes the Frobenius norm of the block residual and that the singular values of the scaled block residual are always nonincreasing. In Section 3.4 we propose a criterion for choosing which directions to take into account when expanding Zj at iteration j, which is mostly based onSection 2.9as well as [103] and [78]. Then, inSection 3.6 we show that DMBR using the proposed criterion never breaks down, thus guaranteeing convergence (considering a large enough restart size) along with other properties. InSection 3.5we use DMBR as a framework to describe existing algorithms as BGMRES, BGMRES-R and BFGMRESD and in which situations these algorithms could be considered as equivalent to DMBR. Then inSection 3.9we demonstrate the effectiveness of DMBR on three academic illustrations and one real life application, showing that in practical cases, none of these methods are algebraically or numerically equivalent to DMBR. These conclusions are later extended with further experiments inSection 4.6. Finally we draw some conclusions inSection 3.10.