X∗L2=ZjL1,
showing that one can write a part of X∗as a linear combination of Zj.
Although an analogous of the following corollary can also be found in [103, Corolary 1], we reproduce it here for the sake of completeness. It is a direct implication ofProposition 2.3.3andProperty 2.2.7.
Corollary 2.3.4. At each iteration j of a MBR method using block Krylov subspace as correction subspace, it holds that
null Rj = p − pj.
2.4
Preconditioning the Correction Subspace
A practice even more common than choosing Zj as Kj(A, B)is to choose a “preconditioned block Krylov subspace” instead. It consists in choosing a nonsingular preconditioning matrix M ∈ Cn×nsuch that AM presents better numerical properties, to then solve the system
AM M−1X = AM T = B (2.4.1)
instead. Once T is known, we take X = M T to retrieve the solution of the original system. For solving (2.4.1) one then uses
Zj =M × span h B AM B ... (AM )(j−1)Bi =M Kj(AM , B) (2.4.2) as correction subspace5.
An even more general concept is the use of nonlinear variable preconditioners. It consists in applying a (nonlinear) preconditioning operator which depends on the iteration. For a given iteration j and matrix V ∈ Cn×p, we represent the application of the preconditioner on V with
Z = Mj(V ) (2.4.3)
where Z ∈ Cn×p is the preconditioned V . Also, for the sake of simplicity, we always consider that the operator Mj(.) is rank-preserving for every j (that is, rank (V ) = rank (Z)) and that it satisfies the following definition.
Definition 2.4.1(Rank-Preserving Variable Nonlinear Preconditioner). Suppose we dispose of a matrix Vj =V1 V2 ... Vj
where each Vi ∈ Cn×ki for some sequence of values of ki with 1 ≤ i ≤ j, and that we obtain Zj =M1(V1) M2(V2) ... Mj(Vj)by applying the sequence of (possibly) nonlinear operators Mi, 1 ≤ i ≤ j. If rank Zj = rank Vj
, we say that the sequence of preconditioner operator Mi, 1 ≤ i ≤ j is rank-preserving.
We then establish the following lemma.
Lemma 2.4.2 (Equivalent Preconditioner Matrix). Given Vj a full rank matrix, for any sequence of rank-preserving preconditioner operators Mi, 1 ≤ i ≤ j, there is at least one nonsingular matrix Mj ∈ Cn×n such that
Zj=MjVj (2.4.4)
holds.
5 because range (T ) ⊂ K
`(AM , B) according to Corollary 2.3.2, we know that range (X) ⊂ M K`(AM , B); thus
M K
Proof. One example of such a matrix is
Mj=hZj Zj⊥i hVj Vj⊥ i−1
where Z⊥
j (respectively Vj⊥) is the orthogonal complement of Zj (respectively Vj) in Cn.
Lemma 2.4.2also implies that there is a matrix Mi such that Zi= MiVi
for every 1 ≤ i ≤ j. We prefer the notation on (2.4.4) over the one in (2.4.3) throughout this thesis, but they are equivalent.
Both flexible and fixed preconditioners are relevant and widely applied, and the choice between either is problem dependent. Perhaps the most well known class of fixed preconditioners are the incomplete fac- torizations [109, Chapter 10]. Concerning variable preconditioners we can exemplify inner-outer iteration methods (see [113]). Quoting [113] about variable preconditioners, “one can also consider preconditioners which might improve using information from previous iterations” (see [8, 48, 75]). Because the variable preconditioner also covers fixed preconditioner and unpreconditioned case, in this whole thesis we con- sider that a variable preconditioner is being used unless otherwise noted, and we always suppose that the matrices Mi∈ Cn×n, 1 ≤ i ≤ j as well as Mj were properly chosen.
The block Krylov correction subspace using a variable preconditioner is thus introduced as follows Zj= span
n
M1B M2AM1B ... MjA...AM2AM1B o
, (2.4.5)
for a given sequence of nonsingular preconditioning matrices Mi∈ Cn×n. It is not trivial to write (2.4.5) as a (block) Krylov subspace, but the following theorem shows that if the sequence of preconditioning operators Mi is rank-preserving this is always possible.
Theorem 2.4.3. Define the sj-dimensional subspaces Zj= span
n
M1B M2AM1B ... MjA...AM2AM1B o
,
where the sequence of variable preconditioners Mi ∈ Cn×n, 1 ≤ i ≤ j is rank-preserving (see Defini-
tion 2.4.1). There is always a nonsingular matrix Mj ∈ Cn×n such that Zj =MjKj (AMj, B).
Proof. For the theorem to be proved, there should be a matrix Mj ∈ Cn×nsuch that the subspace Zj=MjKj(AMj, B)
=Mj× span n
B AMjB . . . (AMj)j−1B o
equals (2.4.5). One possible case in which this is true is when there is a matrix Mj such that all the equalities MjB = M1B MjAMjB = M2AM1B Mj(AMj)2B = M3AM2AM1B ... Mj(AMj)j−1B = MjA . . . M3AM2AM1B (2.4.6)
2.4. PRECONDITIONING THE CORRECTION SUBSPACE 15 hold. However, substituting the first equality in the subsequent, we can rewrite the requirement as
MjB = M1B
MjAM1B = M2AM1B
MjAMjAM1B = M3AM2AM1B ...
Mj(AMj)j−2AM1B = MjA . . . M3AM2AM1B.
Substituting the second equality on the subsequent, and doing so recursively implies that this requirement can be written as MjB = M1B MjAM1B = M2AM1B MjAM2AM1B = M3AM2AM1B ... MjA . . . M3AM2AM1B = MjA . . . M3AM2AM1B. Denoting ˘Sk= AMk−1. . . AM1B, for 1 < k < j and ˘S1= B we finally have that
MjS˘1= M1S˘1 MjS˘2= M2S˘2 MjS˘3= M3S˘3
...
MjS˘j= MjS˘j.
This means that we are looking for a nonsingular matrix Mj such that Mj h ˘ S1 S˘2 . . . S˘j i =hM1S˘1 M2S˘2 . . . MjS˘j i . (2.4.7)
Because we assumed that the sequence of variable preconditioners Mi ∈ Cn×n is rank-preserving, rank h ˘ S1 S˘2 . . . S˘j i = rank h M1S˘1 M2S˘2 . . . MjS˘j i
allowing us to use Lemma 2.4.2to show that there is always a nonsingular matrix Mj such that (2.4.7) holds, proving the theorem.
The following corollary is another way to stateTheorem 2.4.3. Corollary 2.4.4. Define the sj-dimensional subspaces
Zj= span n
M1B M2AM1B ..., MjA...AM2AM1B o
,
where the sequence of variable preconditioners Mi∈ Cn×nis rank-preserving (seeDefinition 2.4.1). There is always a nonsingular matrixTj ∈ Cn×n such that
Zj = Kj(TjA, Z1). where Z1= M1B.
Proof. Analogous toTheorem 2.4.3.
To the best of our knowledgeTheorem 2.4.3(followed byCorollary 2.4.4) is the first demonstration on how to describe subspaces of the form (2.4.5) as a (block) Krylov subspace for both single and multiple right-hand side case, being one of the main theoretical contributions of this chapter of the thesis.
Notice thatCorollary 2.4.4clarifies what was stated in [114, p.27] for the single right-hand side case: “there may not exist any Krylov subspace containing Zj”. In fact, the proof ofTheorem 2.4.3 relies on the rank-preserving assumption of the variable preconditioner, and in a general case, we do not guarantee that Zj can be written as a block Krylov subspace. Although theoretically speaking this assumption is rather strong, in practice, a rank-deterioration rarely happens due to the preconditioner application, thus justifying our assumption.
Also, it is not trivial to define the concept of block grade (cf. Corollary 2.3.2) for a flexibly precondi- tioned (block) Krylov subspace because the matrix Mj changes every time we expand the subspace Zj. i.e
Zj =MjKj(AMj, B) but Zj+16⊂MjKj+1(AMj, B).
However, as long as the subspace Zj grows with j, the respective MBR method is convergent (cf. [114, p.27]). In practice, when using a variable preconditioner, we expect to build a variable preconditioned block Krylov subspace such that
range (X∗) ⊂MkKk(AMk, B) ⊂ K`(A, B)
where dim(MkKk(AMk, B)) < dim(K` (A, B)) but such inequality holds only if the variable precondi- tioning matrices Mi were properly chosen, and such a choice is highly problem dependent.
We rewriteProposition 2.3.3for the flexibly preconditioned case. Proposition 2.4.5. At each iteration j of a MBR method using
Zj= span n
M1B M2AM1B ... MjA...AM2AM1B o
, (2.4.8)
as correction subspace for a given sequence of rank-preserving variable preconditioning nonsingular matri- ces Mi∈ Cn×n, it holds that a linear combination of p − pj solutions lies inside Zj.
Proof. Thanks toTheorem 2.4.3we know that Zj=MjKj(AMj, B)for some nonsingular matrix Mj ∈ Cn×n. Thus
dimrange (B) ∩ AMjKj(AMj, B)
= dim range (B) + dimAMjKj (AMj, B)
− dimrange (B) + AMjKj(AMj, B)
But
range (B) + AMjKj(AMj, B) = Kj+1(AMj, B). Moreover from the nonsingularity of AMj we have that
dimAMjKj(AMj, B) = dimK j(AMj, B) therefore, dimrange (B) ∩ AMjKj (AMj, B)
= dim range (B) + dimK
j (AMj, B) − dimK j+1(AMj, B) = p + sj− dim Kj+1 (AMj, B) .
2.5. THE BLOCK ARNOLDI ALGORITHM 17