Multiple Subspace Correction

Various recently developed enhancements of the basic MR and OR schemes presented above are based on introducing additional subspace corrections besides those associated with the stepwise increasing correction spaces. Existing approaches include generating such auxiliary projections from spectral information on the operator A gained during the iteration process or from additional inner iteration or restart cycles. In addition, time and storage constraints often make it necessary to form these projections only approx-imately, while at the same time keeping this approximation as effective as possible. In order to better describe and compare these new developments, we first formulate the basic projection steps required to combine two subspace corrections. Again, such a combina-tion of two or more subspace correccombina-tions is a commonly used device in other areas of numerical analysis, e.g. in the field of additive Schwarz methods for solving boundary value problems, in which the different correction spaces correspond to finite-dimensional spaces of functions with support on subdomains of the original domain (which can also be interpreted as a block-Jacobi method). In the following Section 3.4, we then discuss how subspace information may be quantified in order to construct effective approximate projections.

Consider an initial approximation x₀ to the solution of (1.1) for which we seek the MR approximation x₀ + c with c selected from the correction space C . We assume C to be the direct sum C = C1 ⊕ C² of two spaces C1 and C2, and our goal is to obtain the MR approximation as the result of two separate projection steps involvingC1 and C2, respectively. This task is equivalent to finding the best approximation w = Ac ∈ W = AC = W1 ⊕ W² to r0, where Wj := ACj, j = 1, 2.

If, in a first step, we obtain the best approximation w₁ = P_W1r₀ in W1, then the best approximation in W is obtained by introducing the orthogonal complement Z :=

W ∩ W1^⊥ of W1 inW , in terms of which W has the direct and orthogonal decomposition W = W1⊕ Z . The global best approximation is now given by

w := P_Wr₀ = (P_W1 + P_Z)r₀ = P_W1r₀+ P_Z(I− PW1)r₀. (3.12) The last expression shows that the contribution from the second projection consists of the orthogonal projection onto Z of the error (I − P_W1)r₀ of the first approximation.

Expressing all spaces in terms ofC1 and C2 and noting that Z = (I − PAC1)AC2, we conclude that the correction c associated with the residual approximation w satisfies

Ac = w = P_AC1r₀+ P_{(I−PAC1)AC2}(I− PAC1)r₀. The global correction is thus of the form c = c₁+ d , where

Ac₁ = P_AC1r₀ (3.13)

Ad = P_{(I−PAC1)AC2}(I− PAC1)r₀. (3.14) The solution c₁ of (3.13) is simply the MR solution of the equation Ac = r₀ with respect to the correction spaceC1. To obtain a useful representation of d , we note that the right hand side of (3.14) may be viewed as the MR approximation with respect to C2 of the equation

(I− PAC¹)Ac = (I− PAC¹)r₀. (3.15)

Lemma 3.3.1. The operator (I − PAC1)A restricted to C2 is a bijection fromC2 to Z . Proof. The assertion follows by showing that the operator in question is one-to-one: (I− P_AC1)Aec = 0 for ec∈ C2 implies Aec∈ AC1 ∩ AC2 ={0}.

The solution d of (3.14) yielding the second component of the combined correction c may thus be obtained by first determining the MR solution c₂ of (3.15) and then evaluating

d = A⁻¹(I− P^AC1)Ac2 = c2− A⁻¹PAC1Ac2. (3.16) Lemma 3.3.2. The operator P := A⁻¹(I− PAC1)A restricted to C2 is the oblique projec-tion onto A⁻¹Z along C1.

Proof. The projection property follows immediately upon squaring P . Since A is non-singular, N (P ) = A⁻¹W1 = C1 and R(P ) = A⁻¹(AC1)^⊥. Restricted to C2, the range reduces to the preimage under A of the orthogonal complement of AC1 with respect to AC2, i.e., A⁻¹Z .

At first glance, the evaluation of d as given in (3.16) appears to require a multiplication by A as well as the solution of another equation involving A with a right hand side from AC1, in addition to the computation of the two projections. In fact, we show how d can be calculated inexpensively using quantities generated in the course of the two MR approximation steps.

AssumeC1 has dimension m and that Algorithm 3.2.3 (FGMRES) has been employed to obtain the MR approximation to the solution of Ac = r₀ with respect to C1. If Cm⁽¹⁾ = [c₁⁽¹⁾, . . . , cm⁽¹⁾] denotes a basis ofC1, then, besides the MR approximation c₁, which has the coordinate representation c₁ = Cm⁽¹⁾y₁ with y₁ ∈ C^m, FGMRES also constructs the Arnoldi-type decomposition (3.9), which we write here as ACm⁽¹⁾ = V_m+1⁽¹⁾ Hem⁽¹⁾. The QR decomposition eHm⁽¹⁾ = eQ⁽¹⁾m R⁽¹⁾m (cf. (2.49),(2.50)) makes available the Paige-Saunders basis bVm⁽¹⁾ (cf. (2.63)), which forms an orthonormal basis of AC1. Note also that, in view of relation (3.11), there holds

A⁻¹Vb_m⁽¹⁾ = C_m⁽¹⁾R⁻¹_m . (3.17) The orthogonal projection PAC1 may be expressed in terms of bVm⁽¹⁾ as bVm⁽¹⁾

 Vbm⁽¹⁾

^∗ , and, denoting the residual of the first MR approximation by r₁ := r₀ − Ac1, equation (3.15) may be written



I− bV_m⁽¹⁾

Vb_m⁽¹⁾^∗

Ac = r₁.

Applying Algorithm 3.2.3 to equation (3.15) using the basis C_k⁽²⁾ = [c₁⁽²⁾, . . . , c_k⁽²⁾] of the k-dimensional correction space C2 thus produces the decomposition



I − bV_m⁽¹⁾

Vb_m⁽¹⁾^∗

AC_k⁽²⁾= V_k+1⁽²⁾He_k⁽²⁾ (3.18)

as well as the MR approximation c₂ = C_k⁽²⁾y₂, y₂ ∈ C^k. The solution d of (3.14) as given in (3.16) can now be expressed as

d = c₂− A⁻¹P_AC1Ac₂ = C_k⁽²⁾y₂− A⁻¹Vb_m⁽¹⁾

which shows that the action of A⁻¹ in (3.16) is effected by the inverse of the (small) triangular matrix R⁽¹⁾m . We further observe that the evaluation of Ac2 in (3.16) is accom-plished through the m× k matrix 

Vbm⁽¹⁾

^∗

AC_k⁽²⁾, which is available at no extra cost as a by-product of the orthogonalization process carried out in the second MR step to obtain (3.18). In fact, (3.17) and (3.18) can be combined to yield the global decomposition

Ah with respect to C . We summarize the coordinate representation of these two successive projections in

Theorem 3.3.3. The MR approximation of the solution of Ac = r₀ with respect to the correction space C = C1⊕ C2 is given by

c = C_m⁽¹⁾y₁+ C_k⁽²⁾y₂+ C_m⁽¹⁾R_m⁽¹⁾−1  Vb_m⁽¹⁾∗

AC_k⁽²⁾
y₂,

where the coefficient vectors y1 ∈ C^m and y2 ∈ C^k satisfy the least-squares problems

Remark 3.3.4. The decomposition (3.19) is a slight modification of the standard Arnoldi-type decomposition (3.9), which, translated to the present context, would have the form

Ah

with an upper Hessenberg matrix eH_m+k ∈ C^{(m+k+1)×(m+k)} composed of the submatrices H_m+1,k ∈ C^(m+1)×k, the upper triangular matrix eR_k ∈ C^k×k and the upper Hessenberg matrix eHm⁽¹⁾ ∈ C^(m+1)×m associated with the Arnoldi decomposition of A with respect to Cm⁽¹⁾. The modified decomposition (3.19) is obtained from (3.20) by substituting the QR decomposition (2.49) and introducing the Paige and Saunders basis (2.63), which also reveals the relations

(Note that the last equation contains two different block partitionings of the (m+k+1)×k matrices: that on the left is split into an (m + 1)× k and a k × k block, while the blocks on the right are of dimension m× k and (k + 1) × k, respectively.)