6.4 Numerical Examples
7.1.1 Multigrid Methods
The first tool which we have to describe is the Multigrid approach.
Multigrid methods have been developed for solving partial differential equation and, more in general, for solving linear systems of equations of large size. The basic idea of the multi- grid is to create a sequence of linear systems which get smaller and smaller by consecutive projection. In this way the computational effort can be reduced and the convergence speed can be improved up if the smaller linear systems are properly chosen.
Let us start with the linear system of equation
Ax = b, (7.1)
where A ∈ Rn×nis invertible and x, b ∈ Rn.
It is well known that iterative methods first converge in the well-conditioned space and that the convergence can be very slow in the ill-conditioned space. Differently, while direct meth- ods are not affected by this kind of problem, they are usually much more expensive and are more sensible to error propagation.
Remark 7.1. The definition of well- and ill-conditioned space is not formal.
Let V ⊂ Rnbe a linear subspace of Rn. We define the conditioning number of A restricted to V by
κV = sup x∈V
kAxk kxk .
The well-conditioned space is the space W such that κW is not too large, whereas the ill-conditioned space I is the one where κI is very large.
For matrices deriving from the discretization of compact integral operator we have that W corresponds to the low frequency space and I is the high frequency space.
The idea of the Multigrid method is to combine the positive aspects of both direct and itera- tive method.
7.1. Preliminaries 107
The TGM is an iterative algorithm. Let xkbe an approximation of the solution of (7.1) at the kth step, apply ν1steps of an iterative method to xkobtaining
˜
xk=Pre-Smooth(A, b, xk, ν1).
This step is called pre-smoothing, since it is done before everything else and, in the context of differential equations, damps the error in the high frequencies, i.e., it smooths the error. We then compute the residual
rk= b− A˜xk,
since we are moving to the error equation in order to compute a refinement term for ˜xk. Let 0 < n1 < n, we call R ∈ Rn1×nthe restriction operator. This operator projects a vector from a grid of size n to a grid of size n1.
Define P ∈ Rn×n1 the interpolation operator. This operator interpolates a vector from a grid
of size n1to a grid of size n. Usually R = Pt.
We can now define the restricted operator using the Galerkin approach as
A1 = RAP ∈ Rn1×n1.
Let us assume that both R and P are of full rank. This implies that A1is invertible. Comput- ing the refinement term for ˜xkas
hk= P A−11 Rrk, we obtain the refined version of ˜xkas
ˆ
xk= ˜xk+ hk= ˜xk+ P A−11 R(b− A˜xk).
The procedure that computes ˆxkfrom ˜xkis called Coarse Grid Correction (CGC). Let us call C the iteration matrix of the CGC, i.e.,
C = I − P (RAP )−1RA,
it is possible to show that C is a projector and hence λ(C) = {0, 1}, where λ(C) denotes the spectrum of C. Therefore, the TGM algorithm, without any smoothing step, can not converge to the solution of (7.1), cf. [24].
Finally, to obtain the (k + 1)th approximation, we apply ν2 steps of an iterative method
xk+1 =Post-Smooth(A, b, ˆxk, ν2),
which can be different from the pre-smoother. This is called post-smoothing.
It is possible to show that, under mild conditions, this method converges to the solution of (7.1).
The problem of the TGM algorithm is obviously the computation of hksince it requires the inversion of A1 which, if n1 is large, can be extremely expensive. The multigrid method stems from this observation. Since A1 can be very large the idea is to restrict consecutively the grid until it is so small that the inversion can be easily performed.
FIGURE7.1: V-cycle scheme
Let n = n0 > n1 > · · · > nL > 0, we call Ri and Pi the ith restriction and interpolation operator respectively so that Ri∈ Rni+1×ni and Pi ∈ Rni×ni+1, for i = 0, . . . , L − 1; then
Ai=
A if i = 0
Ri−1Ai−1Pi−1 i = 1, . . . , L. (7.2)
We proceed to project for L levels then we directly solve the system. The idea is that nL is so small that AL can be inverted directly. For instance, for images of size 2L × 2L we have n = 22L and picking up a pixel every two in each direction it holds n
i = 22(L−i), for i = 0, . . . , L, thus nL= 1.
Summarizing the single step of the multigrid iteration goes as follows
yi=MGM Single Step(xi, bi, Ai, i, L)
if (i = L) then yL =Solve(ALyL= bL)
else x˜i =Pre-Smooth Ai, bi, xi, ν1 ri+1= Ri(bi− Aix˜i)
ei+1=MGM Single Step(0, ri+1, Ai+1, i + 1, L) ˆ
xi = ˜xi+ Piei+1
yi =Post-Smooth Ai, bi, ˆxi, ν2
end
We call this iteration V-cycle since when represented graphically the single iteration resem- bles a V, see Figure 7.1. Iterating the process we obtain the Multigrid Method
x=MGM(x0, b, A, L)
for k = 1, 2, . . .
xk=MGM Single Step(xk−1, b, A, 1, L)
end
Remark 7.2. For well-posed problems, in [4] the authors showed that for certain matrix algebras related to BTTB and under some hypothesis that links the smoothers and the coarsening strategy, the Multigrid method has a linear convergence rate, i.e., the number of iterations does not depend on the dimension of the problem.
Intuitively the idea is that the restriction operator should map into the subspace where A is ill- conditioned. In this way the multigrid is able to deal simultaneously on both the ill-conditioned and the well-conditioned subspaces, on the first the matrix is inverted directly on the second the smoother damps the error very fast.
Conversely, for ill-posed problems, the projection in the ill-conditioned subspace results to be dangerous and the grid transfer operator has to be chosen differently [56]. The reason of this is twofold: firstly the matrix A is usually not invertible and thus, if projected on the ill-conditioned subspace, the restricted operator is not invertible as well; secondly the order
7.1. Preliminaries 109
of the zero of the singular values can be very high, even exponential, and thus an eventual (pseudo-)inversion on the ill-conditioned subspace may lead to a dramatical amplification of the noise.
On the other hand the projection in the well-conditioned subspace allows the direct inversion of the coefficient matrix, since, if L is chosen big enough, the restricted operator becomes in- vertible. Because we assume L large, ALis very small, usually just a scalar, and so numerical stability is not an issue.
Using a projection into the well-conditioned subspace, the multigrid which we are going to construct does not have a linear convergence rate since it does not fulfills the hypothesis in [4], but it shows a very stable convergence which is also fast thanks to the preconditioned smoother.