4.3 A Wavelet Method for Elliptic Problems
4.3.2 Fast Solution over Finite-Dimensional Subspaces
Up to this point, we have been working purely in the infinite-dimensional context, first discussing varia-tional problems over Sobolev spaces and then switching to an equivalent wavelet formulation in `2. To make this formulation accessible for computations, we need to identify finite-dimensional subproblems which approximate the original problem. To this end, we present the concepts of uniform and adaptive wavelet discretisations and comment on the implications on the expansions of functions in wavelet bases and the structure of the stiffness matrix. We explain our notion of a fast solver, and demonstrate that wavelet discretisations deliver all ingredients for the conception of such a fast method.
Finite Wavelet Discretisations
The wavelet bases introduced in Chapter 2 consist of an infinite collection of hierarchically ordered functions which are indexed over `2(IIH) and span the full space H. Finite-dimensional subspaces are selected by a reduction of the index set, i.e., by choosing a finite subset Λ ⊂ IIH. Hence, all wavelet basis functions whose indices are not in Λ are discarded, and we obtain a finite basis ΨΛ spanning the finite-dimensional subspace SΛ⊂ H. Note that at this point the structure of Λ is arbitrary.
Consequently, all vectors of wavelet coefficients are truncated by deleting the entries which are not in Λ, and similarly all matrices in wavelet representation are shrunk by deleting all rows and columns which do not belong to Λ. For example, the solution y and the right hand side f are obtained as
yΛ= hy, ˜Ψ1Λi , fΛ= hf, Ψ1Λi . (4.3.13) The vectors yΛ and fΛ have NΛ = #Λ ∈ N entries. The truncated stiffness matrix is denoted by AΛ, it has the dimensions NΛ× NΛ, it is still symmetric, and it inherits the uniformly bounded condition number from the infinite-dimensional setting.
Corollary 4.8. The condition number of the truncated stiffness matrix is bounded by the condition number of the infinite-dimensional problem,
κ(AΛ) ≤ κ(A) ≤ CA
cA ∼ 1 . (4.3.14)
Proof. Above relation follows from (4.3.8) and (4.3.9).
It remains to solve the finite and well-conditioned, symmetric linear system of equations over RNΛ,
AΛyΛ= fΛ. (4.3.15)
The structure of the matrix AΛdepends on the strategy by which the finite number of wavelet coefficients is selected. Since wavelet bases are built over a multiresolution analysis (see Definition 2.1), we may choose a particular level of resolution j and define the subspace SΛj:= Sj ⊂ H according to (2.2.3). To this space corresponds the truncated wavelet basis Ψ(j) (2.2.16). We refer to this approach as uniform discretisation, it is analogous to multi-level finite element methods.
Iterative solvers repeatedly apply the stiffness matrix Aj:= AΛj to a vector. Making use of the identities (2.2.19) and (2.2.54) we obtain the representation
Aj = D−1WTja(Φj, Φj)WjD−1. (4.3.16) To compute the product Ajv, we can subsequently apply the matrices on the right hand side of (4.3.16).
The matrix a(Φj, Φj) is the standard stiffness matrix in the finite element setting, which contains O(Nj) nonzero elements. As the multi-scale transformations Wj (2.2.20) for both of the wavelet constructions 71
Chapter 4. Wavelet Methods for Linear Elliptic Partial Differential Equations
that we have encountered in Chapter 3 may also be applied in linear time, we conclude that Aj as a whole can be applied in O(Nj) arithmetic operations. When using the operator-adapted diagonal Da
instead (4.3.11), its entries can be precomputed in O(Nj) operations once at the beginning of the process.
While the subspaces of the uniform discretisation are selected level-wise, it is also possible to base the choice of the active wavelet indices on other criteria. In view of the norm equivalence (4.3.2), we could select only the N largest coefficients of the wavelet expansion, irrespective of their level. This is called best N -term approximation and leads to the concept of adaptive discretisation. For solutions which do not have the full regularity required by (4.2.16), it offers the potential to achieve the same accuracy as the uniform procedure with less coefficients, which reduces memory and time requirements. We will develop such a method in Chapter 7, including references to the necessary theory on adaptive wavelet methods.
A Fast Solver
For the solution of the finite system (4.3.15), we employ the method of conjugate gradients (CG), originally conceived in [90] as a direct solver for symmetric positive definite matrices. It has been found later that it can also be used as an efficient iterative method, see e.g. [22] for a discussion in the context of modern numerical methods. Its convergence rate depends on the condition number of the system matrix,
kyΛ(k)− yΛkAΛ ≤ 2
pκ(AΛ) − 1 pκ(AΛ) + 1
!k
ky(0)Λ − yΛkAΛ, (4.3.17)
where yΛ denotes the exact solution for (4.3.15), and y(k)Λ the iterative solution in step k. The type of energy norm used here is defined as follows,
kvk2AΛ:= vTAΛv ∼ kvk2, (4.3.18) where the equivalence relation on the right follows from (4.3.14) and is thus specific to the wavelet setting.
This allows us to derive the following estimate in `2,
ky(k)Λ − yΛk <∼ ρ(AΛ)kky(0)Λ − yΛk with ρ(AΛ) := pκ(AΛ) − 1
pκ(AΛ) + 1 ≤ ρ(A) , (4.3.19) which means that the convergence rate ρ(AΛ) is independent of the choice of the index set Λ. This guarantees that the reduction of the error by a fixed proportion η requires a constant amount Kη of iterations irrespective of the number of unknowns,
Kη <
∼ − log(η) . (4.3.20)
To determine the overall computational cost of the iterative solver, it remains to quantify the effort of one single iteration. We derive this for a uniform wavelet discretisation here, defining
yj:= yΛj, yj(k):= y(k)Λj , Aj:= AΛj. (4.3.21) The central step of one CG iteration consists in the multiplication of the matrix Ajwith a vector, which in view of (4.3.16) costs O(Nj) arithmetic operations. The memory consumption is thus proportional to Nj, and the computation time for a reduction η is proportional to KηNj.
Hence, it is possible to solve the discrete system for a given level j to an arbitrary high accuracy of y(k)j by increasing the number of iterations k. Yet we have to keep in mind that the discretisation error between the exact full solution y and the exact discrete solution yj persists. Since the full error of the numerical scheme is composed according to
ky − y(k)j k ≤ ky − yjk + kyj− y(k)j k , (4.3.22) 72
4.3. A Wavelet Method for Elliptic Problems
Algorithm Nested (A, b, J, ) → x: Solves Ax = b up to accuracy ∼ 2−J. (i) Initialisation for coarsest level
(1) Compute start value xj0:= A−1j0 bj0 to machine precision.
(2) Set j := j0. (ii) While j < J
(1) Prolongate xj→ x0j+1, Set j := j + 1.
(2) Solve Ajxj= bj iteratively, using the start value x0j, up to accuracy 2−(j−J).
(iii) Accept xj→ x.
Algorithm 4.1: We display the nested iteration algorithm Nested for the solution of an elliptic boundary value problem. The prolongation in the wavelet setting is trivially executed by padding the vector with zero coefficients.
This algorithm needs O(2J) operations.
only the rightmost term tends to zero for k → ∞. The left part is the discretisation error, which can be derived from the wavelet norm equivalence (4.3.2), Cea’s Lemma (4.2.15) and (4.2.16) as
ky − yjk ∼ ky − yjkH1 <
∼ infv∈Sjky − vkH1 <
∼ 2−j|y|H2. (4.3.23) We conclude from this that to prescribe higher accuracies for the CG method than ηj:= 2−j would be a waste of computing power. In other words, a stopping criterion of ηj for the CG method is most efficient to obtain a convergent series of discrete solutions,
ky − y(Kj ηj)k <∼ 2−j. (4.3.24) Concluding from (4.3.20) that Kηj <∼ −log(ηj) = j, we arrive at a computational cost of O(jNj), which is not yet the optimal result which we ultimately aspire. To remove the logarithmic factor j ∼ log(Nj), we use a strategy which is known as nested iteration, see e.g. [104], which works as follows (see Algorithm 4.1 for a complete listing). The system is solved to machine accuracy on the coarsest level. We then prolongate the coarse solution to the next level j and use it as a start value for the iterative solver. The solver is stopped at discretisation error accuracy 2−j, and the temporary solution is prolongated again to gain a start value for the next level j + 1. This scheme is repeated until the highest level J is reached.
Theorem 4.9. The nested iteration algorithm features a memory and time complexity of O(NJ), where J is the finest level of discretisation.
Proof. This result essentially follows from the summation of a geometric series, see (2.2.6),
O
J X
j=j0
Nj
= O(NJ) . (4.3.25)
where we use the fact that only a constant amount of iterations per level is required for a reduction of the error by a factor of 2.
Since prolongation and restriction are trivial operations in the wavelet setting, there are no additional dif-ficulties with respect to the implementation. We have thus constructed an iterative solver for the wavelet discretisation of the elliptic boundary value problem which achieves optimal computational complexity.
73
Chapter 4. Wavelet Methods for Linear Elliptic Partial Differential Equations
(−∆)0 (−∆ + 1)0 −∆ + 1
j D Da D Da D Da
3 232 229
4 103 103 93.7 93.7 350 244
5 166 118 151 107 393 255
6 207 129 188 117 433 262
8 244 137 221 125 493 271
10 263 141 239 128 531 276 12 274 144 249 130 557 278
n = 1 (−∆)0 −∆ + 1
j D Da D Da
3 532 519
4 51.7 51.7 697 627
5 175 101 739 646
6 570 337 768 664
8 1222 738 798 681 n = 2
(−∆)0 −∆ + 1
j D Da D Da
3 1238 1103
4 34.6 34.6 3410 1917 5 2956 1015 3930 2228 6 14600 5476 4330 2459
n = 3
Table 4.1: We show the condition numbers κ(A) (4.3.6). We provide three tables for the spatial dimensions n = 1, 2, 3. The suffix ()0 refers to homogeneous boundary conditions, while D and Da designate the type of diagonal scaling according to (2.2.48) and (4.3.11).
Condition Numbers in Uniform Discretisation
The convergence rate of the solution of (4.3.15) depends on the condition number of the stiffness matrix.
Although it is uniformly bounded and therefore independent of the level of resolution j, we are still inter-ested in its actual values on the various levels, since smaller values generally induce shorter computation times. These values depend on the choice of wavelet basis.
We have selected the construction of biorthogonal spline wavelets detailed in Section 3.3, with d = 2 and ˜d = 4. In Table 4.1, we have collected the condition numbers of the stiffness matrices for three different situations, namely for the operators −∆ and −∆ + 1 with homogeneous boundary conditions and the operator −∆ + 1 with inhomogeneous boundary conditions. We have devoted separate tables to the spatial dimensions 1, 2 and 3. For each combination, we list the condition numbers with either the classical diagonal scaling (2.2.48) or the operator-adapted scaling from (4.3.11).
First of all, the results confirm that the condition numbers are uniformly bounded. Secondly, we assert that the use of Da is always superior to the standard diagonal scaling by a factor between about 1.6 and 2.7. Finally, the condition numbers increase exponentially with the spatial dimension as expected by the tensor product approach.
In Table 4.2, we examine the effect of the transformations to the nodal basis from (3.3.74) and (3.3.75) on the condition numbers. This transformation does not lead to reduced condition numbers with homo-geneous boundary conditions. However, for inhomohomo-geneous boundary conditions the improvements are significant. In this scenario, the condition numbers of the stiffness matrix increase only with a factor less than 2 with the spatial dimension. These results confirm the observation made in the previous chapter that the transformation to the nodal basis is generally advantagous for free boundary conditions.
74
4.3. A Wavelet Method for Elliptic Problems
Table 4.2: In this table, we present condition numbers of the stiffness matrix with the transformation to the nodal basis from Section 3.3.5. The notation is the same as in Table 4.1. The transformation Cj (3.3.74) is only applicable for free boundary conditions, while Kj (3.3.75) can be applied in any case.