A Multigrid Tutorial part two

A Multigrid Tutorial

part two

William L. Briggs

Department of Mathematics University of Colorado at Denver

Van Emden Henson

Center for Applied Scientific Computing Lawrence Livermore National Laboratory

Steve McCormick

Department of Applied Mathematics University of Colorado at Boulder

Outline

• Nonlinear Problems

• Neumann Boundary Conditions • Anisotropic Problems

• Variable Mesh Problems

• Variable Coefficient Problems • Algebraic Multigrid

Nonlinear Problems

• How should we approach the nonlinear system and can we use multigrid to solve such a system? • A fundamental relation we’ve relied on, the

residual equation

does not hold, since, if A(u) is a nonlinear operator,

f

u

A

(

)

=

r

e

A

v

A

f

v

A

u

A

−

=

−

Þ

=

e

A

v

A

u

A

(

)

−

(

)

≠

(

)

The Nonlinear Residual Equation

• We still base our development around the residual

equation, now the nonlinear residual equation:

• How can we use this equation as the basis for a solution method?

v

A

f

v

A

u

A

(

)

−

(

)

=

−

(

)

r

v

A

u

A

(

)

−

(

)

=

f

u

A

(

)

=

Let’s consider Newton’s Method

• The best known and most important method for

solving nonlinear equations! • We wish to solve F(x) = 0 .

• Expand F in a Taylor series about x :

• Dropping higher order terms, if x+s is a solution, • Hence, we develop an iteration

0

=

F

(

s

)

+

s

F

′

(

x

)

∴

s

=

−

F

(

x

)

/

F

′

(

x

)

x F x F x x ) (′ ) ( − ←

F

(

x

+

s

) =

F

(

x

) +

sF

′(

x

) +

s

2

F

″ ( ξ)

Newton’s method for systems

• We wish to solve the system A(u) = 0. In vector

form this is

• Expanding A(v+e) in a Taylor series about v : u u u f u u u f u u u f u A ö ÷ ÷ ÷ ø ⋅⋅ ⋅ æ ç ç ç è = ö ÷ ÷ ÷ ÷ ÷ ø ) , … , , ( ⋅⋅ ⋅ ) , … , , ( ) , … , , ( æ ç ç ç ç ç è = ) ( 2 1 2 1 2 2 1 1 N N N N 0 0 0

e

v

J

v

A

e

v

Newton for systems (cont.)

• Where J(v) is the Jacobian system

• If u=v+e is a solution, 0 = A(v) + J(v) e and

• Leading to the iteration

J₍ v_{) =} è ç ç ç ç ç ç ç ç ç ç ç æ ∂f ₁ ∂u₁ ∂f ₁ ∂u₂ … ∂f ₁ ∂u_N ∂f ₂ ∂u₁ ∂f ₂ ∂u₂ … ∂f ₂ ∂u_N M M ⋅ ⋅⋅ M ∂f _N ∂u₁ ∂f _N ∂u₂ … ∂f _N ∂u_{N ø}÷ ÷ ÷ ÷ ÷ ÷ ÷ ÷ ÷ ÷ ÷ ö u ₌ v e _{= −} J₍v₎ −1A₍v₎

v

←

v

−

J

(

v

)

− 1

A

(

v

)

Newton’s method in terms of the

residual equation

• The nonlinear residual equation is

• Expanding A(v+e) in a two-term Taylor series about v :

• Newton’s method is thus:

v

←

v

+

J

(

v

)

−

1

r

v

A

f

r

=

−

(

)

r

v

A

e

v

A

(

+

)

−

(

)

=

r

v

A

e

v

J

v

A

(

)

+

(

)

−

(

)

=

r

e

v

J

(

)

=

How does multigrid fit in?

• One obvious method is to use multigrid to solve

J(v)e = r at each iteration step. This method is

called Newton-multigrid and can be very effective. • However, we would like to us multigrid ideas to

treat the nonlinearity directly.

• Hence, we need to specialize the multigrid components (relaxation, grid transfers, coarsening) for the nonlinear case.

What is nonlinear relaxation?

• Several of the common relaxation schemes have

nonlinear counterparts. For A(u)=f, we describe the nonlinear Gauss-Seidel iteration:

– For each j=1, 2, …, N

• Set the jth component of the residual to zero and solve for

v_j . That is, solve (A(v))_j = f_j .

• Equivalently,

– For each j=1, 2, …, N • Find s ∈ ℜ such that

where is the canonical jth unit basis vector

=

)

)

ε

+

(

(

A

v

s

_{j j}

f

_j ε

How is nonlinear Gauss-Seidel done?

• Each is a nonlinear scalar equation for

v_j . We use the scalar Newton’s method to solve! • Example: may be

discretized so that is given by

• Newton iteration for v_j is given by = ) ) ( ( A v _j f _j , = ) ( + ) ( ″ − u x u x eu( x) f = ) ) ( ( A v _j f _j = + − + − f e v h v v v j v j j j j _{− +} 2 1 1 2 j 1 ≤ j ≤ N − 1 v e f e v v v j v j v j h v v v j j ) + ( + − + − ← − + − j j j j j _{− +} 2 1 1 2 2 1

How do we do coarsening for

nonlinear multigrid?

• Recall the nonlinear residual equation

• In multigrid, we obtain an approximate solution v h

on the fine grid, then solve the residual equation on the coarse grid.

• The residual equation on appears as

r

v

A

e

v

A

2h

(

2h

+

2h

)

−

2h

(

2h

)

=

2h Ω2h

r

v

A

e

v

A

(

+

)

−

(

)

=

Look at the coarse residual equation

• We must evaluate the quantities on in

• Given v h_{, a fine-grid approximation, we restrict}

the residual to the coarse grid

• For v 2h _{we restrict} v h _by • Thus, Ω2h

r

v

A

e

v

A

2h

(

2h

+

2h

)

−

2h

(

2h

)

=

2h v A f I r2h _{= h}2h ₍ h ₋ h ₍ h _{) )}

v

I

v

2h

=

_h2h h v A f I v I A e v I A2h( _h2h h + 2h) = 2h ( _h2h h) + _h2h( h − h( h) )

We’ve obtained a coarse-grid

equation of the form .

• Consider the coarse-grid residual equation:

• We solve for and obtain

• We then apply the correction:

f

u

A

2h

(

2h

)

=

2h v A f I v I A e v I A2h_{( h}2h h ₊ 2h_{) =} 2h _{( h}2h h_{) + h}2h₍ h ₋ h₍ h_{) )}

f

2h

All quantities are known

u

2h coarse-grid unknown f u A2h₍ 2h_{) =} 2h u2h ₌ I_h2h vh ₊ e2h v I u e2h ₌ 2h _{− h}2h h h h h h

=

+

2

FAS, the Full Approximation

Scheme, two grid form

• Perform nonlinear relaxation on to obtain an approximation .

• Restrict the approximation and its residual • Solve the coarse-grid residual problem

• Extract the coarse-grid error

• Interpolate and apply the correction

e

I

v

v

h

=

h

+

h 2h f u Ah₍ h_{) =} h

v

h v I v2h _{= h}2h h _r2h _{= Ih}2h _{( f} h _{− A( v}h _{) )}

r

v

A

u

A

2h

(

2h

)

=

2h

(

2h

)

+

2h

v

u

e

2h

=

2h

−

2h

A few observations about FAS

• If A is a linear operator then FAS reduces directly to the linear two-grid correction scheme.

• A fixed point of FAS is an exact solution to the fine-grid problem and an exact solution to the fine-fine-grid problem is a fixed point of the FAS iteration.

A few observations about FAS,

continued

• The FAS coarse-grid equation can be written as where is the so-called tau correction.

• In general, since , the solution to the FAS coarse-grid equation is not the same as the solution to the original coarse-grid problem .

• The tau correction may be viewed as a way to alter the coarse-grid equations to enhance their

approximation properties.

f

u

A

2h

(

2h

)

=

2h

+

τ

2_hh

τ

_h2h ≠ τ_h2h 0 u2h

f

u

A

2h

(

2h

)

=

2h

Still more observations about

FAS

• FAS may be viewed as an inner and outer iteration: the outer iteration is the coarse-grid correction, the inner iteration the relaxation method.

• A true multilevel FAS process is recursive, using FAS to solve the nonlinear problem using . Hence, FAS is generally employed in a V- or W-cycling scheme.

And yet more observations about

FAS!

• For linear problems we use FMG to obtain a good initial guess on the fine grid. Convergence of

nonlinear iterations depends critically on having a good initial guess.

• When FMG is used for nonlinear problems the

interpolant is generally accurate enough to be in the basin of attraction of the fine-grid

solver.

• Thus, one FMG cycle, whether FAS, Newton, or Newton-multigrid is used on each level, should provide a solution accurate to the level of

discretization, unless the nonlinearity is extremely strong.

u

I

₂h_h 2h

Intergrid transfers for FAS

• Generally speaking, the standard operators (linear interpolation, full weighting) work effectively in FAS schemes.

• In the case of strongly nonlinear problems, the use of higher-order interpolation (e.g., cubic interpolation) may be beneficial.

• For an FMG scheme, where is the

interpolation of a coarse-grid solution to become a fine-grid initial guess, higher-order interpolation is always advised.

u

I

₂h_h 2h

What is in FAS?

• As in the linear case, there are two basic possibilities:

• is determined by discretizing the nonlinear operator A(u) in the same fashion as was employed to obtain , except that the coarser mesh spacing is used.

• is determined from the Galerkin condition where the action of the Galerkin product can be

captured in an implementable formula.

• The first method is usually easier, and more common.

A

h

(

u

h

)

A

2h

(

u

2h

)

A

2

h

(

u

2

h

)

A

2h

(

u

2h

)

Nonlinear problems: an example

• Consider

on the unit square [0,1] x [0,1] with homogeneous Dirichlet boundary conditions and a regular

Cartesian grid.

• Suppose the exact solution is

− ∆

u

(

x

,

y

) + γ

u

(

x

,

y

)

e

u( x, y)

=

f

(

x

,

y

)

Discretization of nonlinear example

• The operator can be written (sloppily) as

• The relaxation is given by

1 1 4 1 1 1 f e u u h2 + γ = ö ÷ ø − − − − æ ç è i j u h j i h j i_{, , ,} h j i_,

u

A

h

(

h

)

e u f u A u u u h j i h j i j i h h h j i h j i , , , , , ) + ( γ + − ) ) ( ( − ← ₄ 2 1 j i_,

FAS and Newton’s method on

• FAS • Newton’s Method − ∆u₍ x_, y_{) + γ}u₍ x_,y₎ eu(x,y) ₌ f ₍ x_,y₎ 1 10 100 1000 convergence factor 0.135 0.124 0.098 0.072

number of FAS cycles 12 11 11 10

1 10 100 1000

convergence factor 4.00E-05 7.00E-05 3.00E-04 2.00E-04

number of Newton iterations 3 3 3 4

γ

Newton, Newton-MG, and FAS on

• Newton uses exact solve, Newton-MG is inexact Newton with a fixed number of inner V(2,1)-cycles the Jacobian problem, overall stopping criterion

− ∆u₍ x_, y_{) + γ}u₍ x_,y₎ eu(x,y) ₌ f ₍ x_,y₎

Outer Inner

Method iterations iterations Megaflops

Newton 3 1660.6 Newton-MG 3 20 56.4 Newton-MG 4 10 38.5 Newton-MG 5 5 25.1 Newton-MG 10 2 22.3 Newton-MG 19 1 24.6 FAS 11 27.1 < || ||r ₂ 10−10

Comparing FMG-FAS and FMG-Newton

− ∆u₍ x_, y_{) + γ}u₍ x_,y₎ eu( x,y) ₌ f ₍ x_,y₎

We will do one FMG cycle using a single FAS V(2,1) -cycle as the “solver” at each new level. We then

follow that with sufficiently many FAS V(2,1)-cycles as is necessary to obtain ||r|| < 10-10_.

Next, we will do one FMG cycle using a

Newton-multigrid step at each new level (with a single linear V(2,1)-cycle as the Jacobian “solver.”) We then

follow that with sufficiently many Newton-multigrid steps as is necessary to obtain ||r|| < 10-10.

Comparing FMG-FAS and FMG-Newton

− ∆u₍ x_, y_{) + γ}u₍ x_,y₎ eu( x,y) ₌ f ₍ x_,y₎

Cycle Mflops Mflops Cycle

FMG-FAS 1.10E-02 2.00E-05 3.1 1.06E-02 2.50E-05 2.4 FMG-Newton FAS V 6.80E-04 2.40E-05 5.4 6.70E-04 2.49E-05 4.1 Newton-MG FAS V 5.00E-05 2.49E-05 7.6 5.10E-05 2.49E-05 5.8 Newton-MG FAS V 3.90E-06 2.49E-05 9.9 6.30E-06 2.49E-05 7.5 Newton-MG FAS V 3.20E-07 2.49E-05 12.2 1.70E-06 2.49E-05 9.2 Newton-MG FAS V 3.00E-08 2.49E-05 14.4 5.30E-07 2.49E-05 10.9 Newton-MG FAS V 2.90E-09 2.49E-05 16.7 1.70E-07 2.49E-05 12.6 Newton-MG FAS V 3.00E-10 2.49E-05 18.9 5.40E-08 2.49E-05 14.3 Newton-MG FAS V 3.20E-11 2.49E-05 21.2 1.70E-08 2.49E-05 16.0 Newton-MG 5.50E-09 2.49E-05 17.7 Newton-MG 1.80E-09 2.49E-05 19.4 Newton-MG 5.60E-10 2.49E-05 21.1 Newton-MG 1.80E-10 2.49E-05 22.8 Newton-MG 5.70E-11 2.49E-05 24.5 Newton-MG

||

Outline

• Nonlinear Problems

• Neumann Boundary Conditions • Anisotropic Problems

• Variable Mesh Problems

• Variable Coefficient Problems • Algebraic Multigrid

-1 N+2

Neumann Boundary Conditions

• Consider the (1-d) problem

• We discretize this on the interval [0,1] with grid spacing for j=1,2, …, N+1 .

• We extend the interval with two ghost points

)

(

=

)

(

″

−

u

x

f

x

0 < x < 1 u u _{′ (} 0_{) = ′ (} 1_{) =} 0 h j x_{j =} N h + = 1 ₁ 0 1 2 3 … j ... N-1 N N+1 0 x 1

We use central differences

• We approximate the derivatives with differences, using the ghost points:

• Giving the system

-1 0 1 j-1 j j+1 N N+1 N+2 h u u u _{(′ ) ≈} − 2 0 1 −1 h u u u′(1) ≈ N+₂2 − N h u u u x u_{″( j) ≈} − j − + j − j+ 2 1 1 2 = − + − f h u u u j j j j _{− +} 2 1 1 2 h u u_{1 1} = − ₋ 0 2 h u u_{N +2 N} = − 0 2

1

0

≤

j

≤

N

+

Eliminating the ghost points

• Use the boundary conditions to eliminate ,

• Eliminating the ghost points in the j=0 and j=N+1

equations gives the (N+2)x(N+2) system of equations:

u _{− 1} u_{N +2} h u u_{1 1} = − ₋ 0 2 u u_{N +2 = N} h u u_{N +2 N} = − 0 2 u u_{−1 = 1} = − + − f h u u u j j j j _{− +} 2 1 1 2

₀

≤

j

≤

N

+

1

2 2 f h u u 0 2 1 0 = − − 2 + 2 _{= f} h u u N N N + + 1 2 1

We write the system in matrix

form

• We can write , where

• Note that is (N+2)x(N+2), nonsymmetric, and the system involves unknowns and at the boundaries.

f

u

A

h h

=

h Ah ₌ 1 h2 è ç ç ç ç ç ç ç æ 2 −2 −1 2 ₋1 −1 2 ₋1 ⋅⋅⋅ ⋅⋅⋅ −1 2 ₋1 −1 2 _ø÷ ÷ ÷ ÷ ÷ ÷ ÷ ö

A

h

u

₀h

u

_Nh _{+ 1}

We must consider a compatibility

condition

• The problem , for and with is not well-posed!

• If u(x) is a solution, so is u(x)+c for any constant c. • We cannot be certain a solution exists. If one does,

it must satisfy

• This integral compatibility condition is necessary! If f(x) doesn’t satisfy it, there is no solution!

)

(

=

)

(

″

−

u

x

f

x

0 _< x _< 1 u u _{′ (} 0_{) = ′ (} 1_{) =} 0 − ó_õ 0 1 u_″(x₎ dx ₌ ó õ 0 1 f ₍x₎ dx − [u_′(1_{) −}u_′(0_{) ] =} ó õ 0 1 f ₍x₎ dx 0 ₌ ó õ 0 1 f ₍x₎ dx

The well-posed system

• The compatibility condition is necessary for a

solution to exist. In general, it is also sufficient, which can be proven that is a well-behaved operator in the space of functions u(x) that have zero mean.

• Thus we may conclude that if f(x) satisfies the

compatibility condition, this problem is well-posed:

• The last says that of all solutions u(x)+c we choose

∂ ∂ − ₂ 2 x

)

(

=

)

(

″

−

u

x

f

x

0 _< x _< 1 u u _{′ (} 0_{) = ′ (} 1_{) =} 0 = ) ( õ ó1 0 x d x u 0

The discrete problem is not well posed

• Since all row sums of are zero,

• Putting into row-echelon form shows that hence

• By the Fundamental Theorem of Linear Algebra, has a solution if and only if

• It is easy to show that

• Thus, has a solution if and only if • That is,

A

h 1h _∈ NS₍ Ah ₎

A

h

1

S

N

m

i

d

(

(

A

h

)

)

=

NS₍Ah _{) =} span ₍ 1h ₎

A

h A S N f h _{⊥ ( (} h₎ T₎ c A S N _{( (} h₎T_{) = (}1_/2_, 1_, 1_{, . .. ,} 1_, 1_/2₎ T

f

u

A

h h

=

h c f h _{⊥ (} 1_/2_, 1_, 1_{, . .. ,} 1_, 1_/2₎ T 0 2 1 2 1 f f f ₁ 1 0 jh Nh N j h _{+ + =}

å

+ =

We have two issues to consider

• Solvability. A solution exists iff

• Uniqueness. If solves so does • Note that if then

and solvability and uniqueness can be handled together • This is easily done. Multiply 1st and last equations by

1/2, giving A S N f h _{⊥ ( (} h₎ T₎

u

h Ahuh ₌ f h uh + c1h Ah _{= (}Ah₎T NS ₍ Ah_{) =} NS _{( (} Ah₎ T₎ Ah ₌ 1 h2_çç ç ç ç æ 1 ₋1 −1 2 ₋1 −1 2 ₋1 ⋅⋅⋅ ⋅⋅⋅ ÷÷ ÷ ÷ ÷ ö

The new system is symmetric

• We have the symmetric system :

• Solvability is guaranteed by ensuring that is orthogonal to the constant vector :

1 h2 è ç ç ç ç ç ç ç æ 1 ₋1 −1 2 ₋1 −1 2 ₋1 ⋅⋅⋅ ⋅⋅⋅ −1 2 ₋1 −1 1 _ø÷ ÷ ÷ ÷ ÷ ÷ ÷ ö è ç ç ç ç ç ç ç ç ç æ u₀h u₁h u₂h M u_Nh u_Nh _+1ø÷ ÷ ÷ ÷ ÷ ÷ ÷ ÷ ÷ ö = è ç ç ç ç ç ç ç ç ç ç ç æ f₀h_/2 f₁h f₂h M f_Nh f_Nh _+1/2_ø÷÷ ÷ ÷ ÷ ÷ ÷ ÷ ÷ ÷ ÷ ö

A

h

u

h

=

f

h

f

h

1

h f f N h_j j h h = = , 1

å

+ 0 = 1 0

The well-posed discrete system

• The (N+3)x(N+2) system is:

or, more simply

= − + − f h u u u j j j j _{− +} 2 1 1 2

₀

1

+

≤

≤

j

N

u_{0 −} u₁ h2 = f₀ 2 − u_{N +} u_{N +1} h2 = f _{N +1} 2

A

h

u

h

=

f

h

å

h i N i + = 1 0 u ₌ 0 uh_, 1h ₌ 0

Multigrid for the Neumann Problem

• We must have the interval endpoints on all grids

• Relaxation is performed at all points, including endpoints:

• We add a global Gram-Schmidt step after relaxation on each level to enforce the zero-mean condition

x_Nh_/2 x₀h x₁h x_Nh x_N2h_/2 x_N2h_/4 x₀2h f h v v₀h_{← 1}h ₊ 2 h₀ v v v h f v_Nh _{+1 ←} v_Nh ₊ h2f _Nh ₊₁ h j h j h j j h _← − + + + + 2 1 1 2 v v v h h h h h h h , , − ← 1 1 1 1

Interpolation must include the

endpoints

• We use linear interpolation:

I₂h_h ö ÷ ÷ ÷ ÷ ÷ ÷ ÷ ÷ ÷ ÷ ÷ / / / / / / / / æ ç ç ç ç ç ç ç ç ç ç ç = 2 1 2 1 1 2 1 2 1 1 2 1 2 1 1 2 1 2 1 1

Restriction also treats the

endpoints

• For restriction, we use , yielding the values

f

f

f

2₀h

=

₀h

+

₁h

4

1

2

1

f

f

f

_N2h_{+ 1}

=

h_N

+

_Nh _{+ 1}

2

1

4

1

f

f

f

f

2_jh

=

₂h_{j 1}

+

h_2j

+

h_{2j 1}

4

1

2

1

4

1

+ −

I

I

_h2h

=

(

₂h_h

)

T

2

1

The coarse-grid operator

• We compute the coarse-grid operator using the Galerkin condition

I

A

I

A

2h

=

_h2h h ₂h_h A2h ₌ 1 4h2 è ç ç ç ç ç ç ç æ 1 ₋1 −1 2 ₋ 1 −1 2 ₋ 1 ⋅⋅⋅ ⋅⋅⋅ − 1 2 ₋1 − 1 1 _ø÷ ÷ ÷ ÷ ÷ ÷ ÷ ö − ε − ε ε ε 2 2 2 2 2 0 2 2 2 0 2 2 0 h h h h h h h h h h h h h 1 1 2 1 0 2 1 0 2 1 1 0 1 I A I h h I A I j2h _{= 0 1}

Coarse-grid solvability

• Assuming satisfies , the solvability

condition, we can show that theoretically the coarse-grid problem is also solvable. • To be certain numerical round-off does not perturb

solvability, we incorporate a Gram-Schmidt like step each time a new right-hand side is generated for the coarse grid:

f

h f h_, 1h ₌ 0 v A f I u A2h 2h _{= h}2h ₍ h ₋ h h₎ f 2h f f f 2 2 2 2 2 2 2 _h h h h h h h , , − ← 1 1 1 1

Neumann Problem: an example

• Consider the problem ,

which has as a solution for any c

(c=-1/12 gives the zero mean solution).

−

=

)

(

″

−

u

x

2

x

1

1

0

<

x

<

u

(′

0

)

=

u

′

(

1

)

=

0

c x x x u_{( ) =} 2 ₋ 3 ₊ 3 2

grid size average number

N conv. factor of cycles

31 6.30E-11 0.079 9.70E-05 9 63 1.90E-11 0.089 2.40E-05 10 127 2.60E-11 0.093 5.90E-06 10 255 3.70E-11 0.096 1.50E-06 10 511 5.70E-11 0.100 3.70E-07 10 1027 8.60E-11 0.104 9.20E-08 10 2047 2.10E-11 0.112 2.30E-08 10 || ||rh ||eh ||

Outline

• Nonlinear Problems

• Neumann Boundary Conditions • Anisotropic Problems

• Variable Mesh Problems

• Variable Coefficient Problems • Algebraic Multigrid

✔

✔

Anisotropic Problems

• All problems considered thus far have had as the off-diagonal entries.

• We consider two situations when the matrix has two different constant on the off-diagonals.

These situations arise when

– the (2-d) differential equation has constant, but different, coefficients for the derivatives in the coordinate directions

– the discretization has constant, but different, mash spacing in the different coordinate directions

− 1 h2

We consider two types of anisotropy

• Different coefficients on the derivatives

discretized on a uniform grid with spacing h . • Constant, but different, mesh spacings:

=

α

−

−

u

x

x

u

y

y

f

N

h

h

_x

=

=

1

h

h

_y x

α

=

h_y

Both problems lead to the same

stencil

− + − α + − + − h u u u h u u u_{j − ,k j,k j + ,k j,k − j,k j,k +} 2 1 1 2 1 1 2 2 ö ÷ ø æ ç è − + − + − + − u u u h u u u h k j k j k j k j k j k j α + , , − , , + , , − 2 1 1 2 1 1 2 2 h Ah ö_÷ ø α − α − + − α − æ ç è = 1₂ 1 2 2 1

Why standard multigrid fails

• Note that has weak connections in the

y-direction. MG convergence factors degrade as _α gets small. Poor performance at _α = 0.1 .

• Consider _{α Þ} 0 .

• This is a collection of disconnected 1-d problems!

• Point relaxation will smooth oscillatory errors in the x -direction (strong connections), but with no connections in the y-direction the errors in that direction will

generally be random, and no point relaxation will have the smoothing property in the y-direction.

h Ah ö_÷ ø α − α − + − α − æ ç è = 1₂ 1 2 2 1 h Ah ö_÷ ø − α + − æ ç è Þ 0 1 2 2 1 0 1 2

0 5 10 15 0 5 10 15 0 0.2 0.4 0.6 0.8 1 l k

We analyze weighted Jacobi

• The eigenvalues of the weighted Jacobi iteration

matrix for this problem are

0 2 4 6 8 10 12 14 16 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 2 4 6 8 10 12 14 16 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 λ_i,l = 1 ₋ 2ω 1_{+ α è}ç æ _sin2 è ç æ i_π 2N _ø÷ ö _{+ αsin2} è ç æ l_π 2N _ø÷ ö ø ÷ ö

i

l

i

l

Two strategies for anisotropy

•

Semicoarsening

Because we expect MG-like convergence for the 1-d problems along lines of constant y, we should coarsen the grid in the x -direction, but not in the y-direction.

•

Line relaxation

Because the the equations are strongly coupled in the x-direction it may be

advantageous to solve simultaneously for entire lines of unknowns in the x-direction (along lines of

Semicoarsening with point

relaxation

• Point relaxation on smooths in the x -direction. Coarsen by removing every other y-line.

• We do not coarsen along the remaining y-lines.

• Semicoarsening is not as “fast” as full coarsening. The number of points on is about half the number of points on , instead of the usual one-fourth.

h Ah ö_÷ ø α − α − + − α − æ ç è = 1₂ 1 2 2 1

Ω

2

h

Ω2h

Ω

h

Ωh

Interpolation with semicoarsening

• We interpolate in the 1-dimensional way along each

line of constant y.

• The coarse-grid correction equations are

v v v₂h_{j,k = 2}h_{j,k + j}2_,h_k v v v v 2 1 2 1 2 1 2 h k j h k j h k j h k j , + , , + , + + + = 2

Line relaxation with full

coarsening

• The other approach to this problem is to do full coarsening, but to relax entire lines (constant y) of variables simultaneously.

• Write in block form as

where and

A

h Ah ₌ è ç ç ç ç ç æ D −cI −cI D ₋cI −cI D ₋cI ⋅⋅⋅ ⋅⋅⋅ −cI −cI D _ø÷ ÷ ÷ ÷ ÷ ö h c ₌ α 2 D = 1 h2_çç ç æ 2+2α −1 −1 2₊2_{α −}1 ⋅⋅⋅ _÷÷÷ ö

Line relaxation

• One sweep of line relaxation consists of solving a tridiagonal system for each line of constant y.

• The kth such system has the form where

is the kth subvector of with entries and the kth right-hand side subvector is

• Because D is tridiagonal, the kth system can be solved very efficiently.

g

v

D

_kh

=

_kh v_kh

v

h ₍ v_kh_{) j =} v_jh_,k ) + ( α + = ) ( v v h f g_kh _{j j}h_{,k j}h_{,k −1 j}h_{,k +1} 2

0 5 10 15 0 5 10 15 -0.2 0 0.2 0.4 0.6 0.8

Why line relaxation works

• The eigenvalues of the weighted block Jacobi

iteration matrix are

ö ÷ ø ö ÷ ø π æ ç è α + ö ÷ ø π æ ç è æ ç è α + ö ÷ ø æ ç è ω − = λ N i l i π , sin ₂ sin ₂ n i s 2 2 1 2 2 2 2 N l N i

i

l

i

0 2 4 6 8

l

10 12 14 16 -0.2 0 0.2 0.4 0.6 0.8 0 2 4 6 8 10 12 14 16 -0.2 0 0.2 0.4 0.6 0.8

Semicoarsening

with

line relaxation

• We might not know the direction of weak coupling

or it might vary.

• Suppose we want a method that can handle either or

• We could use semicoarsening in the x-direction to handle and line relaxation in the y-direction to take care of . A₁h ₌ 1 h2_èç æ − α − 1 2₊ 2_{α −} 1 − α ø÷ ö _A 2h = 1 h2_èç æ −1 − α 2₊2_{α − α} −1 ø÷ ö

A

₁h A₂h

Semicoarsening

with

line relaxation

• The original grid • Original grid viewed as a stack of “pencils.” Line relaxation is used to solve problem along • Coarsening is done by deleting every other pencil

An anisotropic example

• Consider with u=0 on the boundaries of the unit square, and stencil given by

• Suppose so the exact solution is given by

• Observe that if _α is small, the x-direction dominates while if _α is large, the y-direction dominates

=

α

−

−

u

xx

u

yy

f

h Ah ö_÷ ø α − α − + − α − æ ç è = 1₂ 1 2 2 1 f ₍x_,y_{) =} 2₍y ₋ y2_{) +} 2_{α (}x ₋ x2₎ u₍ x_,y_{) = (} y ₋ y2_{) (} x ₋ x2₎

0 0.2 0.4 0.6 0.8 1 0 0.5 1 0 0.02 0.04 0.06 0.08 0.1

What is smooth error?

• Consider _α=0.001 and suppose point Gauss-Seidel is applied to a random initial guess. The error

after 50 sweeps appears as:

0 0 . 1 0 . 2 0 . 3 0 . 4 0 . 5 0 . 6 0 . 7 0 . 8 0 . 9 1 0 0 . 0 2 0 . 0 4 0 . 0 6 0 . 0 8 0 . 1 0 . 1 2 0 0 . 1 0 . 2 0 . 3 0 . 4 0 . 5 0 . 6 0 . 7 0 . 8 0 . 9 1 0 0 . 0 2 0 . 0 4 0 . 0 6 0 . 0 8 0 . 1 0 . 1 2

Error along line of constant x

y x

We experiment with 3 methods

• Standard V(2,1)-cycling, with point Gauss-Seidel

relaxation, full coarsening, and linear interpolation • Semicoarsening in the x-direction. Coarse and fine

grids have the same number of points in the y -direction. 1-d full weighting and linear

interpolation are used in the x-direction, there is no y-coupling in the intergrid transfers

• Semicoarsening in the x-direction combined with line relaxation in the y-direction. 1-d full weighting and interpolation.

With semicoarsening, the

operator must change

• To account for unequal mesh spacing, the residual and relaxation operators must use a modified

stencil

• Note that as grids become coarser, grows while remains constant.

A ₌ è ç ç ç ç ç ç ç ç ç æ ₋ α hy2 − 1 hx2 èç æ 2 hx2 + 2α hy2 ø÷ ö − 1 hx2 − 1 hy2 _ø÷ ÷ ÷ ÷ ÷ ÷ ÷ ÷ ÷ ö

h

x

h

_y

How do the 3 methods work for

various values of

αααα

?

• Asymptotic convergence factors:

• Note: semicoarsening in x works well for _α < .001

but degrades noticeably even at _α = .1

scheme 1000 100 10 1 0.1 0.01 0.001 1E-04

V(2,1)-cycles 0.95 0.94 0.58 0.13 0.58 0.90 0.95 0.95

emicoarsening in x 0.94 0.99 0.98 0.93 0.71 0.28 0.07 0.07

semiC / line relax 0.04 0.08 0.08 0.08 0.07 0.07 0.08 0.08

α

A semicoarsening subtlety

• Suppose _α is small, so that semicoarsening in x is used. As we progress to coarser grids, gets small but remains constant.

• If, on some coarse grid, becomes comparable to , the problem effectively becomes

recoupled in the y-direction. Continued

semicoarsening can produce artificial anisotropy, strong in the y-direction.

• When this occurs, it is best to stop semicoarsening and continue with full coarsening on any further

coarse grids.

h

_x− 2

hy− 2

h

_x− 2

Outline

• Nonlinear Problems

• Neumann Boundary Conditions • Anisotropic Problems

• Variable Mesh Problems

• Variable Coefficient Problems • Algebraic Multigrid

✔

✔

✔

Variable Mesh Problems

• Non-uniform grids are commonly used to

accommodate irregularities in problem domains • Consider how we might approach the 1-d problem

posed on this grid:

− u_″( x_{) =} f ₍ x₎ 0 < x < 1

u₍ 0_{) =} u₍ 1_{) =} 0

x₀ x_{j −1} x_j xj +1 x_N

We need some notation for the

mesh spacing

• Let N be a positive integer. We define the spacing interval between and :x_j x_{j +1}

h_{j +1/2 ≡} x_{j +1 −} x_j j ₌ 0_, 1_{, . . . ,} N ₋ 1

x₀ x_{j −1} x_j xj +1 x_N

We define the discrete

differential operator

• Using second order finite differences (and

plugging through a mess of algebra!) we obtain this discrete representation for the problem:

• where − αh_j u_jh_{−1 + ( αj}h _{+ βj}h₎ u_jh _{− β}h_j u_jh_{+1 =} f _jh 1 _≤ j _≤ N ₋1 u₀h ₌ u_Nh ₌ 0 αh_j = _h 2 j ₋1_/2(hj ₋1_/2 + hj ₊1_/2) β_jh = _h 2 j +1/2(hj −1/2 + hj +1/2)

We modify standard multigrid to

accommodate variable spacing

• We choose every second fine-grid point as a

coarse-grid point

• We use linear interpolation, modified for the spacing. If , then for

x₀h x₀2h x_Nh x_N2h_/2 x₂h_j x2_jh v₂h_{j =} v_j2h v₂h_{j +1} = h_{2j +3/2} v_j2h ₊ h_{2j +1/2} v_j2h₊₁ h_{2j +1/2 +} h_{2j +3/2} vh ₌ I₂h_h v2h 1 _≤ j _≤ N_/2₋ 1

We use the variational properties

to derive restriction and .

• This produces a stencil on that is similar, but not identical, to the fine-grid stencil. If the

resulting system is scaled by , then the Galerkin product is the same as the fine-grid stencil.

• For 2-d problems this approach can be generalized readily to logically rectangular grids. However, for irregular grids that are not logically rectangular,

A

2

h

A

2h

=

I

_h2h

A

h

I

₂h_h

I

h

2

h

=

1

2

(

I

2

h

h

)

T

Ω

2h ( h_{j −1/2 +} h_{j +1/2)}

Outline

• Nonlinear Problems

• Neumann Boundary Conditions • Anisotropic Problems

• Variable Mesh Problems

• Variable Coefficient Problems • Algebraic Multigrid

✔

✔

✔

Variable coefficient problems

• A common difficulty is the variable coefficient

problem, given in 1-d by

where a(x) is a positive function on [0,1]

• We seek to develop a conservative, or self-adjoint, method for discretizing this problem.

• Assume we have available to us the values of a(x) at the midpoints of the grid

− (

a

(

x

)

u

′(

x

) ) ′ =

f

(

x

)

0 _< x _< 1

u

(

0

) =

u

(

1

) =

0

a_{j +1/2 ≡} a₍x_{j +1/2)}

We discretize using central

differences

• We can use second-order differences to

approximate the derivatives. To use a grid spacing of h we evaluate a(x)u’(x) at points midway

between the gridpoints:

(

a

(

x

)

u

′(

x

) ) ′ ≈

( au_{′) |xj +1/2 − (} a u_{′) |xj −1/2} h

+

O

(

h

2

)

x_j

x

_jh _xh j ₊ 1 x_jh_{− 1} x₀h

x

_Nh

We discretize using central

differences

• To evaluate we must sample a(x) at the point and use second order differences:

where (au_{′) |}x_{j +1/2} x_{j +1/2} ( a u_{′ ) |xj +1/2 ≈} a_{j +1/2} uj +1 − uj h ( a u′ ) |xj −1/2 ≈ aj − 1/2 u_{j −} u_{j −1} h a_{j + 1/2} ≡ a( x_{j + 1/2}) h h h

Points used to evaluate (au’ ) ’ at x_j Points used to

evaluate u’ at

The basic stencil is given

• We combine the differences for u’ and for (au’ ) ’

to obtain the operator

and the problem becomes, for −a_{j +1/2} u_{j +1 −} u_{j −1} h + aj −1/2 u_{j −} u_{j −1} h h − (a₍x_j) u_{′ (} x_{j) ) ′ (}x_{j) ≈} 1 h2( −aj −1/2uj −1 + (aj −1/2 + aj +1/2) uj − aj +1/2uj +1) = f j 1 _≤ j _≤ N ₋ 1 u_{0 =} u_{N =} 0

Coarsening the variable

coefficient problem

• A reasonable approach is to use a standard

multigrid algorithm with linear interpolation, full weighting, and the stencil

where

• The same stencil can be obtained via the Galerkin

A2h ₌ 1 (2h)2( −aj −1/2 2h _a j2h₋1_/2+ aj2h₊1_/2 −aj2h₊1_/2 ) a_j2h_{+1/2 =} a2j+1/2 h _{+ a} 2hj ₊3_/2 2

Standard multigrid degrades if

a(x)

is highly variable

• It can be shown that the variable coefficient

discretization is equivalent to using standard multigrid with simple averaging on the Poisson problem on a certain variable-mesh spacing.

• But simple averaging won’t accurately represent smooth components if is close to but far from .

-(a(x)u’)’ = f -u’’(x) = f x₂h_{j +1} x₂h_j x₂h_{j +2} x₂h_{j +1} x₂h_j x₂h_{j +2} x_j2h _xj2h₊₁

One remedy is to apply

operator

interpolation

• Assume that relaxation does not change smooth error, so the residual is approximately zero.

Applying at yields • Solving for −a₂h_{j +1/2} e₂h_{j + (}a₂h_{j +1/2 +} a₂h_{j +3/2)} e₂h_{j +1 −} a₂h_{j +3/2}e₂h_{j +2} h2 = 0 x₂h_{j +1} e₂h_{j +1}

e

₂h_{j + 1}

=

a2j +1/2 h _e j2h + a2hj ₊3_/2ej2h₊1 a₂h_{j +1/2 +} a₂h_{j +3/2}

Thus, the operator induced

interpolation is

• And, as usual, the restriction and coarse-grid operators are defined by the Galerkin relations

v

2

h

j

+

1

=

a

2j +1/2 h

v

j2h

+

a

2hj ₊3_/2

v

j2h₊1

a

₂h_{j +1/2}

+

a

₂h_{j +3/2}

v

2

h

j

=

v

j

2

h

A

2h

=

I

_h2h

A

h

I

₂h_h

I

h2h

=

c

(

I

2hh

)

T

A Variable coefficient example

• We use V(2,1) cycle, full weighting, linear interpolation. • We use and

a

(

x

) = ρ

sin

(

k

π

x

)

a₍ x_{) = ρ} rand ₍ k _πx₎

k=3 k=25 k=50 k=100 k=200 k=400 0 0.085 0.085 0.085 0.085 0.085 0.085 0.085 0.25 0.084 0.098 0.098 0.094 0.093 0.083 0.083 0.5 0.093 0.185 0.194 0.196 0.195 0.187 0.173 0.75 0.119 0.374 0.387 0.391 0.39 0.388 0.394 0.85 0.142 0.497 0.511 0.514 0.514 0.526 0.472 0.95 0.191 0.681 0.69 0.694 0.699 0.745 0.672 a₍ x_{) = ρ} rand ₍k _πx₎

a

(

x

) = ρ

sin

(

k

π

x

)

ρ

Outline

• Nonlinear Problems

• Neumann Boundary Conditions • Anisotropic Problems

• Variable Mesh Problems

• Variable Coefficient Problems • Algebraic Multigrid

✔

✔

✔

✔

✔

Algebraic multigrid:

for unstructured-grids

● Automatically defines coarse “grid” ● AMG has two distinct phases:

— setup phase: define MG components — solution phase: perform MG cycles

● AMG approach is opposite of geometric MG — fix relaxation (point Gauss-Seidel)

— choose coarse “grids” and prolongation, P, so

that error not reduced by relaxation is in

range(P)

— define other MG components so that

coarse-grid correction eliminates error in range(P)

(i.e., use Galerkin principle)

(in contrast, geometric MG fixes coarse grids, then defines suitable operators and smoothers)

● Solve Phase

— Standard multigrid operations, e.g., V-cycle, W-cycle,

FMG, FAS, etc

AMG has two phases:

• Setup Phase

– Select Coarse “grids,” – Define interpolation,

– Define restriction and coarse-grid operators

..

.

,

,

=

,

Ω

m + 1

m

1

2

m

I

_mm _{+ 1}

,

=

1

,

2

,

.

..

I I_mm +1 _{= ( m}m₊₁₎T A m + 1 ₌ I_mm + 1A m I_mm_{+ 1}

● Note: Only the selection of coarse grids does not

AMG fundamental concept:

Smooth error

= “small” residuals

• Consider the iterative method error recurrence • Error that is slow to converge satisfies

• More precisely, it can be shown that smooth error satisfies

e

A

Q

I

e

k+1

=

(

−

−1

)

k

≈

)

−

(

I

Q

−1

A

e

e

Þ

Q

−

1

A

e

≈

0

≈

Þ

r

0

e

r

A

D

−1

«

(

1

)

AMG uses

strong connection

to

determine MG components

• It is easy to show from (1) that smooth error satisfies

• Define i is strongly connected to j by • For M-matrices, we have from (2)

• implying that smooth error varies slowly in the direction of strong connections

e

e

D

e

e

A

,

«

,

≤ θ < , } − { θ ≥ −a a_ik i k j i max _≠ 0 1

)

(

2

1 « 2 2 1

å

i j i i i j i j i ≠ ö ÷ ø − æ ç è ö ÷ ø − æ ç è e e e a a 2

Some useful definitions

• The set of strong connections of a variable , that is, the variables upon whose values the value of depends, is defined as

• The set of points strongly connected to a variable is denoted: .

• The set of coarse-grid variables is denoted C. • The set of fine-grid variables is denoted F.

• The set of coarse-grid variables used to

interpolate the value of the fine-grid variable is denoted .

S

_i

ü

ý

þ

−

θ

>

−

:

ì

í

î

=

j

a

a

_ij i j j i

m

a

x

≠

u

_i

S

_iT

=

{

j

:

j

∈

S

_i

}

C

_i

u

u

_i

u

_i

Choosing the Coarse Grid

• Two Criteria

– (C1) For each , each point should either be in or should be strongly connected to at least one point in

– (C2) should be a maximal subset with the property

that no two -points are strongly connected to each other.

• Satisfying both (C1) and (C2) is sometimes

impossible. We use (C2) as a guide while enforcing (C1).

i

∈

F

C

C

C

C

i

S

j

∈

i

Selecting the coarse-grid points

C-point selected (point with largest “value”) Neighbors of C-point become F-points Next C-point selected (after updating “values”) F-points selected, etc.

Examples: Laplacian Operator

5-pt FD, 9-pt FE (quads), and 9-pt FE (stretched quads)

ö

÷

ø

−

−

−

−

−

−

−

−

æ

ç

è

1

1

8

1

1

1

1

1

1

ö

÷

ø

−

−

−

−

æ

ç

è

1

4

1

1

1

ö

÷

ø

−

−

−

−

−

−

æ

ç

è

2

1

8

4

2

1

1

4

1

5-pt FD 9-pt FE (quads) 9-pt FE (stretched quads)

Prolongation

is based on smooth

error, strong connections (from

M-matrices)

Prolongation :

Smooth error is given by:

i C C C F F F

e

a

e

a

r

_{ij j} N j i i i i

=

+

å

≈

∈ _i

0

∈

,

ω

∈

,

ì

ï

í

ï

î

=

)

(

e

i

F

C

i

e

e

P

k

k

i

C

k

i

i

å

∈

_i

Prolongation

is based on smooth

error, strong connections (from

M-matrices)

The definition of smooth error,

Sets:

Strongly connected -pts. Strongly connected -pts. Weakly connected points.

i C C C F F F C_{i C} D_is D_iw F e a e a _{ij j} i j i i i ≈ −

å

≠ Gives:

e

a

_{ii i}

Strong C Strong F Weak pts.

−

−

−

≈

å

å

å

_{ij j} D j j j i D j j j i C j ∈ _{i ∈ i}s _{∈ i}w

e

a

e

a

e

a

Finally

, the prolongation

weights are defined

• In the smooth-error relation, use for weak connections. For the strong F-points use :

yielding the prolongation weights:

e e_j = _i a e a e _jk C k k k j C k j ö÷ ø æ ç è ⁄ ö ÷ ø æ ç è =

å

å

∈ ∈ _{i i} a a a w n i D n i i a a a D j j i j i + + − =

å

å

∈ ∈ w i m k i C m j k k i s i

å

_∈

AMG setup costs:

a bad rap

• Many geometric MG methods need to compute prolongation and coarse-grid operators

• The only additional expense in the AMG setup phase is the coarse grid selection algorithm

• AMG setup phase is only 10-25% more

expensive than in geometric MG

and may be considerably less than that!

AMG Performance:

Sometimes a Success Story

• AMG performs extremely well on the model

problem (Poisson’s equation, regular grid)- optimal convergence factor (e.g., 0.14) and scalability with increasing problem size.

• AMG appears to be both scalable and efficient on diffusion problems on unstructured grids (e.g., 0.1-0.3).

• AMG handles anisotropic diffusion coefficients on irregular grids reasonably well.

• AMG handles anisotropic operators on structured and unstructured grids relatively well (e.g., 0.35).

So, what could go wrong?

Strong F-F connections: weights are dependent on each other

• For point the value is interpolated from , , and is needed to make the interpolation weights for approximating .

• For point the value is interpolated from , , and is needed to make the interpolation weights for approximating .

• It’s an implicit system!

C k C k 1 1 ∈ ∈ j i C k_{2∈ i} C k_{3∈ i}

i

j k4∈ Cj

e

_i e_j e_j

k

₁

i

k₂ k₂

j

e

_i

k

₁

Is there a fix?

• A Gauss-Seidel like iterative approach to weight definition is implemented. Usually two passes suffice. But does it work?

theta Standard Iterative

0.25

0.47

0.14

0.5

0.24

0.14

0.25

0.83

0.82

0.5

0.53

0.23

Convergence factors for

Laplacian, stretched quadrilaterals

∆

=

∆

x

1

0

y

∆

=

∆

x

1

0

0

y

● Frequently, it does:

AMG for

systems

• How can we do AMG on systems?

• Naïve approach: “Block” approach (block Gauss-Seidel,

using scalar AMG to “solve” at each cycle)

ö ÷ ø æ ç è = ö ÷ ø æ ç è ö ÷ ø æ ç è g f v u A A A A 2 2 1 2 2 1 1 1 u A g A v v A f A u ) − ( ) ( ← ) − ( ) ( ← 1 2 1 2 2 2 1 1 1 1 − −

● Great Idea! Except that it doesn’t work! (relaxation

AMG for systems: a solution

• To solve the system problem, allow interaction

between the unknowns at all levels: and

• This is called the “unknown” approach.

• Results: 2-D elasticity, unifor