Non-Linear Methods

Most non-linear methods involve the iterative search for the global minimum of the objective function

σλ = arg min

σ∈_Rn

Φ=kF(σ)−vk2+λ2Ψ(σ), (2.52)

where the absolute valuesσandvare replaced by changesδσandδv, if differential measure- ments are reconstructed. In each iteration, asearch directionis computed and the algorithm moves in this direction by a certain distance (step size). With the new value forσ, the forward problem is re-evaluated and if the functional value is still above a defined stopping criterion, another iteration is performed.

Steepest Descent

The steepest descent method minimises the function Φ(σ) by finding the steepest descent direction in each iteration stepk, which is the negative gradient−∇Φat pointσk. The gradient

is linearly approximated with the Jacobian matrixJσk at this point

∇Φ(σk) =J>σk F(σk)−v

+λ2∇Ψ(σk). (2.53)

and the solution is then updated according to step sizeτk

σk+1=σk−τk∇Φ(∆σk). (2.54)

The convergence of the steepest descend method can be very slow for problems with an anisotropic optimisation surface (Shewchuk, 1994). Therefore the conjugate gradient method is almost always a better choice.

Conjugate Gradient

The steepest descend method builds a solution by iteratively moving in orthogonal directions, thereby often moving in the same direction several times. Theconjugate gradientmethod aims at moving in each direction just once, which is achieved by Gram-Schmidt conjugation of the residuals, rather than the search directions (Shewchuk, 1997). The Polak-Ribière implementation of the non-linear CG algorithm is given as follows

Algorithm 3Non-linear Conjugate Gradients (Polak-Ribière)

k= 0,σ0 =initial guess

d0 =r0 =−∇Φ(σ0)

whilestopping criterion not fulfilleddo

τk = arg minτΦ(σk+τdk) σk+1=σk+τkdk rk+1 =−∇Φ(σk+1) βk+1 = max r> k+1(rk+1−rk) r> krk ,0 dk+1 =rk+1+βk+1dk k=k+ 1 end while

The advantage of the CG algorithm is a much faster convergence than for steepest descent and also that the Hessian matrix does not need to be calculated, when compared to Newton methods. This makes CG computationally and memory efficient and thus interesting for large systems.

Newton Methods

Since CG only uses the first-order derivative, its convergence can be slow.Newton methodstry to find the unique minimiser of the quadratic approximation of the objective function

Φ(σk+dk)≈Φ(σk) +∇Φ(σk)>dk+

1 2d

>

k∇2Φ(σk)dk, (2.55)

by setting the derivative to zero and therefore

∇2Φ(σk)dk =−∇Φ(σk). (2.56)

The new search direction is thus given by

dk=− ∇2Φ(σk)

−1

∇Φ(σk), (2.57)

where the difficulty lies in the computation of the second derivative of the objective function, the so-calledHessianofΦ. The first and second derivatives are given by:

∇Φ(σk)−→ ∇F>(σk) (F(σk)−v) +λ2∇Ψ(σk) (2.58)

To simplify the computation of the Hessian matrix, theGauss-Newtonmethod approximates it by omitting the second derivative of the forward map

∇2Φ(σk)≈J>σkJσk +λ

2_∇2_Ψ(σ

k). (2.60)

On large meshes the Hessian matrix cannot be held in memory, let alone be inverted. Conse- quently, Krylov subspace methods such as generalised minimal residuals (GMRes) or CG are used to solve the system

∇Φ(σk) =− J>_σ kJσk+λ 2_∇2_Ψ(σ k) dk (2.61) =J>_σ k(Jσkdk)−λ 2_∇2_Ψ(σ k)dk, (2.62)

where the brackets highlight the order of the computations. This way, the Hessian is never computed explicitly, and only the result of its multiplication with a vector is stored. The conductivity is then iteratively updated along the computed search direction

σk+1 =σk+τkdk, (2.63)

where the step sizeτkis either set to one, or chosen according to a line search in thedamped

Gauss-Newton method.

Line Search

All non-linear iterative methods profit from an accurately chosen step sizeτk, which can be

found with a line search. In line search methods, there is a trade-off between the accuracy of the choice ofτkwhich reduces the functional value substantially, and the time spent to compute this

step size. Many methods exist to perform a line search, but they all contain the same two stages: in the first stagebracketsare defined to limit the desirable step sizes and in the second stage a good step size is computed through bisections or interpolation (Nocedal and Wright, 1999).

Brent’sline search method combines the stable bisection methods and the fast interpolation methods (Brent, 1973). It uses golden section bracketing within an initially defined interval

a < τk < bknown to contain a local minimum, and fits a parabola onto the three points. If

the minimum of the parabola is betweenaandb, it is accepted as a new bound, otherwise the

interval halves betweenaandbare again divided by the golden ratio. This process is continued

Direct Methods

The alternative to non-linear iterative algorithms are direct methods. These have the advantage that they are faster than iterative methods.Inclusion detectionmethods search for discontinuities in the conductivity distribution. Instead of reconstructing the conductivity distribution itself, they are testing data on their plausibility when applied to a known background conductivity (which can consist of several tissues). Wherever the data are not plausible, it is assumed that there is a perturbation. The best known of these inclusion detection methods is thefactorisation method, which has already been used in 3D EIT (Chaulet et al., 2014). Inclusion detection methods have the disadvantage that no knowledge of the conductivity of the perturbation is gained. They could, however, be used as preconditioners for other reconstruction algorithm, by indicating the region of interest.

Another direct method which has been applied to EIT is the D-bar method (Siltanen et al., 2000). So far it is mainly of interest for mathematicians, as it is practically limited to two- dimensional applications with simple geometries. Advantages of the D-bar method are that it is fast, gives an analytic solution rather than a finite element based approximation and that it finds the global minimum.