Chapter 4 Reduced basis methods for Bayesian inverse problems
4.6 Bayesian Inverse Problems
As we have discussed the potential of using the RBM over finite element methods, we now in this section aim to couple our new forward solver within inverse problems. Inverse problems require repetitive evaluations of the forward problem, therefore by hopefully implementing the RBM we can acquire a cheaper evaluation whilst maintaining similar accuracy. Our inverse solver for this chapter will be based on the iterative Kalman method. Before we present the algorithm we need to define our inverse problem in a Bayesian setting which will take a similar form as done in [137] but now with a dependence on our parameters ϑ. Following on from this we then given an overview of the iterative Kalman method and some of it’s key properties before defining the method. We now begin this section with a quick review of the Bayesian approach towards inverse problems.
Given two separable Hilbert spacesX andYand a forward operatorG :X → Y we are interested in the recovery of the quantity of interest u ∈ X from noisy observationsy∈ Y which are given by
whereη∼ N(0,Σ) with Σ denoting a positive self-adjoint operator. Trying to invert (4.6.1) can cause difficulty as there is no guarantee with well-posedness through the classical approach. One way to alleviate this is through the Bayesian approach where now we are interested in the posterior distributionµy on the random variable u|ywhich can be evaluated through Bayes’ Theorem. From this we can characterise the posterior as dµy dµ0 (u) = 1 Z exp −Φ(u;y) , (4.6.2) withZ being Z= Z X exp −Φ(u;y) µ0(du), (4.6.3)
where our misfit functional is given as Φ(u;y) = 12y− G(u) 2 Σ.
4.6.1 Iterative Kalman Method
Our inverse solver for this chapter will be the iterative Kalman method, which was proposed by Iglesias et al. [79] as an optimisation based technique to produce stable solutions to constrained PDE problems. The method can be derived from the least- squares formulation, which takes motivation from data assimilation. The main idea behind the method is to represent a noise controlled system i.e. u →u† asη → 0. Assume we have an ensemble of J members
u(0j) Jj=1 ⊂ X. We label this as our initial ensemble at iteration level 0 which is said to be a linear space of our solution space X. We wish to build upon our ensemble
u(nj) J
j=1 which at each iteration
levelnis updated through by combining the artificial dynamics with artificial data
yn, resulting in a new ensemble
u(nj+1) Jj=1 which is achieved by using the ensemble mean ¯ un= 1 J J X j=1 u(j) n , (4.6.4)
to approximate the solution of the inverse problem.The iterative Kalman method can be split into two parts, a prediction step and an analysis step similar to the ensemble Kalman filter [57]. The purpose of the prediction step is to map the ensemble of particles into the observational spaceY implying information is introduced into the forward model.
The analysis step takes the mapped ensemble in the data space and compares it with the data where the the ensemble is modified to better match the data. As stated previously the scheme attains regularisation properties, this is achieved through the discrepancy principle. In order to define the principle a regularisation parameterτ > 1
Algorithm 5 Regularized Iterative Kalman Method Let {u(0j)}J
j=1 ⊂ X be the initial ensemble with J elements. Letρ ∈(0,1) with τ > 1 ρ We wish to generate, u(0j)∼µ0, y(j) =y+η(j), η(j) ∼N(0,Σ). Then for n= 1, . . . Prediction Step
1. Evaluate the forward map,
wn(j) =G(un(j)) for j∈ {1, . . . , J}, and define ¯wn= J1 PJj=1w (j) n . Discrepancy Principle 2. IfkΣ−1/2(y−w¯ n)kY ≤τ η, stop! Output ¯un= J1 PJj=1u (j) n . Analysis Step
3. Define sample covariances:
Cww n = J−11 PJ j=1(G(u (j) n )−w¯n)hG(un(j))−w¯n,·iY. Cuw n = J−11 PJ j=1(u (j) n −u¯n)hG(un(j)−w¯n,·iY.
Update each ensemble member as follows
un(j+1)=un(j)+Cnuw(Cnww+αnΣ)−1(y(j)−w(nj)),
where αn≡αNn satisfies ρkΣ−1/2(y(j)−w¯
n)kY ≤αNnkΣ1/2(Cnww+αNnΣ)−1(y(j)−w¯n)kY,
and where αn is chosen based on αi+1
principle is given as
kΣ−1(y−w¯n)kY ≤τ η, (4.6.5)
where ¯wn = J1 PJj=1G(u (j)
n ). Usually with iterative inverse solvers it is common
to add some regularisation. We note that for all the experiments we will work with the regularized iterative version which is given by Algorithm 5. EKI uses regularisation properties taken from the Levenburg-Marqardt scheme [71]. We note also that this discrepancy principle is slightly different to typical one as it is applied to the average of the output ¯wn. We will now refer to inversion as ensemble Kalman
inversion (EKI).
4.6.2 RB-EKI
As we have discussed both the forward and inverse solver in detail we will now present the coupled scheme. We begin the development of the algorithm by refor- mulating the inverse problem with the RBM.
As we have defined our parametric PDE of interest, we can use this to formulate a Bayesian inverse problem. Recall that our solution p(·;ϑ) ∈ X := H1
0(D) and lj ∈X∗ are continuous linear functionals. Then we can define our observed data
yj =lj(p(·;ϑ)) +ηj, j= 1, . . . , J, (4.6.6)
where{η}J
j=1 ∼N(0,Σ) is Gaussian additive noise. From this we can further define
our forward operatorG:Rk→RJ where
Gj(u) =lj(p(·;ϑ)) = p(xj) ∈D= [0,1] Z D p(xj)gj(x)dx ∈D= [0,1]2,
wheregj(x) is a covariance kernel for our inverse problem. This allows us to rewrite
(4.6.6) as the inverse problem
y =G(u) +η.
Note here we define our domains to be D = [0,1]d for d = 1,2 as these are the