3.5 Image Segmentation
4.1.5 Image Deblurring without Knowledge of the Blur Function
function, known asblind image deconvolution, is a more challenging yet realistic prob- lem. In this section, we present a method for carrying out blind deconvolution of images by simultaneous approximation of the image and blur functions following §3.4.2 and incorporating alternative boundary conditions.
There exist many algorithms for recovering both the image and the kernel simul- taneously. For example, see [78, 121, 120, 216]. Chan and Wong [54] proposed the following alternate minimisation blind deblurring model using total variation:
min
u,h f(u, h)≡minu,h
1 2k[h∗u](x)−z(x)k 2 L2(Ω)+α1 Z Ω|∇ u(x)|+α2 Z Ω|∇ h(x)| (4.3)
to recover the image with no a priori information about the point spread function. Here x = (x, y), z(x) is the received image, h(x) and u(x) are the blur function and image respectfully, andα1,α2 are small, positive regularisation parameters. Since the
total variation method is particularly effective in recovering edges, we present examples of images which have been corrupted by out of focus blur, which is represented by a piecewise constant function. We will attempt to recover both the image and the blur function simultaneously by minimising (4.3).
True Image Received Image Restored Image
True Image Received Image Restored Image
True Image Received Image Restored Image
True Image Received Image Restored Image
Figure 4.4: Illustration of the deblurring of images by the total variation model given Algorithm 8 assuming Neumann boundary conditions. From top to bottom, we have on each row examples of deblurring 1) a Fundus Autofluorescence retina image corrupted by motion blur, 2) a satellite image corrupted by motion blur, 3) a Fundus Autofluores- cence retina image corrupted by Gaussian blur, and 4) a satellite image corrupted by Gaussian blur. On each row, from left to right, we have (1) the original (true) image, (2) the received (blurred) image, and (3) the restored (deblurred) image.
are considered: ∂f ∂h :u(−x)∗(u(x)∗h(x)−z(x))−α2∇ · ∇h(x) |∇h(x)|β = 0, x∈Ω, (4.4) ∂f ∂u :h(−x)∗(h(x)∗u(x)−z(x))−α1∇ · ∇u(x) |∇u(x)|β = 0, x∈Ω. (4.5)
The idea to solve the blind deblurring model is to develop an alternating minimisation algorithm such that f u(i), h(i) decreases asiincreases.
To solve this system, we first make an initial estimate (u0, h0) for the image and blur function (u, h). For the image, it is typical to let the received data provide the initial estimate since it is often the closest approximation available. We then minimise (4.3) by solvingf u(0), h(1)≡minhf u(0), husing (4.4), thenf u(1), h(1)≡minuf u, h(1)
using (4.5). At the general step, givenu(i) and h(i), we solve
u(i)(−x)∗(u(i)(x)∗h(i+1)(x)−z(x))−α2∇ · ∇h(i+1)(x) |∇h(i+1)(x)| β = 0, forh(i+1)(x) h(i+1)(−x)∗(h(i+1)(x)∗u(i+1)(x)−z(x))−α1∇ · ∇u(i+1)(x) |∇u(i+1)(x)| β = 0 foru(i+1)(x).
One problem we have is that this does not necessarily yield physical solutions because the minimisation problem for (4.3) may not have a unique solution. For example, if (u, h) is a solution, then so are
((a2/a1)h,(a1/a2)u), (−u,−h), (u(x±c, y±d, h(x∓c, y∓d))∀c, d∈R
In order to obtain a physical solution, we impose the following conditions on the image u and the blur function h which ensure that the integral of the blur function is equal to one, both the image and blur function are non-negative, and the blur function is rotationally symmetric:
Z
Ω
h(x, y) dxdy = 1, u(x, y), h(x, y)≥0, h(x, y) =h(−x,−y).
The first condition is imposed by the function C1 which normalises the blur function,
dividing it by its integral over the domain, and is given by
C1(h(x, y)) =
h(x, y)
R
Ωh(x, y) dxdy
.
The second condition is imposed by projectionsP(h) and P(u) of the functions onto a non-negative range by setting any negative values equal to zero, given by
P(h(x, y)) =
(
h(x, y) ifh(x, y)>0 0 otherwise ,
a rotationally symmetric approximation given by
C2(h(x, y)) = h(x, y) +h(−x,−y)
2 .
These conditions are imposed after each update of the blur and image functions. Boundary Conditions
We now consider boundary conditions of the image and blur function, as considered in §4.1.4. Recall that the total variation operator L(u) becomes asymmetric under Neumann boundary conditions. We therefore replace this operator by the sum of sym- metric and asymmetric components L(u) = L+S(u) +L−S(u), where L±S(u) is defined above, and modify the solution algorithm to use a fixed-point lagging technique. Incor- porating this idea into the Euler-Lagrange equations (4.4)—(4.5) yields the modified equations
(H⊤H+αL+S(u)u=H⊤z−α1L−S(u)u, (4.6)
(U⊤U+αL+S(h)h=U⊤z−α2L−S(h)h, (4.7)
where u = M(u), h = M(h) and z =M(z) are vectorised counterparts of the m×n discretised image u, blur function h and received image z respectfully given by the mapping M(ui,j) = ui+m(j−1). U and H are dense mn×mn matrices representing the image and blur function respectively such thatM−1(Uh) gives the discretisation of the convolution [u∗h](x, y) and M−1(Hu) gives the discretisation of the convolution [h∗u](x, y). In practice, the large and dense matrices U and H are not formed. As above the operation of convolution is implemented by Fourier transforms but for ease of notation we continue to write the convolution as the matrix-vector product.
Algorithm
The resulting algorithm is presented in Algorithm 9. For brevity, the notation ia b is
used to denote the step i−1 +a/b. Note that there is no requirement for a separate algorithm for the Neumann and Dirichlet cases since the asymmetric component of the tv operator L(·) would simply be equal to zero and the symmetric component would be equal to L(·). The equations (4.6) and (4.7) then reduce to the Dirichlet case and the maximum number of iterations forj in Algorithm 9 should be set to 1.
Experimental results
We include some examples of restoring images from motion blur corruption by Al- gorithm 9 in Figures 4.5—4.6. In the case of the satellite, we use Dirichlet boundary conditions and in the case of the colour Fundus retina example, we use Neumann bound- ary conditions. In both cases, the result is reasonable for a blind deblurring model but still still leaves room for improvement.
Algorithm 9 Blind TV Deblurring (BTVD)
1: function(h,u)←BTVD(z, α1, α2, β1, β2, maxit) 2: u(0) ←z
3: for i= 1..maxit do
4: Obtain approximation of the blur function by solving (4.4): 5: forj= 1..maxitdo
6: for k= 1..maxitdo
7: Using preconditioned conjugate gradient, solve for h(i14,j,k):
U⊤U +αL+Sh(i14,j,k−1) h(i14,j,k) =U⊤z−αL− S h(i14,j−1) h(i14,j−1), 8: end for 9: end for 10: Imposeh(i24) =C1 h(i14) . 11: Imposeh(i34) =Ph(i24) . 12: Imposeh(i) =C2 h(i34) .
13: Obtain approximation of the image by solving (4.5): 14: forj= 1..maxitdo
15: for k= 1..maxitdo
16: Using preconditioned conjugate gradient, solve for u(i12,j,k):
H⊤H+αL+Su(i12,j,k−1) u(i12,j,k) =H⊤z−αL− S u(i12,j−1) u(i12,j−1) 17: end for 18: end for 19: Imposeu(i)=Pu(i12) . 20: end for 21: h←h(i) 22: u←u(i) 23: end function
(a) True Image (b) Received Image (c) Restored Image Figure 4.5: Illustration of the performance of Algorithm 9 with a colour Fundus retina image, using Neumann boundary conditions. From left to right, we have (a) true image, (b) the received image and (c) the restored image. Some improvement can be seen in the restored image which has an improved SNR value (20.79) over the received image (19.15) and an improved PSNR value (27.00) of the received image (25.03).
(a) True Image (b) Received Image (c) Restored Image Figure 4.6: Illustration of the performance of Algorithm 9 with the satellite image, using Dirichlet boundary conditions. From left to right, we have (a) true image, (b) the received image and (c) the restored image. Some improvement can be seen in the restored image which has an improved SNR value (16.10) over the received image (8.95) and an improved PSNR value (30.23) of the received image (22.74).