Regularization Methods at Work: Solving Real Problems
7.5 Deconvolution in 2D—Image Deblurring
Image deblurring is the science of computing a sharper reconstruction of a digital image from a blurred and noisy one, on the basis of a mathematical model of the blurring process. Figure 7.5 shows an example of image deblurring by means of Tikhonov regularization; the reconstruction was computed by means of the MATLAB software from [39]. The basic problem is very similar to the barcode reading problem from Section 7.1, where a clean signal (the sharp image) is deteriorated by blurring in the optical system. The main difference is that everything now involves two-dimensional signals, namely, the sharp and blurred images, as well as the point spread function.
In the continuous setting of Chapter 2, image deblurring is a first-kind Fredholm integral equation of the generic form
1 0
1
0 K(s, t) f (t) dt1d t2= g(s), s ∈ [0, 1] × [0, 1],
in which the two functions f (t) and g(s) that represent the sharp and blurred images are both functions of two spatial variables s = (s1, s2) and t = (t1, t2). Discretization of this problem, e.g., by means of the midpoint quadrature rule, then leads to the discrete image deblurring problem encountered in a variety of applications involving digital images.
For clarity we restrict this discussion to square gray-level images, but all our methods and results also hold for rectangular images as well as color images; see, e.g.,
Downloaded 07/26/14 to 129.107.136.153. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
7.5. Deconvolution in 2D—Image Deblurring 145
Figure 7.5. Top to bottom: Sharp image, blurred and noisy image, and Tikhonov reconstruction.
[39] for the necessary extensions. Let the two N× N arrays X and B represent the unknown, sharp image and the recorded blurred and noisy image, respectively, and let Xi j and Bi j denote the array elements, also known as the pixels in the two images.
Also let the vectors x and b, of length n = N2, consist of the “stacked” columns of X and B, i.e.,
x = Xi j
b = Bi j
0
, where = (j− 1)N + i. (7.8)
Downloaded 07/26/14 to 129.107.136.153. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
146 Chapter 7. Regularization Methods at Work: Solving Real Problems
Figure 7.6. The point spread function is the image of a single bright pixel.
Then a good model for the blurring is very often of the form A x = b, where A is an N2× N2 given matrix.6
In the rest of the discussion we will assume that the blurring is spatially invariant, i.e., the blurring is the same everywhere in the image. While this seems like a strong requirement, it is often a good approximation to the actual blurring, and it makes the deblurring problem much simpler than the general case.
7.5.1 The Role of the Point Spread Function
To arrive at the model A x = b, let us first inspect the point spread function for image blurring, which prescribes how the light from a single pixel in the the sharp image X is spread over a number of pixels in the blurred image. Specifically, we define the array P —called the point spread function (PSF)—as the (possibly small) image of a single white pixel, as illustrated in Figure 7.6. We will use the convention that the elements of P sum to one, which means that the energy is preserved in the blurring process.
The point spread function may arise from a mathematical model of the blurring process, or from actual measurements of the point spread function. The dimensions of the computed or measured PSF array are often much smaller than N, due to the fact that the spreading of the light takes place in a small region around the white pixel. We can always conceptually augment such a small array with zeros to make the PSF array P as large as needed in the equations below.
From the assumption of spatial invariance and the definition of P it now follows that if Xi j = 1 for a certain index (i , j ), and otherwise zero, then the blurred image B is given by
Bkl = Pk−i,l−j, i , j, k, l = 1, . . . , N.
With this convention of indices, the elements of the PSF array P are allowed to have zero and negative indices, and the center of P is the element P00.
Underlying the integral equation formulation is an assumption that the blurring is a linear process. This implies that given any image X, the corresponding blurred image B is the sum of the blurred images of all the pixels in X, as illustrated in Figure 7.7.
Hence, the pixels in X and B are related through a discrete 2D convolution that takes
6In the literature we often find the notation x = vec(X) and b = vec(B) for (7.8).
Downloaded 07/26/14 to 129.107.136.153. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
7.5. Deconvolution in 2D—Image Deblurring 147
Figure 7.7. The blurred image is the sum of all the blurred pixels.
the form
N i ,j =1
Pk−i,l−jXi j = Bkl, k, l = 1, . . . , N. (7.9)
We immediately notice the similarities with the discrete one-dimensional convolution in (7.5). With this definition, where all indices to X and B are in the range 1, . . . , N, we are implicitly assuming zero boundary conditions; cf. the discussion in Section 7.3.
In MATLAB, the discrete 2D convolution in (7.9) is computed by means of the call B = conv2(X,P,’same’), where the array P that represents the point spread function is allowed to have smaller dimensions than the image X, and the input param-eter ’same’ ensures that the output image B has the same dimensions as the input image X. The center of the PSF array, represented by the element P00, is by definition located at the center of the p× q MATLAB array P.
Since the model (7.9) describes a linear relationship between the elements of the sharp and blurred images, it is clear that the linear relationship is preserved when the elements of X and B are rearranged into the vectors x and b according to (7.8).
Hence, there is a unique—but complicated—relationship between the elements of the PSF P and the elements of the matrix A in the linear system A x = b. This relationship is worked out in detail in [39], and in light of the discussion in Section 7.1, it should not come as a surprise that we will encounter Toeplitz structure. More precisely, the matrix N2× N2 A is block Toeplitz with Toeplitz blocks (BTTB), meaning that A consists of N× N blocks arranged in Toeplitz structure, and each block is an N × N Toeplitz matrix. The test problem blur in Regularization Tools creates such a matrix.
Due to the size of the coefficient matrix A, it is necessary to avoid forming it explicitly, and instead we must be able to perform all computations with A via (7.9) and knowledge about the PSF P . Typically, we are given two functions Amult(P,x) and ATmult(P,x) that compute the matrix-vector products A x and ATx through the 2D convolution in (7.9) involving the point spread function stored in P and the image represented by x. For the square matrices considered here, the two functions could take the form
Downloaded 07/26/14 to 129.107.136.153. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
148 Chapter 7. Regularization Methods at Work: Solving Real Problems
function y = Amult(P,x) function y = ATmult(P,x) N = sqrt(length(x)); N = sqrt(length(x));
X = reshape(x,N,N); X = reshape(x,N,N);
Y = conv2(X,P,’same’); P = flipud(fliplr(P));
y = Y(:); Y = conv2(X,P,’same’);
y = Y(:);
In the function ATmult, the command P = flipud(fliplr(P)) rotates the point spread function 180 degrees, and it can be shown that this corresponds to a multipli-cation with AT. With these two functions at our hands, it is easy to use the CGLS algorithm for image deblurring, as shown in Exercise 7.5.
7.5.2 Rank-One PSF Arrays and Fast Algorithms
For certain point spread functions we obtain more structure in the matrix A, which immediately allows us to compute its SVD, similar to the technique developed in Section 7.4. Several such cases are discussed in [39]; here we restrict our discussion to the case where the matrix P is a rank-one matrix, i.e., it can be written as
P = a ¯aT → Pi j = aia¯j, (7.10) where a and ¯a are two vectors whose indices, like those of P , can be zero and negative.
We also assume that both a and ¯a are symmetric about the “middle” element a0 and
¯
a0, i.e., ai = a−i and ¯ai = ¯a−i for all relevant i . The resulting PSF P is doubly symmetric: it is symmetric across both the row and the column passing through its center. The following PSF arrays are doubly symmetric:
⎛
If we revisit the 2D convolution in (7.9) and insert the expression (7.10) for the elements of P , then for k, l = 1, . . . , N we get
It follows immediately that we can write the blurred image B as B = A X ¯AT,
where A and ¯A are two symmetric Toeplitz matrices defined as
A =
Downloaded 07/26/14 to 129.107.136.153. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php