Regularization Methods at Work: Solving Real Problems
7.6 Deconvolution and Resolution ∗
As already mentioned, this model corresponds to the assumption of zero boundary conditions; if we want to use the reflexive boundary conditions introduced in Section 7.3, then we must replace A and ¯A with their STH counterparts ABC and ¯ABC, as defined in the previous section.
To arrive at a fast regularization algorithm we proceed as before, using that ABC
and ¯ABC are both diagonalized by the DCT matrix Wn, ABC= WnD WnT, A¯BC= WnD W¯ nT, leading to relations
B = WnD WnTX WnD W¯ nT ⇔ X = Wn
!D−1(WnTB Wn) ¯D−1"
WnT. Here we recognize /B = WnTB Wn = dct2(B) as the 2D DCT of B, while X = WnC W/ nT = idct2( /C) is the inverse 2D DCT of /C = D−1B ¯/D−1. Thus, again following the ideas presented in the previous section, we arrive at the following efficient MATLAB code for computing a Tikhonov regularized solution (assuming that Abc and Abc_bar hold the matrices ABCand ¯ABC):
c = dct(eye(n,1));
d = dct(Abc(:,1)) ./ c;
d_bar = dct(Abc_bar(:,1)) ./ c);
Q = (d*d_bar’) ./ ((d*d_bar’).^2 + lambda^2);
X_reg = idct2( Q .* dct2(B) );
We recall that for large problems there is no need to store the full ABC and ¯ABC ma-trices, since the first column of each matrix suffices for our computations. Extensions of this approach to other PSF arrays and other boundary conditions are described (including MATLAB algorithms) in [39].
7.6 Deconvolution and Resolution
∗Given a problem with noisy data, and given a specific regularization method, we are interested in studying the resolution—how small are the details that we can reliably recover in the regularized solution? The analysis of resolution is closely connected to the number of SVD components that can be recovered from the noisy data, as well as the spectral properties of the singular vectors (which we discussed in Section 2.5).
For many deconvolution problems this task is simplified by the fact that their singular vectors are samples of simple spectral basis functions. Specifically, for the deconvolu-tion problems with reflexive boundary condideconvolu-tions that we consider in this chapter, the singular vectors are identical to the discrete cosine basis vectors associated with the DCT.
From the analysis in Chapters 4 and 5 we know that spectral properties of a regularized solution are determined by the singular vectors that take part in the solution. This is, of course, obvious for the TSVD method, but it holds for any method that involves a filtered expansion in the SVD basis. Hence, for a deconvolution problem, if we include the first k DCT basis vectors w1, . . . , wk, then the size of the
Downloaded 07/26/14 to 129.107.136.153. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
150 Chapter 7. Regularization Methods at Work: Solving Real Problems
exact k = 10
no noise
k = 20 k = 30
k = 40 k = 50 k = 60 k = 70
k = 80 k = 90 k = 100 k = 110
k = 50 η = 0.1
k = 60 k = 70 k = 80
k = 20 η = 2.0
k = 30 k = 40 k = 50
Figure 7.8. Illustration of the resolution limit for a deconvolution problem with a DCT basis. Top left: The exact solution consisting of three narrow peaks of width 0.004, 0.03, and 0.07, respectively. Remaining plots in top three rows: TSVD solutions to a noise-free problem, where truncation errors in the form of oscillations are visible for large values of k. Bottom two rows: TSVD solutions to two noisy problems; for the smaller noise level η = 0.1 the two widest peaks can be reliably identified in the solutions, and for the larger noise level η = 2.0 only the widest peak is clearly visible.
smallest details that we can hope to recover is determined by the width of a single oscillation in the vector wk with the highest frequency. Assuming the standard interval [0, 1] for the t variable, the width of a half a cosine in wk is 1/(k− 1), which we can define as the resolution limit.
Figure 7.8 illustrates the issues of resolution, using an exact solution consist-ing of three narrow peaks of approximate width 0.004, 0.03, and 0.07, respectively.
The coefficient matrix A is a Toeplitz matrix that models a Gaussian kernel, and we impose reflexive boundary conditions leading to the DCT basis. The top part of the figure shows TSVD solutions for increasing values of the truncation parameter k for a noise-free problem, and we see that as k increases we obtain increasingly better
Downloaded 07/26/14 to 129.107.136.153. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
7.6. Deconvolution and Resolution∗ 151 Exact Noisy and blurred
λ = 0.1 λ = 0.2 λ = 0.4
Figure 7.9. Illustration of the resolution limit for image deblurring in the form of a 2D deconvolution problem with a 2D DCT basis. The color scale is inverted for clarity. Top: The exact image with nine spots of size 1× 1, 2 × 2, . . . , 9 × 9 pixels, and the noisy and blurred image. Bottom: Tikhonov solutions for three different values of the regularization parameter; only six (perhaps seven) spots can be identified in the deblurred images.
approximations to the exact solution. Some oscillations, caused by the truncated expansion, are visible for all values of k.
The second to last row of plots in Figure 7.8 show TSVD solutions for a problem with some noise. The Picard plot (not shown) reveals that k = 60 is a good truncation parameter for this problem, leading to an expected resolution limit of about 1/60≈ 0.017, which indicates that we should be able to recover the two widest peaks only.
The solution plots confirm this: in the regularized solutions we can identify the two widest peaks (although the amplitude of the second is incorrect), while there is no significant visible indication of the thinnest peak.
The four plots in the bottom row of Figure 7.8 show TSVD solutions for a problem with higher noise, where the Picard plot indicates that only about 40 SVD components can be included in the regularized solution. The resolution is therefore about 1/40 = 0.025, meaning that we should barely be able to detect the middle peak.
Again the solution plots confirm this: only the widest peak can be reliably identified (and its amplitude is incorrect for k = 40).
While the situation is conceptually the same for 2D problems, such as image deblurring, the analysis of resolution is complicated by the fact that spatial resolution in 2D is much more complex than in one dimension. It is outside the scope of this book to go into these issues, so we choose to merely illustrate matters with the example shown in Figure 7.9, which shows a similar numerical experiment using a test image with nine small spots of size 1× 1, 2 × 2, . . . , 9 × 9 pixels. Again we use a Gaussian blurring model with reflexive boundary conditions, and the Tikhonov solutions are computed by means of the function tik_dct from [39]. The inverted noise (that
Downloaded 07/26/14 to 129.107.136.153. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
152 Chapter 7. Regularization Methods at Work: Solving Real Problems appears as small high-frequency oscillations in Figure 7.8) now appears in the form of
“freckles” that are visible in all three Tikhonov solutions. We can safely identify six (perhaps seven?) of the nine spots, while the smallest ones are lost in the inverted noise; we conclude that the resolution limit is about 4× 4 pixels in this problem.
7.7 Tomography in 2D
∗Tomography can be characterized as the science of computing reconstructions in 2D and 3D from projections, i.e., data obtained by integrations along rays (typically straight lines) that penetrate a domain Ω—typically a rectangle in 2D, and a box in 3D.
Here we consider a 2D model problem on the square domain Ω = [0, 1]× [0, 1] (in arbitrary units), in which we are given an unknown function f (t) = f (t1, t2) that we wish to reconstruct. We assume that this function represents some material parameter, in such a way that the damping of a signal penetrating an infinitesimally small part d τ of a ray at position t is proportional to the product to f (t) dτ. The data in the tomography problem consist of measurements of the damping of signals following well-defined rays through the domain Ω. See, e.g., [7, Section 7.4] for details of the mathematical formulation.
In this model problem, the i th observation bi, for i = 1, . . . , m, represents the damping of a signal that penetrates Ω along a straight line, which we refer to as rayi; see Figure 7.10 for an example. All the points ti on rayi are given by
ti(τ ) = ti ,0+ τ di, τ ∈ R,
where ti ,0is an arbitrary point on the ray, and di is a (unit) vector that points in the direction of the ray. Due to the above assumption, the damping is proportional to the integral of the function f (t) along the ray. Specifically, for the ith observation, and ignoring a problem-specific constant, the damping associated with the i th ray is given by
bi = ∞
−∞
f! ti(τ )"
d τ, i = 1, . . . , m, where d τ denotes the integration along the ray.
We can discretize this problem by dividing Ω into an N× N array of pixels, and in each pixel (k, ) we assume that the function f (t) is a constant fk:
f (t) = fk for t1∈ Ik and t2∈ I,
where we have defined the interval Ik = [ (k−1)/N , k/N ], k = 1, . . . , N (and similarly for I). With this assumption about f (t) being piecewise constant, the expression for the kth measurement takes the simpler form
bi=
(k,)∈rayi
fkΔL(i )k, ΔL(i )k = length of rayi in pixel (k, )
for i = 1, . . . , m. Again, see Figure 7.10 for clarification, for the case N = 6.
Downloaded 07/26/14 to 129.107.136.153. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
7.7. Tomography in 2D∗ 153
-6
t1 t2
""""""""""""""""""""
rayi :
ti(τ ) = ti ,0+ τ di
I1 I2 I3 I4 I5 I6
I1 I2 I3
I4
I5
I6
f11
f12 f22 f32
f33 f43
f44 f54 f64
f65
Figure 7.10. Example of a discretized tomography problem with N = 6. The i th ray intersects a total of 10 pixels, and thus the i th row of the matrix A has 10 nonzero elements (in columns 1, 2, 8, 14, 15, 21, 22, 28, 34, and 35).
The above equation is, in fact, a linear system of equations in the N2 un-knowns fk. To arrive at a more convenient notation for this system, we introduce the vector x of length n = N2whose elements are the (unknown) function values fk, ordered as follows:
xj = fk, j = (− 1) N + k.
This corresponds to stacking the columns of the N× N matrix F whose elements are the values fk. Moreover, we organize the measurements bi into a vector b.
There is clearly a linear relationship between the data bk and the unknowns in the vector x , meaning that we can always write
bi=
n j =1
ai jxj, i = 1, . . . , m.
With the above definitions it then follows that we arrive at a linear system of equations A x = b with an m× n matrix whose elements are given by
ai j =
ΔL(i )k, (k, )∈ rayi
0 else.
We recall that index i denotes the i th observation (corresponding to rayi) and j denotes the pixel number in an ordering with j = (− 1) N + k. The matrix A is very sparse,
Downloaded 07/26/14 to 129.107.136.153. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
154 Chapter 7. Regularization Methods at Work: Solving Real Problems
0 50 100 150
0
50
100
150
nz = 369
Figure 7.11. An arbitrary 150×150 submatrix of a sparse tomography matrix A for the case N = 50 leading to a matrix of dimensions 2500× 2500.
and the number of nonzero elements in any row is bounded above by 2N. Figure 7.11 shows a typical example of a 150× 150 submatrix of A for the case N = 50.
In the image deblurring problem described in the previous section, one would never form the coefficient matrix explicitly. On the other hand, for tomography prob-lems it is often feasible and convenient to explicitly form the very sparse matrix A.
For both problems one will use an iterative method, and the classical methods from Section 6.1 have proved to be quite efficient for tomography problems (while they may be very slow when applied to general discrete inverse problems with a dense coefficient matrix). Exercise 7.8 illustrates this.