The Maximum a Posteriori solution

A prior over x is usually introduced into the super-resolution model to avoid so- lutions which are subjectively very implausible to the human viewer. Section 2.4 introduced the Maximum a Posteriori (MAP) approach, which we explain here in terms of the generative model and its probabilistic interpretation. We then go on to cover a few of the general image priors commonly selected for image super-resolution

Number of low−resolution images

Standard deviation of noise (in grey levels, assuming an 8−bit image)

K=1 K=2 K=4 K=8 K=16 K=32 K=64 2.5 2 1.5 1 0.5 0

Figure 3.5: The ML super-resolution estimate. Synthetic datasets with varying numbers of images and varying levels of additive Gaussian noise were super-resolved using the ML algorithm. If the number of input images is small, then the system of equations is underconstrained, and the ML solution tends to be reasonably free from the characteristic “chequer-board” pattern, but also lacks much of the rest of the high-frequency information (see the first two columns). When the input images have not been corrupted by noise, the outputs are generally of the same high quality (see bottom row). The problem of ill-conditioning is most obvious when there are intermediate numbers of images, and increased noise, as seen in the top-middle and top-middle-right regions of the set of super-resolution images.

Number of low−resolution images

Standard deviation of noise (in grey levels, assuming an 8−bit image)

K=1 K=2 K=4 K=8 K=16 K=32 K=64 2.5 2 1.5 1 0.5 0

Figure 3.6: The bounded ML super-resolution estimate. These images were were created in exactly the same way as those in Figure 3.5, except that here the pixel values were constrained to lie in the interval [−1/2,1/2]. More of the outline of the window in the scene is visible in the top-middle section of these results than in the set of results without any pixel value constraints. However, the resulting super-resolution images in this problem region are still noisy and unnatural-looking.

in the next section.

The MAP estimate of the super-resolution image comes about by an application of Bayes’ theorem,

p(x|d) = p(d|x)p(x)

p(d) . (3.16)

The left hand side is known as the posterior distribution over x, and if d (which in this case might represent our observed data) is held constant, then p(d) may be considered as a normalization constant. It is, however worth noting that it is also the case that

p(d) =

Z

p(d|x)p(x)dx. (3.17)

Applying these identities to the super-resolution model, we have

p x|y(k),θ(k),λ(k) = p

y(k) |x,θ(k),λ(k) p(x)

p({y(k)_{} | {θ}(k)_,λ(k)_}) (3.18)

If we again assume that the denominator is a normalization constant in this case — it is not a function of x — then the MAP solution, xMAP, can be found by maximizing the numerator with respect to x, giving

ˆ

xMAP = argmax

x

p({y(k)}|x,{θ(k)})p(x). (3.19)

and minimize L with respect tox. The objective function and its gradient are L = −log (p(x)) + β 2 K X k=1 r(k)2₂ (3.20) ∂L ∂x = ∂ ∂x[−log (p(x))]− K X k=1 λ(₁k)W(k)Tr(k). (3.21)

Of course, in order to solve this, we still require a form for the image prior p(x). In general we want the prior to favour smoother solutions than the ML approach typically yields, so it is usual for the prior to promote smoothness by penalizing excessive gradients or higher derivatives. Log priors that are convexand continuous

are desirable, so that gradient-descent methods like SCG [77] can be used along with (3.20) and (3.21) to solve for x efficiently. A least-squares-style penalty term for image gradient values leads to a Gaussian image prior which gives a closed-form solution for the super-resolution image. However, natural images do contain edges where there are locally high image gradients which it is undesirable to smooth out. Figure 3.7 shows the improvement in super-resolution image estimates that can be achieved using a very simple prior on the super-resolution image, x. The super- resolution images were reconstructed using exactly the same input datasets as Fig- ures 3.5 and 3.6, but this time a Huber prior was used on image gradients, and all of the noise present in the ML solutions is gone. We will consider the forms and relative benefits of a small selection of popular MAP super-resolution priors in the next section.

Number of low−resolution images

Standard deviation of noise (in grey levels, assuming an 8−bit image)

K=1 K=2 K=4 K=8 K=16 K=32 K=64 2.5 2 1.5 1 0.5 0

Figure 3.7: Maximum a Posteriori super-resolution images. This figure uses exactly the same input data as 3.5, but uses the Maximum a Posteriorimethod to infer the super-resolution images, which introduces a prior over the super-resolution image pixels. To construct the results above, the Huber prior was used (see Sec- tion 3.4), with parameter α= 0.05 and various settings of parameter β, in order to obtain the sharpest possible results. Notice that in every case, the scene is clearly visible, and the super-resolution images in the upper middle and upper right of the grid no longer exhibit the noise present in the ML super-resolution images. However, the lack of detail on the two left-columns is still present, since this arises from a lack of input images, rather than a conditioning problem.