whereJ(u) provides the regularisation for the problem andα∈R>0, known as theregu- larisation parameter, is a positive parameter which measures the trade-off between data fitting and regularisation. Two ongoing research areas concern the choice of parameter α and the choice of regularisation which may provide a better quality result.
The problem of deblurring, where the blur is assumed to be spatially invariant, may modelling in a similar manner with the blur taken into account. This is considered in more detail later sections.
3.2
Image Denoising
Noise is present in almost any acquired image and can be introduced at the acquisition stage and when storing the data. For example, noise may be introduced by poor illu- mination or inaccurate collection of the data. Noise varies considerably in magnitude, ranging from barely visible specks to heavy noise which is typical in astronomical imag- ing. Even small amounts of noise affect the perceived quality of an image and make fine details difficult to identify. Noise may also be dependent on signal intensity or in- dependent of it. Below, we briefly classify a few of the many types of noise considered in image processing.
3.2.1 Gaussian Noise
Gaussian noise typically occurs during the acquisition of an image, for example by poor illumination. It is typically modelled as additive and independent of the signal intensity. Gaussian noise is also independent at each pixel and can vary between channels in colour images. The probability density function p is assumed to be that of the Gaussian distribution and given by
p(z) = 1 σ√2πe
−(z−µ)2
2σ2
for a given grey level image (or colour image channel)zwhereµis the mean value and σ is the standard deviation. Gaussian noise is said to be white if the mean value µis equal to zero. Some examples of the effects of Gaussian noise introduced into a clean image can be seen in Figures 3.1–3.2.
3.2.2 Other Noise Types
We now briefly introduce some other noise types which are commonly met in image processing.
Poisson (Photon) Noise
In real imaging systems, photon noise and other sensor-based sources of noise contribute in varying proportions at different signal levels, leading to noise which is dependent on scene brightness. Understanding photon noise and modelling it explicitly is especially important for low-level computer vision tasks treating noisy images [[79, 129]] and for the analysis of imaging systems that consider different exposure levels [1, 97, 192] or sensor gains [96].
Photon counting is a classic Poisson process since individual photon detections can be treated as independent events that follow a random temporal distribution. The number N of photons measured by a given sensor element over a time interval t is described by the discrete probability distribution
P r(N =k) = e
−λt(λt)k
k!
whereλis the expected number of photons per unit time interval, which is proportional to the incident scene irradiance. This is a standard Poisson distribution with a rate parameterλtthat corresponds to the expected incident photon count. The uncertainty described by this distribution is known as Poisson noise. Note that since the incident photon count follows a Poisson distribution, its variance is equal to its expectation E[N] = Var[N] = λt. From this, we see that Poisson noise is signal dependent and that its standard deviation increases with the square root of the signal.
Salt and Pepper Noise
Salt and pepper noise orimpulsenoise typically occurs as a result of transmission errors and can also be caused by errors in conversion between analogue and digital formats. It is characterised by dark pixels occurring in bright regions and bright pixels occurring in dark regions and can often be treated by interpolation involving nearby pixels.
(a) Noisy Image (b) Section of (a)
50 55 60 65 70 75 80 85 90 95 100 0.35 0.4 0.45 0.5 0.55 0.6 0.65 x Intensity at (x,76) img z
(c) Intensity Values of Noisy Image compared with the Clean Image along Yellow Line
Figure 3.1: Example of the effect of adding Gaussian noise to give signal-to-noise ratio of 40. From left to right, we have (a) the noisy image, (b) the section of the noisy image outlined by the black square, (c) the intensity values of the noisy image along the yellow line (shown in green) compared with the intensity values of the clean, noiseless image along the same line (shown in blue).
3.2.3 Total Variation Denoising
The authors of [169] proposed a variational approach to the denoising problem of the form (3.2) which employed total variation (tv) regularisation due to it’s improved per- formance over alternatives, such as the L2-norm of the gradient. Their aim was to minimise the tv semi-norm of the image subject to the assumption that the noise is of zero-mean and standard deviation σ which is unknown. The problem is presented as
min u fROF(u) = α 2 Z Ω| u(x, y)−z(x, y)|2dxdy+ Z Ω|∇ u(x, y)| (3.3) where α ∈ R>0 is a small positive parameter which controls the amount of noise to be removed and the smoothness of the result. An important remark is that (3.3) is well-posed and we can guarantee the existence and uniqueness of the minimiser [40].
In order to find the minimiser u of the functional we derive the Euler-Lagrange equation given by α(u−z)− ∇ · ∇u |∇u| = 0 in Ω (3.4) ∇u·ν = 0 onδΩ
where ν is the unit outward normal vector. There are many methods for solving this equation such as gradient descent as used in [169] or those presented in [82, 50, 173, 196].
(a) Noisy Image (b) Section of (a) 50 55 60 65 70 75 80 85 90 95 100 0 0.2 0.4 0.6 0.8 1 1.2 x Intensity at (x,76) img z
(c) Intensity Values of Noisy Image compared with the Clean Image along Yellow Line
Figure 3.2: Example of the effect of adding Gaussian noise to give signal-to-noise ratio of 12. From left to right, we have (a) the noisy image, (b) the section of the noisy image outlined by the black square, (c) the intensity values of the noisy image along the yellow line (shown in green) compared with the intensity values of the clean, noiseless image along the same line (shown in blue).
It is also possible to minimise (3.3) directly as in [48].
Total variation denoising is popular since it can offer very impressive results. See Figure 3.3. However, it is also known for a phenomenon called the staircasing effect caused by the total variation transforming piecewise smooth functions into piecewise constant functions. This causes denoised images to look blocky. Much research has been carried out into finding alternative regularisation techniques which offer similar high-quality results regarding the noise reduction while avoiding the staircasing effect. 3.2.4 Alternative Regularisation
In an effort to improve on the good results of total variation, high-order models have been considered and are known for their ability to well-approximate smoother surfaces [39, 46, 133, 134, 202, 215]. Lysaker et. al [133] proposed the use of the regularisation term J(u) = Z Ω q u2 xx+u2xy+u2yx+u2yydxdy.
When employed in a problem of the form (3.2), we aim to minimise the resulting functional by solving the associated Euler-Lagrange equation which is given by
uxx |D2u| xx + uxy |D2u| yx + uyx |D2u| xy + uyy |D2u| yy +α(u−z) = 0 where |D2u| = qu2
xx+u2xy+u2yx+u2yy. This formulation can yield excellent results
on smooth regions where total variation regularisation can result in an approximation by piecewise constant functions.
Additionally, mixed models for regularisation have been proposed which aim to ob- tain good denoising results while preserving edges which high order models may struggle to do. Such models as those presented in [46, 39, 134, 56] attempt to combine different regularisation techniques in order to benefit from the edge-preserving properties of one
(a) Clean Image (b) Noisy Image (c) Denoised Image
(d) Clean Image (e) Noisy Image (f) Denoised Image Figure 3.3: Illustration of the performance of total variation denoising to restore images from noise corruption. On the top row, we show an example of a Colour Fundus retina image (a) which has been corrupted by noise (b) with signal-to-noise-ratio (snr) of 28.1024 and restored using total variation (c) to snr 35.5171. On the bottom row, we show the camera-man example (a) which has been corrupted by noise (b) with snr of 28.1059 and restored using total variation (c) to snr 32.6741.
regularisation function and the smoothness properties of another. We discuss one such model in more detail in the context of deblurring in§3.3.6.