Second step: data adaptive estimation of the basic denoised image

In this step we obtain the basic estimate using the pre-filtered version of the noisy image. We follow the following procedure for two different block sizes of , and , depending on the noise level present in the examined image. Unless otherwise mentioned, for the rest of this step we will drop the subscript guide and est as the process is similar for both the case.

Let us denote the basic estimate by and the final estimate by . We define Zx

27

Zx is the coordinate of the top left corner of the patch. In other word, we can say that the

patch Zx is located at location in the image . A 3D group or a group of selected 2D

patches is denoted by a bold face capital letter with a subscript representing the set of its grouped patches coordinates (e.g. Zs represents a collection of 2D patches composed of

patches Zx located at ). As we have two near identical steps in BM3D we use

the superscript ht (stands for hard thresholding) and wie (stands for Wiener filtering) to represent the parameters in the first step and the second step respectively. That is, the patch size used in first step is denoted by and the patch size used in second step is denoted by . We process the blocks in sliding-window manner by following a raster scan where each next block is taken with a fixed pixel shift from the previous one. This ensures there is at least one estimate for each image pixel. We denote the currently processed block by and call it reference block.

Firstly, we find the blocks that are similar to the reference one within a fixed neighborhood. We define the similarity based on the block-distance and only consider those blocks whose distance is less than a pre-defined threshold. The distance is obtained by applying a normalized 2D linear transform on both blocks followed by a hard- thresholding on the obtained coefficients.

’

3.7

here, is the distance between the two blocks in consideration. ϒʹ denotes the hard thresholding operator with threshold value and denotes the linear transform.

By applying the above equation and defining a maximum distance to match the similar blocks we get the coordinates of a set of blocks, , that are similar to ,

28

as the reference block matches with itself. It implies that we will find at least one match for each reference block leaving the set , always non-empty. By using the found set we form a 3D group

, stacking the noisy blocks .

Then we perform the collaborative filtering on the formed 3D group by applying hard thresholding in 3D transform domain. This is done by first applying the 3D transform to the group, performing shrinkage of the transform coefficient by applying hard- thresholding, inverting the 3D transformed coefficients and finally returning the estimates of the blocks back to their previous positions,

, 3.9

here, ϒ represents the hard thresholding operator with a threshold value of . The 3D

array

is made of the block-wise estimates located at the matched coordinates . In the subscript x represents the location or the coordinates of this block-estimate and the superscript refers to the reference block.

As the block-wise estimates

of pixels overlap, we obtain a overcomplete representation of the true image. A block can be matched and grouped with multiple reference blocks. For example, and are both located at xp and xq but

matched during the processing of reference blocks located at xp and xq respectively. This

process produces a substantially overcomplete representation of the true image if there is plenty of overlapping block-wise estimates. To get basic estimate at the end of the first stage, we perform aggregation on the corresponding block-wise estimates

by applying a weighted averaging. The aggregation of block-wise estimates is a critical step which relies on the proper selection of aggregation weights. Inspired by [76] and [90], the authors of BM3D proposed aggregation weights that are inversely proportional to the total sample variance of the corresponding block-wise estimates. That is, the more dissimilar or noisy the block-wise estimate is the less the weight is. This ensures higher contribution from the similar block-wise estimates in the final estimation of the basic

29

image. The following weight is applied to the group of the overlapping block-wise estimates for each , , 3.10

where σ is the variance and denotes the number of retained coefficients after the hard thresholding.

Finally, we compute the preliminary basic estimate by a weighted average of the block-wise estimates

located at the same place, using the weights from Equation 3.10,

3.11

where is the characteristic function of the square support of a block located at .

At this stage, unlike BM3D, which computes final basic estimate by a weighted average of the block-wise estimates

located at the same place, using the weights from Equation 3.10, we compute two preliminary estimates (i.e. ) namely, and for the two different block size. We apply the Canny edge detector [91] on the preliminary basic estimation to construct a edge map. The Canny edge detector highlights regions with high spatial derivatives by applying a Gaussian smoothing with a kernel size to reduce the level of noise in the input image, followed by the

processing of derivative in both x and y direction to get the magnitude and direction of gradient,

30

and , 3.13

where D is the magnitude of gradient and θ is angle of gradient. Given the values of the gradient, the algorithm applies non-maximum suppression, which suppresses any pixel that is not at the maximum. This is done by preserving all local maxima in the gradient image, and deleting everything else. Furthermore, the gradient array is reduced by hysteresis that uses two thresholds, and , determined empirically. Pixels with gradient magnitude are discarded, while pixels with gradient magnitude are kept as edges. If the magnitude is between the thresholds (i.e.

), it is only kept as an edge if and only if there is a path from this pixel to a pixel

with gradient . Once we get the edge map by applying the Canny, we binarize

the array values,

, 3.14 where,

represents the Canny edge detection process and represents the binarization method.

The obtained edge guide is then used to combine the values from the two

preliminary basic estimates and to form the final basic estimate . This is done by processing each pixel in a raster scan fashion and obtaining the edge strength for that location based on its neighboring pixels. For this processing, with we denote a block of size taken from , where x is the center of the block; and

with we denote a pixel at location x. The edge strength _{of a block is}

defined as the number of pixels indicating edge over the total number of pixels,

. 3.15

31

Now we select the estimation for our current pixel based on the calculated edge strength _{. We define a threshold and if the edge strength is greater than}

this threshold value (i.e. ) we call this block, , a block with edges or edged block, otherwise (i.e. _{), we call this a smooth block.}

Once the edge strength in the block is calculated for the current pixel of interest we get the value for each pixel of the final basic estimate by the following equation:

. 3.16

Where, the threshold is defined empirically, which we will discuss later in Section 3.2. In the above equation, what we basically do is select the estimation for each pixel from different block size based on the edge activity present in its defined neighborhood .

The obtained basic estimate , together with the pre-filtered noisy image, is used in the third step to process the final denoised image.

3.1.3

Third step: estimating the true image from the basic estimate