4.2 Inverse Problem: Registration
4.2.2 Solving the Image Registration Problem
4.2.2.1 Distance Measures
The similarity criterion measures differences in values at specific pixel/voxel locations between the two images. Brown[33] reported that the differences were measured as volumetric differences. Typ- ically, value changes are the differences in intensity or radiometry. However, more generally, the concept of volumetric is proposed in order to include the wide variety of existing sensors whose values are not intensities.
Intensity-based methods minimise a cost function that measures the similarity between the image intensity of corresponding points of source and target images. The similarity computation is straight- forward for the registration between source and target images, which are acquired by using the same imaging modality. However, it is more complex to deal with multi-modality image registration. The most commonly used similarity measurements are Squared Sum of Intensity Differences (SSD), Cor- relation Coefficient (CCoe), Mutual Information (MI), and Normalised Mutual Information (NMI), which are represented as follows.
SSD =X(R(x, y, z, t0) − T(T (x, y, z), t1))2; (4.17)
CCoe = P(R(x, y, z, t0) − ¯R(t0))(T(T (x, y, z), t1) − ¯T(t1)) pP(R(x, y, z, t0) − ¯R(t0))2P(T(T (x, y, z), t1) − ¯T(t1))2
, (4.18)
in which R(·) and T(·) are the fixed and moving images, and ¯R(·), ¯T(·), T , t0, and t1 represent the
average image intensities, spatial transformation, and imaging time respectively. In addition, x, y, and z are the 3D coordinates of the image pixels.
Mutual information is based on the concept of information theory and expresses the amount of information that one image R contains about a second image T[244]
MISimilarity(R, T) = H(R) + H(T) − H(R, T), (4.19)
where H(R), H(T) denote the marginal entropies of R, T. H(R, T) denotes their joint entropy, which is calculated from the joint histogram of R and T. If both images are aligned, the mutual information is maximised. Because the mutual information depends on the overlap between two
images, normalised mutual information is advocated as[244]
NMISimilarity(R, T) =
H(R) + H(T)
H(R, T) . (4.20)
In 1981, Horn and Schunck [120] proposed the original optical flow registration method, which
assumed that the changes in image intensities are only due to motion. SSD possesses the same basic assumption that both images are identical when they are registered. It is the optimum measure if the images are varied by Gaussian noise only[303] [302]. However, SSD is sensitive to outliers, and it is inappropriate, for instance, when intensities have changed due to the injection of contrast agent. MI, which is a symmetric similarity measurement, was proposed by Viola and Wells III [303], and
Collignon et al. [50] independently in 1995. Previous research showed that MI was very general
because it made an assumption that a predictable relationship existed between the intensities of the two images. Additionally, Studholme et al. [277] [278] showed that NMI is more robust than MI for
intermodality brain registrations when overlap changes are substantial.
4.2.2.2 Numerical Optimisation
Medical image registration algorithms usually estimate the transformation either following a data- driven flow or minimising a certain cost function [76]. Flow methods are very similar to approaches called optical flow in computer vision, which estimate motion in an image sequence such as a video. Here, we will focus on registration via optimisation. Generally, the cost function consists of a similarity measure expressed as follows
ζ?= arg min ζ f (ζ) = S[ T[Tζ(x)], R(x) ] = S[ IInt1[Tζ(x)], IInt2(x) ] , (4.21)
in which T[Tζ(x)] and R(x) denote the source image and the target image; however, there are in-
terpolation operators, i.e., IInt1 and IInt2, processing the images. Here x represents a pixel/voxel
location or coordinates described using lexicographical ordering, and f (ζ) denotes the cost function. In addition, S is an appropriate similarity measure or distance measure, and Tζ is the transforma-
tion. We minimise the cost function with respect to the transformation parameters ζ. Therefore, the desired transformation is a solution of this optimisation problem. More accurately, the trans- formation parameters ζ should be constrained in an admissible region Ωζ [76]. Furthermore, due
have been introduced to alleviate unstable and unfeasible solutions. Consequently, the registration problem in Equation4.21becomes one of minimising the joint functional
ζ?= arg min ζ f (ζ) = S[ T[Tζ(x)], R(x) ] + λR(Tζ(x)) + ξP(Tζ(x)) , (4.22) s.t. ζ ∈ Ωζ ∈ Rn
where the λ and ξ are the parameters of the regularisation (R) and penalty (P) functions. We can use any non-linear optimisation techniques (as seen in Section3.2.3.2and Appendix Dto solve this problem.
4.2.2.3 Ill-Posed and Regularisation
As mentioned before, registration is an inherently ill-posed inverse problem. Because for every spatial location x ∈ Ω ∈ Rn, we try to obtain a vector yζ = Tζ(x) ∈ Rn with only scalar information T(yζ)
and R(x) provided. Normally, we use the L2-norms of derivatives of the displacement field u = y ζ
to express the regularisation term λR(yζ) in Equation4.22. However, other regularisation schemes
such as L1-norm based methods can also be applied. Therefore, let K denote a numerical operator
(e.g., a differential operator), and we can use the L2-norm k · k22 to describe the regularisation as
R(yζ) =
Z
Ω
kK[yζ(x)]k22dx, (4.23)
in which Ω is a feasible spatial boundary of the image. We can use the norm operator, the partial derivatives based diffusion operator, the gradient and divergence based elastic operator, and the curvature operator to model the regularisation functional. For example, in 2D, n = 2, and let K[yζ] = yζ and yζ = [y1ζ; y
2
ζ], then the norm of yζ is
R(yζ) =
Z
Ω
(y1ζ(x))2+ (y2ζ(x))2dx. (4.24)
This is the well-known Tikhonov (Tychonov) regularisation.
The diffusion operator, which groups partial derivatives ∇ynζ(x), measures the variation of yζ,
and the operator has been used in optical flow and demons algorithms, that is
R(yζ) =
Z
Ω
In addition, if we define the divergence as ∇ · yζ(x), the elastic operator is
R(yζ) =
Z
Ω
ι h∇yζ, ∇yζi + (ι + ς)(∇ · yζ(x))2dx, (4.26)
in which ι and ς are the Lam´e constants[204].
By introducing the second order derivatives ∆ynζ(x), we can derive the curvature based regulariser as
R(yζ) =
Z
Ω
(∆y1ζ(x))2+ (∆y2ζ(x))2dx. (4.27)
Furthermore, the penalty term P(yζ) could contain additional constraints including the deviation
from user-supplied landmarks, volume preservation, and local rigidity[76].
Although the concept of regularisation and penalty terms, which changes the ill-posed problem into a solvable one, is easy to understand, the implementation needs to be deliberated. Firstly, the discretisation of the operators needs to be conceived. Secondly, the derivative of the operators needs to be deduced before substituting them into the optimisation process. In addition, the operators in higher-dimensional space may require more computational time. The choice of the parameters (λ and ξ) of these operators considerably affects the results of registration.
4.2.2.4 Validation Methods
In real clinical applications, the ground truth or the real transformations are generally unknown. However, a registration is not feasible for clinical usage without an effective validation. Maintz and Viergever[185], and Jannin et al. [127] reported that the registration validation criteria include accu- racy, reliability, robustness, fault detection, functional complexity, and clinical use. Commonly used evaluation schemes are Target Registration Error (TRE), Consistency Registration Error (CRE), visual inspection, image matching criteria, and clinical usefulness. The TRE is expressed as follows.
TRE(x) = kT(x) − Twarp(x)k, (4.28)
in which TRE is calculated at the corresponding anatomical positions of interest[99], i.e., manually
selected landmarks, in the source image T(x) and transformed target image Twarp(x). In addition,
TRE represents the distance after registration between corresponding points or landmarks, which are not used for calculating the registration transformation [77]. TRE spatially varies across the
image except for simple translational errors. The error images show the spatial TRE distribution, which is useful in analysing registration achievements. The accuracy can then be expressed by the
mean, median, Root Mean Square (RMS), 95% or maximum of the regional TRE distribution.