Validation criteria

Validating the performance of the image registration methods was accomplished through several evaluation statistics quantifying the obtained results. The quantifi- cation was done in several ways, either for the warped images, or with respect to the estimated transformations, or in terms of the biomedical quantities.

Due to the mono-modal data set, the sum of the squared differences (SSDI) is

Figure 4.4: Example of CT lung data used in the experimental validation: (left) the end-inhalation (right) exhalation phase are shown.

Figure 4.5: Example of MRI pelvic area data used in the experimental validation with a ground truth annotation of the organs of interest: bladder (red), rectum (green), and prostate with seminal vesicles (blue).

criterion is defined by: SSDIpIj, Ikq  1 N ¸ ~x pIjp~xq
Ikp~xqq2 (4.1)

where Ij and Ik are the assessed images and N is the number of image points

(pixels/voxels). During the evaluation of the synthetic data with the ground truth deformation field, the sum of the squared differences SSDϕ~ for the estimated trans-

formation ~ϕest and the known transformation ~ϕorg can be calculated. The SSDϕ~ is

defined as follows: SSDϕ~p~ϕorg, ~ϕestq  1 N ¸ ~ x }~ϕorgp~xq
~ϕestp~xq}2 (4.2)

The distance to the known Jacobian of transformation (distJ ac) is computed in the following way:

distJ acp~ϕorg, ~ϕestq 

1

N

¸

~ x

pdetpJp~ϕorgp~xqqq
detpJp~ϕestp~xqqqq2 (4.3)

This criterion is often used to track the deformation field estimation when the ground truth data are available [144, 145]. To enable further comparison of the estimated deformation fields, the determinant of the Jacobian matrix of the transformations is also calculated to track the minimum of the Jacobian matrix determinant (minJ ac). The minimum of the determinant of the Jacobian matrix is defined as follows:

minJ acp~ϕq  min

~

x pdetpJp~ϕp~xqqqq (4.4)

Tracking of minJ ac is important because it reflects the volume changes of a point mapped through the estimated transformation [27]. A positive value of minJ ac indicates that the estimated transformation is one-to-one mapping between image points, while a negative value suggests that the transformation locally folds the domain inside out. The smoothness of the estimated deformation field was measured by evaluating the harmonic energy (HE) criterion [144, 145, 94]. The harmonic energy is defined as:

HEp~ϕp~xqq  1 N ¸ ~ x d ¸ j1 d ¸ k1  Bϕkp~xq Bxj 2 (4.5)

In many applications (especially in medical and biological image analysis), it is commonly assumed that each spatial point in moving image Im is mapped to the

mation i.e. the point from fixed image If has a corresponding point in moving image

Im. Nevertheless, most of the image registration algorithms estimate the forward

transformation that does not invert the backward transformation and vice verse. This inconsistency (described in details in Chapter 3.3) for the estimated transfor- mations can be assessed with respect to the inverse consistency criterion (ICC). A computable criterion reflecting the inability to provide unique description of the correspondence between two images is the inverse consistency error (ICE). The ICE is defined here as an average distance between the original points in one image and their positions in this image after mapping to another image and subsequent mapping back to the original image as follows:

ICEp~ϕf orwp~xq, ~ϕbackp~xqq  }p~x
p~ϕf orw ~ϕbackqp~xqq}2 (4.6)

The inverse consistency error measures only the consistency between the forward and backward transformation, while the accuracy of transformation is not assessed (a detailed discussion is presented in [27, 127]). In the case of the perfectly consistent transformations, the ICE is equal to zero. During the validation the average of the

ICE (aveICE) is also calculated as follows: aveICEp~ϕf orw, ~ϕbackq 

1 2  1 N ¸ ~ x }~x
p~ϕf orw ~ϕbackqp~xq} 1 N ¸ ~ x }~x
p~ϕback ~ϕf orwqp~xq}  (4.7)

and maximum ICE (maxICE) as follows:

maxICEp~ϕf orw, ~ϕbackq 

maxpmax }~x
p~ϕf orw ~ϕbackqp~xq}, max }~x
p~ϕback ~ϕf orwqp~xq}q (4.8)

All aforementioned criteria indicate the quantity with respect to either the image similarity measures or the deformation field statistics, while they somehow do not reflect the anatomical correctness of the results. Thus far, there is a lack of the ground truth data in the clinical applications and alternative criteria for the physical measures of accuracy have to used. From the medical point of view, the algorithms were validated with respect to the Region Overlapping (RO) (or sometimes called Dice Similarity Index (DSI)). The Region Overlapping for organ of interest P is defined as:

ROPpPref, Pwarpq 

numberOf V oxelspPref X Pwarpq

numberOf V oxelspPref Y Pwarpq

(4.9)

organ segmented in the warped image. ROP assesses how well two segmentations of

the organ of interest agree or disagree with each other [127]. It can be also defined as follows:

ROP %pPref, Pwarpq  ROPpPref, Pwarpq  100 (4.10)

Another, common choice of the description for the anatomical registration accuracy is based on the salient, corresponding points between images identified by the medi- cal experts. This enable the target registration error (T RE)to be used by measuring the distance between those selected landmarks before and after registration. The

T RE is defined as follows: T REpHf ix, Hmovq  1 NL ¸ ~ x }~hf ix
~hmov}2 (4.11)

where Hf ix and Hmov are sets of manually selected landmarks in the fixed (refer-

ence) image ~hf ix and in the moving image ~hmov, and NL is the number of selected

landmarks. After image registration, the selected landmarks in the moving image

Hmov are warped using the resulting deformation field and the target registration

error is calculated between the points in the reference image and points mapped by the estimated transformation. In the case of perfect matching, the distance should be zero.