of each patch) plotted against the target intensity as shown in figure 4.8(a). This visual- ization was also used to perform the initial clustering of the data set using a mixture of 30 Gaussian distributions as can be seen in figure4.8(b). The clustering of a 76-dimensional data set would be extremely biased towards the source feature vectors, this is why we re- sorted to cluster on the intensity space, and translate the cluster memberships to the data set elements.
(a) source and target intesities density distribution (b) Initial clustering of the data set for 30 experts
Figure 4.8: Visualization of the densities, on the left is the joint histogram of input and output intensities (we show here only one input intensity for visualization purposes but computations were carried out in a multidimensional input space). On the right we show the initial clustering with 30 experts in the same intensity space as the left for visualization purposes as well.
The testing of the mixture of experts on a new T1-MRI source image can be seen in figure 4.9. Image 4.9(b) has been obtained by the maximization of the conditional probability. We can see that the regressed image is a lot like the actual T2-MRI image in figure 4.9(c), even though the intensity distribution is far from linear as can bee seen in figure4.8(a). Yet the result image presented here would be hard to accurately register to a T2-MRI, some intensities are clearly off in this image. Most notably there is a white halo around the brain. This is easily explainable by the fact that the background black matches the ventricles black in the T1-MRI, maximizing on the conditional probability just misses the non functionality that the black intensity can map to either a white or a black intensity. This precise issue will be discussed in the next section.
4.4
Solving the one to one problem
Such situations can arise where there is not a one-to-one application between a feature vector and an intensity. If we remind ourselves that Ω is the spatial domain for all images,
(a) T1-MRI test source Image (b) Maximum of the conditional probability
(c) Actual T2-MRI target Image
Figure 4.9: Testing Mixture of experts on a new T1-MRI image
x a position vector in Ω, and that J denotes the source image and I denotes the target image, the aim of image regression is to define an operator f such that:
∀x ∈ Ω, f (J (x)) ≃ I (x) (4.26)
It is important to note here that f is not dependent on the spatial position x as is the case in [Hofmann 2008]. Such a model is compact but also ill-posed. The same origin intensity (different spatial positions) could be mapped to numerous different intensities in the target space, or
J (x0) = J (x1) and I (x0)6= I (x1) (4.27) These situations cannot be modeled by a unique function of the input space, we refer to these situations as non-functionality. To cope with such non-functionalities we adopt a two-component approach. First we augment the information space on which the transport function is defined using a feature vector extracted at the point position x as input of the regression function. Following the notation we introduced in 3.1, this feature vector extraction will be denoted as π (I, x). The feature extraction function is assumed to bring
4.4. SOLVING THE ONE TO ONE PROBLEM 61
some context to the extracted feature by taking into account the direct neighborhood of the spatial point extraction. This augmentation of the information on each pixel drastically reduces the occurrences of the situation earlier described. There is a trade-off to consider in he design of the feature extraction function, the bigger the neighborhood it will act upon, the less ambiguity we will encounter, but in the same time the less general the learned function will be. Hence we will still face situations where ambiguities arise as explained in figure 4.10. For medical images, this lack of one-to-one mapping is due to the use of different modalities and to the locality of some artifact in images.
Figure 4.10: The locality of some image features prevents us from assuming a one-to-one correspondence between feature vectors.
We can see this effect in the images used in the previous section, if we take a closer look at figure4.8(a)we can see that one intensity can map to multiple intensities as shown in figure 4.11. We also can see that this problem still arises even when neighborhood information is taken into account in the form of a feature vector, as we saw in the previous section with the problem of the white halo around the brain.
(a) Graphed line is extracted along the red line (b) For some input intensities, 3 output intensities can be found
Figure 4.11: Visualization of the local maxima for one input intensity
4.4.1
Markov Random Field smoothing
As we have seen, mixture of experts provides us with a complete conditional probability profile instead of only a regression function. Using this fact, for each and every point location in the image, instead of the maximum of the conditional probability, we can focus on local maxima of the conditional probability. For each local maximum we have access to the conditional probability of its occurrence which we can easily transform into a score that will help us chose the right local maximum. The decision of taking only local maxima, instead of the full probability profile is only a choice of discretization of the problem with computational efficiency in mind. Optimizing the score would obviously lead to the having the maximum a posteriori (MAP) of the conditional probability and give rise to the same image as the one in the previous section. Instead, we chose to balance this score with a smoothing constraint that forces a decision on an intensity in one position of the image to be consistent with decisions in a defined neighborhood of the image.
Let us assume that we retain M local maxima, for each pixel location x ∈ Ω. Now let us assume that those maxima are ordered according to their probability, and then labeled with L = {ℓ1(x), . . . , ℓN(x) : ∀x ∈ Ω}, where ℓ1 denotes the maximum with highest probability. If we consider the function lmax that extracts the local maxima of the condi- tional probability and order them: lmax (p (I (x) |J (x) , θ) , ℓn(x)), that will be written lmaxn(x) for short, then we have:
p (I (x) = lmax1(x)|J (x) , θ) ≥ . . . ≥ p (I (x) = lmaxN(x)|J (x) , θ) (4.28) Now let us consider the discrete Markov Random Field energy (we refer the reader to section: 2.3.2):
4.5. RESULTS 63