5.2 Future Work
5.2.4 Additional Appearance Models
I propose four alternative appearance models to the model presented in Section 4.2.1. A related proposal about the likelihood function is discussed in 5.2.5.
Multiple Appearance Features
One simple improvement to the existing appearance model is to use image features beyond intensity. Segmentation in ultrasound images has been shown to benefit from incorporating texture features. MR images typically have three available features: T1, T2, and proton density. The methods presented in Section 2.2.1 for modeling multivariate probability distrib- utions could be used to estimate the appearance of such features in each model-relative image region.
The Object-Scale Appearance Model
I propose an appearance model that uses object-scale exterior correspondences defined by other objects being (simultaneously) segmented in the image. The global appearance model assumes that no correspondences are known between the objects or along the surface of each object boundary. The local appearance model assumes that both interior and exterior cor- respondences are implied by the geometric correspondence given by the m-rep shape model. Here, I propose a third appearance model, called the object appearance model. This model does not change interior correspondences, so it could use either local or global interior regions. Exterior to the object, it modifies the global exterior region to be exterior to all of the ob- jects being segmented. Therefore, the current segmentations of the other objects are taken into account. For example, the bladder would use an exterior region that would not include intensities from the interior of the current prostate segmentation. The object appearance model more accurately separates shape and appearance variations by leveraging more of the information supplied by the shape model. For example, if the prostate were to slide along the surface boundary of the bladder, this would cause nonlinear mixture variation in local, external bladder image regions. If the prostate were to move away from the bladder, this would cause variation in the global, external bladder image region. However, this variation should only be modeled by the shape prior. The object appearance model correctly does not model such variation.
For the bladder and prostate segmentation experiments described in Section 4.3.2, this model should be more accurate than the global appearance model. However, since not all
of the objects of interest are modeled and segmented, it may be less accurate than the local appearance model. Therefore, the biggest benefit of the object appearance model may be for the segmentation experiment discussed in Section 4.3.3 for which local image regions are inappropriate. Additionally, in both experiments the pelvic bones and the rectum could also be segmented, which would increase the benefit of this approach over the global appearance model.
Pre-Computing an Approximatated Local Appearance Model
I propose an approximation to the local appearance model that has a likelihood function that could be pre-computed. One drawback of the existing appearance model is its computa- tional complexity. I believe an approximation based on locally linear image regions (see below) could dramatically speedup the optimization performed during segmentation.
The local appearance model presented in Section 4.2.1 uses local image regions centered at every spoke that are defined using the local, curved surface of the object boundary. Instead, I propose defining each local image region using the tangent plane defined by the spoke. Each of these more local image regions is only a function of a spoke’s 3 position and 2 orientation parameters. I believe this 5D space could be adequately sampled and used for segmentation as follows. First, for each sample point in this space, compute QFs for its paired interior and exterior regions. Then, for each spoke end, compute and store their Mahalanobis distances to each sample’s estimated QFs. A straightforward implementation as described above combined with an m-rep with 75 spoke ends will require too much memory to store. In this case, a sampling at every pixel with 100 orientations would result in a file 7500 times larger than the image. However, I believe this could be made manageable using a bounding box in the image and a multi-scale sampling scheme.
An Appearance Model based on Distance Distributions
I propose an appearance model based on distributions on distance variables instead of distributions on intensity variables. This proposed appearance model computes the spatial relationship of many boundary points to gas, fat, tissue, and bone. It is natural to combine
this model with the shape prior since they both model spatial relationships. The shape prior estimates the probability of explicitly modeled objects while this appearance model estimates the probability of objects implied by image intensities.
Local image regions can be viewed as definingp(d, i), the joint distribution of distance and intensity with respect to each spoke end. The local appearance model computes two weighted marginal distributionspint(i) andpext(i) fromp(d, i) using signed distance to compute the con- tribution of each sample to pint(i) and pext(i). Here, I instead propose modeling p
α(d|i), the distribution of unsigned distances of the closest α samples at fixed intensities corresponding to gas, fat, tissue, and bone. Specifically, I propose defining 4 intensities {ig, if, it, ib} corre- sponding to gas, fat, tissue, and bone. The appearance model will consist of 4 QF-represented distance distributions for each spoke end, pα(d|ig)pα(d|if)pα(d|it)pα(d|ib). To compute each distribution, I propose using a piecewise linear weighting scheme that assumes that all inten- sity variation from{ig, if, it, ib}is caused by partial voluming. Additional image normalization may be required to insure the image intensities correctly correspond to {ig, if, it, ib}.
This model has 4 desirable properties. First, it can describe object boundaries that are described by any stable spatial relationship of fat, tissue, and bone. Therefore, it is not constrained to object boundaries at transitions between them, like gradient based methods. Second, it is not constrained to predefined local image regions. This model examines the image as far from the boundary as required in order to find the amount of gas, fat, tissue, and bone specified by α, i.e., it does not have a limited capture range. Third, it can be pre-computed. When each spoke end is considered independently, it has a 3 parameter input space that could be sampled at every pixel position. A straightforward implementation with an m-rep model with 75 spoke ends will have storage requirements of 75 times the input image, which can easily be made manageable. Fourth, I believe it will be fairly invariant to day-to- day variation of correct segmentations while varying linearly with increased deformation from correct segmentations. Roughly, local movement of gas, fat, tissue, and bone relative to a spoke end should be linear while local changes in the amount of each one is nonlinear. For the segmentation of the bladder and prostate I believe day-to-day variation of fat, tissue, and bone can be locally characterized as movement. Therefore this variation should be linear in
distance. Gas, however, needs special handling because its position is highly variable day-to- day and its amount changes day-to-day. For the bladder and the prostate, this can be resolved by pooling fat and gas intensities together. Since they are both always exterior to the bladder and prostate, this will decrease their variability while not effecting accuracy.
An early version of this idea has been implemented for distances to bone by Joshua Stough at UNC Chapel Hill. Limited experiments showed no advantages in segmentation accuracy of the prostate over the QF mixture approach described in Section 4.2 based on thresholding bone intensities and estimating their frequency.
5.2.5 Incorporation of Segmentation Variability: The Ideal Image Likeli-