General Deformable Model Approach for Model-based Reconstruction

General Deformable Model Approach for Model-based Reconstruction

Hervé Delingette and Johan Montagnat INRIA, Epidaure project

06902 Sophia-Antipolis Cedex, BP 93, France

Published in

the proceedings of the IEEE International Workshop on MODEL-BASED 3D IMAGE ANALYSIS (MB3IA’98) January 3rd 1998, Bombay, India

Abstract

In this paper we propose a general approach for per- forming model-based reconstruction. The task consists in deforming a reference shape in order to extract a geomet- ric model from a 3D dataset. To achieve this goal, two complementary approaches have been widely used. The deformable model framework locally applies internal and external forces to fit 3D data. The non-rigid registration framework iteratively computes the best global transforma- tion that minimizes the distance between a model and the data. We first show that applying a global transformation on a surface model, is equivalent to applying some global forces on a deformable model. Second we propose a scheme where we combine the registration framework with the de- formable model framework. This hybrid deformation model allows us to control the amount of deformation from the ref- erence shape with a single parameter. Finally, we propose a general algorithm for performing model-based reconstruc- tion in a robust and accurate manner. We have applied this approach to the reconstruction from both range data and medical images.

1 Introduction

1.1 Model-based Reconstruction

Model-based reconstruction consists in deforming a given contour or surface template in order to fit some dataset. This approach differs from general reconstruction techniques where no a priori knowledge about the geome- try or the topology of the object is known. The inclusion of important prior knowledge for reconstruction is beneficial when the shape variability of the object remains small or when the dataset is difficult to segment. The interest of the method is thus to provide a good initial model while taking into account some shape variations.

The segmentation of human organs from CT or MRI im- ages, is a good example where model-based reconstruction can be applied. In general, a kidney looks very much alike

another kidney, even tough there may be significant shape variation due to differences of gender or size.

Surface Model

3D dataset

Model-Based Reconstruction

Reconstructed Surface

Figure 1. The principle of model-based recon- struction

The model-based reconstruction problem can be summa- rized in the following manner (see figure 1) : let

be a contour or surface model of a given object consisting in a set of vertices

V i

IR ^d

^{, (} i = 1 :::n ^{). Let}

be a tridi- mensional dataset (such as range data or a CT-scan image).

The problem is to find a transformation T ^{such that} T 

]

is an appropriate representation of the object in dataset

. A transformation T associates with each vertex V i ^{a vertex}

T  V i ] ^.

To find the transformation T , two different approaches have been proposed :

1. The registration framework[2]. In this case, the transformation T is searched among a restricted class

T

reg of transformations defined over the whole Eu- clidian space IR ^{d :}

T

^reg

8

P

IR ^d

T  P ]

IR ^{d (1)}

Usually those classes of transformations form a group of transformations with respect to the composition operator. Examples of such classes of transforma- tions

reg are the group of rigid, similarity and affine transformations. We call this transformation a global transformation because it is defined on all Euclidian points. The number of degrees of freedom is lim- ited : 6 DOF for the group of rigid transformations, 9 DOF for affine transformations... Typical registration methods[1, 19] iteratively compute the best transfor- mation through least-squares estimation until some convergence criterion is met.

2. The deformable model framework[7]. In this case, the transformation T is only defined for the vertex position V i of the model

.

8

V i

T  V i ]

IR ^{d (2)}

Because there is no a priori restriction on the type of transformation, this framework is also called free- form modeling.

In all cases, finding a transformation T is equivalent to finding a displacement field D ^{defined as}

D 

] = T 

]

⁽³⁾

Registration Free-Form Deformation

Generality ? ? ? ?

Robustness ? ? ? ?

Efficiency ? ??

Figure 2. Comparison between the registra- tion and free-form deformation framework

The characteristics of the registration and free-form de- formation framework are summarized in table 2. On the one hand, registration methods describe a transformation with far less degrees of freedom than free-form deformations.

Therefore their generality of representation, ie their ability to represent shape variations, is less important than the free- form deformation approach. On the other hand, because of their restricted degrees of freedom (DOF), they tend to be more robust than free-form deformations which require a very close initial position.

The computation efficiency of either method is depen- dent on the application. Free-form deformations require the computation of a closest point thus the computation cost is :

C free

form = C closest ( N ) ⁽⁴⁾

where N is the number of vertices of the deformed model.

For deformable models the computation time is approxi- mately proportional to the number of vertices. Registration

requires an additional stage which is a least square estima- tion of the transformation thus :

C registration = C _closest ( N ) + C _LSE ( D ) ⁽⁵⁾

where D is the number of DOF of the computed trans- formation. For low degrees of freedom transformations,

C LSE ( D ) is negligible and registration is as efficient as free-form deformations. But as the number of DOF rises, the transformation becomes more complex and the compu- tation time may become prohibitively high. Examples of complex transformation are the B-Splines defined over cu- bic lattices.

1.2 Previous Work

Many approaches have been proposed to combine both free-form deformation and registration approaches. For in- stance, Feldmar[6] proposed locally affine transformations for surface-registration. Similarly, researchers have im- proved the robustness of deformable models by applying more global constraints. Terzopoulos and Metaxas consid- ered in [17, 10] the superposition of a rigid component with a finite element mesh. A close approach on deformable contours is proposed in [8]. Modal analysis[13, 18] or Fourier representation[15, 16] aim similarly at controlling the amount of deformation.

1.3 Contributions

We first propose a hybrid deformation scheme that inte- grates both global transformations, local deformations, and where the user can specify the shape adaptability of the model with a single parameter. The hybrid scheme has the advantage to be computationally efficient and simple to im- plement.

Second, we provide a general framework for model- based reconstruction that combines both registration and deformable models approaches. We achieve the reconstruc- tion by increasing the DOF of the model in an intuitive man- ner. The hybrid deformation scheme is used as an interme- diate behaviour between registration and free-form defor- mations.

2 Deformable Models and Registration Framework

2.1 Deformable Models Framework

Deformable models evolve under the action of forces

usually resulting from an energy minimizing criterion. At

vertex V i , the external force f _i ^ext is computed from the data

and the regularizing force f _i ^int is computed from the geo- metric properties of the model. It is possible to control the internal and external forces relative effect through weight coefficients  ^and  . The ratio between  ^and  ^{is equiva-}

lent to the regularization parameter in regularization theory.

Under the fundamental principle of dynamics, the mo- tion of each vertex is governed by :

md

V i

dt

⁼

dV dt ^{i +} f _i ^int + f _i ^ext

By discretizing time and space with finite differences we get the following equation

:

V _i ^t

= V _ti + (1

 )( V _ti

V _i ^t

) + f _i ^int + f _i ^{ext (6)}

where V _ti is the position of V i ^{at time} t . The overall scheme of a deformable model is shown in figure 3.

Compute Internal Forces Compute External Forces

Update Vertices Positions

Final Model Initial Model

Figure 3. The iterative scheme for classical deformable model reconstruction

Between time t ^and t + 1 , the displacement field D ^ap-

plied on the model

corresponds to :

D =

(1

 )( V _ti

V _i ^t

) + f _i ^int + f _i ^ext

2.2 Registration Framework

The registration framework requires the computation of a global transformation G which can be evaluated iteratively using an Iterative Closest Point algorithm [19]. To compute the transformation G , for each vertex model V i , its closest point W i in the dataset is computed. Let

reg be the group of transformations among which G is searched. At each iteration, the ICP algorithm finds G

reg which satisfies the minimization of the least square error criterion

G = arg min _G

2T

reg

(^X

n i

k

G  V i ]

W i

)

(7)

1The time constant ^thas been hidden in the coefficients^and^

Equation (7) provides a general criterion to evaluate the global transformation G ^. Its resolution depends on the group of transformations considered.

Best Rigid Transformation. A rigid transformation of a point P is T rig ( P ) = RP + O ^where O is a translation vector and R a rotation matrix. It can be shown [12] that

O = V

R W ^where V ^and W are the inertial centers of

V i

i ^and

W i

i , respectively, and that R minimizes the criterion

P

n i

R V ^{^} i

W ^ i

^with V ^ i = V i

V ^and W ^{^} i = W i

W ^.

Best Similarity. A similarity (rigid transformation plus a scale factor) can be written T sim ( P ) = SRP + O ^where S is a diagonal matrix whose diagonal terms are all equal to the scale factor s . The optimal rotation and translation are evaluated as for the rigid case. The scale factor is computed independently [12] s = ^Tr

( ^R

)

i

k

V

^with  wv =

_i W i V _Ti ^.

Best Affine Transformation. An affine transformation can be written in matrix form in homogeneous coordinates

T a ( P ) = AP . It can be shown [12] that A is a closed form : A =  wv 

_vv ^with  vv =

_i V i V _Ti ^.

Best B-Spline Transformation. A B-spline transforma- tion of P ^is T spl ( P ) = ( f ^x ( P ) f ^y ( P ) f ^z ( P )) ^{T where} f ^d

is a piecewise polynomial function defined as a tensor prod- uct of B-spline base functions of a given order. The best spline transformation, given a set of pair of points ( V _i W _i ) ^,

should minimize a criterion sum of residual distances and a smoothness term C ( T spl ) = C dist ( T spl ) + C smooth ( T spl )

where  is a weight. Details on solving this system using a conjugate gradient method can be found in [4].

Axial Constraint. We have developed an axial con- straint to deform 3D models based on an axial symmet- rical topology such as vessels. For more details see[11].

As shown in section 5.4 it gives encouraging results in seg- menting vessels from angiographic images.

Other Global Transformations. Many other global transformations could be investigated. The use of transfor- mations with more degrees of freedom (such as high degree B-splines or radial basis functions) would allow to generate a wider range of shapes, however, at a greater computation cost.

The overall scheme of the registration framework is shown in figure 4.

3 General Framework for Model-Based Re- construction

3.1 Registration as a Displacement Field

The purpose of this section is to show the link between computing a global transformation on a model

and ap- plying an external force field to a deformable model. Let

f

V _ti

be the vertex position of a model

at time t ^.

Find Closest Point

Compute Best Transformation

Apply Transformation

Final Model Initial Model

Figure 4. The iterative scheme of registration- based segmentation

In the registration framework, we first compute the clos- est points

W _ti

, then find the best transformation G ^{t and}

finally apply the transformation at each vertex : V _i ^t

= G ^t  V _ti ] . The displacement field applied between two itera- tions is therefore G ^t

I ^where I is identity transformation :

D = G ^t

I =

G ^t  V _ti ]

V _ti

In section 2.1, we have seen that the displacement field applied on a deformable model is equal to

f

(1

 )( V _ti

V _i ^t

) + f _i ^int + f _i ^ext

We can therefore consider that the registration method based on the ICP algorithm is equivalent to having a de- formable model without inertial and internal forces (  = 0 ^, f _i ^int = 0 ) and submitted to the global force :

f _i ^global = G ^t  V _ti ]

V _ti ⁽⁸⁾

3.2 Link between Global and External Forces The computation of external forces is dependent on the type of datasets. In this paper, we suppose that the external force acting on vertex V i is of the form :

f _i ^ext = W _ti

V _ti ⁽⁹⁾ W _i represents the "closest" point on the object where V _i

should be attracted. In practise, we have verified that ex- ternal forces based on the notion of "closest" point lead to more stable behavior than external forces based on the no- tion of gradient of potential field. This is because the poten- tial field extracted from the gradient information may have steep minima that entail oscillations of the surface around those minima. To remove those oscillations, it is neces- sary either to decrease the time-step or blur the image at

the larger scale. The former method has the drawback of slowing down the convergence whereas the latter removes and displaces some minima of potential field.

In section 4, we will see different expression of the exter- nal force for reconstruction from range data and volumetric images.

By comparing equation 9 with equation 8, we can see that the displacement field for a registration-based defor- mation has less DOF than an external force field applied on deformable model since it is constrained by the nature of the transformation.

3.3 Hybrid Deformation Scheme

In previous sections, we have demonstrated the equiv- alence between registration-based deformation and the ap- plication of a global force to a deformable model. We now propose a hybrid deformation scheme, where a deformable model is submitted to global, external and internal forces.

The purpose of this scheme is the following : to have a computer-efficient deformable model with an easy control of the number of DOF.

Our approach is to weight with a single parameter ^the

influence of global forces versus external forces :

V _i ^t

= V _ti + (1

 )( V _ti

V _i ^t

) + f _i ^int + ⁽¹⁰⁾



f _i ^ext + (1

) f _i ^global

We call , the locality parameter. It controls the number of DOF of the deformable model. With = 0 , the model is under the influence of global forces and internal and there- fore has very few DOF. With = 1 , the model is under the influence of internal forces and external forces and therefore has a maximum number of DOF. With intermediate values of , we can control the shape variation allowed during the deformation.

Final Model Update Vertices Positions Compute External Forces

Compute Best Transformation Initial Model

Compute Global and Internal Forces

Figure 5. The iterative scheme for a de-

formable model under hybrid deformation

In figure 5, we show the overall scheme of an hybrid deformable model. Because we can easily change the de- formable behavior of the model from global to local, we can choose a global transformation with few DOF such as an affine transformation.

The hybrid deformation scheme is efficient because the search for closest points has to be performed only once for the computation of the global and external forces. Further- more, we usually can use transformations with few DOF since controls model variability. Thus the least square estimation of the transformation is done efficiently.

3.4 General Model-Based Reconstruction

In section 3.3 we have introduced a hybrid deformation scheme that allows to control through a single locality pa- rameter the amount of deformation allowed. In figure 6, we present the general framework that allows us to achieve model-based reconstruction. The principle of this frame- work is to increase the number of DOF during the deforma- tion in order to combine robustness with accuracy. We first apply iterative registrations with rigid, similarity and affine transformations. We then choose values of lambda between 0 and 1 (usually around 0.2) to increase the shape deforma- tion. Finally, we apply free-form deformation to closely fit the data.

Rigid Registration

Similarity Registration

Affine Registration

Hybrid Deformation

Free-form Deformation

0 < λ < 1 λ= 0.0 λ= 0.0 λ= 0.0

λ= 1.0

Figure 6. General framework for Model-Based Reconstruction

The remaining problem is to determine when to change the transformation class and therefore increase the num- ber of DOF. When considering the model-based registration problem shown in figure 15, we measure the total vertices displacement between two consecutive iterations of the ICP registration. Figure 7 shows the results for four different transformations.

0.0 50.0 100.0 150.0

Iteration 10³

10⁴ 10⁵ 10⁶ 10⁷

Displacement

(a) rigid

0.0 10.0 20.0 30.0 40.0 50.0 Iteration

10² 10³ 10⁴ 10⁵ 10⁶ 10⁷ 10⁸

Displacement

(b) similarity

0.0 10.0 20.0 30.0 40.0 50.0 Iteration

10² 10³ 10⁴ 10⁵ 10⁶ 10⁷

Displacement

(c) affine

0.0 10.0 20.0 30.0 40.0 50.0 Iteration

10² 10³ 10⁴ 10⁵

Displacement

(d) B-spline

Figure 7. Displacement of model

Due to the fast convergence of the ICP algorithm (notice the log-scale on the Y axis), the displacement is sharply decreasing. When the displacement is low, the model does not evolve significantly anymore. Therefore we can set a low threshold to stop a deformation stage and increase the number of DOF by changing the transformation.

We propose two strategies to set the low threshold : Absolute displacement. An absolute threshold value is

provided by the user.

Relative displacement. The threshold is computed as a percentage of the initial displacement. We let the model evolves a few iterations (three for instance), compute the mean displacement and set the threshold as a fraction of the obtained value.

We have runned the same face registration experiment with an automatic thresholding to change transformations : each deformation stage ends when the model displacement is lower than 0.1% of the initial displacement. Figure 8(c) compares the two strategies in term of number of iterations.

The total number of iteration set manually (solid line) is

much higher than the number of iterations needed to meet

the automatic threshold (dashed line). It reveals a signifi-

cant diminution of the total computation time.

rigid similarity affine B−spline

0.0 40.0 80.0 120.0 160.0 200.0

Number of iterations

User controled Automatic threshold

Figure 8. Registration with automatic thresh- olding

4 Deformable Surface Models : Simplex Meshes

4.1 Simplex Meshes

Simplex meshes [5] are meshes with constant vertex adja- cency and have interesting geometric properties. 2-simplex meshes are a natural extension of active contours in 3D and they provide a powerful framework to express regulariz- ing constraints. A 2-simplex mesh of IR

is a 3-connected mesh. Each vertex position V i can be expressed as a func- tion of its neighbors V N

i

^, V N

i

^, V N

i

, its metric pa- rameters

_i ^,

_i ^,

_i and its simplex angle ' i ^.

V i =

_i V _N

_i

+

_i V _N

_i

+

_i V _N

_i

+ L ( r i d i ' i )

i ⁽¹¹⁾

where

 n

i ^{is the normal vector at triangle}

( V _N

_i

V _N

_i

V _N

_i

) ^.



r i is the radius of the circumscribed circle at the tri- angle ( V N

i

V N

i

V N

i

) ^.



d i is the distance between F i =

_i V N

i

+

_i V N

i

+

_i V N

i

and the center C i of the circumscribed cir- cle.



L ( r i d i ' i ) is a function describing the elevation of

V i above plane ( V N

i

V N

i

V N

i

) ^.

The metric parameters and the simplex angle are intrinsic parameters that describe the shape of a mesh with a given topology up to a similarity.

The simplex mesh framework is computationally very efficient. Local force computation does not require a mini- mization step. The example given in figure 9 requires a 50 seconds of computation on DEC Alphastation 500/400Mhz.

4.1.1 Shape Constraint

The metric parameters and simplex angles of simplex meshes allow us to define smoothness as well as shape reg- ularizing forces. Without external forces, a simplex mesh submitted to the shape constraint converges toward its ref- erence shape. An example of a face model iteratively re- turning to its reference shape is given in figure 9.

Figure 9. Shape constraint

4.2 External Force Computation

We compute the external force f _i ^ext as a vector directed along the normal direction proportional to the distance of

V i to the dataset. Whether the force is computed from range data or volumetric images, the external force is of the form :

f _i ^ext =  ( V i M i

i )

i

where M i represents some data point.

4.2.1 External Force Computation on Range Data The computation of the external force is dependent on the notion of closest point. For every vertex V i of the mesh, we search for the closest data point M Cl

i

and the force is then computed as :

f _i ^ext =  _i G

V i M _Cl

_i

D ref



( V _i M _Cl

_i

_i )

i ⁽¹²⁾

where  i is a weight parameter,

i is the normal vector of the mesh at V i ^, G ( x ) is the stiffness function and D ref is a reference distance.

The force f _i ^ext is computed as the projection of the vec- tor V _i M _Cl

_i

along the normal direction. The distance D ref is the maximum distance of attraction of a data point. The stiffness function G ( x ) ensures that the force decreases sharply when the distance between M _Cl

_i

^and V i ^{is greater}

than D ref.

The computation of M _Cl

_i

depends on the nature of the dataset and can be achieved with two algorithms :

Projection Method On dense range images extracted from

triangulation principles, we restrict the search along

a two-dimensional segment in the image, projection

of the tridimensional line passing through V i ^{and di-}

rected along

i . This segment is centered around the projection of V i and is only a few pixels long, de- pending on the value of D ref . Once the closest point along the segment has been found, we search for the closest point in a 5

5 window around that point.

KD-tree When no calibration matrix is available, we com- pute the closest point with a kd-tree [14]. This data structure gives the closest point inside a sphere cen- tered around V i and of radius D ref, at nearly constant time.

4.2.2 External Force Computation on Volumetric Im- ages

On volumetric images, the task of reconstruction usually consists in isolating regions of consistent intensity values.

Therefore, gradient intensity is the main information on which is based the external force. As in [3] and [9], we combine both gradient intensity and edge information for the computation of f _i ^{ext :}

f _i ^ext = f _i ^grad + f _i ^{edge (13)}

The gradient intensity is used for local deflection of the mesh towards the voxels of maximum variation of inten- sity. The edge information, on the other hand, corresponds to maxima of gradient and entails larger deformations of the mesh.

The force f _i ^{grad at vertex} V i relies on the search in a neighborhood of V i , of the voxel of maximum gradient in- tensity. If

is the voxel containing V i , then we inspect all voxels in a m

m

m window around

for the voxel cen- ter G _i , of highest gradient intensity (see figure 10(a)). The force expression is then :

f _i ^grad =  _i ^grad ( V i G i :

i )

i ⁽¹⁴⁾

The gradient force can be made more specific by incorpo- rating additional constraints : gradient direction constraint and intensity value constraint.

The edge image is built by thresholding the gradient in- tensity values. Our approach consists in finding the closest edge voxel in the normal direction of the mesh. At vertex

V i , we find the closest voxel

, and a tridimensional line of voxels is scanned in the direction of

i (see figure 10 (b)).

The maximum number of voxels scanned is given by the reference distance D ref, determined as a percentage of the overall radius of the edge image. If E _i is the closest edge voxel along the normal line, then the edge force is given by :

f _i ^edge =  _i ^edge V i E i ⁽¹⁵⁾

Similarly to the gradient force, we can add constraints to the determination of the closest edge voxel.

N_i F

V

ext

Gi

Voxelν

i

(a)

V

Ei

Normal L

Edge Voxel i

(b)

Figure 10. (a) Computation of the gradient force f _i ^grad ; (b) Search of the closest edge voxel along the normal line for the compu- tation of f _i ^{edge .}

5 Results

5.1 Estimation of Registration Quality using Hy- brid Deformations

In this section, we demonstrate the robustness of the hy- brid deformation scheme. We consider the following ex- periment : given some range data of a foot, we have built the model

of figure 11 (a). The model was manually deformed to get the model

? of figure 11 (b).

(a) (b)

Figure 11. (a) foot model;(b) range data and deformed model

We then fit the mesh

? on the original range data for a fixed number of iterations to get model

. Run- ning the deformation process with different deformation schemes, the quality of registration is evaluated as the point- wise distance between the reconstructed model

and

:

d =

_i

V i

V _i

. A small value of d implies that ver-

tices positions of

are close to those

and therefore

that the transformation is robust and accurate. Figure 12

shows the values of d for a rigid registration (first point) an

affine registration (second point) or a hybrid deformation affinely constrained following a first rigid fit (other points).

rigid affine

rigid + affine (0%)rigid + hybrid (5%)rigid + hybrid (10%)rigid + hybrid (20%)rigid + hybrid (30%)rigid + hybrid (40%)rigid + hybrid (70%)rigid + local (100%)

1000.0 3000.0 5000.0

distance

Figure 12. Distance of the deformed model We get the best results for intermediate value of the lo- cality parameter . Therefore, the free-form deformation ( = 100% ) or the global (affine) transformation ( = 0% ⁾

do not give the best fit.

This result can be interpreted in the following manner : if the model is too constrained, the model cannot deform enough to fit the dataset. On the contrary, if it has too many DOF, the surface vertices can be submitted to large displacements during the deformation. In particular, a point of high curvature is not necessarily deformed into a point of high curvature.

5.2 Results on Range Data

(a) (b) (c)

Figure 13. Reference face (left, center) and range data (right)

We have applied the general reconstruction scheme pre- sented in section 3.4 for recovering face models. In figure 13 (a) and (b), we show the simplex mesh model that will be used to fit the range data of figure 13 (c). We will use the texture of the reference model to study the applied transfor- mations. For instance, we can check if the vertices lying on

the nose of the reference model will be moved on the nose of the range image.

We have positionned the reference model in a slightly different orientation and position than the range data, as seen in figure 14 (a). With a purely deformable scheme ( lambda = 100% ), the shape of the face has been recov- ered but the transformation does not preserve the geometry (figure 14 (c)).

(a) (b) (c)

Figure 14. Initialization (left) and face recov- ered using local deformations (center, right)

Using the general deformation scheme, we can first ap- ply global transformations (first rigid, then affine) to pro- vide a better initialization. The model is then deformed with hybrid and local constraints. Figure 15 (a) (b) (c) il- lustrates a few stages of the reconstruction. The texture of figure 15 (d) clearly shows that the geometry of the face was preserved.

(a) (b) (c) (d)

Figure 15. rigid, affine, hybrid deformations and textured result

We now compare the computation time for global, hybrid and free-form deformation (see figure 16 (a)). In this chart, we can see that global transformations with few DOF such as affine transformations have the same computational cost than free-form deformation despite its high level of DOF.

On the other hand, more complex global transformations

such as B-Spline transformations require a much high com- putational cost. The cost of globally constrained deforma- tion is marginally higher than free-form deformation. This is because the cost the computing the external force (or the closest point) is much higher than computing the best global transformation (here with affine transformation).

rigid registration similarity registrationaffine registration

B−spline registrationaffine constrained free−form deformation 0.0

1.0 2.0 3.0 4.0 5.0

Time per iteration (seconds)

Figure 16. Computational cost of global, hy- brid and free-form deformation

5.3 Results on Volumetric Images

Model-based reconstruction is well-suited for the extrac- tion of anatomical structures from 3D medical images. Be- cause most organs have similar shapes between patients, it is often a good strategy to use an a priori information on the shape to extract. Furthermore, this method allows to recover parts that are hardly visible in the original images.

We show three examples of human organs with which our general reconstruction scheme has been successful : liver, brain ventricles and kidney. The recovered geometric models can then be used for surgery planning or diagnosis.

Figure 17 (a) (c) and (e) show the original liver, kidney and ventricle models of complex shape and topology. The reference template of the kidney does not have any cav- ity because of the large variation between patients. The datasets consist in abdominal CT-scan and MRI data of the head.

CT-scan images have a low contrast and therefore the model must be robust enough to deform smoothly even in presence of low gradient regions. However, it must be de- formable enough to fit the drastic inter-patient variability of abdominal organs.

We performed the segmentation following the general deformation scheme presented in section 3.4. The model is first registered rigidly then affinely (locality parameter

= 0 ). As the fit improves and the model is less sensitive to outliers, degrees of freedom are added by smoothly in- creasing ^{up to} 35% . Segmentation of the brain ventricles is performed in a similar way ( is released up to 30% ^{). For}

kidney segmentation, we refine the mesh based on an area criterion in order to recover the cavities.

(a) (b)

(c) (d)

(e) (f)

Figure 17. Models of the liver (top) brain ven- tricles (middle) and kidney (bottom)

The recovered models are shown in figure 17 (b) (d) and (f). Figures 18 (a) (b) and (c) demonstrates the goodness of segmentation by showing the intersection of the deformed models with an image slice. The segmentation of liver mod- els have been validated on 15 cases with the help of medical experts.

Figure 18. Slices of the recovered shapes

5.4 Vessels Segmentation

Segmentation of vessels around an aneurism is impor- tant for diagnosis and pathological understanding. Using deformable axially constrained models (see [11]) we per- form a segmentation of the aneurism and the surrounding vessels. The extracted model is easy to visualize and it al- lows one to perform quantitative measures such as volumet- ric evolution.

Figure 19 (a) shows the initialization of deformable mod- els inside a volumetric image. The aneurism is modeled by a sphere affinely constrained through the hybrid deforma- tion scheme. Vessels are initialized as cylinders restricted by a hybrid axial constraint. The result of the deformation process is shown in figure 19 (b) and the final model ob- tained after topological operations is shown in figure 19 (c).

Figure 19. Vessel models

6 Conclusion

We have introduced a general reconstruction framework that encompasses both deformable models and registration approaches. The hybrid deformation scheme is an efficient and simple algorithm for controlling the amount of allowed deformation.

In the future, we plan to incorporate some additional sta- tistical information in the reference model, in order to con- strain further the deformation.

Acknowledgments

We thank the IRCAD and Dosigray for abdominal CT- scans, the Brigham and Women’s Hospital for brain MR images and General Electric, Medical Systems Europe, for the 3D images of aneurisms (Courtesy of Dr. Yves Trous- set).

References

[1] P. Besl and N. McKay. A method for Registration of 3D shapes. In Pattern Analysis and Machine Intelligence, pages 239–256, Feb. 1992.

[2] L. Brown. A Survey of Image Registration Techniques.

ACM Computing Surveys, 24(4):325–376, Dec. 1994.

[3] I. Cohen, L. Cohen, and N. Ayache. Using deformable sur- faces to segment 3-d images and infer differential structures.

Computer Vision, Graphics, and Image Processing: Image Understanding, pages 242–263, 1992.

[4] J. Declerck, J. Feldmar, M. Goris, and F. Betting. Auto- matic registration and alignement on a template of cardiac stress and rest spect images. In Mathematical Methods in Biomedical Image Analysis, pages 212–221, June 1996.

[5] H. Delingette. Simplex Meshes: a General Representation for 3D Shape Reconstruction. Technical Report 2214, IN- RIA, Mar. 1994.

[6] J. Feldmar and N. Ayache. Locally affine registration of free- form surfaces. In Computer Vision and Pattern Recognition, 1994.

[7] M. Kass, A. Witkin, and D. Terzopoulos. Snakes: Active Shape Models. Intertnational Journal of Computer Vision, 1:321–331, 1987.

[8] K. Lai and R. Chin. Deformable Contours: Modeling and Extraction. In Computer Vision and Pattern Recognition, 1994.

[9] T. McInerney and D. Terzopoulos. A Finite Element Model for 3D Shape Reconstruction and Nonrigid Motion Track- ing. In International Conference on Computer Vision, pages 518–523, 1993.

[10] D. Metaxas and D. Terzopoulos. Constrained Deformable Superquadrics and nonrigid Motion Tracking. In Computer Vision and Pattern Recognition, June 1991.

[11] J. Montagnat and H. Delingette. A Hybrid Framework for Surface Registration and Deformable Models. In Computer Vision and Pattern Recognition, CVPR’97, San Juan, Puerto Rico, June 1997.

[12] X. Pennec. L’incertitude dans les Problèmes de Recon- naissance et de Recalage. Application en Imagerie Médicale et Biologie Moléculaire. PhD thesis, Ecole Polytechnique, France, 1996.

[13] A. Pentland and S. Sclaroff. Closed-Form Solutions for Physically Based Shape Modeling and Recognition. In Pat- tern Analysis and Machine Intelligence, volume 13, pages 715–729, July 1991.

[14] F. P. Preparata and M. I. Shamos. Computational Geometry:

an introduction Texts and monographs in computer science.

Springer-Verlag, 1990.

[15] L. Staib and J. Duncan. Deformable Fourier models for sur- face finding in 3D images. In Visualization in Biomedical Computing, pages 90–104, 1992.

[16] G. Székely, A. Kelemen, C. Brechbüler, and G. Gerig. Seg- mentation of 2D and 3D objects from MRI volume data using constrained elastic deformations of flexible Fourier surface models. Medical Image Analysis, 1(1):19–34, July 1996.

[17] D. Terzopoulos and K. Fleisher. Deformable models. Visual Computer, 4:306–331, 1988.

[18] B. Vemuri and A. Radisavljevic. From Global to Local, a Continuum of Shape Models with Fractal. In European Con- ference on Computer Vision, 1993.

[19] Z. Zhang. Iterative point matching for registration of free

-form curves and surfaces. International Journal of Com-

puter Vision, 13(2):119–152, Dec. 1994.