S Model-based object recognition

Robinson et al. [Robinson 1994] describe a system which uses a symbolic model of human anatomy in the recognition of brain objects.

Images and Image Processing

The System has been used to provide a labelling of structures near the midline of a three-slice brain MRI data set. The slices are segmented using an edge detector based on the Canny edge detector. Constraints on the size and position of the identified regions are used to select a set of regions as candidate anatomical structures. A shape description module computes a description of these regions in terms of their skeletons and boundaries.

Symbolic knowledge

The model of brain anatomy consists of a set of frames describing the properties of anatomical features. These are connected by three sets of links which form: a spatial adjacency graph, a part hierarchy graph and an inheritance network. Each frame also has associated with it a model of the shape of the anatomical feature it represents. These shape models are derived from a standard set of MRI images known as the Talairach atlas. The shape models are computed from points identified on the images by the investigator, which allow segments representing the skeleton and the boundary to be determined by the computer. The final description is in the form of a lattice containing skeleton and boundary information about components o f the region.

Operation

The labelling of image parts is achieved in the following steps. First, image processing is used to identify a set of regions. Next shape models are computed for

these regions which are then passed to the shape description module. The resulting descriptions are matched against the shape descriptions associated with frames describing midline structures. A good match results in a provisional labelling o f the structure. The system uses a technique which allows the modelling of hypothetical alternatives to construct a set of consistent labellings. This involves the notion o f a viewpoint, or set of consistent labellings. Each provisional labelling sets up a new viewpoint. The application of rules which merge consistent sets of labellings and remove those containing contradictions converges on the interpretation with the highest number of image to model matches.

Combining symbols and signal data

The shape descriptions computed from the Talairach atlas represent an image description derived from signal data. This representation makes explicit some o f the information about the spatial arrangement of features and can be viewed as an inter­ mediate representation between the signal data and the symbolic model. The link between the image data and the symbolic knowledge is made in part when the descrip­ tions are compiled, and in part in the process of matching the compiled descriptions with those stored in the frames.

3.3.4.6 Atlas-based segmentation of MRI data

Collins et al. [Collins 1995] describe a technique for the segmentation o f 3D M RI images of the human brain, based on a model of brain anatomy. The term ‘m odel’ is used to refer to a segmented and labelled MRI image, also referred to as an ‘atlas’.

Images and image processing

The images used are 3D MRI brain scans and the image processing is an iterative hierarchical registration procedure which matches the image to be segmented with the model, an image whose segmentation is known. The registration consists of

intensity and gradient magnitude are matched. In the linear phase the best affine trans­ formation matrix is found in order to maximise the correlation of features. This linear registration is performed at increasing spatial scales, starting with very blurred data. In the non-linear phase the same method is used in small neighbourhoods of the data set. The non-linear warp is composed of a set of local linear transformations which are performed iteratively, revising the estimate of the required deformation.

Symbolic knowledge

The only symbolic knowledge in the system is the labelling of the model.

Operation

The segmentation of the brain MRI scan is performed by registering the scan with the model, using the linear and non-linear transformations described above. The labels in the model are then mapped onto the image through the inverse of the transfor­ mation determined by the registration procedure.

Combining symbols and signal data

The link between a symbolic representation and one in signal data is achieved through the manual labelling of manually segmented signal data.

3.3.4.7 Knowledge-guided segmentation of CT scans

Kobashi and Shapiro [Kobashil995] present a detailed account of a knowledge- based system for the accurate segmentation of CT scans.

Images and image processing

The following image-processing operations are performed on abdominal CT scans: thresholding, morphological opening and closing, set operations, connected components operations, measurements of shape, position and size.

Symbolic knowledge

The knowledge base includes a representation of the spine, aorta, kidneys, spleen and liver. The following properties are recorded for each: rank order of brightness, grey-level range, height of grey-level cliff, location relative to landmarks, adjacency with other organs, relative size, relationships with slices at other levels, positive and negative shape constraints. In addition to this “domain knowledge”, the system contains a representation of knowledge about the application of the image- processing operators for this problem. This knowledge was established experimentally and determines first the order in which the organs are identified; second, the order in which slices are processed and third, the sequence in which the image-processing operators are applied to each slice.

Operation

The system identifies each of the required organs in turn, in each of the slices in turn, starting with the slice that is two fifths of the organ’s expected height from its bottom-most expected slice. The segmentation can involve a sequence of up to eleven image-processing operations before the outline of the target organ meets the set criteria.

Combining symbols and signal data

The organ properties in the symbolic knowledge base are formulated so that they are directly computable. Symbols and signal data are combined by the set of image-processing operations which are applied to the image and return a value which is used in the symbolic processing.