Expert systems with image processing

Another class of systems consists of those which provide an expert system for an image interpretation task and incorporate some image-processing operators into the system, so that at least a proportion of the data required by the expert system can be obtained automatically from the image.

3.3.3.1 AUTOMEX: knowledge-based classification of breast lesions

Stewart [Stewart 1994] presents a knowledge-based system for automatically classifying breast lesions.

Images and image processing

The system is for use with digitised mammograms and includes image- processing routines for measuring four characteristics of circumscribed lesions: density, contour, size and form. It is not clear from the published account how the abnormalities are detected.

Symbolic knowledge

Knowledge about the abnormalities found on mammograms and their diagnostic significance is represented as production rules.

Operation

The system works by backward chaining into a system of menus which elicit from the user the required characterisations of mammographie signs in order to prove

or disprove a hypothesis about the sign. The hypothesis will normally be a diagnosis. In addition, the system includes an image-processing module which measures four features of images of circumscribed lesions, namely density, contour, form and size. Thresholds were determined to split the measurement range into sub-ranges corre­ sponding to radiologists’ assessments.

Combining symbols and signal data

Circumscribed lesions, as defined in the symbolic knowledge base, have a number of properties and each of these has a number of values. For four properties there is a mapping between the symbol and the output of an image-processing operator which computes a value for a relevant property of the photometry. The values obtained are split into sub-ranges which correspond to the symbols for property values in the knowledge base. This correspondence was established experimentally.

3.3.3.2 Rule-based reconstruction of 3D angiograms

Smets et al. [Smetsl990] describe an expert system for the labelling and 3D reconstruction of coronary arteries from coronary angiograms representing two projec­ tions.

Images and image processing

The images are stereo pairs of 2D angiograms. The image processing delineates the blood vessel segments using a ridgepoint detector which finds the maximum intensity point in all one-dimensional segments of an image. Since a blood vessel appears as a high intensity region bordered by two almost parallel lines, a region- growing algorithm is used to identify the parallel lines on either side of the centre found by the ridgepoint detector.

Symbolic knowledge

The system contains rules which capture constraints on the possible arrangement of blood vessels. Some of the constraints concern properties of individual segments: position, direction, grey-level and thickness. Some concern relations between segments: above, thicker-than, left-of, connected-to and same-direction.

Operation

The process has three stages. First the images are processed to delineate the blood vessel segments, second the coronary tree is identified using a rule-based model of normal and abnormal anatomical structures and finally the two projections are used to form a 3D reconstruction. All segments are first initialised to all possible interpreta­ tions and the rules used in the recognition of the coronary tree serve to eliminate impossible or unlikely interpretations. In general the output at this stage leaves one or two remaining solutions. The 3D reconstruction requires matching the two projections, this is trivial if the output from the previous phase is unambiguous, otherwise it involves calculating the permitted disparities, finding potential matches and calculating goodness-of-fit based on: diameter (which is the same in all projections), similar grey value and longest matching length.

Combining symbols and signal data

The signal-symbol mapping takes place in two stages. First a set o f abstractions is created from the image data, that is, the segments. Next a labelling operation is performed, proceeding on the basis of rules which eliminate impossible labellings. Some of the rules include signal data, as shown in Figure 8.

(view^id RAO)

{ (s e g m e n tj c a ^id <id>^lad T

^begin <begin> ^end <end>) < segm ent> } (it_is_so^that (s e g m e n t_ lie s jn <begin> <end>

1 1 1 1 1 1 0 0 1 1 1 1 1 0 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00000000))

(it_is_so^that (vertical <begin> <end>))

— >

(modify < segm en t> ^lad F))

Figure 8: one o f the rules in Smets et al. ^ system. The rule consists o f a condition and an action, to be perform ed when the condition is fulfilled. A complete explanation o f the syntax would be inappropriate here, but this figure shows how image data is actually incorporated into the symbolic representations: the pattern o f zeros and ones represents a portion o f the image and ‘segm entJliesJin’ is the name o f procedure which identifies the position o f the segment in the image. The condition requires that the segm ent lies in the top left co m er o f the image.

This suggests a mapping from spatial to symbolic information which makes very particular assumptions about where symbols will be on the input image. Such assumptions can be made in certain classes of medical image in certain applications, where the same projections are used and where images even from strongly abnormal patients share certain key features.