The measurement of tissue density

One of the properties of mammograms measured using image-processing techniques is the density of normal tissue. There are a number of reasons for being

interested in this: in order to be able to identify asym m etries [M iller 1992], because d enser breasts are harder to read and m ay require a different interpretation protocol [Jackson 1993] and because density may be related to risk [O z a l9 9 3 ],

breast outline variable width patch 128 pixels rightm ost •«i» foreground pixel 32 pixels

Figure 17: the m ethod used to measure the density o f tissue. Thresholding is used to give an estim ate o f the brea st outline and then an autom ated procedure is used to identify a region o f interest behind the nipple. The measure, based on the asym m etry o f gray- level histogram s, is ap p lied in tiles over the region o f interest. The diagram is not to scale.

E lsew here [Taylor 1994] I have presented a m ethod for m easuring the density of norm al tissue in the breast. This m ethod involves first locating a region o f interest above and behind the nipple, w here, if the breast is dense, dense tissue will norm ally be found and then m easuring the asym m etry o f grey-level histogram s in non-overlapping

16x16 pixel w indow s w ithin this region.

Measurement: the evidence for each argument may include the result of meas­ uring a component of the interpretation of an image.

The model required in this task distinguishes the breast region from the background. The finding is depicted as the ratio of lighter to dark pixels and measured using an operator which constructs histograms of small regions within the breast area, measures the skewness - the asymmetry of the histograms - and takes the mean of the skew values. We require facts stating the model:

The M odel, to assess risk we must identify a region within the breast.

A set of facts is also required, recording how the regions in the model are depicted, identifying detectors for those depictions, representing how ‘tissue density’ is depicted in the region of interest and how that depiction can be measured.

Measurement: the evidence for each argument may include the result of measuring a component of the interpretation of an image.

If region( Patient, D ecision, Finding, region( R egion, Im ageR egion ))

an d m easure_of_property( Finding, M easure)

an d apply_m easure( Im ages, M easure, Im ageR egion, Value )

Then find_evidence_for_argum ent( m easurem ent. Patient, D ecision, Finding, [ region( Region, Im ageR egion ), V alue ])

M o d e l to assess risk we must identify a region within the breast.

m odel( a s s e s s m e n t of risk, mammography, [ breast region ]).

Facts D escribing D etectors: the measure used is the mean of local skew­ ness values.

m easure_of_property( sk ew n ess_of_local_h istogram ( Bound ), m ean( local s k e w n e s s ))

Table 9: the rules and fa cts used to describe the measurement o f tissue density. The model identifies the region which must be detected; the measurement o f tissue density is perform ed by applying the appropriate measure in this region.

Two image-processing procedures are applied. First a thresholding operation is applied in the interpretation of the image according to the model. Secondly the skewness operator is applied to measure the tissue density in the region identified in the interpretation.

4.6

Discussion

The purpose of this work is to solve a number of problems associated with the provision of decision support based on information obtained from two distinct infor­ mation sources and represented in radically different ways: as digital images and as symbolic knowledge. This chapter has described an approach which incorporates infor­ mation obtained from image processing in a knowledge-based decision support system. The approach is based on a symbolic model of decision making which is used to propose candidate solutions for a decision problem and then to generate arguments for and against them. This model has been extended to allow arguments to be based on the results of image-processing operations carried out on the image under the control o f an extension to the decision procedure. This represents a different way of looking at image processing - as a source of information to help in making a decision - and a new framework within which to present the information which image-processing operations can obtain from images.

The model represents the processes of decision making and image interpre­ tation at a very general level. The hope is that this makes possible the implementation of systems which support different decisions and different image interpretation tasks and consider different classes of image while sharing a common structure and consistent interface. As a demonstration of at least a measure of generality, two prototype systems have been developed using the model. These are described in Chapters Seven and Nine.

Im age Processing PA CS/H IS/RIS

IP K now ledge K now ledge Base

and depictions(... )

and detecto r(... )

and detector_applied_to_image(... )

Then find_evidence_for_argument(...) p a tie n t_ h a sjm a g e s(... )

Figure 18: the operation o f the rule describing the detection task. The first line represents a query to he addressed to an external information system - perhaps a PACS system o f the kind d escrib ed

in Section 1.3 - and the second looks up the required depiction in the know ledge base. The third identifies the appropriate image p rocessing operator. The fourth line invokes the operator, which is a p p lied to the image. The result o f the image processing operation then p rovides the evidence.

O ne feature o f the m odel outlined that once decision support is requested the im age-processing operators are applied autom atically by the system . T he m odel is intended to fit w ithin a decision support system and the m ode o f operation o f the system has been assum ed to be passive: the system only provides decision support on request. It should, how ever, be possible to use a m odel o f this kind w ithin an autom atic system o f the sort described in the account of the dom ino m odel in Section 4.2.

C onsider how such a system m ight be used in the m easurem ent o f tissue density. O ne reason for m easuring tissue density is the hypothesis that m ore skilled radiologists perform better at the interpretation of dense m am m ogram s, w hereas the

interpretation of fatty mammograms can be adequately performed by a trained radio­ grapher, or another less expensive member of the clinical team. If the protocol followed in screening programmes was based on this hypothesis a decision would have to be made, once the images were taken, as to who would interpret them. The system would have a knowledge base which included a representation of the protocol and would be able to apply the model to obtain the information required for the system to make a decision - the measurement of tissue density. This decision could then be taken automatically, which would then move on to the next activities in the plan and schedule these. This is a potentially interesting application of the model, but one which is radically different from the notion of a decision support system adopted in this thesis, where the emphasis is on assistance for decision-makers rather than autonomous decision making.

One reason for adopting the Symbolic Decision Procedure as a means of combining image processing and knowledge-based reasoning was that the decision procedure allows a flexible approach to the handling of uncertain information. Recall the approach taken in LA where arguments are associated with some measure of their force. This measure could be drawn from any one of a number of dictionaries and different approaches to aggregation could be taken depending on which representation was used and what assumptions could be made about the independence o f different arguments. I have used the Symbolic Decision Procedure to organise the presentation of decision support information in the form of evidence for arguments; no facility is provided for combining numbers associated with different arguments, or with different pieces of evidence for an argument.

This is an important area for future research: the integration of measures of uncertainty associated with the application of image processing to digital images. Uncertainty arises through the fallibility of the image processing, the imprecision o f the radiological descriptions it seeks to capture, the uncertainties of the image-forming

aetiology. Further difficulties stem from the problem of how uncertain information from different imaging modalities, or even different areas of a single image, should be combined.

The prototypes have to deal with uncertainty in a minimal way, however, because the procedure, as presented here, contains no threshold for what constitutes useful information. Thus, in the detection task, the relevant evidence consists of the response of the calcification detector at every point in the image. If this was presented to the user as a set of arguments, the result would be disastrous. This problem is handled by implementing an all-or-nothing calcification detector. In fact, most systems which attempt to detect abnormalities take this approach. Even where the detector returns a value which indicates the likelihood of an abnormality being present, the only information which is passed to the user is whether the obtained measure was above or below some threshold.

In the classification task, an elongated appearance is an argument in favour of the hypothesis of intraductal carcinoma: malignant calcifications. The rule used in the classification task controls the application of an image-processing operator which represents a measure of elongation. W hether or not a particular value for this measure means that a calcification is elongated or not is determined by a simple threshold and no other information about the degree of elongation is passed to the user.

A consequence of using the decision procedure to control the invocation of image processing is that computationally intensive image-processing measures may be invoked, slowing down the system in a way that is unacceptable if it is to be used inter­ actively. The implementation of the extended decision procedure copes with this problem in two ways: by requiring a user who requests decision support to indicate a region o f interest and by storing with each image a set of facts representing the results

of expensive computations. This cache of facts is added to as further processing is performed under the control of the decision procedure.

The aim of this chapter has been to describe the extended decision procedure which forms the basis of the approach to combining signal and symbol information. The next chapter describes a generic architecture for decision support systems built using this approach.