The interpretation rules

In the extension to the decision procedure described here, evidence is obtained from images. The rule for D ecision Support calls a procedure evldence_for_argum en t,

which returns the evidence for a finding, given the decision and the patient. The evidence is obtained from the image by applying the rules which describe the three interpretation tasks - detection, classification and measurement - but first a fact identi­ fying the appropriate task for a given decision must be consulted. Table 4 contains the definition of the rule which describes how the Interpretation Rule is chosen by consulting the facts stating which interpretation task is appropriate for a given decision.

Task Selection: evidence to justify arguments is found by applying the rules defining the appropriate image interpretation task for the decision.

if find_evidence_for_argum ent( Task, Patient, D ecision, Finding, E vid en ce )

and decision_task( D ecision, T ask )

Then evidence_for_argum ent( D ecision, Patient, Finding, E vidence )

D ecision Tasks: the facts recording which interpretation is required for each kind of decision.

decision_task( screen in g, detection )

decision _task ( feature classification, classification ) decision _task ( risk a s s e s s m e n t, m easurem ent )

Table 4: the fa cts that identify which o f the three different image interpretation tasks is appropriate f o r each decision

The three generic image interpretation tasks described in Section 4.3 must be characterised using a set of logical rules, which - in their procedural interpretation - can be used to obtain evidence from images automatically. Consider first the infor­ mation that must be represented in order to allow symbolic rules to control the appli­ cation of image-processing operators and use the result o f image processing in symbolic reasoning. The various findings depicted in images are represented as symbols in the medical knowledge database and the representation must include a mapping between these symbols and the values returned from the image-processing operators. If the rules are to be used to control the application o f image-processing operators, then the representation must include a link between each finding and the image-processing operators. It would be possible to represent this directly: for example

by recording that a given operator is used in the detection of calcifications. However, image-processing operators do not respond to findings, but rather to characteristic properties of the ways in which findings are depicted in images. A clearer represent­ ation, therefore, will include a set of facts recording how the required medical findings are depicted in classes of image and which image-processing operators perform appro­ priate detection, classification and measurement operations for various depictions.

The rule describing detection will therefore use the facts about depiction to identify the depiction of a required finding, then use the facts about detectors to identify the appropriate image-processing detector for the required depiction and finally apply the detector to the appropriate image. The results of applying the operator will be used as the evidence for the existence of the finding.

Detection: the evidence for each argument may include the value returned by a detector applied to the image to detect a feature which depicts the required find­ ing.

Similarly, the rules for classification and measurement will use facts about depiction to identify the depiction of a required finding, and facts about classifiers and measures to identify an appropriate image-processing operator for the classification or measurement of the required depiction and to apply that operator to the appropriate image. The results of applying the operator will be used as the evidence for the existence of the finding.

Query Evidence

Has Ms Smith region of

calcifications? mammogram

I

Imaging Knowledge

depictions of calcifications include sm all areas of high

attenuation

detectors of small areas of high attenuation include the

outlier operator

Mammogram Outlier applied to mammogram

Figure 16: the procedu ral interpretation o f the rule describing detection. In this example, the ‘o utlier' opera to r is a p p lied to a mamm ogram and the region identified by the o p erator is used as evidence f o r the existence o f calcifications.

The key difference betw een the detection task, on the one hand, and the m easurem ent and classification tasks, on the other, is that in the classification and m easurem ent tasks a representation o f at least part o f the relevant anatom y has to be constructed: if you w ant to m easure or classify som ething, you m ust have to detect it. A

m odel o f anatom y is required w hich lists the structures that m ust be detected before a particular classification or m easurem ent decision can be taken, as w ell as inform ation about how the structures are depicted in the im age and w hich im age-processing o p era­ tions can be used to detect these depictions. This perm its the required detectors to be

anatomy nor the resulting interpretation needs to be complete; they need only represent what is required for the particular task.

Region o f Interest: the region corresponding to a finding is any region recorded in the interpretation of an image with features which match the depiction of the finding.

If patient_has_im ages{ Patient, Im ageType, Im ages )

an d m odel( D ecision, Im ageType, Model )

an d interpretation( Im ages, Model, Interpretation )

an d depictions( Finding, Im ageType, region(R egion, F eatureSet ))

and includes( Interpretation, region(lm ageR egion, F eatu reS et ))

Then region( Patient, D ecision, Finding, region(R egion, Im ageR egion ))

Interpretation: the interpretation of an image according to a model is the list of regions identified by detectors which respond to the depictions of the ana­ tomical features listed in the model.

If first_item_of_list( Structures, Structure, Rem ainingStructures )

an d first_item_of_list( R egions, R egion, R em ainingR egions )

and interpretation_of_item( Im ages, Structure, R egion )

an d interpretation( Im ages, Rem ainingStructures, R em ainingR egions )

Then interpretation^ Im ages, Structures, R egion s )

If detector_applied_to_im age( Im age, Detector, R egions, V alue )

an d detector( Depiction, Type, D etector )

and depictions( Item, Type, Depiction )

Then interpretation_of_item( Image, Item, region( Item, R egion s ))

Table 5: the rules describing the use o f a m odel to identify a region o f interest f o r the classification or measurement tasks. The m odel contains a list o f items. The third rule perform s the detection task f o r an item, the second ensures that this task is perform ed f o r every item in a list. The rule Region o f Interest takes the list o f regions returned when the interpretation o f every item identified in the m odel has been performed, and identifies the region with properties corresponding to the depiction o f the finding. This region is then used as the region o f interest.

Region o f Interest: the region corresponding to a finding is a region recorded in the interpretation of an image with features matching the finding’s depiction.

Interpretation: the interpretation of an image according to a model is the list of regions identified by detectors which respond to the depictions of the anatomical features listed in the model.

In a classification task, facts are required which relate findings in the knowledge base to properties of the image, as well as a matching process which ascertains if the values obtained for these properties are suggestive of the presence of the finding.

Classification: the evidence for each argument may include the result o f match­ ing the features depicting a finding against the properties of relevant items in the interpretation of an image.

The rule for measurement is analogous except that the finding is defined in terms of a measure and this measure is applied to the interpretation of the image.

Measurement: the evidence for each argument may include the result o f meas­ uring a property of an item in the interpretation of an image.

In addition to the three rules describing the image interpretation tasks, the extended decision procedure includes another rule which allows information entered by the radiologist onto the patient record to be used as evidence.

R a d iologist’s Report: the evidence for each argument may include items recorded by a radiologist.

D etection: the evidence for each argument may include the value returned by a detector applied to the image to detect a feature which depicts the required finding.

If patient_has_im ages( Patient, Im ageType, Im age )

and depjctions( Finding, ImageType, Feature )

and detector( Feature, ImageType, D etector)

and detector_applied_to_im age( Im ages, Detector, R egion, Value )

Then find_evidence_for_argum ent( detection. Patient, D ecision, Finding, [ Detector, Region, Value ] )

Classification: the evidence for each argument may include the result of matching the features depicting a finding against the interpretation o f an image.

If region{ Patient, Decision, Finding, region( R egion, Im ageR egion) )

an d m easure_of_property( Finding, M easure )

an d apply_m easure( Im ages, M easure, Im ageR egion, R esult )

an d depiction_m atches_interpretation{ M easure, Result, Value )

Then find_evidenceJor_argum ent( classification. Patient, D ecision, Finding, [ region( Region, Im ageR egion ), V alue ] )

Measurement: the evidence for each argument may include the result of measuring a component of the interpretation of an image.

If region( Patient, Decision, Finding, region( R egion, Im ageR egion ) )

an d m easure_of_property( Finding, M easure )

an d apply_m easure( Im ages, M easure, Im ageR egion, V alue )

Then find_evidence_for_argum ent( m easurem ent. Patient, D ecision, Finding, [ region( Region, Im ageR egion ), V alue ] )

Table 6: the rules describing the three image interpretation tasks

The extended decision procedure, therefore, consists of four elements from the original decision procedure:

• a set of logical rules describing the process of decision structuring and decision making

• a set of facts about different kinds of decision

• a set of facts describing the domain in which the decision is to be made • a set of facts describing what is known about the particular case

augmented with:

• a set of logical rules describing the three image interpretation tasks

• a set of facts listing the structures to be identified in specific image interpre­ tation tasks

• a set of facts describing how findings and other relevant structures are depicted in medical images

• a set of facts describing how image-processing operations are used to detect certain kinds of depiction

In this section I have introduced rules describing the process of decision struc­ turing and rules describing the generic image interpretation tasks. In the next three sections I illustrate how these rules are applied in three different decisions, and give examples of the additional task specific and domain specific facts which are required.

4.5

Supporting Image Interpretation Tasks

In this section I give examples of how the extended decision procedure outlined above is used to control the application of image processing in a detection, a classifi­ cation and a measurement task. The examples given are taken from the mammography system, described more fully in Chapter Seven.