execution plan.
Figure 6-14 illustrates the query processing for a spatial predicate with
similar-tooperator for the target image as shown in Color Figure-4. The system allows user relaxation control input to overwrite the one provided by the user model. Since neither relaxation control policy nor BASED ON subclause is provided by the user in this example, the relaxation con- trol and matching policy specied in the user model are used (details of the matched user model are shown in Figure 6-10). Thus, SR(lesion,late- ral ventricle) is the rst candidate TAH to be relaxed. Based on the TAH shown in Figure Color Figure-2, the value ranges for retrieving similar images are: (43:91SR(lesion;lateral ventricle):d c 71:31), (0:85SR(lesion;lateral ventricle): c 1:54), (4:0SR(lesion;lateral ventricle):x c 49), (?27SR(lesion;lateral ventricle):y c 57):
These conditions correspond to the value range of the TAH node two lev- els higher than the matched leaf node (see Figure Color Figure-2). The retrieved images are shown and ranked with therelaxation errorattached to each retrieved image (see Color Figure-4). There is anexplanationwin- dow which displays the selected features and spatial relationships used for the matching, the relaxation level, and the number of matched images for this TAH node. During the relaxation process, if the relaxation of a TAH reaches a certainrelaxation error threshold, then the system, according to the relaxation policy, selects another TAH for relaxation. Users can also re- trieve images based on complex query conditions by selectively combining the TAHs with logical operations (e.g., AND, OR, etc.).
6.8 Implementation
Figure 6-15 illustrates the overall ow and key components of our system. Our raw data set is the images stored in the UCLA Picture Archiving and Communication System [20]. These images are sent through segmentation routines to generate contours of interested objects in the images. Methods of the proposed feature extraction technique (see Section 6.3) compute im- age features and spatial relationships from the object contours. These fea- tures are stored in afeature databaseand mapped with image objects and spatial relationships in the Semantic Layer. The evolutionary constructs are used to describe the temporal characteristics of object development.
180
Chapter 6
Objects extracted from the target image
Select the matched user profile from the user model (mandatory matched objects are highlighted by thick−lined box)
Retrieve the TAH(s) from the TAH directory that match the selected features. Locate the TAH nodes in the TAHs such that their value ranges are most close to the target data values to start the query relaxation
The query constraints are relaxed based on user input or the relaxation policy from the user model. The value ranges in the finalized TAH nodes are used to retrieve similar images
The matched user profile is used to select the features and spatial relationship for representing tumor similarity
Lesion Lateral Ventricle Brain
Tumor
Lesion Lateral VentricleLateral Ventricle (1) SR(l, lv) (1) SR(t, f) Frontal lobe (2) SR(l, b) Brain Tumor SRtl
Lesion Lateral Ventricle (1) SR(l, lv) (2) SR(l, b) Brain Tumor SRtl
Lesion Lateral Ventricle (1) SR(l, lv) (2) SR(l, b) Brain TAH for SR(l,b) TAH for SR(l, lv) (dc =62.63, Oc = 1.49, Xc = 39, Yc = 49) Tumor SRtl
Lesion Lateral Ventricle (1) SR(l, lv) (2) SR(l, b) Brain TAH for SR(l,b) (0.85 <= Oc <= 1.54, 43.91<= dc <= 71.31, 4 <= Xc <= 49, −27 <= Yc <= 57) TAH for SR(l, lv)
Figure6-14 Processing steps for Query 4 (the TAH forlesionNearbylateral ventricleis shown in Figure 6-4)
In addition, the features are classied by the MDISC clustering algorithm to produce the type abstraction hierarchies required for knowledge-based query answering.
A knowledge-based spatial evolutionary query language (KEQL) is devel- oped to express spatial evolutionary queries which provides direct manip- ulation of image objects.
The knowledge-based query processor parses the KEQL statement and uses type abstraction hierarchies for the approximate and conceptual matching of image content specied in the KEQL statement. The query result is returned for further presentation and visualization.
Content-Based Image Retrieval
181
− segmentation − feature and content extraction clustering algorithm (MDISC) TAH selection, query relaxation image
retrieval feature retrieval selecting
objects of interest − image objects and relationships − temporal constructs − evolutionary constructs
database schema
images featuredatabase
Queries input (expressed in KEQL) knowledge−based query processing TAHs knowledge representation of features and content query
results GUI
Figure6-15 Data ow and system architecture.
Using GemStone2, an object-oriented database, and VisualWorks as the
application development tools, a prototype medical image management system [7, 19] has been implemented at UCLA to demonstrate the feasi- bility of the approach. Currently, the system runs on a Sparc 10 with a RAID of 16 gigabyte disk storage. Using VisualWorks, we have developed a graphical interface, MQuery [14], with pull-down menus and \point-click- drag" features for querying the database. With graphical representation for both objects (e.g., icons) and streams (used for temporal and evolution- ary constructs), various KEQL predicates can be directly specied on the objects or streams to formulate a query. Buttons are included in the query window for schema navigation, object display, and query conrmation. At present, the prototype system has a few hundred hand X-ray and MR brain image studies. Feature extraction for hand X-rays is performed au- tomatically for patients between the ages of 3 and 11 years [32]. A com- mercially available MR image segmentation and rendering system from ISG Corporation is used to extract features from brain MR images. Con- tours of objects showing good signal-to-noise ratios can often be automat- ically acquired using available minimum/maximum thresholding methods. The system also includes a semiautomatic region growing program after
2We selected GemStone because it provides richer data types than relational database
management systems. This is essential to developing the new modeling constructs proposed in this chapter. Further, GemStone has a gateway to Sybase, which allows us to retrieve the stored patient demographics at the UCLA medical center.