Complex and variable natural scenes: the example of medical

imaging

Heuristics and the familiarity of experts had long been the basis for interpreting medical im- ages as well. These, like planetary images, display complex natural scenes, prone to noise, acquisition limitations, and significant variability in the appearance of features. Unlike for planetary imagery, much work has been done in the field of automated processing of medical images, driven by the much wider use of medical imaging, the significant costs associated with assigning physicians and radiologists to visually interpret images, and the potential for hu- man error. Decades of research have led to very succesful automated interpretation algorithms for autonomous segmentation and classification of complex images with noise, poorly-defined

boundaries between regions, and other challenges to automated interpretation. Planetary scenes have been much less studied for the purposes of automated interpretation, but their interpreta- tion faces many of the same difficulties, and techniques which have shown success in medical imaging may well provide guidance to the interpretation of geological and atmospheric scenes. Medical images concern the interior of the human body, which presents a complex collec- tion of tissues having variable appearance and irregular boundaries. Regardless of the modal- ity used – x-ray tomography (XRT), nuclear magnetic resonance (NMR), positron emission tomography (PET), or other – the problem is usually one of segmentation and classification, with the goal of identifying tissues to find tumours, inflammation, or tissue features indicat- ing trauma or disease. Such an effort represents the segmentation and classification of natural scenes having a highly complex and variable appearance and very large parameter space [3]. As in imagery of geological scenes, medical images are also affected by variations in acqui- sition, leading to differences in speckle, contrast, brightness, and missing boundaries [4]. In geological imagery, similar difficulties are presented by changes in lighting, weathering and dust covering on rocks, vegetation (on Earth), and instrument differences.

Meeting or mitigating these challenges to achieve useful automated image processing is a difficult task. In the medical field, ultrasound (US) is a particularly challenging modality, due to the high incidence of speckle, poor signal-to-noise ratio, low contrast resolution, and frequent discontinuity of boundaries [5]. Patient motion during imaging adds to the difficulty in localizing boundaries [6], and is nearly inevitable in imaging of cardiac, pulmonary, and other tissues. Such challenges have made direct analytical image analysis techniques difficult, and led to the use of statistical and pattern-recognition techniques in US image processing, of which [7] give a comprehensive review.

The challenges are similar to those for planetary scenes, where contrast and noise are lim- ited by lighting conditions that cannot be controlled, and where boundaries can be obscured for a variety of reasons. Contacts – the boundaries between geological units – are often gradational, rather than sharp. Processes in impact crater formation and elsewhere can cause brecciation

– fragmentation, mixing, and re-lithification of materials – leading to fragmentary and broken boundaries between rock types, while the edges of pre-existing geological units can be mod- ified by the heat and pressure of new molten materials solidifying adjacent to them. Surface weathering, the overlay of dust, and erosion of the rock surface contribute to the noise in the image – such effects reduce the strength of the visual signal corresponsing to the rock itself, mixing it with other components. Atmospheric imaging faces similar problems, particularly in the dusty atmosphere of Mars, where the dust contributes to obscuration and effective noise, and where the clouds are of poorly-predictable, inconsistent, and dynamic morphology. Here, segmentation of individual clouds from the dust and the background sky, and classification of cloud types, will face similar challenges of noise, contrast, variability of appearance, and boundary ambiguity in geological and US imagery.

A great variety of techniques have been used for segmenting US images. Artificial neu- ral networks (ANN) are very common, performing well at segmenting images of a variety of tissues, including in the heart [8], prostate [9], bloodstream [10], brain stem [11], liver [12] and elsewhere. Techniques using shape, physical, and other priors have also met with success, and may in some cases have analogies in geology. Knowledge about the expected clast shape in sedimentary rocks or breccias, or about the orientation of sedimentary beds, for example, could be used as prior information to inform a classification algorithm dealing with these kinds of materials. Similarly, models of cloud type and convective dynamics inform classification algorithms for atmospheric features. Energy-minimization methods are also used in US image analysis, as are Bayesian, level set, and active contour techniques, with [7] giving a compre- hensive overview of the application of each.

Given the difficulties in managing the complex feature space of geological scenes, and the difficulty in reverse-engineering the visual heuristics of an experienced field geologist, such techniques may find similar utility in geological image interpretation as well, or, indeed, in many types of scenes to be found in planetary exploration applications.