Image processing

We use the term image processing to mean enhancements applied to pixel displays of gridded data, primarily to

emphasise particular features within in the data and create a more easily interpretable product. It is distinct from data enhancement where the emphasis is on numerical process- ing to highlight particular characteristics of the data or to improve the signal-to-noise ratio. Our description applies equally to the voxel displays used to present data volumes. At its most fundamental level, digital image processing involves the representation of each grid (data) value as a coloured pixel. The resultant image is known as a pseudo- colour display, because the colours are not the true colours of the parameter displayed. The physical size of each pixel and the number of the pixels in the display depends on the hardware used to display the image. Depending on the resolution of the display device and the amount of data to be displayed, each node in the data grid is assigned to a pixel in the display. The amplitude of the data value is used to control the colour of the pixel. This raises two funda- mental issues: how the display colours are selected, and how these colours are assigned to the (grid) data values. 2.8.2.1 RGB colour model

The number of different colours available, and therefore the ability to accurately represent the amplitude variations of the data, depends on both hardware and software considerations. Digital display devices use the primary colours of light: red (R), green (G) and blue (B) (Fig. 2.31a), which can be mixed to create a pixel of any colour. For example, mixing equal amounts of two of the primary colours creates the secondary colours yellow, cyan and magenta. Every colour is represented by its location in the RGB colour space which can be visualised as a Cartesian coordinate system (Fig 2.31b). In Fig. 2.31b, black occurs at the origin (R¼0,

G¼0, B¼0), with shades of grey and white plotting on the ‘grey’ or ‘intensity axis’, along which all three primary colours are mixed in equal amounts (R¼G¼B). White is produced by mixing the maximum amounts of the three primaries.

Typically an eight-bit hardware architecture is used to represent the amount of each primary colour in a three- colour RGB display, so each primary colour has 28(256) discrete intensity levels (0 to 255) and each pixel carries 24 (3  8) bits. By mixing the three colours, up to 2563 (16,777,216 ¼ 224) different colours can be specified, referred to as 24-bit colour. Several techniques are used for assigning colours to pixels.

Hue–saturation–intensity

Colours in the RGB colour space can also be defined in terms of hue (H), saturation (S) and intensity (I)

(Fig. 2.31a). The HSI model is based on the polar coord- inate system whose origin is that of the RGB colour space. Colours are represented by hue and saturation, which are defined in terms of a colour wheel centred on the intensity axis of the RGB colour model. Hue varies around the circumference with the primary and second- ary colours occurring at equally spaced intervals

(Fig. 2.31c). Saturation is the radial distance from the

axis, so lines of equal saturation form concentric circles about the intensity axis. Zero saturation occurs on the intensity axis. Hue can be thought of as the dominant colour itself – orange, red, purple etc. – whilst saturation represents the ‘whiteness’ of the colour. For example, a pure red colour has high saturation, but pastel shades or lavender or pink have a lower saturation. Moving the colour wheel down the axis reduces intensity, so the colours darken but are otherwise unaltered, whilst moving up the axis produces brighter colours. Note that although a pixel’s colour may be defined in terms of HSI, it has to be converted to the equivalent coordinates in the RGB space for display, since RGB is the basis on which the hardware creates the colours.

2.8.2.2 Look-up tables

The human eye can perceive many thousands of different colours; however, the enormous number of colours avail- able in the 24-bit RGB colour model is unnecessary in practice as the human eye is unable to resolve them all. Instead, this can be reduced to 256 selected colours, as there is normally only minor visible difference between an image displayed using the full 24-bit colour spectrum and one using 256 colours selected from across the full spectrum. This also significantly reduces the computa- tional resources needed to produce the image. The 256 colours together constitute a colour map or look-up table (LUT).

The LUT is simply a table containing the colours defined by their coordinates in the RGB colour space, with each colour being identified by its position in the table (0 to 255). Numerous LUTs are in common use, with many image processing systems allowing the user to design their own. A simple example is a grey-scale look-up table, where the colours vary from black to white via increasingly lighter shades of grey (Fig. 2.32a). In terms of the RGB colour space, the 256 grey-scale values all lie on the intensity axis. c) b) a) Green (Cyan) Colour wheel (Green) (Red) (Magenta) (Yellow) (White) (0,255,255) (255,255,255) (0,255,0) (255,255,0) Blue Red Green Green Yellow Cyan

Red Magenta Blue

White Red (255,0,0) (255,0,255) (0,0,0) Blue Intensity (grey, R=G=B) (Blue) (0,0,255) Saturation Hue (Black) Grey Hue Saturation Green C ya_n B l ue M ag en ta R e d Y ello w

Figure 2.31 Hue, saturation and intensity. (a) The primary and secondary colours of light. (b) The red–green–blue (RGB) and hue–saturation–

intensity (HSI) colour models. Redrawn, with permission, from Drury (1987). (c) Colour wheel showing changes in tone across a circle

whose axis is the intensity axis. Hue changes around the circumference, saturation varies from pure spectral colour at the circumference through pastel colours to grey at the centre, and intensity changes along the central axis.

a) e) b) c) f) g) d) h) 2 Kilometres 0 High Low TMI High Low TMI ‘Sun’ ‘Sun’ High

Low TMI ‘Sun’

‘Sun’

High

Low TMI

Figure 2.32Images illustrating the use of different look-up tables applied to the same dataset, and the improvements in resolution achievable with shaded relief. (a) Grey-scale, (b)–(d) various rainbow-style displays, (e) to (h) shaded-relief grey-scale displays illuminated from different directions. Data are aeromagnetic data from the vicinity of the Mount Polley alkalic Cu–Au deposit, British Columbia, Canada. Pit outlines are

shown. TMI– total magnetic intensity. Data are reproduced with the permission of the Minister of Public Works and Government Services

Canada, 2006, and courtesy of Natural Resources Canada, Geological Survey of Canada.

The human eye can only differentiate about 30 shades of grey, so if more are used in the display the variation appears continuous. Grey-scale displays are useful for some types of data, but generally more detail is revealed when colour is used.

The most common and probably most effective LUTs comprise a rainbow-like spectrum of colours, i.e. the order of the colours in the visible spectrum, although not all are necessarily included. Images created with three different spectrum-based LUTs are shown in Figs. 2.32b, c andd. Lower values in the LUT are assigned to purples and dark blues; intermediate values assigned to shades of lighter blue, green and yellow; and higher values assigned to oranges, reds and finally white. This kind of colour map associates cold colours such as blues with low data values, and warm colours such as oranges and reds with high data values. When displaying data where polarity are important, such as seismic data, a colour map comprising two colours separated by a thin central band of white is very effective (seeFig. 6.12).

There is no firm basis for a particular colour scheme being superior to another; the choice really depends on the nature of the data being displayed, the nature of the inter- pretation and the interpreter’s personal preference (and colour vision). In general, schemes producing greater vari- ation in colour allow more subtle detail to be seen and are preferable for geological mapping. On the other hand, simpler schemes can be effective when the primarily aim is to identify anomalous responses which may constitute targets. It is worth noting that the human visual system is not equally adept at seeing variations in different colours (Welland et al., 2006). It is worst at recognising different shades of red and blue, so LUTs comprising predominantly these colours should be avoided.

Colour stretch

Once the LUT has been selected, the 256 colours are then assigned or mapped to variations in the data. An important tool for controlling this is a frequency histogram. The data arefirst assigned to one of 256 class intervals. Data values falling between the maximum and minimum values of the interval are assigned to the interval. Each interval has the same width, and this width is chosen such that the 256 intervals span the entire range of the data, or at least most of the range. To reduce the influence of outlier data values the class interval width may be scaled so as to extend across a fraction of the entire data range, 95% of the range for example. The data histogram shows the number of data

values assigned to each class interval (Fig. 2.33). The vari- ations in the data are now no longer represented by their true values; instead they are represented by the class inter- val in which they fall, 0 representing the lowest value class interval, 255 the highest. The 256 data classes are mapped to the 256 colours comprising the colour map using a stretch function, i.e. the available colours are ‘stretched’ across the data histogram.

A linear colour stretch across the full range of 256 data classes maps each class to its corresponding colour value in the LUT, with the lowest data class (0) mapping to the lowest colour value (0) through to the highest data value (255) mapping to the highest colour value. In other words, there is a linear relationship between the data classes and the colour values. When the data histogram shows that the majority of the data fall within a comparatively small number of classes, which is often the case for geophysical data, a large proportion of the data are assigned to just a few of the display colours. The result is an image which is composed predominantly of just a few colours. In this case, a few extreme data values, the outliers, are exerting a disproportionate influence on the image producing a dis- play which prevents variations across the full range of the data from being recognised. In other words, the data values do not make full use of the available colour values, so the display is said to lack contrast. This is not a problem when anomalous readings are the principal interest, as is the case for anomaly detection, but for more general interpretations of the whole data grid, e.g. for mapping, a different stretch function is required.

A simple solution for improving the contrast is to redefine the linear stretch so that it only extends over that part of the data range where most values occur.

Figure 2.33a shows a linear function chosen to span the

main range of variation in the data. In this case, an arbitrarily chosen data value (G) maps to colour value 209, a mid-range red. The few low-valued data points below the lower limit of the linear stretch all map to the lowest colour value (0) and, similarly, high-valued points beyond the upper limit all map to the highest colour value (255).

It is useful to illustrate the effects of the stretch using a display histogram, which is a frequency histogram of the colour values. It can be plotted next to the data histogram. Both have 256 class intervals; in the data histogram they are populated by the data values and in the display histo- gram by the colour values. For the case of the localised linear stretch shown inFig. 2.33a, the display histogram is

broadened relative to the data histogram. This means there is a greater range of colours comprising the image.

Alternatively, the histogram equalisation or histogram linearisation stretch adjusts the size of the data class inter- vals so that the display histogram has approximately equal number of data points in each class, i.e. the data points are more uniformly distributed across the classes (Fig. 2.33b). The flattened display histogram shows that the image

contains approximately equal numbers of pixels of each colour. Value G now maps to display value 249, a pale red. The histogram normalisation stretch adjusts the size of the data class intervals so that the display histogram resembles a normal (Gaussian) distribution (Fig. 2.33c). This increases the number of colours assigned to the central part of the data range, where usually most variation occurs. Value G now maps to colour value 195, a shade of orange. a) b) c) 2 Kilometres 0 Stretch function 209 Data histogram Linear Class values Histogram equalised 249 Class values 0 High Low TMI High Low TMI High Low TMI Histogram normalised 255 Look-up table Display histogram Display histogram Display histogram Data histogram Data histogram G 195 Colour values 0 255 Look-up table Colour values 0 255 Look-up table Colour values Min Max G Min Max Class values G Min Max

Figure 2.33Images of the data shown inFig. 2.32created using the same look-up table, but with different colour stretches. (a) Linear stretch, (b) histogram equalised, (c) histogram normalised (Gaussian stretch). The frequency histogram of the data and the display histogram are

shown for each image. Note how class value‘G’ is mapped to different colour values with the various stretch functions. Data are aeromagnetic

data from the vicinity of the Mount Polley alkalic Cu–Au deposit, British Columbia, Canada. Pit outlines are shown. Data are reproduced with the permission of the Minister of Public Works and Government Services Canada, 2006, and courtesy of Natural Resources Canada, Geological Survey of Canada.

These three colour stretches are commonly used for imaging geophysical data, although most image processing systems allow the user to modify the stretch function in a variety of ways. Note that the data values can also be adjusted during data processing. For example, using the logarithm of the data values reduces the influence of high amplitude regions and raising the data to a power, such as squaring them, increases higher values relative to lower values (see Amplitude scaling inSection 2.7.4.4).

2.8.2.3 Shaded relief

Short-wavelength variations within an image can be high- lighted by applying a visual enhancement known as shaded relief, also called sun shading, hill shading and artificial illu- mination.Figure 2.32illustrates the spectacular improvements obtainable with shaded-relief enhancements, shown in (e) to (h), compared with the unshaded grey-scale display in (a).

The physical process mimicked by the shaded-relief enhancement is sunlight illuminating topography simulated from the grid values, i.e. higher values form the hills and lower values form the valleys. Inclined surfaces facing the ‘sun’ are illuminated more than surfaces oblique to the illumination direction. This is determined by both the dip and strike of the surface, and means that the display has a directional bias and acts as a form of gradient enhancement. The enhancement is effective because human visual

perception of shapes and objects relies heavily on resolving areas of light and shadow (Ramachandran,1988).

The reflectivity of a surface in the shaded relief enhance- ment can be quantified. The most common type is Lam- bertian reflectivity, where the incident light is reflected, and equally in all directions. The reflectivity depends on the orientation of each part of the‘topographic’ surface relative to the position of the sun. Since the surface’s orientation depends on the relative values of the neighbouring pixels, the calculated value does not depend on individual pixels. The calculated value is used to assign a shade of grey to each pixel (grey-scale displays should always be used for shaded-relief displays). A popular type of shaded relief is the‘wet-look’. It reduces the contrast between mid-range values and emphasises the brightness of highly illuminated areas, making the illuminated surface appear as if it were ‘wet’ or ‘glossy’. As shown inFig. 2.36b, it is particularly effective in highlighting subtle detail when used as part of a composite display (seeSection 2.8.2.4).

Shaded relief is a very useful image enhancement, but it does have several associated drawbacks. What is perceived as positive and negative‘topography’ can vary from person to person, and also whether the illumination is from the ‘top’ or the ‘bottom’ of the image as displayed on the page; compare Fig. 2.32hwith Figs. 2.32e, f and g. The shaded relief image is also subtly distorted because the illumination

2 Kilometres 0 b) a) ‘Sun’ ‘Sun’

Figure 2.34 Illustration of the change in apparent cross-cutting relationships between anomalies for shaded relief displays with different illumination

directions. This is a composite image combining a pseudocolour display of amplitude with grey-scale shaded relief (seeSection 2.8.2.3). Image

position determines where topographic peaks and troughs are perceived to occur. It also has a strong orientation bias, with features oriented at a high angle to the direction of illumination being enhanced more than those trending towards the illumination direction, i.e. shaded relief is a powerful directionalfilter. Note the suppression of northerly trending features inFig. 2.32f, which is illuminated from the north. The directional bias of shaded relief means it is important to vary the illumination azimuth when analysing images. It is also important to alter the elevation of the‘sun’. With the ‘sun’ ‘lower in the sky’, shadows are more pro- nounced and more detail is evident, but the directional bias is increased. When the‘sun’ is ‘too high in the sky’, subtle detail may not be evident. An elevation from about 25° to 45° usually produces good results. Figure 2.34 shows a geologically important consequence of the directional bias associated with shaded relief displays. The apparent cross- cutting relation, i.e. relative age, of the source of the inter- secting linear anomalies changes with illumination direc- tion, with anomalies striking perpendicular to the illumination direction appearing to cross-cut those that are more parallel. This is an example of how geophysical responses must be treated differently from geological data.

Most image processing systems allow the position of the ‘sun’ to be adjusted in real time, with the display changing accordingly. This allows the optimum illuminations to be identified, although it is stressed that at least two (approxi- mately) perpendicular illuminations of the data should always be analysed and that different directions may be required for different parts of a large dataset.

2.8.2.4 Composite displays

It is often convenient, and sometimes necessary, to display several datasets together, allowing an integrated interpret- ation. Also, different presentations of the same dataset, selected to emphasis different characteristics of the data, can be combined. This can be achieved by integrating the datasets in colour space to produce a single composite image. For the RGB model, the most common applications are the display of multichannel data such as radiometrics, electromagnetics and the different spectral bands in remote sensing data. Up to three datasets can be displayed; one dataset (or channel) is displayed as 256 variations in red, the second in green, and the third in blue. This is known as a ternary image (Fig. 2.35) and it makes direct use of the a) b) High Low K c) d) High Low High Low