Is Grey Perceived Differently?

The ternary isotrions differ from their binary counterparts by the presence of a third level, grey. Therefore, a natural questions is whether the grey token of ternary textures has special properties with regard to texture perception. i.e: according to the current literature on lightness/brightness perception, is grey perceived differently to black and white?

According to the standard terminology, lightness refers to the apparent reflectance of a surface within a visual scene, whereas brightness refers to the apparent luminance of a patch within the image itself (Evans and Bartley,

75

1948, Kingdom, 2011). Lightness constancy in natural scenes is generally good. Although we rarely experience lightness errors, as pointed out by Kingdom (Kingdom, 2011) "…the study of lightness perception…has been dominated by an exhaustive examination of its errors". Such errors, elicited by synthetic stimuli, provide clues as to how the visual system functions (Kingdom, 2011).

It is now well established that the perceived brightness of a region of visual space is not solely related to that regions' luminance; it also depends on the luminance of adjacent regions. This phenomenon is known as brightness induction and includes both brightness contrast and assimilation effects. Brightness contrast effects occur when the brightness of a test region shifts away from the brightness of adjacent regions. The canonical example of this is the simultaneous brightness contrast illusion (SBC), first reported by Chevreul (Chevreul, 1855). In this illusion, a grey patch on a white background looks darker than a grey patch of equal luminance on a black background (Figure 19) (Heinemann, 1955, Williams et al., 1998).

76

Figure 19. An example of the simultaneous brightness contrast illusion (SBC). Although they appear to differ in brightness, in reality the luminance of the two grey patches is equal (Purves et al., 2001).

Another influential class of lightness error is brightness assimilation. Brightness assimilation is the opposite of brightness contrast, in that lightness appears to shift towards that of the surround. Thus, a grey patch on a black background appears darker than a grey patch of equal luminance on a white background. One of the most intensely researched brightness assimilation illusions is White’s Effect (White, 1979). In White’s Effect, the two sets of grey bars have the same luminance, but differ markedly in their perceived brightness, which shifts towards that of the flanking bars (Figure 20) (Anderson, 2003), seemingly opposite to the result observed in Figure 19.

Figure 20. An example of White’s Effect, a brightness assimilation illusion. The perceived brightness of the two grey bars appears to shift

77

towards that of the flanking bars; the ground truth is that both bars are equiluminant (Anderson, 2003).

First described by Cornsweet, a third lightness illusion is the Craik-O’Brien- Cornsweet (CCOB) (Cornsweet, 1970). In this case, illusory brightness values are afforded to regions based on the perception of edges (Figure 21) (Purves et al., 1999). Note how the region to the right of the CCOB edge looks slightly lighter than the region to its left. The ground truth is that the brightness of both areas is the same. Thus, the CCOB is a brightness induction phenomenon in which the central edge of an opposing pair of luminance gradients makes adjoining equiluminant regions appear dissimilar (Cornsweet, 1970).

The CCOB effect has traditionally been interpreted as evidence for the filling- in of lightness information (Kingdom, 2011). Filling-in describes a process that begins with edge detection and is followed by a propagation of neural activity which fills in the intervening regions. According to this model, brightness values are determined by what happens at the edges of image elements (Cohen and Grossberg, 1984).

Examples of brightness induction and brightness assimilation demonstrate the profound influence that context plays on brightness perception. Research into their neurophysiological basis has uncovered details regarding how the visual system functions.

78

Figure 21. An example of the Craik-O’Brien-Cornsweet effect (CCOB) (Cornsweet, 1970). In this brightness induction phenomenon, the regions to the left and to the right of the CCOB edge have equal luminance, but they appear to be different (Purves et al., 1999).

The prevailing wisdom was that lightness was encoded in the retina. For example, Cornsweet championed the idea that SBC resulted from reciprocal interactions between retinal neurons (Cornsweet, 1970). By this account, SBC is caused by lateral inhibition by neighbouring receptive fields at the contrast boundaries; this has the secondary effect of enhancing edge detection (Purves et al., 2001, Sekular and Blake, 1994, Coren, 2003). Therefore, any target surrounded by an area of higher luminance should be perceived as darker than the ground truth, and vice-versa (Cornsweet, 1970).

However, according to more modern accounts, lightness and brightness perception is a multi-stage process involving both retina and cortex. Whereas

79

the retina normalizes luminance variations and encodes rudimentary contrast information, cortical neurons in the ventral stream explicitly signal lightness and brightness. In Section 1.5.4 below, we discuss various studies involving gain control mechanisms (Graham and Sutter, 2000, Heeger, 1992b, Maddess et al., 1988, Shapley and Enroth-Cugell, 1984).

The illusions discussed above can be explained by models based on low- level spatial filtering (Dakin and Bex, 2003, Blakeslee and McCourt, 2008, Blakeslee and McCourt, 2004, Blakeslee and McCourt, 2012). One such model has been described in a series of papers by Blakeslee and McCourt. Their model, called the Oriented Difference of Gaussians (ODOG), can explain most brightness illusions (Blakeslee and McCourt, 2008, Blakeslee and McCourt, 2004, Blakeslee and McCourt, 2012). The core idea of the ODOG model is as follows: although the model cells possess filters tuned to very low spatial frequencies, their range is necessarily finite so they need to incorporate a gain control mechanism. This means that, in the case of artificial images whose low spatial frequency content has been greatly reduced, the model cells increase their gain and thereby restore low spatial frequency content to physiologically expected, prior values; this has the side- effect of inducing lightness effects.

Note that in these high pass filtered images, the residual low frequency information is located near high contrast borders. Like cortical cells, the units in ODOG are orientation specific (Figure 22) so the gain changes can also be orientation specific. This can explain illusions like White’s effect (White, 1979, Blakeslee and McCourt, 2004). Consequently, reconstituted images lack some of the low spatial frequency information which is present in the

80

ground truth image. The loss of these frequencies results in brightness induction (Blakeslee and McCourt, 2008, Blakeslee and McCourt, 2004, Blakeslee and McCourt, 2012).

Blakeslee and McCourt posit that the most responsive filters to White’s Effect stimuli are appropriately tuned high spatial frequency, vertically-oriented filters. Contrast normalization attenuates the response of these vertical filters, relative to those tuned to horizontal orientations. In response to White's Effect stimuli, horizontally-oriented filters pool the luminance of the flanking bars with those of the test patches. However, because the responses of the vertical filters have been disproportionately attenuated, the relative contribution of the flanking bars is enhanced. This results in the brightness assimilation illusion observed in White's Effect (Figures 20 and 22) (Blakeslee and McCourt, 2008, Blakeslee and McCourt, 2004). A closely related model has been described by Dakin and Bex’s (Dakin and Bex, 2003), wherein the contrast normalization stage equates filter responses across spatial-frequency, not orientation.

81

Figure 22. ODOG model as applied to White’s Effect. (a) filter profile; (b) filters of different spatial scales and orientations are summed at a given orientation; (c) filter gains as a function of centre spatial frequency; (d) orientations of combined filters; (e) White’s Effect stimuli; (f) the result of convolving each stimulus with d; (g) contrast normalization equates root mean squares of filter outputs; (h) outputs are summed across filter orientation (Blakeslee and McCourt, 2004).

Taken together, these studies indicate that grey regions are perceived differently in that they frequently describe broad regions of no structure, while

82

high contrasts near borders of images define structure. Thus, we may expect ternary textures on trico axes with a grey bias to produce different performance functions than those for the more traditional black and white contrasts. To put it another way, in the human colour space, grey represents the origin and not one of the cardinal colour and luminance sensory axes. Therefore, grey may be something of an invalid token for colouring a texture and not comparable to tokens selected from the principle sensory axes. These are questions we are interested in answering and they will be pursued in more depth in Chapter 4 in the context of ternary texture discrimination.