as it incorporates achromatic colours additional information is preserved (e. g. red → brown in the dark, → pink in the highlights). The identied colour name was joined in a bag of words scheme with other information (here: some shape information) to identify certain categories of objects in a scene.
This colour name approach was very successfully tested against a simple colour seg- mentation identifying soccer teams or owers in scenes. For these purposes of classifying limited, colour-coded items in scenes it worked very robustly. But it does not allow to deduce information on the illumination, surface reectance or object chromaticity.
3.6.4 Genetic Programming Based Approaches
Ebner has taken the approach of building a neural architecture. He arranged an array of processing elements for each pixel of the image. Each processing element has got access to the full pixel values of its own pixel and the four neighbours' luma values (similar to neural connections between spatially arranged colour processing cells in the visual cortex). A set of operation functions was given, and through genetic programming a suitable algorithm for computing a colour corrected image evolved [1,40].
Due to the extreme long processing times for the genetic programming (10 days per run) and due to the fact that six out of ten times the genetic algorithm was stuck in a local optimum the approach does not seem very promising. However, it did show that genetic evolution based on biologically inspired boundary conditions can come up with suitable solutions for the given problem. Upon analysis of the evolved best candidate algorithm, it also showed (due to colour channel independence) processing similarities with the Retinex theory, which averages data from neighbouring elements.
3.7 Adaptation Method Comparison
The general problem of usability of Fairchild's spectral approach with general image cap- turing systems is obvious, as the spectral colour information is lost, and therefore it is only applicable for very specic and costly systems. Further limitations due to required precision of a scene segmentation apply as well.
The computational colour constancy methods as described in the summary of Barnard's Ph. D. thesis [2] and the more general approaches in Ebner's book based on his habilitaton [1] depend on a static analysis of the current scene to deduce illuminant information. It does not consider additionally information available in live image analysis.
Histogram based inter-camera calibration as performed by Porikli [29] demands sin- gle initial calibration of distinct device pairings. The number of possible distinct pairings explodes exponentially with the number of available devices. The approach only matches
information between the devices, but does not allow to deduce information on estimated chromaticities. The mappings for each channel are very exible, but as these functions only operate on one channel at a time, no shearing, warping or rotation of the colour space is possible. Colour management based colour corrections allow for similar types of correction possibilities while still allowing for more complex colour space distortions.
Many methods are known to be used that are based on a priori knowledge of scene properties, like distinct colours of object markings (e. g. the ball colour, team markings, goal marks, etc. in robot soccer), colour names, etc. Use of these is denitely helpful, but their presence cannot be expected in a scene at all times under more general conditions.
Generally, the idea of learning colour constancy sounds promising towards a more dy- namic and exible solution. Either through statistical approaches or by means of Articial Intelligence. Earlier attempts from the beginning of the 1990s were partially based on biologically-inspired implementations. Though these yielded good results, the theory on which they were based is more derived from guesswork of understanding the processes in the human visual cortex, and lacking true certainty. The approach taken by the implementation, however, is validly usable. Even though neural networks, as used in [9,35,36], are learning how to adapt colour, they do so only during a training phase. So they are still not able to adapt at run time to changing conditions.
Probabilistic methods [37,39] as outlined are not used in an adaptive way. Their models may be updated statistically more easily, though, than a neural network can be re-trained. Genetic programming promises to be the ultimate key to adaptive exibility. But it is by far impracticable as its evolution consumes too much computational time to be useful even in a decoupled side-line process.
Common approaches based on colour management (see Sect. 3.1.1 and Chap. 5) yield comparable results as some of the above mentioned ones. Furthermore, they operate on a prole connection space of a colour encoding that is very suitable for further colour analysis. However, it demands an initial, individual and static characterisation of every device within a process. This step must be performed for every encountered condition, or upon every change into an unknown or new condition.
As all algorithms outlined in this review have been implemented with dierent use cases in mind, it is rather dicult to compare these directly. Researchers use a variety of dierent colour spaces for their computation. Reading the publications, it is obvious that many researchers are not even aware of certain problems with the used colour spaces. These can be for example a severe non-linearity (e. g. gamma corrected RGB), visual non-linearity, device dependence, etc. We can expect, that several of the approaches could perform better than stated in the publications, if a more suitable colour encoding would be used, and if adaptations to e. g. discretisations would be made accordingly. Especially as almost all algorithms make use of colour dierencing for discretisation, thresholding or quality metrics, so potential improvements are possible.
3.7. ADAPTATION METHOD COMPARISON 39 Almost all the dierent methods of handling colour constancy outlined or referenced here provide items in the researcher's toolbox that may contribute valuable functions in a novel tool-chain towards solving the problem of Dynamic Chromatic Adaptation. There- fore, it has been chosen to explore a hybrid approach, that is based on the robustness of colour management with its device-independent, linearised prole connection colour space, which in corporates learning of current conditions based on input from the analytic methods described.
Chapter 4
Colour Constancy with
L∗a∗b∗
Colour constancy algorithms aim at correcting colour towards a correct perception within scenes. To achieve this goal they estimate a white point (the illuminant's colour), and correct the scene for its inuence. In contrast, colour management performs colour transformations on input images according to a pre-established input prole (ICC prole) for the given constellation of input device (camera) and conditions (illumination situation). The latter case presents a much more analytic approach (it is not based on an estimation), and is based on solid colour science and current industry best practises. Unfortunately, it is rather inexible towards cases with altered conditions or capturing devices. The idea as outlined in this chapter is to take up colour correction on visually linearised and device-independent CIE colour spaces as used in colour management, and to try to apply them in the eld of colour constancy. For this purpose, two of the most well known colour constancy algorithms White Patch Retinex and Grey World Assumption have been ported to operate on colours in the CIE LAB colour space. Barnard's popular set of imagery for testing colour constancy was used with the original implementations as a reference as well as the modied algorithms. The results appear to be promising, but they also revealed strengths and weaknesses.
4.1 Introduction
Colour constancy refers to the everyday perception, that the colours of objects remain unchanged across signicant changes in illumination colour and luminance level [20]. Colour constancy is served by the mechanisms of chromatic adaptation and memory colour. The study of colour appearance and the derivation of colour appearance models are aiming to quantify and predict colour constant perception. Such studies generally take place in the arena of computational colour constancy.
In digital imaging systems, colour management is the controlled conversion between the colour representations of various devices, such as image scanners, digital cameras, monitors,
TV screens, lm printers, computer printers, oset presses and corresponding media. The primary goal of colour management is to obtain a good match across colour devices; for example, a video which should appear the same colour on a computer LCD monitor, a plasma TV screen, and on a printed frame of video. Colour management helps to achieve the same appearance on all of these devices, provided the devices are capable of delivering the needed colour intensities [21].
RGB (and other) spaces model the output of physical devices rather than human vi- sual perception. Due to their immediate availability, most colour constancy research is conducted using these device-dependent colour spaces. Colour constancy algorithms derive information based on the colour distribution in the RGB colour space, then shift and stretch colour along the colour primaries of this colour space. Dierences in colour are quantied in terms of Euclidean distances in these non-linear spaces. Therefore, they may not be opti- mal for quantitative colour comparisons. Visually meaningful colour dierences are usually expressed as a∆Evalue derived from CIE LAB, and are considered to be current good prac- tise [20] (based on theL∗,a∗andb∗coordinates of the CIE LAB colour space, see Sect.2.3.4). CIE LAB aspires to perceptual uniformity, and its L∗ component closely matches human perception of lightness. It can thus be used to make accurate colour balance corrections by modifying output curves in the a∗ and b∗ components, or to adjust the lightness contrast using theL∗ component. It seems like a good idea to apply colour constancy algorithms on a device-independent and linearised colour space as CIE LAB for an exemplary evaluation. In colour management, the adaptation from one illuminant to another is performed by von Kries type transformations. Commonly accepted now as being the best candidate for these so called chromatic adaptation transformations (CATs) is the Bradford transforma- tion [22,23]. Additionally to the direct transformation on theL∗a∗b∗colour coordinates, we will be evaluating a Bradford CAT.
The work described in this chapter is trying to evaluate the possibility of implementing variations of colour constancy algorithms in opponent colour spaces particularly in the device-independent, visually linearised CIE LAB colour space. It is not the goal to nd a competitor for the Best Colour Constancy Algorithm Award.
The following section will introduce colour constancy algorithms in general, and speci- cally the ones used here for comparison. In Sect.4.3these algorithms are modied towards being useful in an opponent colour space (CIE LAB). Sect.4.4explains some details towards the implementation and evaluation, with nally Sect.4.5presenting comparative experimen- tal results between the dierent implementations.