12.2 Future Work
In the initial assumptions, we have envisioned to use a background/foreground segmentation to identify segments that are static in the scene composition. For the domain orf robot soccer, we have experienced that it is more suitable to track foreground objects, as these can be ideally identied and mapped to known colours of the initial conditions. The long term observations rst used an implementation of a foreground/background segmentation. The implementation adapted on time scales that proved to be inappropriate for observations over hours. Instead, a probabilistic approach was taken, discarding outlier measurements from rectangular scene sections that hinder a good regression tting. Therefore, the concept of a scene segmentation has been preserved. However, in some scenarios the approach of a background/foreground scene segmentation can still be very suitable. Future work should include this type of scene segmentation.
Another very promising concept to evaluate is a segmentation of the scene based on distinct object surfaces (of the background). This would reveal trackable colour samples with much higher chroma (saturation), and therefore lead to less clustered samples. That should strongly improve the quality of corrective transformations.
Also, fall back strategies in the case of the failure of assumptions have been mentioned. An implementation of heuristics to detect the failure of these assumptions, along with the fallback strategies, would likely improve the robustness of the overall colour correction sys- tem.
Many colour correction systems utilise Articial Intelligence (AI). Some of these tech- niques are briey mentioned in Chap.3. AI as a means for deriving models of colour space distortion, distortion parameters, etc. could potentially open further possibilities. Particu- larly the learning of responses for quicker adaptation could be helpful, if certain illumi- nation changes are predominant and reoccurring. The system could react to such changes much more quickly by reusing pre-tinned responses.
Lastly, the adaptation of the ICC prole and the application of the prole are already independent from each other. In Chap. 6 we have outlined an envisioned system, in which these two are decoupled into distinct processes. It should be rather trivial to implement a system, in which these two processes are communicating by means of a service oriented architecture. The prole adaptation which is not time critical could then be easily deployed on a dedicated machine, to reserve the computation cycles for the payload of running a dedicated application operating on the colour corrected imagery.
This is, of course, an incomplete list of future enhancements. Other potential improve- ments have been mentioned in the dierent chapters. This list, however, can be seen as a summary of the most obvious and pressing topics for future research and implementations based on the research presented.
Glossary
3D three dimensional
CAT chromatic adaptation transformation, the con-
version of a colour tristimulus for a dierent il- luminant
CCD charge-coupled device, sensor type used in digi- tal cameras
CIE International Commission on Illumination
(Commission Internationale de L'Éclairage)
CIE CAM02 CIE Colour Appearance Model (2002), cor-
relates for the six technically-dened dimen- sions of colour appearance: brightness (lumi- nance), lightness, colourfulness, chroma, satu- ration, and hue
CIE LAB see CIEL∗a∗b∗
CIE L∗a∗b∗ CIE 1976L∗a∗b∗is a device-independent colour space, shortly also called CIE LAB; based di- rectly on non-linearly compressed CIE 1931 XYZ colour space coordinates
CIE LCH see CIEL∗C∗h∗
ab
CIE L∗C∗h∗ab Polar description of the CIE L∗a∗b∗ colour space. L∗ is the luminance (lightness), C∗ the chroma (saturation) andh∗ab the hue.
CIE LUV see CIEL∗u∗v∗
CIE L∗u∗v∗ CIE 1976L∗u∗v∗is a device-independent colour space, shortly also called CIE LUV; based di- rectly on non-linearly compressed CIE 1931 XYZ colour space coordinates
clipping limiting colour values to the minimum or max- imum intensity for a channel if a colour falls outside the representable interval of a colour en- coding
CLUT Colour Lookup Table (multidimensional)
CMM Colour Management Modules, an implementa-
tion for ICC compatible colour transformation computations
CMOS complementary metal-oxide-semiconductor,
sensor type used in digital cameras
CMS Colour Management Systems
CMYK CMYK (cyan, magenta, yellow, black) colour
space; used in subtractive primary colour sys- tems e. g. for printing
CRT cathode ray tube, a displaying concept used for
computer screens and TVs
D50 CIE Standard Illuminant, corresponds to a
colour temperature of 5003 K
D65 CIE Standard Illuminant, corresponds to a
colour temperature of 6504 K
gamma correction A nonlinear operation used to code and decode colour channel values in colour space encoding systems. A gamma correction is, in the simplest cases, dened by a power-law expression
gamut The range of colours that a monitor, scanner,
printer, camera, etc. can detect and/or repro- duce.
HLS Hue, Lightness, Saturation colour space
HSV hue, saturation, value based colour space, an
opponent colour space
ICC International Color Consortium
ICC prole a prole that describes the colour attributes of a particular device or viewing requirement by dening a mapping between the source or tar- get colour space and a prole connection space; specied by the International Color Consortium
Glossary 167 L∗a∗b∗ see CIEL∗a∗b∗
LMS long, medium, short value based colour repre-
sentation in the characteristic of the three types of light sensitive cone cells of the human eye
LUT Lookup Table
L∗u∗v∗ see CIEL∗u∗v∗
PCS prole connection colour space, usually
CIE XYZ or CIE LAB
posterisation color quantisation artifact, also known as band- ing; may occur when the color depth, some- times called bit depth, is insucient to accu- rately sample a continuous gradation of colours
RAW image format for not rendered, raw sensor read-
ing of digital camera images
rg-chromaticity chromaticity description using the red and green channel, normalised through division by the sum or all (red, green, blue) colour channels
RGB RGB (Red, Green, Blue) colour space
SPD spectral power distribution
sRGB standardised RGB (Red, Green, Blue) colour
space
UML Unied Modeling Language, a standardised
general-purpose modeling language in the eld of software engineering
YUV colour space consisting of luma (Y) and two
chrominance components (U, V), mostly used in TV and video standards
Appendix A
Python/Native Code Integration
Throughout this thesis we are mentioning primarily the use of the Python programming language for specic implementations. These often make use of native code libraries, com- piled to binaries that are directly executable by the CPU of a computer. There are various reasons for using such libraries: Maybe because an existing library is to be reused without reimplementation; The execution speed could be critical and a higher performance than pure Python is required; Or the run time needs access to low level interfaces for which access is not provided by Python directly.One of the grand fundamentals in software engineering is to use the tools that are best suited for a job, and not to prematurely decide on an implementation. That is often easier said than done, in the light of some complimentary requirements (e. g. rapid/easy implemen- tation vs. needed speed of execution or vs. low level access to hardware). The traditional way [84] of binding native code to Python through extending or embedding is quite tedious and requires lots of manual coding in C.
For binding native code to Python by now a larger variety of tools and technologies are available. A categorisation of some of these approaches is given by Bagnall [85]. But we are also providing here a brief overview of a variety of binding tools. The eld has been limited to tools that are widely in use today within the community, and that are promising to be future proof as well as not overly complicated to use. These are the contestants with (very brief) notes for use cases that suit their particular strengths:
• Use Ctypes [81], if you want to wrap pure C code very easily.
• Use Boost.Python [86,87], if you want to create a more complete API for C++ that also reects the object oriented nature of your native code, including inheritance into Python, etc.
• Use cython [79], if you want to easily speed up and migrate code from Python to speedier native code (Mixing is possible!).
• Use SWIG [88], if you want to wrap your code against several dynamic languages. Kloss has given several talks on the wrapping of codes with Boost.Python1. That talk covered integrating the native code sensibly into Python by inheriting from C++ classes, overriding functionality and automatically generating the bindings using the Py++ code generator and the software build tool SCons.
In this given research we are dealing with the example of wrapping the Little CMS (LCMS) library [42,43]. LCMS is implemented in pure C, so for the ease of wrapping it, and being able to use the bindings for multiple platforms, we have decided to use Ctypes for this project.
A.1 Wrapping Little CMS with Ctypes
This section focuses on wrapping shared C libraries using Python's default Ctypes pack- age [81], which is by default part of Python since version 2.5. Particularly tools to ease the process (by using code generation) and some best practises will be stressed. We will try to tell a step-by-step story of the wrapping and development process, that should be transferable to similar problems.
As an example the creation of a wrapper for the Little CMS colour management li- brary [42,43] is outlined. The library oers excellent features, and ships with ocial Python bindings (using SWIG [88]), but unfortunately with several shortcomings (incom- pleteness, un-Pythonic API, complex to use, etc.). So out of need and frustration the initial steps towards alternative Python bindings were undertaken. An alternative would be to x or improve the bindings using SWIG, though for its simplicity and practicality we have chosen to use Ctypes for the wrapper to the plain C library.
Of course, wrapper code can be written manually, in this case directly using Ctypes. We do not provide a tutorial on how to use Ctypes. But the familiarity with this package is necessary when attempting to undertake serious library wrapping. The Ctypes tutorial and Ctypes reference on the project web site [81] are an excellent starting point for this. For extensive libraries and robustness towards an evolving API, code generation proved to be a good approach over manual editing. Code generators exist for Boost.Python as well as for Ctypes to ease the process of wrapping: Py++ [89] (for Boost.Python) and CtypesLib's [81] h2xml.pyandxml2py.py.
Three main reasons have inuenced the decision to approach this project using ctypes: • Ubiquity of the binding approach, as Ctypes is part of the default distribution. 1Talk slides available online under http://www.slideshare.net/XEmacs/
thinking-hybrid-pythonc-integration, http://www.slideshare.net/XEmacs/ thinking-hybrid-pythonc-integration-247208 and http://www.slideshare.net/XEmacs/ thinking-hybrid-pythonc-integration-368593
A.2. THE EXAMPLE 171