Illumination Normalization

Changes in ambient illumination, and the resulting variations to facial appearance, are known to significantly deteriorate the performance of FR systems. Accordingly, several techniques have been proposed for illumination invariant FR (Sharma et al., 2014). Zou et al. (Zou et al., 2007b) presented a survey of techniques according to passive and active approaches. Passive

approaches focus on the visible spectrum images where face appearance has been altered by

illumination variations. They include illumination variation modeling, illumination invariant features, photometric normalisation, and 3D morphable model techniques. Active approaches employ active imaging techniques to obtain face images captured under consistent illumination condition, or images of illumination invariant modalities. Additional devices (optical filters, active illumination sources or specific sensors) are usually involved to actively obtain different modalities of face images that are insensitive to or independent of illumination change. Those modalities include 3D face information and face images in those spectra other than visible spectra, such as thermal infrared image and near-infrared hyper-spatial image.

There are three main types of techniques to produce illumination invariant facial images under the passive approaches: those applied at the pre-processing, feature extraction and classifica- tion levels (Struc and Pavesic, 2011). Pre-processing techniques seek to produce facial images that are free of illumination induced facial variations prior to feature extraction. They can be applied within any FR system, since they make no prior assumption that influences feature extraction or classification procedures. Feature extraction techniques seek to compensate for appearance variations in facial images using descriptors or representations that are stable un- der different illumination conditions. However, different empirical studies with LBP, Gabor wavelet-based features, and other descriptors have shown (Marcel et al., 2006) that none of

29

these can ensure illumination invariant FR given severe illumination changes. Classification- level techniques compensate for illumination changes according to the type of face model or classifier employed in the FR system. First, some assumptions regarding the effects of illumina- tion on face models or classification procedure are made, and then based on these assumptions, counter measures are undertaken to obtain illumination invariant face models or illumination insensitive classification procedures. Managing the effects of illumination at the feature ex- traction level is debatable. Classification level techniques may impose difficult requirements on the design data. They may provide the most efficient approach to illumination invariant FR. However, a large training set must usually be acquired under a number of lighting condi- tions and is computationally expensive. In this section a brief overview on some of the passive illumination normalization techniques is explained.

Most of the existing passive illumination normalization techniques are related to the retinex theory (Land and McCann, 1971). The latter is based in comprehending the main process of the image formation and perception. Mathematically speaking, an image I(x,y) can be presented as a product of the reflectance R(x,y) and the luminance L(x,y)

I(x,y) = R(x,y).L(x,y) (2.19)

R(x,y) consists in the characteristic of the object in the image and it is based on the reflectivity.

Meanwhile, L(x,y) is based on the amount of illumination in the image. By this, R(x,y) is the representation of the original image that is not dependent to the illumination (i.e invariant). The luminance is considered to vary slowly with the spatial position and can therefore be estimated as a smoother version of the image I(x,y).

R(x,y) = I(x,y)

L(x,y) (2.21)

For example the Single Scale Retinex (SSR) algorithm calculates the luminance factor L(x,y) by performing smoothing with a single Gaussian filter into the image I(x,y). The reflectance

R(x,y) is calculated with the expression using algorithm in equation . However this technique

seems to be insufficient. As a fact, images having large illumination discontinuities may have visible halo on the reflectance. Then a newer extension is the Adaptive Single Scale Retinex algorithm (ASSR) which is based on adapting the process of smoothing using iterative convo- lution that applies two discontinuity measures: local inhomogeneity and spatial discontinuity (Park et al., 2008). Another extension of this work was the Multi-Scale Retinex (MSR) (Jobson

et al., 1997) where in this case multiple Gaussian filters are implemented in terms of widths

and the different values of the reflectance for each case are then combined providing a global reflectance R(x,y).

Self Quotient image (Wang et al., 2004a) was proposed as another technique for combating illumination issues. It combines the image processing technique of edge preservation filtering with the theory of retinex. A newer version of this approach was introduced in the same work called Multi-Scale Image (SQI) where instead of using Gaussian filter the use of anisotropic filter was used for smoothing.

The Non-local mean (NL) consists in smoothing an image by computing at each pixel value a weighted average of surrounding pixels. The weight is a similarity function that calculates the tendency of similarity between the neighbouring pixels and the target one having constant variables. As for the newer version of this approach called Adaptive Non-local means (ANL), the smoothing parameter found in the weight function is dependent on local contrast instead of having preselected values In this case, more smoothing is performed when contrast is low at a region and vice versa. Details can be found in (Gross and Brajovic, 2003).

31

In the past few years recent normalization techniques have appeared such as the "Tan and Triggs" (TT) technique (Tan and Triggs, 2010). It merges multiple approaches: robust illumi- nation normalization that contains series of stages in order to cope up with variations such as local shadowing and highlights. The stages can be resumed in applying Gamma correction as a start then filtering with Difference of Gaussian (DoG), finally performing masking in order to mask out irrelevant variables. Afterwards, local textured base feature extraction Local Ternary Pattern (LTP) which is based on the principle of LBP is employed. Finally, a feature fusion technique is applied. The TT technique shows to perform well when applied to FR especially using LBP feature representation.