Illumination Source Measurement

To generate an estimate of the illumination sources, a direct measurement of the incident illumination in the scene is required. This is performed by measuring the incident illumination either directly through the use of a downwelling irradiance sen- sor, or indirectly through a calibration panel. The downwelling irradiance sensor is advantageous in that the sensor can be positioned on the sensor platform, so no hardware needs to be placed in the environment. However, the small field-of-view of the sensor means that there are often times of day where the terrestrial sunlight is not being measured due to a lack of line-of-sight visibility. Calibration boards can be easily orientated in a way that the incident illumination is a combination of terrestrial sunlight and skylight, however it requires placing the board in the scene. This may not be a viable option in hazardous situations. Once a measurement is obtained, an estimate of the illumination source can be obtained by combining the estimates at each initialisation point pair.

Calibration Board

When performing field based measurements, calibration boards such as those shown in Figure 4.10 are typically placed in the scene. These boards have a precisely known spectral reflectance curve and are therefore useful in providing a reference measure- ment in the scene. The radiance at a point p on the panel is described by:

Lp(λ) =

ρp(λ)

π [Vp,sunEsun(λ)τ (λ) cos θp+ ΓpEsky(λ) +

N X

j=1

FpjLj(λ)π], (4.16)

= ρp(λ)

π [Vp,sunEsun(λ)τ (λ) cos θp+ ΓpEsky(λ) + Eind,p(λ)], (4.17)

where ρp is known from laboratory analysis of the panel.

Figure 4.10 – Calibration boards of known reflectance are placed in the scene to

provide reference measurements when calibrating the image.

gives:

Esky(λ) =

Lp(λ)π

ρp(λ)

Γp+ Vp,sunT2(λ) cos αp

− Eind,p(λ) + Vp,sunT1(λ) cos αp Γp+ Vp,sunT2(λ) cos αp

| {z }

optional

. (4.18)

The terrestrial sunlight spectra can then be obtained by substituting Equation (4.18) into Equation (4.15).

Downwelling Irradiance Sensor

Another hardware based method of obtaining a measurement of the incident illumi- nation in the scene are downwelling irradiance sensors. Typically, these sensors are situated at position s on the sensing platform and their radiometrically calibrated measurement can be described as:

Es(λ) = [Vs,sunEsun(λ)τ (λ) cos αs+ ΓsEsky(λ)]. (4.19)

In this formulation, indirect illumination is assumed to have a negligible influence on the measurement. This is a valid assumption as in most scenarios, the terrestrial sunlight and skylight will dominate the measurement, and indirect illumination from

the scene will enter the sensor at grazing angles leading to a low weighting.

Simultaneously solving Equation (4.15) and Equation (4.19) gives an expression for skylight spectra: Esky(λ) = Es Γs+ Vs,sunT2(λ) cos αs − Vs,sunT1(λ) cos αs Γs+ Vs,sunT2(λ) cos αs | {z } optional , (4.20)

and the terrestrial sunlight spectra can be obtained from Equation (4.15).

Therefore, given initialisation points and measurements from either a calibration board or downwelling irradiance sensor, the terrestrial sunlight and diffuse skylight illumination sources can be estimated in terms of colour and intensity. This allows illumination invariant radiometric normalisation to take place, which assists the per- formance of high level algorithms. Compared to performing flat-field correction using the hardware based measurements, the proposed method provides robustness against shadow artefacts that may appear in the image due to the surrounding geometry. The approach is flexible enough to allow the inclusion of indirect illumination as a light source, depending on whether the application requires it. It is important to note that the method is not limited to the use of calibration boards and downwelling irradiance sensors. As long as the measurement of the incident light can be modelled using the outdoor illumination model, then an explicit solution to the illumination sources can be obtained.

4.4

Summary

Radiometric normalisation is a key component of remote sensing systems as it allows measured data to be converted to a format where it can be compared against labora- tory data. Accurate normalisation methods are required in for high level supervised algorithms such as clustering and classification to operate reliably.

Traditional normalisation methods include the use of calibration panels and down- welling irradiance sensors, which operate without any regard to the geometry of the

scene. This means that after normalisation, identical materials appear differently based on their incident illumination which is problematic for high level algorithms. In this chapter, a large scale sky factor and indirect illumination approximation method was proposed that reduces the number of full calculations and form fac- tors required. Finally, an illumination invariant radiometric normalisation procedure was proposed that can operate with or without estimates of indirect illumination. The proposed novel pipeline utilises in situ hardware based approaches for direct measurement of the incident illumination and generates an illumination invariant normalised image through the use of fused image and point cloud data. The benefit of this is that identical materials appear similar, independent of their location and orientation in the scene. This is performed without the need for highly parameterised atmospheric radiative transfer models and user selection of a known number of ma- terials by inferring the illumination sources from the image. The methods proposed in this thesis for attaining spatial and temporal illumination invariance in outdoor imagery, and illumination invariant radiometric normalisation are evaluated in Chap- ter 5 on a number of datasets.

Experimental Results

The proposed system for attaining illumination invariance in outdoor imagery through accurate modelling of the physical processes involved is evaluated on a number of datasets. The experiments are designed to first assess the performance of the algo- rithm under ideal conditions such as simple geometry and narrow-band sensors, be- fore testing on complex scenes and wideband sensors. The datasets are also gathered under different weather conditions in order to show the robustness of the proposed approach.

Section 5.1 describes the metrics used to evaluate the performance of the proposed approach, with a summary of the datasets being presented in Section 5.2. A brief overview of the implementation of the algorithm is given in Section 5.3. The experi- ments evaluate the spatial (Section 5.4) and temporal (Section 5.5) properties of the automatic relighting system presented in Chapter 3. Section 5.6 evaluates the large scale sky factor approximation approach. Finally, the illumination invariant radio- metric normalisation approach proposed in Chapter 4 is evaluated in Section 5.7. It should be noted that since relighting is performed on linear images with respect to diffuse skylight, visualisation of the images is hampered by the low intensity. There- fore, a gamma value of 1.5 is applied to the images to assist in visualisation, but the experimental validation of the proposed algorithms utilises the non-gamma corrected images.

5.1

Metrics

A variety of metrics are used in this thesis to evaluate the performance of the illu- mination invariance system. These metrics vary from those that are used to analyse the similarity between the colour and intensity of a pixel, to assessing the quality of high level algorithms.