Component Evaluation

Li(λ) = Le,i(λ)+ ρi(λ)

π [Vi,sunEsun(λ)τ (λ) cos θi+ ΓiEsky(λ) +

N X

j=1

FijLj(λ)π], (2.16)

where Γi is referred to as the sky factor and is equal to the form factor Fi,sky.

The causes of illumination variation in a scene and its influence on the appearance of a material can be seen through Equation (2.16). The visibility, sun angle and sky factor terms dictate the intensity of the direct sunlight and diffuse skylight illumination

sources. If the reflectance of the material is constant and these terms vary, the

appearance of the material captured by the camera will also change as shown in Figure 2.4.

2.1.2

Component Evaluation

Radiance

One of the key components for generating an illumination invariant representation of the scene is to obtain observations of the scene radiance. This is obtained through the use of imaging sensors such as consumer grade RGB and multispectral cameras, as well as hyperspectral line scanners. These sensors differ in the portion of the electromagnetic spectrum that they are able to sense, as well as their respective sensor response functions.

Consumer grade cameras typically record measurements in three colour bands; red, green and blue, with the spectral response of each channel being wideband in nature. Following the measurement of the scene radiance, the raw sensor data is typically processed by removing sensor noise using dark current subtraction, debayering, white

Ratio 500 600 700 800 900 1000 5 10 15 20 25

Increasing sun angle Sky factor is constant

Wavelength(nm)

(a) Decreasing the sun angle not only changes the inten-

sity, but also the shape of the spectra. Each illumi- nation spectra is normalised by the 90◦ sun angle spectra. Ratio 500 600 700 800 900 1000 1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3

Increasing sky factor Sun angle is constant

Wavelength(nm)

(b) In this example, increasing the sky factor predom-

inantly affects the magnitude of the spectra as the sun angle is small. Each illumination spectra is nor- malised by the 0 sky factor spectra.

Figure 2.4 – Influence of the sun angle and sky factor on the appearance of a material

of green reflectance.

balancing, gamma correction and conversion to a colour space. The latter three steps represent non-linear forms of post-processing.

Hyperspectral line scanners offer greater spectral resolution compared to consumer grade cameras through the use of narrow band sensor responses, but this comes at the cost of data storage as well as price. Typically, these scanners measure the radiation at many spectral bands at each pixel and can sense various regions of the electro- magnetic spectrum including visible, near, short and long wave infra-red regions. This makes hyperspectral scanners extremely useful for characterising the chemical constituents of scenes, as different materials possess unique spectral signatures [71]. During processing, sensor noise is removed using dark current subtraction, the image is corrected for smear and the data may be converted to radiance units if required using precise calibration data.

While Equation (2.16) expresses the radiance as a function of the material and illu- mination conditions, it is not required that the data be radiometrically calibrated. As long as the pixel intensity of the camera is linearly related to radiance, the ren- dering equation still holds. Therefore, for consumer grade cameras, the image prior to the gamma correction and colour space conversion can be used. White balancing algorithms that operate on each channel independently can be used to improve the range of the image. For hyperspectral scanners, the data can be used both before and after calibration.

Visibility and Sun Angle

Modern robotics and remote sensing vehicles perceive the environment using a mul- titude of sensors. The navigational sensors typically found on board these vehicles include Global Positioning System (GPS) and Inertial Measurement Units (IMUs). These allow the pose of the vehicle to be determined with respect to the earth and the point cloud generated by the LIDAR sensor can therefore be geo-registered. Given the location of the vehicle and the time at which the scans are taken, the position of the sun can be accurately determined [63]. Therefore, ray tracing from each position in the scene to the sun location and determining whether it is occluded or not allows

Once the position of the sun is determined, the sun angle θi is trivial to compute using

the dot product between the sun position vector and the normal at each position in the point cloud.

Sky Factor Calculation

Determining the sky factor for each region is vital in determining the influence of skylight on appearance. There are several methods used to calculate the sky factors

that differ based on their accuracy and computational time. In this section the

common approaches to sky factor estimation are detailed with the full calculation of sky factors referring to the computationally expensive, but accurate methods used to determine sky factors.

Assuming that the sky dome is of uniform colour and intensity, the sky factor integral (Equation 2.12) can be simplified:

Fi,sky = Z

Ω

cos θi

2π V (ω, dAi)dω. (2.17)

This assumption allows the influence of skylight to be determined without needing to calculate the skylight strength at each position on the sky dome. The uniform assumption is commonly used in the remote sensing community and for some com- puter vision applications [50]. It is valid for most daylight hours, with the exception of times near dawn and dusk when the horizon can appear red.

In order to evaluate Equation (2.17), a sampling procedure can be used. A number of rays are sent from region i in the direction of points on the sky dome chosen from a uniform distribution. Rays that are not occluded before they reach the sky dome are weighted by their angle of incidence and accumulated. After a large number of rays are traced, the result is normalised by 2π and this determines the sky factor. As is standard with Monte Carlo based approaches, the larger the number of samples, the more accurate the estimation becomes. Cosine distributed sampling of the sky dome can also be used, in which case the number of samples reaching the sky dome is just normalised to estimate the sky factor.

In remote sensing contexts, the sky factor is typically found by selecting a larger number of points on the sky dome, and determining the visibility to each point. The sky factor is then estimated as:

Γi =

Ni

Ntotal