In the previous chapter, an abstract level of image acquisition pipeline was shown. Here to further explain steps involved we reshow the image acquisition flow diagram in Figure 2.1. Each step in the flow diagram comprises many tasks. To begin, the light reaches the sensor after passing through the lens, aperture, shutter and any fil- ter used. This can be graphically represented in Figure 2.21where sensor irradiance contains radiant power passing through camera lens, aperture, and shutter. Assum-
Figure 2.2: Graphical representation of light reaching imaging sensor through lens, aperture, with a desired shutter speed.
ing the lens effects the incoming light linearly, given aso,αas aperture size and κas
the effect of any filters used, the spectral irradiance reaching the sensor is given as
E= Loκα. (2.1)
Here L represents the scene irradiance. Note that these values are considered per wavelength channel and are integrated across the electromagnetic spectrum for a combined effect. For trichromatic RGB cameras, often color filters are placed over sensors to form a color filter array. The Bayer pattern is widely used for the color filter array. Assuming the sensor size as constant andTbeing shutter speed (the time the sensor is exposed) the sensor exposure for a given wavelength is given as
X=ET. (2.2)
Charged couple device (CCD) sensors release photoelectrons when exposed to light. Each pixel location of the CCD contains a “well00 where these photoelectrons are stored. The upper bound of photoelectrons well is called the full well capacity of
the pixel (Janesick [2001]). In case of higher exposure, pixel full well capacity often reaches its maximum, resulting in spill of photoelectrons to the neighboring pixels which causes image saturation and blooming effects (Klinger [2003]). After being stored in pixel wells, these photoelectrons are then counted and given a digital num- ber called count for the given pixel location. The ratio between the incoming photons to the photoelectrons released by the CCD is known as quantum efficiency, whereas the conversion ratio of the photoelectrons number to a digital count is referred as gain of the CCD. Theoretically, if the shutter is closed and we capture an image with a short exposure time, there should not be any photons arriving at the CCD. How- ever, in practice, the count is not null. This is due to the presence of dark current and CCD bias. Dark current is the small electric current flowing through the CCD when theoretically no photons are entering the camera. Dark current is dependent on the temperature of the CCD, where a high CCD temperature will result in higher dark current values. Bias often appears as a regular pattern on the image which arises from the pixel-to-pixel variations of the offset level on the CCD count. The dark cur- rent and bias should be subtracted from the image in order to properly estimate the photons and their conversion. Using the quantum efficiency, dark current and bias, we can express the count or data number as
ODN =
1
γ(EQT+ψT+β), (2.3)
whereODN is the observed data number or count, γis the gain, E is the irradiance
impinging on the CCD,Qis the quantum efficiency,T is the exposure time,ψis the
dark current andβis the bias.
To remove the effect of dark current and bias, the common practice is to take an image with a closed shutter, such that the photon flux becomes zero,i.e., EQT → 0. The new equation becomes
O0 = 1
γ(ψT+β). (2.4)
If we subtractO0 fromODN our observed data number will be
O= 1
The expression above implies that the effects of dark current and bias can be safely ignored and removed automatically at start-up or with a periodic dark current ac- quisition routine. Note that this subtraction also assumes that that the effect of dark current and bias remains at the linear part of the camera response function.
The pixel values obtained are usually non-linear with respect to the incoming irradiance. The limited dynamic range of cameras pushes them to adjust the ex- posure according to the dynamic range of the scene. The pixel well capacity at saturation state adds to the nonlinearity of the camera response. Similarly, camera manufactures apply photo-finishing before saving the image in the final pixel values, (Nguyen et al. [2014]), which adds to the non-linearity of the final image values. This non-linear mapping is referred to as the camera response function. With this camera response function, the final pixel values is given as:
I = f EQT
γ
. (2.6)
These equations form the foundation of problem solving for exposure control, radiometric calibration and dynamic range compression. Depending on the nature of the problem, lens effect, aperture size, quantum efficiency, gain and filter effects are taken constant, and at times ignored, when the desired estimate is assumed up to a scale or normalized.