To evaluate different stationary component removal algorithms, I implemented both of the techniques outlined in Section 4.2 as well as mean-image background subtraction in C++ using the Insight Toolkit [ISNC05] (ITK) as a framework. Some implementation details deserve mention.
4.3.1
Enforcing Integrability
The GLT method requires an integration of the stationary gradient estimate in Equa- tion 4.13. The gradient estimates, however, are not guaranteed to be integrable. That is, choosing a different order of integration will result in a different transmission map.
Frankot and Chellappa investigated this problem in research on determining the geom- etry of a surface from shading cues (shape from shading) [FC88]. I adopt this approach to construct a transmission map, and therefore briefly summarized the method here.
The problem is to compute the 2D surface z(x, y) given estimates zx(x, y) = ∂z(∂xx,y)
and zy(x, y) = ∂z(∂yx,y) of the derivatives of the surface z. The image derivatives have
discrete Fourier transform coefficients Cx and Cy, such that
zx(x, y) = X ωx,ωy∈Ω Cx(ωx, ωy)ei(ωxx+ωyy) (4.16) zy(x, y) = X ωx,ωy∈Ω Cy(ωx, ωy)ei(ωxx+ωyy), (4.17)
where Ω is the range of frequencies (ωx, ωy) used in the Fourier transform. Presuming
that the derivatives are estimated by a discrete central difference,
zx(x, y) =
1
2[z(x+ 1, y)−z(x−1, y)], (4.18) Frankot and Chellappa show that the periodic surface that has integrable derivatives closest to the estimated zx and zy is given by the Fourier coefficients
C(ωx, ωy) =
−isin(ωx)Cx(ωx, ωy)−isin(ωy)Cy(ωx, ωy)
|sin(wx)|2 +|sin(wy)|2
. (4.19)
The elegance provided by this solution is that a simple combination of Fourier coefficients from the derivative estimates has simultaneously projected the surface derivatives into the nearest integrable subspace and performed the integration.
Note, however, that this Fourier-based integration finds a periodic surface as an estimate of the stationary transmission map. This presents a problem if the actual transmission map is non-periodic, as is the case for most natural images. To improve the behavior of the GLT method, I pad each log-intensity image with a border of half the image size in each direction and apply a Hanning window ([Har78]) that apodizes
intensity values in this padded region to smoothly transition between the pixel value at the image boundary and the mean image value at the border boundary. This embeds the image data within a signal that smoothly transitions at boundaries, which eliminates discontinuities in the infinite repetition of the signal considered by the Fourier transform. After Fourier-based integration, I crop the resulting surface estimate to the original im- age region. This windowing technique applies only to the GLT-median implementation; the GLT-mean implementation uses the simplification of Equation 4.15.
4.3.2
Handling Zero Intensity
Because log(0) is undefined, zero intensity values require special attention when com- puting the logarithm in Equation 4.9. At low intensities, the measure from an image sensor is dominated by noise [HK94]. Because the intensity estimate is less accurate at the low intensity range, it is least harmful to treat zero-valued and one-valued pixels the same. I therefore clip the minimum input image intensity to one. The alternative approach—offsetting all intensity values by one—would destroy the relative intensity ratios for all pixels.
If a pixel records zero intensity values for all frames of a video, there is total occlusion of light at that pixel. The transmission map would require infinite gain to repair the intensities at that pixel, and the repaired image intensity would have infinite error. In other words, the repaired image intensities should not be trusted at all for pixels that record low intensity values over the entire video. Because the gain scales noise along with image intensity, the transmission map functions as both a transmission gain and a confidence measure—intensity values with high transmission map gain should be trusted less than intensity values with low transmission map gain. It may be desirable to mask out any pixels whose transmission values fall below a certain threshold, or pass along the transmission map as a confidence measure for further image analysis processes.
appears, for example, in the flat-field calibration of Section 2.3, Figure 2.7. This is due to misalignment of the image sensor in the camera housing, and this situation should be apparent during camera calibration. Any affected pixels should be cropped before image analysis.
4.3.3
Transmission Map Scaling
The integration in Equation 4.13 introduces an integration constant to the estimate of the logarithm of stationary transmission. Exponentiation in Equation 4.14 transforms this free parameter into a scale factor. So, the GLT method recovers the constant transmission up to a scale factor, and the task remains to select an appropriate value for that scale factor.
An unoccluded image location should have a transmission factor of one. It is un- known a priori that any particular location is unoccluded—in fact, the whole image plane could be partially occluded. But such a situation is equivalent and indistinguish- able from inserting a neutral density filter into the light path or reducing the intensity of the microscope lamp. Intuitively, the image location that receives the most light over the course of the video would be an obvious choice for an unoccluded location. The pixel with the maximum transmission metric, then, is a good candidate, and this choice reinforces the assumption that objects in the light path only absorb light. The computed stationary transmission map is therefore be rescaled to have a maximum of 1.
Highlights in the image due to diffraction, such as at the center of beads, may set this maximum transmission value too high, overly dimming the output images. In this case one may select some other reference value, such as the 90thpercentile of the transmission
map, to scale to 1.
Another possible solution scales the mean transmission to one. Although violating the assumption that light is only absorbed, this choice maintains the mean intensity between input and output images, and preserves visual consistency especially when
occlusions cover a small portion of the image plane.
A similar situation occurs in background subtraction. Mean image subtraction in- volves a shifting of intensity values. As this process does not preserve the intensity ratios between pixels anyway, an arbitrary intensity shift can be applied to put the output image intensities into some desired range. In my background subtraction im- plementation, I maintain the mean image intensity between input and output image sequences to preserve visual consistency.