Section 2.1 discussed how bright-field microscopes form images; this section discusses how these images are digitized for computer storage and analysis. Fourier optics explains that the image formed by a microscope is a complex EM field. When projected onto a
surface and observed, a microscope image is a continuous function of irradiance—the power of light energy incident on a surface area [FP03, Ch. 4]. A charge-coupled device (CCD) image sensor is commonly used in image acquisition to sample the continuous distribution of light to produce a digital image.
A digital image is a collection of data organized in a structured, regular grid, for example a two-dimensional array of intensity values [FvDFH97, Ch. 14 & 17]. A digital image is composed of picture elements—pixels—each of which has a coordinate position and data value defined at that coordinate location. A digital image is therefore a discrete function over the domain of grid locations.
In the convention adapted for this work, pixel coordinates specify the center of a pixel and the pixel value is strictly defined only at the center of pixels. This work is concerned with the analysis of time sequences of two-dimensional (2D) images. Other types of images may be generated through this analysis, for example 2D vector images that describe motion fields.
2.2.1
Charge-Coupled Devices
The image formed by a CCD is a sampled representation of a continuous distribution of light. A CCD is composed of a regular grid of photosensitive elements that operate under the principles of the photoelectric effect. When light is incident on a metal, photons may interact with electrons in the metal. A photon can transfer its energy to an electron, possibly ejecting the electron from the atom to which it is bound. A freed photoelectron can be trapped in a potential energy well associated with a photosensor site on a CCD.
A CCD captures freed photoelectrons over the exposure time period [HK94]. At the end of the exposure, the electrons in each potential well are transfered sequentially to a readout register where a voltage signal proportional to the number of photoelectrons captured is created. The analog sequence of voltage signals is converted to a digital
stream by an analog-to-digital converter (ADC). This stream of digital pixel values is transferred from the camera to a storage device or computer for storage and analysis.
If the purpose of a CCD is to measure photoelectrons created by a distribution of light intensity on a surface, the ideal value, I(x, y), recorded by a camera for an individual sensor element would be proportional to the number of photons incident on the sensor integrated over the exposure period, E(x, y):
I(x, y) =kE(x, y), (2.26)
where k is the device-specific constant of proportionality. Of course, it is impossible to accurately measure the incident intensity function E(x, y) exactly—several types of noise corrupt the image acquisition process [HK94, TRK01, FP03]. Understanding the character of these noise sources is important for accurate image analysis and simulation tasks, such as those described in this dissertation. Below I construct the CCD imaging model used by Healey and Kondepudy while explaining each source of noise.
The arrival of photons at a sensor site is a Poisson-distributed random process [HK94] that leads to the random effect known asshot noise. The photoelectron count at a site is therefore a sample of a Poisson random variable, Ns with variance equal to the ideal
number of photoelectrons collected:
I(x, y) =Ns(kE(x, y)). (2.27)
Healey and Kondepudy represent shot noise with a Poisson distribution that is shifted to have zero mean, implicitly making the following manipulation:
I(x, y) = kE(x, y) + (Ns(kE(x, y))−kE(x, y)) (2.28)
= kE(x, y) +Ns0. (2.29)
Shifting the shot noise to have zero mean affords a convenience to the analysis that follows: when all noise sources are zero-mean and additive, the expected value of a pixel’s intensity can be obtained by averaging many observations. For this reason, I adopt this notation here.
Fixed pattern noise arises from different physical sensors having slightly different sizes and quantum efficiencies, therefore counting a different number of photons for the same incident light intensity. The fixed pattern noise can be incorporated by the scaling factorkin Equation 2.26;k(x, y) has a normal distribution with a mean of 1 and variance
σ2
k:
I(x, y) =k(x, y)E(x, y) +Ns0(x, y). (2.30) Thermal energy in the image sensor can also free electrons that become trapped in the potential energy wells associated with a CCD sensor element. These dark cur- rent electrons are indistinguishable from photoelectrons and so contribute an additional source of noise in the image measurement:
I(x, y) =k(x, y)E(x, y) +Ns0(x, y) +Nt(x, y). (2.31)
The dark current increases with exposure time and sensor temperature.
The circuitry that reads out the potentials stored at each sensor site contributes a small, zero-mean, Gaussian-distributed noise, modeled by Nr:
I(x, y) = k(x, y)E(x, y) +Ns0(x, y) +Nt(x, y) +Nr(x, y). (2.32)
Equation 2.32 represents the number of electrons collected as the input signal to the ADC process. The ADC applies a gain, A, that converts this electron count in to an intensity level in arbitrary units, often called “counts”. Quantization of the signal by
the ADC involves an additional noise term,Nq:
I(x, y) = (k(x, y)E(x, y) +Ns0(x, y) +Nt(x, y) +Nr(x, y))A+Nq(x, y). (2.33)
Healey and Kondepudy argue that Nq is approximately zero-mean and uniformly dis-
tributed over [−q
2,
q
2], where q is the quantization step size. This equation represents
the actual imageI(x, y) recorded by a CCD in response to an ideal photoelectron count function E(x, y). Rearranging a few terms yields a convenient form:
I(x, y) = µ(x, y) +N(x, y), (2.34)
µ(x, y) = k(x, y)E(x, y)A+µt(x, y)A,
N(x, y) = Ns0(x, y)A+Nr(x, y)A+Nq(x, y).
Here, µ(x, y) is the expected value of I(x, y), µt(x, y) is the expected value of the dark
current, Nt(x, y), which is the only non-zero-mean noise source, and N(x, y) is a zero-
mean random variable that encapsulates all other temporal noise sources.
In summary, the formation of digital bright-field microscopy images consists of the following key components:
1. K¨ohler illumination ensures that a uniform light field illuminates the specimen.
2. The microscope’s objective gathers light transmitted through the specimen, form- ing an image of the specimen convolved with the microscope’s PSF on the image sensor.
3. The image sensor integrates the light irradiance distribution function over an ex- posure time by counting photoelectrons collected in a regular grid of potential energy wells.
4. The camera scans the charges collected at each sensor site and digitizes these
sensor values.
5. A computer saves the collected digital image for further image processing.