Choosing Optimal Pattern Frequencies

Assuming that the pattern and specimen are moving independently, the pattern provides the only information in an image for tracking with structured illumination microscopy. To provide the most accurate tracking, then, one needs to maximize the information present in the pattern (the signal) with respect to the other factors in the image (the noise). In this sense, noise includes both camera noise and the specimen, as both of these elements interfere with pattern tracking.

Chapter 2 discusses the role the microscope point-spread function (PSF) plays in im- age formation. The image that a microscope forms of a planar object is the convolution of the image predicted by geometrical optics with a slice of the 3D PSF determined by where the object is placed in front of the objective lens. Equivalently, in the frequency domain, the magnitudes of the frequencies in the object are attenuated by the Fourier transform of the microscope’s PSF. From Equation 2.21,

Ii(fX, fY) =H(fX, fY)Ig(fX, fY), (6.8)

whereI(fX, fY) is the Fourier transform ofI(x, y) andHis the Fourier transform of the

normalized PSF, also known as the optical transfer function (OTF).

Analysis of the microscope OTF enables choosing the optimal pattern frequencies for a tracking application. The OTF is the frequency domain representation of the normalized point-spread function (PSF)—it determines by how much pattern frequencies are attenuated. So, maximizing the SNR for a pattern involves choosing the pattern that, when multiplied by the OTF, has maximum magnitude at the pattern frequencies. The

selection of possible patterns and OTFs from all possibilities is driven by the constraints of the experimental parameters. The critical experimental parameters include objective lens characteristics, image sensor size, the expected axial distance between specimen and pattern, and the composition of the specimen itself.

Figure 6.1 shows a frequency-focus slice of magnitudes in the OTF of a simulated 40X, 0.65NA objective lens. Here focal depths range within z = [0. . .25]µm and fre- quencies range within f = [0...1₂] cycles per pixel. For this lens and sensor pair, the Abbe frequency limit is above the Nyquist frequency. This figure illustrates why one cannot select a set of pattern frequencies that provide accurate tracking capabilities when the pattern is in focus, and expect tracking to perform uniformly throughout the focus range. Different frequencies will be attenuated by different amounts at each focal depth, and this attenuation is not monotonic across frequency or depth.

Because the OTF is radially symmetric for a focal slice, in the following analysis, I restrict the pattern design to 1D laterally. When designing a 2D pattern one would select the same frequencies determined by 1D analysis, possibly distributing different frequencies along each axis.

Let P(x) be a pattern defined by a set of frequencies f := {fi | i = 0,1, . . . n}.

Let P(f) be the Fourier transform of P(x). The magnitude at frequency fi is given by

|P(fi)|. LetH(x;z) be the normalized PSF of an objective lens at a focal planez. Here,

the semicolon specifies that the focus axis is treated differently from the lateral axis on which the pattern resides. Specifically, H(f;z) is the Fourier transform of a slice of the PSF, one slice of the OTF; the magnitude at frequency fi is given by|H(fi;z)|.

The total magnitude at all frequencies in a pattern at a distance, z, from the focal plane is given by

S(P;z) = X

i

|P(fi)||H(fi;z)|. (6.9)

Equation 6.9 provides a scoring function for how strongly a particular pattern with frequenciesfi is represented at a particular focal depth, and this indicates how well this

(a)

(b)

Figure 6.1: The frequency-focus plane of the optical transfer function (OTF) for a 40X, 0.65NA objective lens from 0 to 25µm. a) One slice of the OTF displayed as an image; magnitudes are mapped with a logarithmic scale to make all magnitudes visible; bright values indicate high magnitudes. b) The OTF as a height map with same scaling, to better show relative values.

pattern will trackable compared to other patterns with different frequencies.

There is usually a known range of distances from the focal plane to the pattern in a given experiment. This distance could be fixed, or it could stay centered around a value with some variation, or it could span a whole range with equal probability. For this reason it is sensible to introduce a weighting function W(z) that weights the importance of focal distances in an experiment. This can be thought of as a probability density function for axial stage positions.

Combining the pattern scoring function for individual focal depths and the weighting function for the importance of focal depths, a metric for how well a particular pattern is suited to tracking in an experiment is given by:

S(P) =X z W(z)S(P;z) = X z W(z)X i |P(fi)||H(fi;z)|. (6.10)

Note that the weighting function in Equation 6.10 is multiplied by all frequencies in the pattern, and the result is summed across all focal depths. Rearranging terms therefore provides a function that specifies the score for all frequencies in the OTF, irrespective of the magnitude in a particular pattern:

S(f) =X

z

W(z)|H(f;z)|. (6.11) The frequency selection thus far has not taken into account the noise present in the system—the camera noise and interference from the specimen. Camera noise has constant magnitude at all frequencies, so one needs to ensure that any frequency selected for the pattern does not get attenuated by the OTF to near or below the noise floor. Equation 6.11 treats all frequencies equally—any frequency that is heavily attenuated by the OTF will receive a low score—and so provides the set of best frequencies to choose. Section 6.1.4 will discuss determining whether the pattern is trackable.

Spectral analysis of the specimen to be observed provides a way to consider the track-

ing interference caused by its presence in images. Because the specimen can have any arbitrary orientation, the magnitudes at frequencies at all orientations should be consid- ered. Given a specimen image, I(x, y), and its Fourier transform in polar coordinates, I(f, θ), the radially-summed magnitude provides a measure of specimen interference:

Q(f) = X

θ

|I(f, θ)|. (6.12)

To adjust this to the same scale as the axial weighting function, it is sensible to express the magnitudes on a logarithmic scale, and shift the scores to fall withinQ(f)∈[0. . .1]. This weighting term can now apply an additional constraint to Equation 6.11 that is the inverse of the specimen magnitude at all frequencies, Q0(f) = max(Q(f))−Q(f):

S(f) =Q0(f)X

z

W(z)|H(f;z)|. (6.13) The frequency weighting function can also be truncated to enforce the minimum and maximum pattern frequencies from Equations 6.6 and 6.7.

Now that a frequency scoring criterion has been established, the final task is to use it to select which frequencies to include in the pattern. Choosing the n frequencies with maximum score maximizes Equation 6.13, but these frequencies are likely to be bunched around whichever frequency had the maximum score. Section 6.2 discusses how to determine focus, with the observation that focus accuracy increases when the frequencies used are spread out. For this reason it makes sense to apply some additional constraints to the frequency selection. For example, one could select the frequencies that have local maxima in the scoring function; this distributes the pattern frequencies throughout frequency space, but uses the best one in each area. Or, one could select the frequency with maximum score and some of its harmonics; this ensures the pattern is periodic over a small window.

1. Select an objective lens and compute its OTF.

2. Construct a focus weighting function, W(z), that specifies the importance of dif- ferent axial distances from the pattern.

3. Construct a frequency weighting function, Q(f), that specifies the importance of different frequencies, using the parameters of the imaging system and the Fourier transform of an image of the specimen.

4. Multiply the OTF by the focus and specimen weighting functions.

5. Sum the weighted OTF along the focus axis to determine the frequencies that will provide the best tracking signal.

6. Select the n frequencies that maximize this signal.

Figure 6.2 provides some concrete examples of weighting functions applied to the OTF of Figure 6.1. In Figure 6.2a, the only constraints applied are the minimum and maximum frequency thresholds. The high frequency cutoff occurs because of the noise filtering requirement, whereσn= 1.6, as determined for a Pulnix camera in Section 2.3.

For this scenario, the predominant feature of the frequency scoring function is due to the attenuation of higher frequencies by the OTF. Low frequency patterns will provide the best tracking signal. The vertical dashed lines in the frequency score plot indicate the four local maxima with highest frequency, but in this case there is little to distinguish these points from any others.

In Figure 6.2b, the focus range has been restricted to fall within dz = [10. . .15]µm of the pattern. The frequency score again shows a preference for low frequency patterns, with local maxima at 0.025 and 0.05 cycles per sample that would be good options for patterns that have frequency spread throughout frequency space.

In Figure 6.2c, the focus range is restricted within dz = [0. . .15]µm and the fre- quency weighting has been constrained by the inverse spectrum of a frog brain tissue

(a)

(b)

(c)

(d)

Figure 6.2: Optimal frequency selection for different experimental conditions. In all cases, the OTF is computed for a 40X 0.65NA objective lens. Vertical dashed lines in the Frequency Score plots represent the local maxima with best score. a) Constraints imposed only by maximum and minimum pattern frequency for optical system. b) Constraints for imaging pattern at depths with [10. . .15µm]. c) Constraints for imaging a frog brain tissue within [0. . .15]µm of the pattern. d) Constraints for imaging frog brain tissue where the mean distance to the pattern is 17µm with standard deviation 2µm.

specimen. Because the tissue has a predominance of low frequencies, the optimal fre- quencies to include in the pattern are pushed out to around 0.2 cycles per sample.

In Figure 6.2d, the frequency weighting is the same as above, but the focus weighting has been provided by a normal distribution of stage positions, withdz ∼N(17µm,2µm). In this case, a mixture of low frequencies and high frequencies would provide the best tracking pattern.