Determining motion from images is a fundamental problem in image analysis, and serves as the basis of many other image analysis tasks, such as structure estimation, object segmentation, and video compression [TV98, Ch. 8]. A significant portion of motion analysis research relies on a brightness constancy assumption: the image of a scene object maintains constant intensity over time, at least for small time steps. This assumption has the following properties:
• Objects that emit light do not change their emittance; objects that reflect light do not change their reflectance; objects do not appear or disappear.
• The illumination of the scene remains constant.
• Surfaces do not occlude each other.
• Surfaces are Lambertian—radiance is independent of viewing direction.
In 1981, Lucas and Kanade proposed an image registration method that is the basis for many feature-based motion techniques [LK81]. This method relates the motion in the direction of the image gradient to the temporal difference between images. The method can be applied to images as a whole or to image patches locally, with the assertion in either case being that motion over the observation window is constant and can be described with a parametric model. Applied to small image patches, this technique provides an estimate ofoptical flow, the projection of scene motion onto the image plane. The computation is often constrained to features with high gradients in both directions, as these are locations where the local tracking is likely to succeed [ST94, SMB00]. Applied over the whole image plane, this technique provides a single parametric model of image motion for all image locations.
Horn and Schunck use the same image gradient constraint in an energy minimization problem and introduce a regularization term that enforces smoothness of the computed
velocity field [HS81]. The result is a technique that provides a dense estimate of optical flow through an iterative solution. Bruhn et al. embed the local motion estimate of Lucas and Kandade into the energy minimization function of Horn and Schunck to provide a combined local and global optical flow computation [BWS05]. This approach provides a balance between using local motion information where it is available and filling in smooth flow fields in areas of low image contrast. An implementation of this combined local and global optical flow computation appears in ImageTracker, a software package described in Appendix A.
In multiple layer imaging, gradient-based techniques fail because image gradients are not preserved in overlapping transparency layers. In other words, an edge appearing in one image can be due to a feature in any component layer or the alignment of features in multiple layers. As features move in each layer independently, edges in the resultant image will appear and disappear. The two remaining popular motion computation techniques are explored in this dissertation—frequency-based methods and correlation- based methods [BB95]. These techniques share a low-dimensional motion model that is robust to information from problematic local areas.
Phase correlation is a frequency-based motion computation that leverages theFourier shift property—translation in the spatial domain is equivalent to a phase shift in the frequency domain. Phase correlation readily provides whole-pixel translations between two images, and much recent research on phase correlation has addressed extending the technique beyond translation transforms and to subpixel accuracy. Reddy and Chatterji explain how to extend phase correlation to recover rotation and scaling in addition to translation using a coordinate transform [RC96]. Forooshet al. show how to obtain ac- curate subpixel translations from downsampled images [FZB02]. Hoge provided another subpixel registration technique that uses singular value decomposition to separate the phase estimation onto the coordinate axes [Hog03]. Takita et al. use low pass frequency domain weighting function to restrict the spectrum of frequencies used in phase correla-
tion [TAS+03]. They further fit the peak indicating translation in the spatial domain to
the analytical functions predicted by the weighting. Recently, Ho and Goecke proposed a phase correlation method that computes parametric scale, rotation, and translation transforms within image blocks using a subpixel function fitting [HG08]. They fur- ther smooth the flow fields across image blocks using a weighted vector averaging. In Chapter 5, I propose a method to determine subpixel phase correlation based on phase estimation of individual frequencies known to be dominant in the images.
Correlation-based methods compute parametric image motion by finding the trans- form parameters that maximize a spatial comparison metric of the intensities in two images [BB95]. Like gradient-based image registration, this matching approach can be applied to small image patches or the entire image frame. Computing the correlation for all possible transforms between images is computationally expensive, so practical methods depend on coarse-to-fine refinements and optimization strategies to drive the parameter search towards the optimal solution [ISNC05]. In Chapter 5, I propose a method to perform correlation-based tracking using a pattern known to exist in an im- age layer and using a model of microscope image formation to predict layer appearance.