Figure 5.6: Spatially one-dimensional schematic diagram of a set of filters adapting their temporal tunings to lie on the line (in two spatial dimensions; plane) in frequency space corresponding to the velocity of the local image signal.
Dynam ics o f Adaptation
A t each tim e step, m easured velocity is fed back to adapt the filters. The
feedback m echanism im plem ented is a vector form ulation of the LM S algorithm o f W idrow and Stearns (1985). Treating the locally m easured velocity as the desired response vector, the tem poral state of the filter kernels is adapted on the basis o f the difference equation:
e =
V- CQ
(5.22)
where e is a vector error signal. Follow ing W idrow and Stearns (1985), we take the instantaneous value o f the squared error, lei", and m inim ise it with respect to
Q to arrive at a gradient descent update equation:
(5.23)
Q f i = O f - T i e f C , (5.24)
schem e converges in one iteration. In the sim ulations described below, using noisy input im ages, r\ is set at 0.1 to ensure convergence.
Results
The perform ance o f the adaptive schem e described above is illustrated with results from tw o image sequences, one synthetic and one real. The synthetic diverging tree sequence (Barron et al, 1994; Fleet & Langley, 1995) has a known 2-D m otion field, allow ing a quantitative com parison o f the perform ance o f the adaptive and non-adaptive versions o f the algorithm at various levels o f additive white spatiotem poral noise. Results are presented graphically in the form o f needle diagram s and confidence maps, and num erically as error statistics for the synthetic images.
Im plem entation
The spatial filters im plem ented are com plex band-pass G abor filters and their derivatives. The filters are tuned to each o f 6 spatial orientations, with
centre-frequency spatial tunings of 0.2 cycles per pixel and envelope standard deviations o f 2.5 pixels. Each spatial filter feeds into a fixed low -pass tem poral filter, as described in equation 5.15 but with Wo= 0, and a pair o f band-pass tem poral filters o f equal and opposite peak tuning. U nder the adaptive schem e, the centre-frequency tunings o f the band-pass tem poral filters are adapted from an
tem poral tunings o f the band-pass filters are set to 0.2 cycles per fram e, equivalent to a velocity tuning o f 1 pixel per fram e in the preferred direction. The tem poral filter time constant (see equation 5.15), b '\ is set at 1.25
fram es, giving an im plicit delay of 3 fram es in m easurem ents of velocity. F ollow ing F leet and Langley (1995), the spatiotem poral window, W (x,t), in equation 5.9 is a G aussian in space (with a standard deviation o f 1.2 pixels) and an exponential in time (with a tim e constant o f 3.33 frames). U nder the adaptive schem e, q (see equation 5.24) is set at 0.1.
E rror m easures
Follow ing Fleet and Jepson (1990), velocity is viewed here as spatiotem poral orientation and error is m easured as an angle in space-tim e. If velocity, v = ( u ,v ) \ is represented as a unit vector, v, in space-tim e:
^ V. I f , (5.25)
+ V +1
then the error, ij/g, between the correct velocity, v^, and an estim ate, y g, is given by:
F or results on synthetic im age sequences a m ean error and a density are quoted. The error is calculated as the mean angular error of all points satisfying an arbitrary confidence threshold. The density gives the percentage of points
satisfying that threshold. The confidence m easure used is the sm allest eigenvalue o f the spatial covariance matrix (see equation 5.10), which depends on the
m agnitudes o f the spatial gradients and the range o f their orientations.
M easured flo w fie ld s
D uring the diverging tree sequence the cam era m oves along its line of sight, the focus o f expansion being the centre o f the image. Speeds are up to 1.4 pixels per fram e on the left o f the image and 2.0 on the right. Figure 5.7a shows a frame from the sequence, and figure 5.8a the true flow field. The m easured motion field under the adaptive scheme is shown in figure 5.8b to correspond closely to the true flow field. Encouragingly, the m ost noticeable deviation from the veridical field occurs in a region where confidence estim ates are low. C onfidence
estim ates are shown in figure 5.7b, with regions o f high confidence being represented by bright points in the image.
To generate im age noise, the noiseless sequence, Iq{x, t), was linearly scaled and added to the random variable, n(x, t), draw n from a distribution uniform over the range o f values of the noiseless sequence. The noisy image, /„(x, t), was created
(a) (b )
F i g u r e 5 . 7 : ( a ) F r a m e 20 of t h e d iv e rg in g t r e e s e q u e n c e : ( b ) c o n f id e n c e m a p o b t a i n e d u n d e r t h e a d a p t i v e s c h e m e w ith no a d d e d noise. T h e d iv e rg in g t r e e s e q u e n c e is a s y n t h e t i c i m a g e s e q u e n c e d u r i n g w h ich t h e c a m e r a m oves a lo n g its line of sig h t t o w a r d s t h e c e n tr e of t h e im a g e . T h e sc e n e c o n s is ts of a h ig h ly t e x t u r e d tre e on a less w e ll - s t r u c t u r e d b a c k g r o u n d . R eg io n s of h ig h c o n fid e n c e in v e lo c ity m e a s u r e m e n t s , r e p r e s e n t e d b y b r ig h t p o in ts in t h e c o n fid e n c e m a p , a r e see n to c o r r e s p o n d to a r e a s of s p a t i a l s t r u c t u r e in t h e im age.
\ \ \ \ \ \ \ \ \ N \ \ \ \ \ \ \ \ \ 1 I \ \ \ \ \ \ \ \ \ 1 I \ \ \ \ \ \ \ \ \ \ t l i l t / / f f / / / / / I / / / / / / I f / / / / / / % X / X X / X X / X X / / X X X / / X X X / / / / X / / / / / / / / / / / / / / / / I I I S \ \ \ N \ \ \ \ \ \ \ I 1 \ \ \ \ \ \ \ \ I 1 I \ \ \ \ \ \ \ I I \ \ \ \ \ I I \ \ \ \ \ 1 1 \ \ \ \ I N . / / / I I / / / / I I I I / / / \ \ \ I I I I / / / / \ \ \ I I I / / / / \ I I I / / / / / / / / / • / / / ^ ^ y ' y y ' ^ / / / % / / / / / / / / / / / / / / / / / I I / / / / / / I / / / / / / f f / / / / / / / / / / / / / / /
F i g u r e 5 .8 : ( a ) C orrect m o tio n field for fram e 20 of th e diverging tre e sequence; ( b ) m e a su re d m o tio n field u n d er th e a d a p tiv e schem e w ith no ad d ed noise. T h e m o st n o ticeab le erro rs occur in a region w here confidence in th e m easu red velocity is low (see figure 5.7b).
= (1 - « « (a:,/) , (5.27)
w here 100a is the percentage noise level. N um erical results from the fixed and adaptive schem es show that, at noise levels exceeding 5%, the adaptive schem e yields low er mean errors at a higher density o f m easurem ent (Table 5.2).
Fixed filters A daptive filters N oise (%) error (°) density (%) error (°) density (%)
0 2.04 43.6 2.44 44.7 5 2.33 41.1 2.71 42.1 10 3.19 39.1 3.10 40.0 15 4.96 37.9 3.99 38.7 20 7.71 37.4 5.91 38.1 25 11.31 37.6 9.22 38.3
T a b le 5.2: M ean errors obtained fro m fra m e 37 o f the diverging tree sequence, a t various levels o f added noise, using fix e d and adaptive filters.
The perform ance o f the adaptive schem e approaches that of its non-adaptive counterpart at low -levels of noise, w hilst show ing an advantage w hich increases w ith noise level for 10% noise and above. W e w ould expect to be able to im prove the perform ance o f the fixed schem e by incorporating additional tem poral filters, but even under the proposed HR im plem entation this would involve a considerable increase in com putational storage.
T he rotating R ubik cube sequence (Barron et al, 1994) is a real image sequence in which a Rubik's cube is rotating anti-clockw ise on a
turntable. Figure 5.9a shows a fram e from the sequence and figure 5.9b the flow field m easured under the adaptive schem e after confidence thresholding.
C onclusion
An algorithm has been proposed for com puting image motion from the phase output o f centre-frequency adaptive HR filters. The algorithm may find application in biom edical im aging, for exam ple in profusion or cardiac studies w here im age noise levels are potentially high. The main limitation o f the technique is that some error is necessarily introduced into the velocity estim ates by the process o f adaptation itself, although for noisy image sequences this factor is outw eighed by the im proved signal-to-noise response o f the adaptive filters. Indeed, as hardw are developm ents allow faster fram e-rates for real-tim e
im plem entation of motion analysis algorithm s, so the proportion o f tim e during w hich significant adaptation occurs will decrease, and the benefit of superior im age signal representation will outw eigh the cost of adaptation even at low levels o f noise.
/
F i g u r e 5 . 9 : ( a ) F r a m e f r o m t h e r o t a t i n g R u b i k c u b e s e q u e n c e ; ( b ) m e a s u r e d flow field, a f te r th r e s h o l d i n g , u n d e r t h e a d a p t i v e s c h e m e . T h e r o t a t i n g R u b i k c u b e s e q u e n c e is a r e a l im a g e s e q u e n c e in w h ic h a R u b i k c u b e is r o t a t i n g a n ti- c lo c k w is e o n a t u r n t a b l e . V e lo c ity m e a s u r e m e n t s w h ic h s a tis fy t h e co n fid e n c e t h r e s h o l d c o m e f r o m t h e s p a t i a l l y s t r u c t u r e d re g io n s of t h e im a g e , a n d c le a rly show t h e r o t a t i o n p r e s e n t in t h e i m a g e se q u e n c e .