When tracking of individual targets is not possible due to a large volume of possible targets (e.g. a crowd of people at a train station), it may still be feasible
to look at the overall motion pattern of these targets. The basic idea is to extract some motion information from a sequence of images and compare that sequence with a prior sequence or even a template. This can be achieved using the motion flow fields method, which originates from the need to track motion patterns of crowds in malls, shopping centres and marathons (Hu et al., 2008).
Motion flow fields method uses a dense optical flow to extract flow vectors for
M frames. The optical flow procedure generates a large number of vectors, many of which can be attributed to noise. The noise is removed by thresholding with all vectors below the threshold are set to zero. The resulting flow vectors are combined into a Point Flow Field (Ali and Shah, 2008). The Point Flow Field for point i is defined as Zi = (Xi, Vi), where X = (xi, yi) represents location
of the point and Vi = (vx, vy) represents velocity components of the motion. Vi
is the mean of optical flows calculated from M frames, which means the Point Flow Field contains combined motion information about a number of frames and additionally, it has an effect of smoothing out the camera motion. To decrease the number of vectors from thousands to hundreds, Hu et al. (2008) used the Gaussian ART algorithm (Williamson, 1996). This achieves a decrease in the number of flow vectors without affecting the geometric structure of the flow field. Once the Point Flow Field is established, a sink seeking process is carried out to determine motion tracks within the field. For each flow field vector Zi,
the sink seeking process is defined by a series of points Zei,t = (Xei,t,Vei,t), where
t = 1,2, .... The process was initialised with Zei,1 = Zi and the points for t > 1
are defined by:
e
Xi,t =Xei,t−1+Vei,t−1 (4.40)
e Vi,t = P n∈N eighbour(Xei,t)VnWt,n P n∈N eighbour(Xei,t)Wt,n (4.41) The weightsWt,nare calculated using kernel based estimation similar to the mean
shift approach (Comaniciu and Meer, 2002) and are defined by:
Wt,n =exp − e Vi,t−1−Vn h 2 (4.42)
where h is the bandwidth of the kernel. This process continues until a stopping condition is satisfied. The stopping condition can be the amount of progress made from the last point. If the algorithm stalls and makes no progress, it is terminated. An additional stopping condition is when the point moves outside the Point Flow Field.
The next step of this process is to cluster sinks together and create super tracks. Clusters are initialised with an empty set. A candidate sink Zi∗ =
(Xi∗, Vi∗), which is part of path PZi is matched with each cluster C
k. If a match
is found then the sink is added to the cluster, otherwise a new cluster is formed. Within each cluster the candidate sink is compared with all sinks within the cluster using three measures: i) distance between sinks, ii) similarity between directions (cosine similarity) and iii) Hausdorff distance between sink paths (Hu et al., 2008). These metrics are be defined by:
i) Dx(ZI∗, Ck) =maxZj∗∈Ck Xi∗−Xj∗ (4.43) ii) Dv(ZI∗, Ck) = minZ∗ j∈Ck < Vi∗, Vj∗ > kVi∗kkVj∗k (4.44)
iii) Dp(ZI∗, Ck) =maxZ∗j∈CkHausdorffDist(PZi∗, PZ ∗
j) (4.45) Finally, these three metrics are combined together to determine which cluster is the closest, idx=max(D1
x
1
DpDv), where idx is the index of the cluster.
4.5
Summary
The choice of a tracking and data association method depends on the applica- tion. In practice, the approach is to first select the simplest method possible and determine if that method performs in a satisfactory manner. The Kalman filter is a very popular and effective tracking method but is restricted to lin- ear systems, though with careful design, the basic KF filter can deal with slight deviations from linearity. For more non-linear cases, the Unscented Kalman Fil- ter is recommended in preference to the Extended Kalman Filter. The Particle Filter can deal with non-linear systems through sample approximation. How- ever Particle Filters have their own difficulties regarding performance (due to the number of particles required to successfully approximate distributions), the choice of importance density and multi-target tracking. However the ability to deal with unpredictable motion patterns is appealing when tracking fish in tanks, and colour based Particle Filters may be able to detect fish as they try to adapt to tank surroundings.
In data association, the Global Nearest Neighbour is the preferred method when dealing with a small number of targets. However fish interactions in tanks cause frequent occlusions which take several time steps to resolve. This might warrant the use of more advanced methods like Multidimensional Assignment.
This thesis looks to apply tracking concepts in the aquaculture context in the following methodology chapters. In sea cages (Chapter 5) there will be lots of targets but they only need to be tracked for short periods of time, just enough to extract velocity data from the tracker, and the emphasis will be on real-time
performance of the system, so the tracking system must fit into this requirement. In tanks there are two tracking objectives. The first is to track fish when possible to determine their positions and velocities (Chapter 8). The second (far more difficult to achieve) objective is to track fish for long periods of time, with correct identification at all times and without an error (Chapter 7).
CHAPTER 5
ESTIMATING FISH MOVEMENT IN
SEA CAGES
5.1
Introduction
The previous chapter reviewed concepts behind tracking of targets in cluttered backgrounds. The aim of this chapter is to describe the development of a com- puter vision system to track fish in commercial sea cages. In the following two chapters the research focus is on observing large groups of fish in sea cages, in order to interpret fish behaviours based on their movement.
The task of the tracking system is to detect as many fish as possible from the video file and calculate an aggregate velocity profile for the whole population. Individual descriptors of the profile such as the mean and standard deviation of speed and direction, are then used to identify patterns in fish behaviour over time. The system consists of three major stages: the segmentation stage, the tracking and data association stage, and the data logging and display stage. The system has been designed using MATLABR /SIMULINKR software, a tool
used to model dynamic systems which uses block based design instead of a usual programming interface. In addition, some of the functions were written in the C programming language to increase the computational speed of the system.
Figure 5.1 shows the SIMULINKR model of the tracking system and this
chapter explains the computational steps used to extract fish sub-images from video sequences, manage fish tracks and calculate the above mentioned descrip- tors of the velocity profile. Section 5.2 describes details of image segmentation and object extraction with several segmentation methods covered. Section 5.3 in- troduces the implementation of the tracking methods and association of multiple measurements to multiple tracked fish. The system also has a target management module to create, maintain and delete targets (Section 5.4) and a module to gen- erate velocity statistics from each sampling period (Section 5.5). The accuracy of the system is discussed in Section 5.6.
Another important factor is the movement of the camera and Section 5.7 deals briefly with the issue of camera stabilisation. Section 5.8 describes how optical flow can be utilised to detect changes in movement patterns when individual fish cannot be detected and tracked.