The system described in this chapter was developed to model fish behaviour in sea cages through estimation of fish swimming speed and direction. The accuracy of the system can be compared with manual observations in terms of individual fish and the fish population.
The first approach is to look at individual fish and compare the automated speed and direction estimation of the system with manual analysis. This will determine if estimates are accurate but more importantly it will provide the error between manual and automated estimation. If the system can produce 20-80 samples per minute during daylight hours, then this can amount to over 8000 samples daily. Any individual inaccuracies will be averaged and it may be possible to calculate an average error of estimation. The system design focused on detection of changes in swimming speed and direction rather than providing very accurate estimates, but knowing the error of estimation allows the results to be adjusted to provide more accurate adjusted estimates. This approach is investigated in this section.
The second approach is to examine whether the system estimates the move- ment of the whole population of fish, given that the camera is positioned properly (at a right depth and location in the cage), and if the system discriminates be- tween fast or slow fish or fish closer or further from the camera. This issue has not be covered in depth in the thesis but casual observations of the working system have shown that the detection is improved by proximity of fish to the camera. Due to the turbidity of the water, fish further from the camera are blurry and the segmentation process may not differentiate these fish from the background. Another concern may be that the system has a bias in regards to fish that swim faster or slower. Casual observations have not revealed any bias but further stud- ies will be required to ensure that the system acquires unbiased samples from the field of view of the camera.
5.6.1
Recording Details
The test recording used here to check the accuracy of the system was a 7 hour recording of the daily routine on the 13th of March 2008. The recording started
at 7:55am and the camera was at a depth of 8 meters. No environmental or feeding data were available for this recording but tidal information was available from the Australian Bureau of Meteorology (BOM).
5.6.2
Accuracy Test
Five randomly selected 5 minute videos were extracted from the test recording. From each video, 20 random samples of automatically tracked fish were extracted (a total of 100 samples). For random video sequence selection, the MATLAB ran- dom number generator was used. For sample selection, a virtual die was used from http://www.random.org. When the result was 6, the sample was analysed, otherwise it was ignored. For each selected sample, only the last five frames were examined. Manual length measurements were made from the head to the tail of the fish using the MATLAB measurement tool. Length was recorded for each frame and the average of five frames was used as the final length. Speed measure- ments were usually carried out by determining the location of the tip of fish’s head in each frame. There were times when fish manoeuvres caused the head to move in a different direction to the centre of the mass. In this case pectoral fins were used as points of reference for speed measurements. Having coordinates of the same point in five frames, row and column inter frame differences in dis- placement were calculated. These differences were then used to calculate inter frame velocity. The final velocity was calculated based on the difference between the first and the last frame. Inter frame velocities were compared with the fi- nal velocity to eliminate erroneous data entry and help evaluate fish motion and camera motion. Unusual fish motion and severe camera motion could cause high error of estimation. The final velocity was broken down into speed and direction components and these values were matched against the last automated estimate from the Kalman Filter.
Differences between automated and manual results were calculated. To avoid the influence of outliers, median and inter-quartile range (IQR) were used as preferred descriptive statistics (Table 5.2).
The system provided a sufficiently accurate estimation of direction (Fig. 5.3i), as any difference within 60◦ meant that fish were travelling in a generally similar direction within the field of view of the camera (median = 14◦, IQR = 20◦). Higher variations in direction could be due to camera movement.
Table 5.2: Descriptive statistics of differences between automated and manual estimations. Median and inter-quartile range (IQR) were used to reduce the influence of outliers.
Median IQR Direction difference (deg) 14 20 Length difference (pixels) -3 14 Speed difference (pixels/s) -8 17 Speed difference (bodylengths/s) -0.1 0.39
10 30 50 70 90 110 130 150 170 0 10 20 30 40 50 60 Difference (deg) (i) −30 −20 −10 0 10 20 0 5 10 15 20 25 30 35 40 Difference (pixels) (ii) −100 −80 −60 −40 −20 0 20 40 0 5 10 15 20 25 30 35 40 45 50 Difference (pixels/s) (iii) −20 −1.5 −1 −0.5 0 0.5 1 1.5 5 10 15 20 25 30 35 Difference (bl/s) (iv)
Figure 5.3: Histograms of differences between manual estimates and automated estimates. (i) Direction, (ii) Length, (iii) Speed in pixels/second, (iv) Speed in body lengths/second.
Estimation of fish length was the most difficult measurement to perform accu- rately due to imperfections in the segmentation process (Fig. 5.3ii). The system often extracted only a portion of fish shape, underestimating the length of fish (median = -3 pixels, IQR = 14 pixels). The Kalman Filter was designed to assign less significance to these measurements and therefore there was less opportunity for these poor measurements to affect the final estimate. A substantial number of automated length estimates (20%) were higher than their manual counterparts due to the system detecting multiple fish as one while the manual analysis would correctly identify only one fish.
Speed in pixels per second tended to be underestimated by the system in comparison with manual estimation (median = -8 pixels/s, IQR = 17 pixels/s) (Fig. 5.3iii). This could be due to manual estimates being taken only during the last 5 frames before the track was lost, while the automated system may have tracked the fish for much longer prior to the comparison. Another reason is that during the analysis, the motion of the camera may have affected the estimate. Because only the last 5 frames were manually analysed, the camera movement could have occurred then or earlier in the automated tracking. The automated tracking would deal at that time with the camera movement by smoothing the estimate (using the Kalman Filter). The manual analysis did not attempt any smoothing.
The speed in body lengths per second was being used to standardise the measurement, given that no attempt has been made to detect the distance of a tracked fish from the camera. Because this measurement is a combination of speed in pix/s and the length, the error of estimation can be compounded as seem in Fig. 5.4. However the median difference of−0.1bl/sis satisfactory for the purpose of this system (Fig. 5.3iv). Table 5.3 indicates other possible sources of error within the system. Of these, the two most important are the segmentation process and camera movement. Improvements in the segmentation process may decrease the errors in length estimation and increase the duration of tracking. The camera movement can at times have a significant effect on the estimation process. Looking at this problem frame by frame, the tracked fish may appear to be moving forwards in the first few frames while in the following frames, due to the camera movement, it will appear to move backwards. The estimation process will attempt to account for this error to a degree but it can nevertheless have difficulty in determining the correct direction of movement and provide incorrect estimates. However the use of validation gates (see Section 4.3.1), especially when motion of the camera is severe, means that the track will be terminated rather than continue to produce poor estimates.
0 10 20 30 40 50 60 70 80 90 100 −100 −50 0 50 100 150 200 250 300 Sample % Length Speed pixel/s Speed bl/s
Figure 5.4: Influence of length and speed errors, for each sample, on the body lengths per second errors.
Table 5.3: Source of errors within the tracking system.
System errors Problem domain errors
Segmentation Natural variability in fish movement Data association Camera movement
Estimation errors Variability in environmental conditions Human interference