using motion features and the length, width and size ratio of the objects. The results show that some increased object separation is achieved, with the white object on the side of the road characterised differently than when just motion features are applied. Some of the road cracks are also defined separately and more clearly. The samples on the Matlab figures are individual objects, such that the size of each object and their bounds is not clear. As seen in the WISE edge linking phase, the car and bike consist of several separate objects that are defined by the textures of each surface (the car has a white roof and dark bonnet) Figure 6.6(c) shows the result of this clustering clearer with the colour overlay on the original image, so that the extent of the characterisation can be seen overlayed onto visual objects. In this image, the long lines that are the road verge, the markings and long road cracks are characterised as the same type of object. The white object and disturbed earth are typed differently and the smaller cracks on the road are also separate from the larger, longer cracks. Both the car and the motorbike are typed the same, different to the rest of the surroundings. The second video sequence is limited in the number of permitted frames, figure 6.7. The clustering is applied to motion features only because in this scenario, there is one small moving object of interest with the remainder being object detections of scene objects inherent to the WISE technique. The Matlab plot, figure 6.7(a) shows several different motion types. The motion type of the object of interest has been circled (orange cluster). Because the motion of the UAV (and hence the camera), is in all 8 degrees of freedom for three-dimensional movement, when the camera rotates or pans with lateral motion, many of the objects in the scenario are moving in the same real direction but in a different relative direction. A filter is applied to compensate for this differential in relative motion, and the results of the remaining motion are shown in figure 6.7(b), where the results have been overlayed onto the original frame, with the objects separated by motion highlighted. The result in this case, shows that there are a couple of moving road objects, the moving object of interest, and detects the motion of the smoke emanating from the chimneys.
6.5
Discussing Object Characterisation
The additional clusters seen in figure 6.6 that do not represent moving objects yet are clustered as moving separately to the other stationary objects can be explained by three dimensional camera movement. For the most part, the camera motion appears to be translational, but there are some small deviations in the rotational and z-axis planes. With objects being in different locations, the direction and magnitude of the motion of each object will vary when the camera movement is outside translational movement. Some objects will appear to move differently to the other stationary background, but the motion is the relative differential
6.5 Discussing Object Characterisation 139
perceived by the camera when moving in a three dimensional plane. Some of this incorrect motion perception is carried over to the second feature set that also uses object size features. The only difference between the two cluster groups is the white object on the side of the road, meaning the disturbed earth and the smaller cracks in the road appear as separate object characterisations because of the camera motion differential, not as a result of sufficient size deviation. The filter to counteract the motion differential in the UAV scenario helps to remove this misconception of motion due to camera movement, and can be explained using an example. In a simple rotational motion the objects in the top left have similar magnitude but different relative rotation (in a 360 degree sense) to objects in the other corners. The filter used reduces the direction range to a single quartile (90 degrees). This is achieved by inverting directions in the opposite quartile (flipping), and offsetting the directions in the adjacent quartiles by either adding or subtracting 90 degrees depending on the quartile. This does not compensate for all variations in three dimensional movements, but it does allow the filtering of rotational variances based on location from the frame centre. Figure 6.7(b) shows an overlay of the UAV frame, with the direction filtering applied to the clusters. The frame only shows objects identified to have different-to-stationary motions, and is not showing clustering differences between these other types. That is because when clustering is applied after the direction filtering, even a slight deviation of an object from another object can cause a separate cluster. This may be useful in some scenarios but it was considered to confuse the point the frame overlay is making.
Chapter 7
Conclusions and Future Work
7.1
Summary of the research
The research developed a real-time detection algorithm in a moving camera environment capable of feature rich object analysis and identification. The direction of the research ended up focussing on the development of the novelty detection algorithms Edge flow and WISE. The result of the research is real-time novelty detection algorithms that detect both static and moving objects in moving or static camera scenarios. As demonstrated in chapter 5, WISE is capable of real-time performance operating in the region of ten frames per second, without the need for a GPU or brute force processing. Due to the lengthy investigation work into the novelty detection aspect, limited progress was made in object identification and analysis. Built into the WISE method is the output of rich features that describe the characteristics of the objects, such as texture gradient, object composition, size and ratio, along with 2D image based velocity components.