clustering method separates the large motion differences from the stationary objects, and creates separate clusters to illustrate this, figure 6.2. This graph shows all the samples over all the frames (dark blue points), green points represent the active samples for the frame, and the coloured circles represent the micro-clusters that make up a cluster (clusters of the same colour are the same cluster). The samples over the entire sequence are shown such that over each of the frames here it can be seen where the samples are overall, and the evolution of clusters as active samples influence the clustering. There are two outlier micro clusters with the moving person, and the stationary grass creates separate clusters along the x-axis. Clusters close to the x-axis are likely to be due to camera shake because the magnitude of motion due to shake is small, but the angle of motion will be in different directions; if it is rotational shake top left objects will appear to move in a different direction to bottom right pixels. There are a number of active samples that are not clustered (green). They are not clustered because the density is insufficient to create a new micro-cluster. A possible explanation is that whilst the person was running quickly (high value on y-axis) the micro clusters were formed, but as the person is slowing down (lower values on the y-axis) new samples are yet to be incorporated until the person remains for a few frames at this slower speed.
6.1.3
Discussing the CEDAS Method
The test results from using CEDAS shows that an adaptive separation of object movement is possible. As the person moves around the scene the clusters update and mostly stay as a cohesive cluster - but updating as the motion of the person changes. This has the benefit of being able to maintain objects as being the same object over a series of frames as opposed to just between two consecutive frames. In this particular test, motion magnitude (y-axis) and motion direction (x-axis) were the features used. If there were two objects moving with the same motion pattern but in different spatial locations, this method would have clustered them together, which is undesirable unless we just want to characterise motion and not separate the objects. By adding spatial location information to the feature set of the clustering it should be possible to separate out the two objects.
6.2
Characterisation of objects
The aim of object characterisation is to apply a type to each object detected, based on its feature set. This is achieved through the use of clustering. The objects that are detected by WISE have a rich feature set that could be used to discriminate and assign a type to each of
6.2 Characterisation of objects 134
the objects. A large feature set implies that there can be greater division (number of clusters) between objects, due to the increase in variance higher dimensions bring.
6.2.1
Available Features for Object Characterisation
The output from the WISE algorithm has several different features which provide a detailed description of candidate objects, the internal texture and the perceived motion. There are also features that can be mathematically derived to provide more dimensions to improve object type separation (where needed). The following features have been extracted from the detected objects:
• Length - A pixel-wise measure of the objects length. This is in the 2-dimensional perspective of the camera
• Width - A pixel-wise measure of the objects width. This is in the 2-dimensional perspective of the camera
• Area - A pixel-wise measure of the objects area. This is in the 2-dimensional perspec- tive of the camera
• Number of pixels - The number of pixels that constitute the object.
• Size ratio - The height - width ratio of the object, in the 2 dimensional perspective of the camera. Used in conjunction with motion, this could be used to derive 3-dimensional size.
• Motion magnitude (pixels) - A measure of the optical flow magnitude for the object.
• Motion direction - A 2-dimensional orientation for the optical flow of an object. A 3-dimensional orientation could be achieved through the use of homography, similar to that in 3
• Mean edge gradient - the mean gradient value constituting the perimeter of the object
• Standard deviation of edge gradient - the standard deviation of the perimeter gradient of the object
• Mean colour (RGB) - the mean colour of the pixels constituting the object
• Standard deviation colour (RGB) - the standard deviation of the pixels constituting the object
6.2 Characterisation of objects 135
• Spatial location (x, y) - the location of the object in the frame. This location is the centre of the object, defined by the intersection of diagonal lines from the maximum and minimum x,y coordinates of the object.
6.2.2
Clustering of features
In order to establish separation between object types, some method of clustering the object features is required. If all the features are clustered it is likely that every object that has been detected will be determined as a different type (similar objects will likely have a different background). This means the features being clustered need to be pre-selected based on some characterisation requirement based on operator or user interest. For example, a user may be looking for small regular sized objects and thus the clustering could be performed on the length, width, area and size ratio features. The clustering method used needs to be real-time, and parameter free such that the selection of cluster parameters does not influence the characterisation. An example of undesirable parameter selection could be cluster radius or number of expected clusters because the spread of the data or the number of object types in a scene is unknown. This removes some clustering techniques from consideration such as k-means and fuzzy c-means which both use parameter selection to define the number of clusters expected. Subtractive clustering and similar derivatives are not real-time and also use a cluster radius parameter. Evolving c-means clustering is data order dependant which is undesirable in an unknown environment, and mean-shift clustering is also dependant on known data. The data is clustered on a frame by frame basis, so the technique used does not have to be adaptive or on-line; all the data samples that need clustering will be available at the time of clustering. On-line clustering methods typically require samples to arrive sequentially, and with several objects and pixels to analyse, this can slow the processing down. Therefore a clustering method that processes the entire frame of samples as a batch is required. A new density based clustering technique named DDC is capable of clustering in real-time and does so in a batch manner. It requires an initial parameter of cluster radius but the radius adapts based on the data distribution, and therefore this initial parameter is not as limiting as previously suggested.
6.2.3
Test videos
The results here are testing the capability of DDC applied to some of the features of objects extracted by WISE.
Helicopter Video
6.2 Characterisation of objects 136
interference or complexity. It also has a large variation of sizes in objects in order to test the characterisation using size based features. Additionally this video sequence has been used in previous tests and will provide some level of comparison with previous steps in the chain.
Fig. 6.4 Helicopter police chase video
UAV VideoThis video is a UAV video with multiple axis of motion. This video has an extremely small moving aircraft, along with other some moving objects originating from the ground such as smoke and cars. This video was used to test the discrimination in a complex moving environment, with objects of interest that are extremely difficult detect. The original image is in greyscale, and this will also test the performance of the WISE and characterisation algorithms on non-RGB frames.