Object analysis and advanced tracking

When novelty detection is complete clustering techniques can be used to identify the individ- ual objects in the screen. Each object will have certain characteristics or behaviour that can help contribute to the classification of the object.

Object identification accounting for occlusion

A significant difficulty in object identification is when two objects move behind each other and become occluded.

Objective

Distinguish objects in an occluded environment

PrR

Mature novelty detection algorithm as defined in section 2.5.1

KPI

1. Separate two objects that are occluded in a simple, sparse scene.

2. Apply to multiple objects in a simple, sparse scene.

3. Distinguish multiple occluded objects in a complex, busy scene

4. Maintain analysis performance of <100ms for a 2 MP frame.

Analysis of object velocity

One of the objectives of detecting objects and novelties in a scene is to derive their behaviour. Analysis of the velocity of the objects is an important feature to be able to determine behaviour.

Objective

Successfully identify the velocity of objects traversing the scene.

PrR

Mature novelty detection algorithm as defined in section 2.5.1

KPI

1. Determine the image / pixel velocity of an object.

2.5 Research Objectives 40

3. Determine the absolute real world velocity of an object.

4. Achieve analysis performance of <10ms for a 2 MP frame.

Assessment of object behaviour

The analysis of the behaviour is a variable concept as objects can have eccentric movements within a scene. The purpose of this is to extract features to enable the classification of objects based on their behaviour.

Objective

Extract features that define the behaviour of an object within a scene.

PrR

Mature velocity model defined in section 2.5.2 Robust classification method available

KPI

1. Analysis of the behaviour of a single object in a simple scene.

2. Analysis of the behaviour of multiple objects in a simple scene.

3. Classify the objects based on extracted behavioural features with a minimum of 80% classification accuracy. This value arrived at from what can be expected from the ground truth of human observation (see table 5.1)

4. Analysis and classification of object behaviour to be within the performance envelope of <20ms per 2 MP frame.

Auto object classification utilising rich feature set

The previous objectives extract rich features from detected novelties and objects. This feature set can to enable improved autonomous classification of objects that are visually similar, but have distinctly different behavioural patterns.

Objective

Autonomously classify objects within a scene in real time utilising the advanced feature sets

PrR

Rich feature sets available in a mature state

2.5 Research Objectives 41

1. Separate classification of two similar objects that have different behaviours.

2. Classification of multiple objects within a scene based on behaviour with a minimum of 80% classification accuracy.

3. Achieve classification online, within a performance envelope of <10ms for a 2 MP frame.

Classification of objects based on dynamic change in shape or size

Some objects exhibit dynamic changes in size or shape (despite being the same object), either due to activity or change in camera perspective. Detecting this change proves to be challenging, even for the human visual system [120], the difficulty is recognising the object as being the same as a previously seen object despite some dimensional change. There is scope to investigate using the dimensional variance as a separate feature set for behaviour, and classify based on object shape / size variance.

Objective

Autonomously classify objects within a scene based on features derived from object change.

PrR

Mature novelty detection technique as defined in section 2.5.1

KPI

1. Re-classify a single object in a simple scene as the same object after changing its physical dimensions.

2. Re-classification of the object in a complex scene

Chapter 3

Methodology and Initial Approach

3.1

Methodology

In order to address the research questions and hypotheses proposed in 2, the approach will initially explore existing research and the limitations on the capability. Because the work is focussed on video streams, the research that focuses on analysing video streams will be used. Despite reviewing the work in 2, this exploration will underpin why the limitations of these techniques exist; operation in moving camera scenarios or why only moving objects are detected. Each technique will be applied to a series of videos, and the results of the detections analysed. The analysis is both objective and subjective - the objective results are the number of objects detected. Subjectivity comes in when determining what a detection is; does it represent an object or is it a representation of noise. Measures of accuracy against processing speed will be made such that an appreciation of an algorithms real-time capability can be made. The plan of the research, once this assessment is made, is to draw parallels with the way human eyes work. This is because human eyes work in real-time and are excellent at detecting and discriminating between static and moving objects. By drawing parallels, the adaptation or development of a new novel approach to object detection should be possible. The new approach that is developed will be compared and contrasted with a wide ranging set of existing methods which operate in single frame analysis or video stream analysis. The reason for comparing the both single frame analysis and video stream analysis methods is the detection capability is generally better in a single image detection method where as the video stream analysis maintains a temporal (and therefore) motion component. The research aim, as stated in the hypotheses, is to achieve similar or better performance in terms of single frame detection whilst maintaining the characteristics of a temporal video stream in order to establish object motion. The results will be analysed against a somewhat subjective outcome. Each test image or video sequence will have a ground truth established