Attribute Value Assessment

To function as a foundation for object tracking, Computer Vision image analysis and object detection outputs must consist of two features. Not only is knowledge of the object of interest required, but the knowledge must be measurable. Measurable outputs supply the means to locate and track objects from frame-to-frame. Each Computer

Vision image analysis model supplies a data set representing attributes of the image that subsequent processes may or may not be interested in. When attempting to locate an OoI in successive video frames, the likelihood of locating exact attribute values is remote. A reliable process is required to match discovered attributes from the current windowed search area to the attributes from the prototype.

For this research, it is assumed that there is minimal interframe motion, allowing only a sub-set of an image to be analysed. Utilising selection windows to build up feature points can employ multiple pixel attributes such as corner, edge colour, gradient direction, etc without the need to interrogate an entire image. The variety of attributes available is not a priori and can be discovered in-situ.

9.2.1 Windowing

Based on the histogram segmentation method, and inspired by the mean-shift [228] object tracking model, windowing improves tracking response times and computational costs over existing CV models tested in this research. All tracking methods within this works employ windowing to reduce the computational costs, while maintaining the root tracking mechanism. The Region of Interest (ROI), or tracking mask, represents a rectangle which consists of an area within the image to track. It holds attributes

pertaining to the OoI. During tracking, the tracking mechanism will scan within the search window (see Figure 9-6- Left Image) of the new frame for rectangular regions containing similar attributes from the previous frame. That is, a match of the prototype signature is searched within the search area.

As shown in Figure 9-7, in which the tracking windows have been turned on for illustration, the Region of Interest (ROI) maintains focus on the OoI (Orange rectangle) while the search area (shown in yellow) is used during the search of the next frame. Windowing assumes that an object exhibits only small spatial variations between each frame; therefore, candidate object positions can be located within the search area. Matching the target attributes (whether object attributes are obtained via segmentation, feature point or other tracking models) involves tracking matching algorithms. These are explained below.

9.2.2 Segmentation Methods

Regardless of the image segmentation model, output data will consist of a binary classification. Each pixel will be classified as either foreground or background. Background pixels are ignored by follow-on image processing systems. Three segmentation Computer Vision object detection models are included in this research, each listed below:

• Histogram segmentation (2D Colour Histogram & HSI Histogram) • Clustering (DBSCAN)

• Frame Subtraction

For each segmentation model, binary pixel classification is the resultant output. Like all CV image processing models, there remains outliers to be ignored. From the

resultant output, as shown in Figure 9-8, a hotspot is determined and used for further processing. With the addition of Windowing techniques (see Section 9.2.1, Windowing), attention is focussed so that segmentation and hotspot detection only occurs within the search area. Pixels associated with the selected attributes within the search area are collected to determine the mean distance between them, which results in the discovery of a new centre of mass for the current frame. The ensuing centre of mass is determined to be the new location of the OoI, tracking the object throughout the successive frames.

9.2.3 Parametric Methods

Parametric Computer Vision object detection models provide a unique set of attributes associated with the object of interest. Attributes associated with the individual Computer Vision image analysis or object detection models consist of comparable signal values, ideal for analytical assessments. Output attributes from the following CV models require parametric comparisons from one of the methods listed in the flowing sub-sections:

• Histogram (2D Colour Histogram & HSI Histogram) • Interest Points (Edge & Corner Detectors)

• Feature Points & Territory Signatures • Template

9.2.3. (a) Chi-Squared Tracking

Discrete attribute values for a tracked prototype will very rarely coincide with values within a search area. As previously detailed in Section 4.2, Image Noise Sources, individual pixel values vary considerably from frame-to-frame, affecting matching techniques. Histogram matching creates a tolerance for attribute variations due to accumulation bins sizes accepting a range of attribute values. Matching the prototype’s histogram to histograms of the search area only requires simple chi-squared (𝜒2_{) test}

to validate the goodness-of-fit or homogeneity for a successful match. 9.2.3. (b) Least Sum of Squares Difference

Overcoming attribute value variation may also be moderated through calculating the Sum of the Squares Difference (SSD) for each of the pixel attributes (or feature point), and selecting the smallest value as the closest match to the attributes of the prototype.

Equation 9-2 shown the sum of each attribute for the X-Y coordinates, where the current attribute value for the prototype is 𝑎_𝑝, and 𝑎_𝑠 is the attribute value for the corresponding location within the search area.

The least Sum of Squares Difference overcomes the difficulty in exact matches between the attributes of the prototype and the search area. However, it may perform badly in situations where other methods would indicate a loss of track. The smallest value from the SSD is not always the appropriate match, and caution must be taken when utilised. This is demonstrated in Section 8.3, Operational Parameter Analysis Tracking Trials. Selecting poor object features also leads to poor object tracking. Unless tracking models employ a sufficient number of robust features, recovery from tracking failures will not be possible [129]. Feature point object detection models become robust with a diverse range of characteristics, which improves tracking. Highly accurate tracking is very difficult [116], so selecting effective registration reference points is just as important to successful tracking.

9.3 Summary

Computer Vision object detection attribute matching methods are a critical element within the object tracking process. Attribute matching is responsible for the location of the object within the new video frame. Synchronising knowledge of the attributes from the previous frame to current frame is the primary task. Poor attribute matching is responsible for losing track of the object of interest, loss of knowledge regarding key reference points and the failure of higher level processes such as Augmented Reality Remote Access Laboratory systems.

Classifying object analysis and tracking processes as either segmentation or parametric models has produced two primary methods to match object attributes. Contributions from this research have determined the types of output data to expect from the various CV object detection models, and effective means to measure and compare those values for the purpose of object tracking. Knowledge of the parametric measure methods is a key factor when applying Computer Vision image analysis and object detection

SSD = ∑ attribxy(ap− as) 2