Methodologies

Augmented Reality and Computer Vision research directs the focus for this study. Reviews of CV image filter technology, object detection methods, and object tracking processes are critical for formulating new or novel hybrid CV models and processes to solve issues surrounding AR for RAL systems. Each of the three technologies is explained in Chapters 4, 6 and 11 respectively. Models from previous research are formed into software implementations to trial against baseline test images and video streams, measuring their effectiveness and response times. Hybrid and unique solutions to various research questions are uncovered through considered analysis of model response functions and their performance analysis.

Analysis of computer (software) implementations of pre-existing CV models, along with newly discovered and hybrid models, form the majority of this research and are the primary form of the research contributions. Structured development of CV models within a test, validation and verification environment is achieved through the construction of a development console. The AR RAL Development Console is presented in Chapter 1.1Appendix K, which is a C# application, created for this research to test individual new and existing models as well as new hybrid CV models. Functionality of the AR RAL Development Console is shown in Construction of the Augmented Reality Remote Access Laboratory Development Console when detailing the capabilities of the various CV models.

Microsoft’s’ Visual Studio’s 2012 (specifically C# - .NET 4.5) was used to develop all the software implementations of existing and new CV research models. Information on the methods of software implementation are found in Construction of the Augmented Reality Remote Access Laboratory Development Console. It is unsuitable to dedicate a chapter to each of the fifty CV models implemented in the course of this research, however the test methodology is detailed in each chapter, and the test schedules are described and listed in Schedule of Tests.

There are no specific contributions from standalone CV filter models; however, there are pre-processing modules which precede CV image analysis and tracking systems, and are included within the research to provide discussion on their effects when applied to other systems. Assessment of current CV image analysis and tracking models is

necessary to establish baseline measurements for use when assessing new or hybrid CV models. Image Analysis is found in Chapter 6.

Contributions to CV object detection exists with the development of a novel colour histogram object identification and image segmentation method. Applying the new colour histogram model to object tracking has also produced simple yet robust object tracking. Alternative non-segmentation tracking methods, using the newly discovered colour histogram models, are also described, in which statistical histogram comparison techniques are used. Methods implemented to perform attribute comparison and matching within object tracking processes, are also unique to this research. From baseline components, the primary contribution of simple, fast and robust object tracking is achieved, which is suitable for AR within a RAL framework.

4

4Computer Vision Filter Functions for Augmented Reality

Systems

This chapter presents computer vision image filter functions which are suitable, within the augmented reality remote access laboratory environment, for the improvement of image signal-to-noise ratio’s.

Computer Vision aims to derive information within the constituent digital data sets of a video stream. Computer Vision filter functions provide a service to normalise data sets in a consistent manner to ensure that image attributes are durable for post-filter processes. Augmented Reality systems, dependant on the analysis of live video streams to ascertain properties of the displayed environment, require a stable, reliable and speedy CV sub-process to provide real-time interaction with the real and virtual objects within the video stream. Consistency of imagery is of primary concern for all vision dependant AR systems. Alleviating the sources of irregularity within the images (video frames) requires careful consideration of the sources of irregularity, and is a major factor within CV research. The major contribution of this chapter is to determine CV filters which provide improved signal levels necessary to support AR RAL configurations. This involves CV filters which operate within the video stream frame rate, and are suitable for object detection and tracking.

A general understanding of the sources of noise and the consequence they cause within images is an important competence when considering effective visual AR development. Video streams from RAL experiments contend with many natural sources of noise. Image noise contributions appear from two distinct sources; environmental noise and internal noise. Should the various signal deficiencies be ignored, AR processes such as image analysis, object detection and tracking operating inadequately. Failures in the image analysis functions within the AR processes manifest themselves through the user

interface; objects fail to track, virtual objects fail to align with real features, and real and virtual objects fail to synchronise. Such failures from the AR systems destroy any improvements the incorporation of AR was supposed to deliver. This chapter defines the sources of irregularity for images sourced from the live video streams of Remote Access Laboratories, and discovers the appropriate Computer Vision filter functions, both existing and hybrid, to improve the signal-to-noise ratio such that Augmented Reality systems may successfully detect and track objects.

Defining the types and sources of image noise provides the basis for the assessment of current Computer Vision image filter models. The processing resources are considered within Section 4.1, Filter Function Processing Considerations. Image noise sources are defined in Section 4.2, Image Noise Sources, with three primary techniques to address image noise detailed in Section 4.4, Filter Techniques - Statistical Filtering and Section 4.5, Filter Techniques - Colour Filtering. The testing regime and results are presented in Sections 4.6 and 4.7.