This section will briefly describe the content and motivation of each chapter. For a visual, one page summary of the various sub-projects involved in this thesis, and how they fit together, refer to the flow chart inFigure 1.6on the next page.
Chapter 1 This introduction gives an outline of the main motivations of this research and gives a brief description on some of the core aspects. It also lays out the primary contributions of the thesis and provides a guide to the contents of each chapter.
1.4 thesis outline 16
ibeo LUX LIDAR Bumblebee 2 Camera
Gray world color correction. Section 5.1 Interpolation of projected points. Section 3.7 Data fusion, producing enchanced depth map.Section 3.7 Transform 3D data to image plane. Section 3.1
Point Gray disparity algorithm. Convert to grayscale. Section 5.3 Convert to HSV colour space. Section 5.3 Depth map segmentation. Section 5.2.2
Image edge detection, cleaning and segmentation. Section 5.2.1 EDISON segmentation. Section 5.2.3 Extract segment point clouds. Section 5.5 Perform plane fitting.Section 5.5
Extract hue and saturation data. Section 5.4 Perform GLCM analysis. Section 5.4 SVM classification. Section 6.2.1 Combined segmentation. Section 5.2.4
Raw 3D scan data Rectified left,right images
Left image only
Neural network classification. Section 6.2.2 K-Means Clustering. Chapter 7 All segment data merged
into description vector
Hardware details. Chapter 4
Figure 1.6: This flowchart maps the most significant steps in the data processing from raw inputs to segment description vectors. Section links are provided for each step for quick navigation. Blue boxes relate toLIDARdata collection and alignment. Red boxes relate to image processing, segmentation and analysis. Green boxes relate to using the data for classification and the purple box relates to the novel unsupervised method.
1.4 thesis outline 17
Chapter 2 The literature review gives specific details on the work of other authors which has formed some part of the foundation for this research. This includes meth- ods which have been incorporated into this work in either modified or unmodified forms and the conclusions of others which have influenced implementation de- cisions during this work. Of particular importance is the theory of depth from stereo, sensor calibration for sensor fusion, machine learning and previous works on semantic labelling.
Chapter 3 This chapter presents a novel method for determining both intrinsic and extrinsic calibration parameters required for accurately projectingLIDARscans
onto the image captured by the camera. The method described here was also published inMeasurement Science and Technology[139]. This necessary step allows
the stereo camera and theLIDARto be used together to produce a more accurate
depth map of the scene. The sensor fusion technique that merges the two data sources is presented along with examples and a short evaluation of the accuracy.
Chapter 4 This chapter on data collection first describes the hardware used for the experiments including some devices which were evaluated and found to be un- suitable for this application. Secondly some of the challenges encountered during implementation are discussed and how they were overcome. Finally a brief descrip- tion of each of the datasets created is given and some samples to give an idea of their contents. The datasets cover the three primary road environments, motorway, rural and urban settings.
Chapter 5 This chapter details how images are pre-processed, segmented and analysed. There are several steps to the segmentation using both image and depth data. A hybrid implementation is presented using the strengths of each segmentation method to get the most effective result. Each type of segment feature, which will be later used for learning, is also described including how it is calculated. Finally
1.4 thesis outline 18
an investigation into the information provided by each metric is presented using a Principal Component Analysis (PCA).
Chapter 6 This chapter presents the first experiment which focuses on using super- vised learning for segment classification. It details how data is hand tagged to provide learning targets, how training and testing sets were chosen and provides details of data manipulation. It then gives a comparison of different learning meth- ods includingSVMand neural networks before evaluating the relative performance.
Chapter 7 This chapter presents the second experiment using unsupervised learning to attempt to automatically cluster the segments into similar groups. Details on clustering methods are given followed by a performance evaluation using the hand tagged data.
Chapter 8 The final chapter will sum up the main conclusions and findings in each of the preceding chapters. There will also be recommendations for future work to address any limitations that were encountered during this research.
Appendix A This appendix contains some potentially useful snippets of Matlab code which can be useful to others as stand alone functions without heavy dependence on external data, other custom functions or environment configuration. It also has a description of the included result video, giving some details on how it is made and the purpose of each section. The video can be found on the included CD or online at the address given inSection 6.3on page177.