Navigation Device by a Camera-based Landmark Detection Approach
Master Thesis
Submitted in Fulfilment of the Requirements for the Academic Degree
M.Sc.
Dept. of Computer Science Chair of Computer Engineering
Submitted by: Kashif Rashid Jumani Student ID: 385965
Date: 07.06.2017
Supervising tutor: Prof. Dr. W. Hardt
Supervising (External): Mr. Thomas Foerster University Advisor: Mr. Michael Nagler
The main objective of this paper is to discuss a platform which can provide accurate information to moving objects like a car in poor environmental conditions where the use of signals of GPS is not possible. This approach is going to integrate imag- ing sensor data into an inertial navigation system. Navigation systems are getting smart and accurate but still, an error occurs at long distances causing a failure to find out accurate location. In order to increase the accuracy front camera in a car is proposed as a sensor for the navigation system. Before this problem is solved with the help of extended Kalman filter but still, the small error occurs. In order to find out, accurate location landmarks will be detected from the real-time envi- ronment and will be matched with the landmarks which are already stored database.
Detection is the challenge in an open environment in which object must be illu- mination invariant, pose invariant and scale invariant. Selection between algorithms according to the requirement is important. SIFT is a feature descriptor which creates the description of features in an image and known as the more accurate algorithm.
Speeded up robust features (SURF) is another algorithm in computer considered as fast and less accurate than SIFT. Most of the time it is not a problem with given algorithms but the feature is not detected or matched because of illumination, scale, and pose. In this condition use of filters and other techniques is important for better results. Better results mean required information from images must extract easily, this part is obtained with the help of computer vision and image processing.
After creating matched images data, this data is given to navigation data calcu- lation so that it can produce an exact location based on matched images and time calculation. Navigation data calculation unit has the connection with Landmark Database so navigation system can compute that at this point landmark is present and it is matched and assure that given location is accurate. In this way accuracy, safety and security can be assured.
Keywords: Computer vision, machine learning, image processing, nav- igation systems, OpenCV, C++, distance measurement.
Contents . . . 3
List of Figures . . . 5
List of Tables . . . 7
List of Abbreviations . . . 8
1 Introduction . . . 9
1.1 Motivation . . . 10
1.2 Navigation systems . . . 11
1.2.1 Categories of Navigation systems . . . 12
1.2.2 Dead Reckoning . . . 12
1.2.3 External signal based Navigation . . . 13
1.2.4 Mapping based Navigation . . . 14
1.3 Traffic sign detection . . . 14
1.4 Need of camera based navigation system . . . 17
1.5 Structure of thesis document . . . 19
1.5.1 Introduction . . . 19
1.5.2 State of Art section . . . 19
1.5.3 Concept section . . . 20
1.5.4 Implementation section . . . 20
1.5.5 Results section . . . 21
1.5.6 Conclusion . . . 21
2 State of Art . . . 22
2.1 Diversity of navigation systems based on working Principle . . . 22
2.1.1 Inertial Navigation System . . . 22
2.1.2 Global Positioning Systems . . . 27
2.1.3 Automotive Navigation System . . . 31
2.2 Different algorithms for Feature detection and matching . . . 32
2.2.1 HAAR like feature . . . 34
2.2.2 Scale-invariant feature transform (SIFT) . . . 36
2.2.3 Speeded-up robust features (SURF) . . . 39
2.3 Camera based distance measurement . . . 41
2.3.1 Distance measurement based on one camera . . . 42
2.3.2 Distance measurement through stereo camera . . . 44
3 Explanation of concept . . . 48
3.1 Imaginary sensor as input sensor . . . 49
3.2 Feature detection and matching . . . 50
3.2.1 Feature matching: . . . 51
3.2.2 Feature description: . . . 52
3.2.3 Feature recognition: . . . 53
3.2.4 Feature alignment: . . . 54
3.3 Creation of datasets . . . 55
3.3.1 Saliency of an object . . . 55
3.3.2 Repeat-ability of an object . . . 56
3.3.3 Absence of an object in negative images . . . 57
3.4 Measurement of distance under this project . . . 58
3.5 Integration of data with navigation system . . . 59
3.6 Scientific challenges . . . 59
3.6.1 Positioning sign at different scales . . . 60
3.6.2 Illumination Invariant . . . 61
3.6.3 Scale Invariant . . . 63
3.6.4 Rotation Invariant . . . 64
3.6.5 Assigning coordinates to positioning signs at real position . . . 65
4 Implementation of Camera based navigation systems . . . 67
4.1 Steps for development of HAAR cascade classifier . . . 68
4.1.1 Management of positive and negative images: . . . 69
4.1.2 Samples creation by using OpenCV utility . . . 70
4.1.3 Creation of object information from positive images . . . 71
4.1.4 HAAR Training . . . 72
4.1.5 Creation of XML file . . . 72
4.2 Distance Measurement approach . . . 73
4.3 Creation of database (XML File) . . . 74
4.4 Integration of whole Data . . . 76
5 Results . . . 79
5.1 Object detection based on HAAR cascade classification . . . 79
5.1.1 Illumination invariant: . . . 80
5.1.2 Pose invariant: . . . 81
5.1.3 Scale invariant: . . . 81
5.2 Distance measurement with single camera . . . 83
5.3 Distance measurement at defined location: . . . 84
6 Conclusion . . . 85
Bibliography . . . 89
1.1 Display for detected positioning signs . . . 11
1.2 Traffic sign assistance [4] . . . 17
2.1 Block diagram of INS . . . 23
2.2 Inertial measurement unit . . . 24
2.3 General accelerometer structure [31] . . . 25
2.4 Basic construction of Gyroscope [12] . . . 26
2.5 MEMS gyroscopes. . . 27
2.6 Three segments of GPS [10]. . . 28
2.7 Signal Synchronization of GPS. . . 29
2.8 Example of traffic signs [12] . . . 33
2.9 HAAR like features [36] . . . 35
2.10 Stages of cascade classifier [42] . . . 36
2.11 Extraction of interesting patches [9] . . . 37
2.12 Effect of Low pass filters [9]. . . 38
2.13 Example of BLOB detection at maxima. [46] . . . 40
2.14 Distance measurement system by Tao et al. [44] . . . 42
2.15 Creation of reference system [24] . . . 43
2.16 Image stitching for relation between two images. . . 45
2.17 Distance measurement to an object with two cameras [30]. . . 46
3.1 Example of feature matching . . . 50
3.2 Example of Window function [46] . . . 51
3.3 Principal Component Analysis . . . 52
3.4 RANSAC Approach . . . 54
3.5 Set of points . . . 54
3.6 Contrast Example . . . 55
3.7 Object contrast efficiently . . . 55
3.8 Effect of Illumination on an object . . . 56
3.9 Effect of pose on an object . . . 57
3.10 Example of positive image . . . 57
3.11 Example of negative image . . . 57
3.12 Capturing of samples with camera. . . 60
3.13 Control of focal length and aperture. . . 61
3.14 Simple descriptor at pixel level. . . 62
3.15 Characteristic scale of a positioning sign. . . 63
3.16 Example of Rotation invariant corner. . . 64
3.17 Lines of latitude and Longitude [18]. . . 65
3.18 Location of signs by a GPS device. . . 66
4.1 Concept for implementation of system. . . 68
4.2 Text file for negative images. . . 69
4.3 Working of Object Marker. . . 71
4.4 Example of XML file. . . 75
4.5 Integration of basic data to Navigation. . . 76
4.6 Integration of basic data to Navigation. . . 77
5.1 Positioning sign at different light intensities. . . 80
5.2 Positioning sign from different poses. . . 81
5.3 Positioning sign detection from different scales. . . 82
5.4 Actual real time distance. . . 83
5.5 Distance measurements. . . 83
5.6 Distance measurement of a positioning sign at specific location. . . . 84
2.1 Types of navigation systems . . . 22
2.2 Applications using feature matching . . . 32
2.3 Names of algorithms for detection and matching. . . 34
2.4 Approaches for distance measurements. . . 41
4.1 Creation of important parameters in our project. . . 67
5.1 Important factors for considerations. . . 80
IMU Inertial Measurement Unit INS Inertial Navigation Systems MEMS Micro Electro-Mechanical
Systems
XML Extensible markup language GPS Global Positioning systems DGPS Differential Global Positioning
systems
RGB Red Green Blue
ADAS Advanced Driving Assistance System
GRPS Geo-Referenced Positioning Signs
DoG Difference of Gaussian LoG Laplacian of Gaussian
SIFT Scale-invariant feature transform SURF Speeded-up robust features GLOH Gradient location-orientation
histogram
HOG Histogram of Oriented Gradients PCA Principal Component Analysis
This Master Thesis project is done under the supervision of Preh Car Connect GmbH and TU Chemnitz, in which possibilities are checked for increasing the po- sition precision of a navigation device by a camera-based landmark detection ap- proach. Positioning signs are considered as landmarks. In order to increase the ac- curacy computer vision, machine learning and distance measurement with the help of imaginary sensor is used. By using these three studies, important inputs are gen- erated which can be used and can be extended to further programs which can result in great help for users of navigation systems. This data can be used for creating a hazard for the driver when any sign is missed to check by him. It is possible by using object detection and recognition. The output of the system is Traffic sign detection, its position in real world and distance between sign and car through a single camera.
As long as detection is concerned it is done with help of matching learning re- sults in an XML file called HAAR like feature. This file is trained under this project for each positioning sign. Later on, it is compared with other detection methods and results in fast and accurate detection than other methods. During designing a HAAR cascade own datasets are used but only for negative images in which object is not present, such images are also taken from other datasets available on the internet.
For distance measurement single camera is used, which is providing sufficient re- sults but not accurate as required for real-time challenges. This system has some limitations which are discussed in this document with their further possibilities to solutions. detection and distance are used to reduce the error produced by dead reckoning and in a situation in which signally is not present like in the tunnel or between tall buildings. In such kind of condition, an input is required to the system which can result in an accurate location in absence of such kind of systems like INS and GPS. More accurate results are possible by combining GPS, INS and imaginary sensor data. As far as distance measurement is concerned, it is required to reduce the distance error by providing measuring it.
This data is created by different sensors, then a system is implemented which can compute positioning signs, distance, location and create hazards. Such type of warning can reduce the number of accidents. This system can be extended in order to facilitate ADAS functions as well as provide advanced functions for navigation systems.
1.1 Motivation
Preh Car Connect GmbH is a long-term supplier of car infotainment, vehicle net- working and telematics for the automotive industries. It creating the solutions through its R&D center in Dresden Germany. Preh Car Connect GmbH also has other development sites in China and USA so that it can overcome current market trend. On the other hand, production is done in Dippach (Thringen) and Oborniki (Poland). As the result of it, their systems are used in more than 8 million vehicles from renowned automobile manufacturers. In Preh Car Connect GmbH, more than 1200 employees are working at 8 different worldwide locations. More development in San Carlos in the San Francisco Bay Area and in the Shanghai China.
In order know the get a person from point A and point B by assuring safety is a goal.
It is possible by mobile control based on features in live routes with imaginary data called real-time data and providing all current information about weather, available routes upon request. Providing all this information is no longer a future concept.
Advanced technologies are in the market these days, companies are searching for more advancements and this thesis project is going to provide one more possibility for increasing advance options in the systems of Preh Car Connect GmbH.
Navigation Systems based on landmark detection can provide more position ac- curacy, quick guidance, warning messages, sign display options, safety assurance, and this data future can be used to extend the system. Data produced by this sys- tem is basic requirement for the ADAS as well, so that environment can be perceived The first part of the motivation is accuracy problem even after a long time and it is part of interest to remove this error and make the system more accurate by different approaches. One approach is discussed to overcome this problem under this document in order to create a system which can be proved as an efficient system for safety as well as for position accuracy. Many studies are conducted in order to in- crease the accuracy by different scientific work which will be discussed and explained according to their system’s art of designs. Most of the factors behind the error in accuracy are mentioned so that it can create an easiness to find an alternative ap- proach. For increasing the accuracy of navigation by adding imaginary sensor data in existing navigation can result in more accuracy.
The second part of the motivation is, according to a research, Due to road acci- dents 1.3 million causalities takes place every year [1]. In order to avoid accidents, more safety is required. As systems are getting more efficient and working auto- matically, in this case, assurance of safety is required. Old navigation systems can be extended as the intention is changing according to increasing requirements. So, these systems can be extended by connecting more technologies with old systems.
Car companies are looking for new techniques and technologies in order to assure accuracy, efficiency and reasonable cost for making things easier. With the help of
this system, important information on the track cannot be missed by the eye of a camera. Detected signs will be set on display as shown in figure 1.1:
Figure 1.1: Display for detected positioning signs
Social reasons are not only the reason behind the modification of navigation systems.
Modifications take places in order to cover latest technologies, as per requirement of the more efficient system, for increased safety and at the end new functions to perform. Day by day navigation systems is getting smarter and cost-efficient for use.
1.2 Navigation systems
The word navigation is used to determine position as well as velocity with the help of mass centered in a moving vehicle. Current navigation systems are considered as the integrated collection of position and orientation sensors (Gyros and Accelerometers) which are taking part in computing and communication hardware results as a great help for the movement of people, vehicle and for other moving objects to reach their destination. The main goal of navigation is determining accurate position and measurement of the distance that is traveled. Navigation can be land-based or space-based in which it helps an object to reach from point A to point B in space.
Our focus is the use of navigation system in automobiles that is beginning from the 20th century. From that time these devices are capable of computing current location of an object and continuously updating the position. The navigation system is a combination of hardware and software in order to facilitate automatic position determination for users. Basically, It is a derivation of a state vector with the help of six components of position and velocity, so it is called six component state
vector [22]. There are different kinds of navigation systems working with different techniques:
1.2.1 Categories of Navigation systems
1. Determination of position and orientation without using external resources, this technique is called Dead Reckoning.
2. Determination of position and orientation by using external resources, this technique is called External signal based Navigation.
3. It is working by recognizing the images then used to compare its observations to a stored database in order to find location called Mapping based navigation systems.(Concept)
1.2.2 Dead Reckoning
For simple understanding, The navigation system based on dead reckoning is used to compute orientation, speed, heading and resolve speed in navigation coordinates, this data is further integrated to obtain a position. The first heading sensor is magnetic compass also known as (Flux gate). Another heading sensor that is more complex and current approach in the field of navigation systems. It is considered as a stand-alone unit that is used to determine position and orientation without using external resources. At this level, a simple problem to understand is that we need some reference to point out a starting point. When the starting point is known, the direction in which object is moving is computed as well as with how much speed we are traveling in this direction. So, By computing, such parameters read reckoning technique is able to determine our position in the space. By reading it seems easy but it is considered as a challenging task in order to compute accurate location. In early stages, the direction was determined by the magnetic compass and distance traveled was predicted by the time of travel with respect to speed. But these days different sensors are used to measure direction and orientation. Such kind of sen- sors are magnetic compasses, transmission pick sensors, accelerometers, differential odometer and the important one is gyroscopes [41]. This approach is used in the Inertial navigation system and this technique is widely used in different applications.
First inertial guidance was used by German V2 rockets in 1942. After that time it is used in many applications. Many new applications using Inertial Navigation Sys- tems are appeared in many domains like in automotive (suspension and navigation systems), Industries and robotics (automation). In order to know about the variety of applications, few examples are given as follows:
Ships inertial navigation systems:
This is demanding application enables navigation through Inertial Navigation Sys- tems in naval ships and submarines. It enables the ships and submarines to find
their location without using any external sources (Satellites and Stations). It is increasing the safety chances in the harsh situation which is created because of com- munication loss, data errors and external sources. Inertial Navigation System in this domain is specially designed for purposes like it makes the location private as it is a stand-alone unit but the chances are the only possibility of finding location by opponent source through Radar and second purpose is it helps to cover long distance navigation even in polar regions and oceans where communication with external sources is not possible [49].
Attitude and heading reference systems:
This is the type of Inertial Systems, which is used to provide angular information.
For finding angular information, it uses lower cost and lower performance inertial sensors. Angular information in these systems is determined not through an external source. In this case, it is again stand-alone unit which can find angular information by its Inertial Measurement Unit in which inertial sensors are used.
In order to get more navigational data, this system can be connected to another system like Radar. It can provide information to other systems those are providing an environment for a complete navigation system and can be proved as cost effective [49].
1.2.3 External signal based Navigation
For precise understanding, Navigation systems determining position with the help of external sources like GPS is considered under this category. It is determining the position with the help of a network of 24 Satellites which are controlled and monitored by many ground stations. Such kind of navigation systems is often called Satellite-based navigation systems. Navigation depending on satellite signals, solv- ing the problem of positioning with the help of signals from multiple satellites.
In case of GPS (NAVSTAR), determine fix position in 3D environmental require 4 signals from satellites. Accurate position depends on the relative position predicted by multiple satellites. It uses 1575.42 MHz and 1227.6 MHz radio frequencies. This type of GPS consists of 2 types of signals which are separated because of security reasons as well as performance such types are divided because for signals which can be used for civil use and military use. Use of military requires more accuracy so it provides more accuracy and efficiency to the military as compared to civil use.
DGPS (Differential Global Positioning systems) are introduced to increasing the accuracy of GPS system for present positions at a location and transmitting signals to GPS receivers. This program is introduced by the WAAS (Wide Area Augmen- tation System) that is known as the large group of DGPS stations(around 25 to 50) specially designed for increasing the accuracy of GPS in US areas. WAAS improving the accuracy by error correction caused by obstructions between the signal source
and GPS receivers [22].
1.2.4 Mapping based Navigation
Mapping based navigation systems are involved in such processes which are map- ping the output given by positioning systems (GPS) called map matching. It is a mapping between digital map and vehicle location given by positioning system.
It depends on the accuracy of a digital map as well as vehicle location, in case of accurate data simple algorithms can be used based on data provided by digital map and vehicle position. But in many cases, the situation is more complex and complex map matching algorithms are used involved more sources, parameters, and data [35].
To enhance the accuracy of positioning system can be done with the help of a good map matching algorithm. Mapping is really important as there is the possibility of a big network of roads towards our destination, in order to identify the path according to our requirements, Mapping helps us to do that. It is possible by the relation of the trajectory of a vehicle and the digital information in the form of a digital map.
Correlation of trajectory of a vehicle is possible by an input from various sensors involved in measuring the vehicle trajectory with the available digital information as a map(database). Under this method, vehicle presence is continuously compared with the available routes in the routes network. So, vehicle position is determined with the help of map through pattern recognition and matching. The database con- sists of road coordinates and matched locations can use which results in limiting the magnitude of errors [52].
As given above, Map Matching is processed in which vehicle path is interacting with digital road information in the form of Map which means find the location of a vehicle with reference of this digital information. Map matching is based on soft- ware technology that is involving the use of digital information extensively.There are some rules and must be kept in consideration in order to avoid errors. The accuracy of the digital map is important, that is used for matching if it is not accurate which can result in the inaccurate output. The performance of the whole system is affected [25].
1.3 Traffic sign detection
Tools working for artificial intelligence are a point of interest which can be used for traffic sign recognition with the help of computer vision techniques and can be inte- grated with navigation systems. This system can be extended for more applications in automotive industry. The main advantage of this system will be safe driving and position accuracy leads to destination without any trouble. Main components of this system will be camera integrated with computer vision techniques and image processing as well as navigation unit for position accuracy.
As traffic signs are present on roads which can be used for creating an assistance system which can alert drivers of hazards ahead in case driver missed it. This traf- fic sign detection system providing information from real-time environment about current road condition and create warning sign. In this work, traffic sign detection is linked with a navigation system in order to avoid accidents. Many possible tech- niques will be discussed in this report in order to achieve efficient results of detection as well as for positioning. The solution will be given for facing some challenges like illumination, scale, and pose invariant solution. Different techniques will be applied and tested so that obtaining results can be checked and decide which result is better and why.
In my case, the challenging task is to collect images for creating a dataset in order to make classifier. Plenty of images is required so that HAAR cascade can be created with the help of positive images and negative images. Positive images are in which object is present and negative images are in which object is not present. With the help of testing car with a front camera by Preh Car Connect GmbH so that dataset can be created. This challenge consists of two parts, in the first part given detection through image frame captured by car camera and recognize the sign in the various environmental conditions.
Under this Master thesis project, various methods are studied in order find an efficient way for detection, Results of different approaches are analyzed so that suc- cessful and efficient road sign detection can be carried out. Few points are considered during making an application for traffic sign detection:
1. Different research papers are studied for traffic sign detection.
2. Image processing methods are applied in order to remove illumination.
3. Existing available methods are also considered for road sign detection.
4. Analyze and design machine learning based traffic sign for proposed system.
5. Implementation of efficient algorithm for detection.
After performing study, It is came into knowledge that detection is possible by various methods based on some factors like color and shape:
1. Model based on Gray scale road sign detection.
2. Model based on Adaboost.
3. Model based on classification.
Gray scale road sign detection:
Under this section, there are two field which are considered and differentiated from
each other based on performance and speed of detection. No doubt color-based de- tection approach helps us to remove false results during recognition process but it is slow because it requires more processing and color based approach is done with the help of segmentation by threshold given in color spaces like RGB and others [23, 39, 14].
Another approach is Neural networks that are used for image filtering and sign recognition which comes under Ghica research [13]. The geometric approach is used for gray image in order to recognition and detection. Most common transformation is Hough transforms.
Model based on Adaboost:
It is known as a general framework to combine classifiers and comes under super- vised learning used for specific pattern recognition problem [15]. Basic steps which important under this approach are:
1. Select a classifier that is weak in nature for a specific problem.
2. Modify the states where the classifier fails.
3. Combine the classifier with other classifier performing same steps.
It selects a weak classifier, which can call feature selection where each component is an image pixels values. In this case, weak classifiers are providing recognition of patterns [47].
Model based on Classification:
There are three important things which vehicle have to detect on the road considered as important:
1. Lane detection.
2. Obstacle detection.
3. Traffic Signs Recognition.
Sign recognition is a problem which requires studies which require studies because it includes color-based as well as shape-based approaches which can result in clas- sification. It provides drivers a very useful information about the roads which can generate warnings in order to avoid accidents appears as a safe and easy system.
As we can predict future of autonomous cars, their functionality can be dependent on the traffic sign detection and their classification. It is mainly possible with two types of technologies, The first possibility is traffic sign detection based on the color threshold for segmentation of an image. The second possibility is with the help of neural networks. Neural networks are hard to implement but results are really impressive.
1.4 Need of camera based navigation system
Mainly there are many threads require the need of Camera-based navigation in order to assure security as well as safety. As mentioned, Many reasons require navigation systems but few reasons are taken really serious like:
1. Navigation consistency in absence of GPS signals.
2. Navigation consistency during INS long-distance error.
3. Independent camera-based navigation system.
The data is very important for driving with a safety like information about the road ahead, back view information, traffic signs for the speed limit, turning curves infor- mation and important places ahead. In such kind of situation, the importance of camera-based navigation is increasing. Another reason is speed limit information, during driving the speed limit to be detected through the camera and provide this information on navigation screen so that violation of speed limit can be avoided. see figure # 1.2 for details in which speed limit is detected with the help of a camera and this data can be used for assistance. Like this other kind of signs can be detected can be processed.
Figure 1.2: Traffic sign assistance [4]
Navigation consistency in absence of GPS signals:
This is the main concept behind this thesis project when there is no any input signal in some circumstances, Camera-based navigation system has to provide input with
the help of the camera. It is a situation like in tunnels or behind tall buildings where there is no any direct access to satellites to GPS receivers in order to provide the location. In such kind of situations, Computer vision navigation system can bring accuracy in case of no input signal by GPS.
Navigation consistency during INS long-distance error:
It is a problem in Inertial navigation systems, which have long distance error and error because of temperature drift. This problem can be solved with the help of camera-based navigation system. Distance measured with the camera can decrease the error in the position of INS. INS itself is a stand-alone unit but it requires some other unit in order to solve this long distance error.
Independent camera-based navigation system:
It is a navigation system using an imaginary sensor in order to achieve an accurate position and helpful guidance to users. It can be considered as an additional unit for current navigation systems. For starting position GPS is used in order to inform the system about starting point from where an object is starting to move. Now there are two possibilities for continuous measurement of position in order to find location.
In the first case, through GPS, in this case, GPS receiver is getting signals contin- uously and updating location every time. In the second case, with the help of INS measuring the position with the help of IMU which consists of sensors. these sensors are measuring the position in order to provide location continuously. Continuous use of INS creating error after covering the long distance. So in this condition, this problem can be handled with the help of imaginary sensor by detecting a known object and some distance measurement approaches.
It can be proved as an essential part for autonomous cars. The basic requirement for such kind of navigation systems is high accuracy which can help an autonomous car in order to itself in the environment without any single error. This error can result as a big problem. In order to get high accuracy, standard dead reckoning approach and imaginary sensor data can be integrated into the system called camera-based navigation system. Few other things are also considered in order to make navigation system operations such as, the location of positioning signs in order to computer de- sired accuracy in the particular area of operation. detection of positioning signs and their validation comes into account at the imaginary sensor level by using a normal existing camera. Many positioning signs must be trained for the detection part and their location must be known because, with the help of positioning signs, high accu- racy can be achieved by camera-based detection at the particular known position.
With the help of dead reckoning approach integrated with other approaches can result in finding positioning errors [34]. Dead reckoning sensors help us to provide proves for the prediction of the trajectory of cars [43, 40]. For the solution of INS, continuous transmission can be helpful in order to achieve better results and error can be reduced from 1cm. to 100m. But this accuracy cannot be guaranteed every time which depends on environmental conditions. So, an alternative can be used.
1.5 Structure of thesis document
In order to define this approach, the structure of documentation plays an important role. In beginning, motivation must be created for the readers for their interest. The structure must be created which starts with the introduction of each and everything that is going to be the part of this approach. No any single point must not be skipped which can cause a trouble for understanding of readers. Structure is created according to particular standard of TU Chemnitz Master thesis guidelines. This document consists of 6 chapters and each chapter consists of several parts related to the main topic. Structure of this document consists of chapters:
1.5.1 Introduction
This section (Introduction) is all about to give basic idea about what is going to be discussed in this document. Under this section, Motivation is also created in order to get the interest of readers towards this document. Motivation is based on current solutions in order to achieve high accuracy in the measurement of position. This part of introduction tells about the company as well, in which this implementation about this master thesis is performed. A basic introduction is also given to the current problems and present solution under this introduction part is given. Under introduction part, Introduction of navigation systems is given according to current trends and standards. This part explains the working and categories of navigation systems as well. Navigation system part is discussed in details which includes dif- ferent approaches to reduce accuracy error. Approaches which are defined under this sections are Dead reckoning, GPS, and mapping based navigation systems. For further possibilities, this section is managed in a way that additional unit which can be added to such systems can result in more accuracy. Importance of traffic sign detection is also given with different existing methods which are used for traffic sign detection. At the end of this section, the importance of this project is explained that why we need this system and how it is going to be useful for us. Navigation general problems are mentioned under this section. The situation in which navigation sys- tem have problem and how it can solve with the help of camera-based navigation system. The importance of camera-based navigation systems is explained with the perspective of ADAS as well. Under this section, it is important to know that what is required for ADAS and how this system can provide the required inputs. Different types of input can be obtained from Camera-based navigation systems which are going to be results in a great help for the ADAS. An example which can be given in this instance is that, input that is providing distance measurement which can help the autonomous car to stop before hitting the object at front.
1.5.2 State of Art section
This section consists of three kinds of state of arts, which includes state of art based on Diversity of navigation systems depends on working principle used in the differ-
ent types of navigation systems which help us to name and recognize the navigation system. This section is providing more details about INS, GPS and Automotive navigation system. These are points which force a creation of another system like camera based navigation system which can be used as additional unit for such kind of navigation systems.
The second state of art is based on traffic sign detection algorithms. Under this section, different algorithms are analyzed in order to get a robust detection algo- rithm for traffic sign detection. This area is considered as more crucial because the system must detect these traffic signs under challenging environmental conditions.
Three things are considered so that algorithm must be compared on the basis of Illumination, Scaling and pose.
The third state of art is based on the approaches used for measuring the distance.
Under this section two approaches are considered, one of them is based on single camera measuring the distance with help of image scaling from different distances.
second of them is stereo cameras. It is an approach that is going to provide accurate results as compared to measuring the distance with the help of single camera.
1.5.3 Concept section
Under this section, each and every step that is taken or studied under this master thesis project is explained. At the beginning of this section, camera providing data considered as input for the navigation and this input can be used for different pur- poses. with the help of camera, distance is measured and this kind of data provided by camera is useful in many cases like measuring distance between car and position- ing sign in order to generate a warning on screen, maintain distance between car in front in order to avoid collision. Once this input is given by camera it is saved in a file so that other different programs can access this data. Datasets are important for training a feature which must be detected. Under this section, important things will be discussed in order to create a robust detection with the help of useful datasets.
During capturing the image in order to datasets, there are many rules which must be considered. In case of missing a rule, it will be results in failure of detection.
The role of distance measurement under this project is very important. Distance measurement will help in generating a warning in described time frame. Once all the inputs are obtained then this data is stored in a file then navigation program is going to access it. There are many challenges which are discussed under this section and their solution as well. images for creating datasets in order to perform training.
1.5.4 Implementation section
This section shows the implementation tasks which are performed in order to com- plete the implementation of the Camera-based navigation system. In the beginning of implementation, steps are explained for training in the form of creating a HAAR
cascade for a particular positioning sign. For training, there are few steps which are given under this section which can help to detect any object of your choice.
Once this training is done, a program is written in order to detect this object in a live video stream. In our case it is a positioning sign that is detected. After the designing of the HAAR cascade, a program is written that is used to measure the distance between the positioning sign and camera. After detection and measuring the distance, Geo-coordinates are assigned to positioning at which this sign is de- tected in order to calculate the accurate position with the help measuring distance with positioning sign and camera so that error can be reduced. This information is saved in an XML file that is further accessed by the main navigation system.
1.5.5 Results section
Under this section, obtained results after the selection of different algorithms and concepts are shown. In this section, Robust detection is shown with the help de- signing of HAAR cascade. After detection, results of distance measurements are given in which camera is detecting a positioning sign and measuring the distance between positioning sign and camera. After measuring the distance, location is checked where this sign is available. All results which are shown in this section are the results which are obtained after the implementation of this system.
1.5.6 Conclusion
Under this section, results are discussed. Advantages and disadvantages are the part of interest under this section. There are many positive and negative points under this concept which are studied thoroughly so that overview of this master thesis project can be given. This section is more focused on the possibilities of this system in the real world which have some advantages and disadvantages.
2.1 Diversity of navigation systems based on working Principle
My focus is to make stand-alone unit more efficient so that no any external signal will be required to an efficient system. Accuracy is possible by using an imaging sensor with existing navigation systems. This kind of system is going to enable the vehicles to find the direction to a destination even in poor environments in which signally is not present due to urban canyons or tunnels. Many types of navigation systems are present these days according to existing requirements and environmental conditions.
No. Types of Navigation Systems Description
1. Inertial Navigation Systems Stand alone unit used to find location with the help of Gyros, accelerometers and equation.
2. Global Positioning Systems working device based on external signals and find the location of device.
3. Automotive Navigation Systems considered as third party device used to find direction to destination.
4. Mapping based Navigation Systems method by which an object enable to construct floor plan and localize itself.
Table 2.1: Types of navigation systems
2.1.1 Inertial Navigation System
Inertial navigation system (INS) is a device which manipulates the data such as velocity, orientation and position of an object in real time, without any need for external references. The basic components used in INS are gyroscopes and ac- celerometer. The block diagram of INS is shown in Fig#2.1. INS consists of inertial measurement units (IMU) and navigation equations which in terms manipulate to give the desired navigation to the system. IMU consist of three gyroscopes and three accelerometers each arranged perpendicularly to each other in pairs. During turns, it measures the each and every movement of the system. By processing the data
from these sensors it is possible to capture the position of a device.
INS system has a broad range of application such as navigation of airplanes, missiles, submarines, and ships. The recent developments in Microelectromechanical Systems (MEMS) possible to make this device light and small, However, this revolution in the technology has opened gates for the wider range of application.
Figure 2.1: Block diagram of INS
Working principle of INS:
Inertial navigation system basically consists of motion sensors such as accelerom- eters. It also consists of Gyroscope and a computer. The working of the system is based on few principles which we will be discussing in detail stepwise. Initially, the INS unit has to set the current location with the help of GPS. Once the system is initialized with the start point and the destination point. The system does not require any help from external source.
Dead reckoning is one of the navigation technique where you know where we started, what direction we are flying and how long have we been flying and how fast are we heading towards our destination all this data is calculated in order to trace our position on the map. We are using the accelerometer in order to calculate the ve- locity and gyroscopes to calculate the relative change in the orientation and finally the computer compiles all the data coming from the inertial measurement unit to calculate the current position in real time.
Inertial Measurement Unit:
The techniques used in INS are based on Sir Isaac Newtons second law namely that a vehicle continues to move with the same velocity unless acted upon by an external force and that such a force will produce a proportional acceleration of the body. The
initial information is provided to the INS about a location and with the calculated data coming from IMU we are able to trace the current location. We will discuss in more detail about the working of individual units [49]. All units are given in figure#2.2:
Figure 2.2: Inertial measurement unit
Acceleration Sensor:
The basic principle behind the working of the acceleration sensor can be explained by Mass spring damper system as shown in Fig 2.3. Most accelerometers are Mi- croelectromechanical sensors. The working of the accelerometer is explained with the help of Mass spring damper system. The Small proof mass is suspended by two complaint beams which are anchored to the fixed frame. The proof mass has a mass of M. The supported beam acts a spring with spring constant K, damping factor (D) affecting the dynamic movement of the mass. When an acceleration is applied to the system. A force F is developed which in return displaces the mass, the spring is strained and displaced with x distance and there is a change in damping factor. There is a restoring force from the spring to restore the system to its initial position. So the change in force can give us the amount of change in acceleration of the system by F=m*a. This is derived from Newtons second law of motion. The formula can also be written in terms of displacement and spring constant. F= K*
x. MEMS acceleration sensors are a technique with a huge commercial capability.
They provide less power consumption, small in size and sense. For accurate data, multiple sensors are often combined.
In an inertial navigation system, we use three accelerometers aligned orthogonal to each other. To calculate the velocity and position of the system we need to perform successive mathematical integration with the given measurement of accel- eration. The reference frame is used to determine the given position of the system for the purpose of the navigation. The true north and the local gravity vector is
Figure 2.3: General accelerometer structure [31]
defined by the local geographic frame. If we need to achieve this we need to have a real-time computation of this data regarding the orientation of the accelerometers with the reference frame. The signals received from the accelerometer can be re- solved in the navigation reference frame prior to the computation. This is achieved with the help of gyroscopes in an Inertial navigation system.
The use of gyroscopes with accelerometers should be synced a small error could be fatal in long run of the navigation system. This device either provide a direct measure of altitude with respect to the reference frame, defined by the angular mo- mentum vector of a spinning mass, or a measure of the angular rate of the vehicle, from which the attitude may be deduced by a further integration process.
Typically inertial navigation system is made up of three gyroscopes and accelerom- eters as discussed above in order to get the change in velocity, position, and ori- entation in three dimensions. Accelerometers here provides a measure of the non- gravitational accelerations which a vehicle experiences. The reason behind this is that the proof mass of the accelerometer and the vehicle mass both are subjected to the same gravitational field.
Gyro Sensor:
A gyroscope is a device which used to determine the orientation by using the help of earth gravity. The design is simple, A gyroscope is made up of a rotor at center surrounded by three rings called gimbals. Rotors on high quality bearing surfaces produce torque can be exerted on the inside rotor as shown in figure#2.4 [12].
Figure 2.4: Basic construction of Gyroscope [12]
Initially, INS was made up of Mechanical Gyroscopes which were bulky and greater in size. Sperry gyroscopes one of the primitive forms of the gyroscope and now many modern era gyroscopes utilize the same rotating momentum of the wheel attached to a gambling structure. However, this kind of working principle had many disad- vantages. Such as friction in the bearing which caused wear and tear of the system.
Recent development in the MEMS technology has made the gyros cheap as it can be manufactured in bulk process.
MEMS gyroscope has been introduced recently which has made the gyroscope cheaper and opened the doors for a vast number of applications. We will discuss the application in later part. The MEMS gyroscope vibrates on its axis unless until there is a change from the external source. When its vibrating in its own axis there is no change in the orientation of the system. When there is an external change the mass vibrates out of phase of its axis. The data is computed and the change in the orientation is calculated.
We make use of three gyroscopes which are arranged perpendicular to each other.
Each axis gives us some information with regards to its orientation position and velocity of the system. Roll pitch and yaw are the three parameters which give us the information in particular axis. Rotation controlled front to back with the help of ailerons is called the roll. Rotation controlled side to side with the help of elevator is called pitch. Rotation controlled vertical axis is called yaw. These three moments are shown in figure # 2.5 with respect to the color of arrows:
Figure 2.5: MEMS gyroscopes.
2.1.2 Global Positioning Systems
The Global Positioning System (GPS) is a constellation of 24 satellites that rotate the earth twice a day at 12000 miles in altitude by transmitting the accurate position and time (Longitude, Latitude, and altitude) information [21, 16]. On the ground, there are 5 stations which constantly monitor the satellite system. This system is providing the 2-Dimensional and 3-Dimensional positioning 24 hours a day. In this rotation, the 3 bits data is being transmitted that includes; satellites number, the position in space, and the time the information is sent. This data is used to measure the distance between the GPS receiver AND GPS satellites.
GPS navigation system started in the early 1970s by the Department of Defence (DoD). They launched the first Navigation System with Timing and Ranging (NAVS- TAR) system in 1978 [29].
GPS system provides two types of services; For civilian access the service names SPS (Standard Position Service) and for exclusive military usage it is Precise Posi- tioning Service (PPS).
3 SEGMENTS OF GPS:
There are 3 segments of GPS system as shown in figure # 2.6 namely; Space, Con- trol, and User segment.
Figure 2.6: Three segments of GPS [10].
The space segment:
It includes 24 satellites orbiting the earth at 12000 miles altitude, as this height makes it cover the larger area of the earth. The arrangement of satellites is done in their orbits to obtain a GPS signal at any time from at least 4 satellites. The low radio frequency signals are being transmitted by every satellite which is coded uniquely over the different range of frequencies so that the GPS receivers can iden- tify the transmitted signals. The actual reason to provide the coded signals is to allow the calculation time for travel from the satellite to the GPS receiver. In this way, the traveling time is being multiplied by the speed of light which is equiva- lent to the distance between the satellite and GPS receiver. However, these signals are low powered and are unable to travel through the solid objects, therefore, it is mandatory to have a clear view of the sky.
The control segment:
The purpose of this segment is to provide the corrected orbital and time data by tracking the satellites. It consists of one master control station and four unnamed control stations. The four unnamed stations collect the data from the satellites and then sends that information to the master control station for correction and after- ward, it is sent back to the GPS satellite.
The user segment:
This segment consists of GPS receivers and the users which are handheld or they can be placed in a vehicle. These GPS receivers are programmed with the almanac in their computer which provides the location of the GPS at any time [2].
Working principle of GPS:
The work flow of GPS system is based on the mathematical principle known as Tri- lateral. When the GPS receiver is turned on, it will download the orbital information regarding all satellites. At the beginning, the process can take time approximately 12.5 minutes to collect the required data. Once the data is received then this infor- mation is being stored in the memory of the receivers. The GPS receivers already are familiar with the satellite information but for the sake of calculating the precise distance, it must know the actual distance from the satellites where it is receiving the signals. The formula for calculation of the distance is to multiply the velocity of the transmitted signal with the time which it takes for the signal to reach the receiver. The velocity is denoted as the speed of the radio wave which is equivalent to 186,000 miles per second (speed of light).
For determining the time formula, the satellites transmitted code is being matched by the receivers code which is compared to identify the delay time. Finally, the delay time is being multiplied by the speed of the light for getting the distance.
The clock of the GPS receiver Is not precise as much as the atomic clock, for that reason all the collected measurements are rectified due to GPS internal clock error.
Synchronization can be seen in figure # 2.7:
Figure 2.7: Signal Synchronization of GPS.
Errors or Limitations to GPS:
GPS receivers may experience few problems in positional accuracy; categorized ac- cordingly:
1. User Equivalent Range Error (UERE): are those errors that are related to the path and timing calculations which are caused by hardware or atmosphere interference. Following is the list of UERE.
a) Satellite Clock: NAVSTAR satellites are using precise clock known as an atomic clock but it can drift a millisecond alteration which can cause the difference in accuracy. However, to minimise the error, the clock correc- tion calculations are done at monitoring stations and these are transmit- ted towards the GPS signal to follow accordingly.
b) Upper atmosphere (Ionosphere): When the GPS signal passing through the upper atmosphere which is ionosphere (50-1000km) above the surface, due to the phase the signals causes delays and deflection. On the other hand, this density is different in some of the spheres which can cause more delays. By evaluating the properties of ionospheres, the calculations are being made to remove errors and are transmitted to the receivers. Just about three-quarters of error can be removed, as ionosphere is the second largest contributor in GPS error.
c) Receiver clock: There are the quartz crystal clocks inside the GPS re- ceivers which are not as much stable as atomic clocks in NAVSTAR satel- lites. To remove the receive error, the arriving time of signals from two satellites is compared whose transmission time is already known.
d) Solar orbit: as the GPS receiver is calculating the co-ordinates by knowing the locations of the satellites, however, it is a difficult task which consists of analyzing the geometry of the satellite orbits and their velocities. The control segment, on the other hand, is calculating the orbit eccentrici- ties, monitoring the satellite location and documenting the complied files known as ephemeris is monitoring the locations every time. This file is managed for every satellite and is broadcasted with a satellite signal. The errors are caused due to a processing of ephemera by the GPS satellite receivers.
e) Lower atmosphere: There are three layers of the atmosphere from earth surface at an altitude of 50km known as tropopause, troposphere, and stratosphere. This atmosphere delays the signals which add the delay to the calculated distances between the receiver and satellites.
f) Multi-path: due to the interference of many objects and reflected signals from them, it is important to discriminate between the direct signals and other signals. For this problem, antennas are designed to reduce the interference of the signals which are received reflected from below, but it is hard to eliminate the error if the signals are reflected from above. The strategy which is used to minimize this error is to track those satellites which are at least 15 above the horizon and this threshold is called as mask angle.
2. Dilution of Precision: The accuracy of GPS positioning is also affected by the arrangement of the satellites in the sky. The ideal arrangement of the satellites is one satellite directly overhead and other three are equally spaced near the horizon (if considering the case of 4 satellites). When satellites are clustered together in the sky they cause the dilution of precision (DOP), which is a parameter of uncertainty associated with UERE to be multiplied. However, the calculation of DOP is concerned with the cases ranges from 1: which is the best possible case and does not expand UERE, and more than 20: which is the worst case consisting of huge errors and making data useless [20].
2.1.3 Automotive Navigation System
It is a type of navigation used to control the vehicle by combining GPS, INS, and mapping. Typically, GPS is used by the automotive navigation systems. under this system, position is computed by satellites signals and where these signals are not reachable to GPS receiver this location is computed with the help of INS. But when INS have this long distance error due to temperature drift then a solution in order to overcome this problem. No doubt this approach is providing sufficient results in order to get accurate guidance and results. Under this approach, the vector map is used to create a database consist of roads data. Vector map consists of data about the routes in the real world and also other information at the different level of detail.
It is based on the coordinate reference system. Under this approach main goal is to find way-point.
Way-point is the collection of Geo-coordinates used to find different points in the real world is actually a system of Geo-coordinates. The whole working of an automotive navigation system is explained under the starting with GPS in which initial input is given to navigation system through GPS. It is very important to tell the system about starting point that is identified by GPS. After the starting point system can switch to dead reckoning approach in which location can be measured with the help of IMU which consists of sensors used to measure the location continuously. Map matching is used under this approach that is used to estimate our location as well.
Automotive navigation system is a unit of infotainment in vehicles which consists of other options of entertainment like colored display which can be used for purpose of television and DVD movies, It can be result in more options which are common in smartphones like Hands-free option, Messaging service can be enabled, Bluetooth can provide connectivity and also Wi-Fi can be used. It provides other facilities for storing personal data.
In order to solve two types of problems, An automotive navigation system is in- troduced. These problems are at different stages. under the beginning stage, it is used to solve the problem in the entry to reach a destination. that is solved with the help of GPS. The second problem or second stage is route guidance is solved
with the help INS and map matching. In order to design a system, there few things which are considered as important for the basics of a navigation system [45].
1. At the beginning, a starting point and ending point must be defined.
2. Display option must be designed carefully by considering readability.
3. Voice commands must be enabled in order to create a warning.
4. Rerouting must be considered if given option is not followed [45].
2.2 Different algorithms for Feature detection and matching
There are various techniques used successfully for object detection with better re- sults from various kind of images and movies. It is possible with the help of computer vision. Computer vision is a field in which computer get an ability to judge envi- ronment. It is possible the collecting and interprets visible surrounded information.
Computer vision is all about image understanding. In the figure # 2.8, computer vision can figure it out that how many positioning signs are there in this picture, what is the meaning of each sign, which one is near which one is far and what is their purpose. These all the question can answer by computer vision because it is all about image understanding. Computer vision is not about image processing, It is very important to understand the difference between image understanding and im- age processing. Image processing is all about to modify the properties of an image.
Properties like brightness, depth, contrast, and color but it is not about understand- ing them. Actually, image is first filtered in order to remove the noise with the help low pass filters. after removing the noise with the help of low pass filters, An image is processed further with the help of high pass filters which are used for detection of various patterns and various shapes and used to find out what is in the picture [33].
The main idea behind the feature matching is to fetch interesting or required fea- tures from the image like blobs, corners, and edges. It is checked or matched across the image. There are some applications using this approach such as:
No. Types of applications 1. Image stitching
2. 3D reconstruction 3. Motion Tracking 4. Object recognition 5. Robot Navigation
Table 2.2: Applications using feature matching
Object detection is a function provided by the computer in order to identify and discover the presence of an object. During the object detection, image processing can be applied in order to increase the quality an object so that it can be detected easily.
There are many functions of computer vision, one of them is feature detection and matching.
Figure 2.8: Example of traffic signs [12]
There are 4 basic steps which are required for robust feature detection and recog- nition. Under this master thesis project, these steps are considered in order to get robust Feature detection and recognition. These steps are:
1. Feature detection.
2. Feature description.
3. Feature recognition.
4. Feature alignment.
These are the main steps which are performed by the most of the algorithms in order to get detection as well as recognition. Few of them are studied and compared by considering their performance and efficiency. As far as detection is concerned, it is possible with the help of color and shape. Such kind of approaches has negative impacts on the results. If there is any system using detection based on color only will result in great failure when there are any minor changes in color because of light [28].
A better way to provide descriptor that us useful in order to detect any object according to its trained description. It starts with finding a set of interesting fea- tures which are descriptor and detection is performed according to these selective features. After selection of features, Defining a region is required around these points which are defined before. After defining the region around these points, extraction takes place that will be used to create the description of feature which is required to detect through this description. Under this part normalization also takes place so that local descriptor can be created. Now there is the turn of the feature matching in which our local descriptors are matched with each other. Feature detection is very important under this master thesis project because the detected object is re- quired in order to extend this system. If detection is not robust it is detection other objects at the instance or missing any object to detect it can result as a failure of this system.
No. Types of algorithms 1. HAAR cascade classification
2. SIFT (Scale-invariant feature transform) 3. SURF (Speeded-up robust features)
Table 2.3: Names of algorithms for detection and matching.
2.2.1 HAAR like feature
Due to the working problem with color intensities, that is making the calculation of each pixel of an image computationally expensive and slow in processing. As the result of this, problem solved with the help of HAAR like features based on Viola and Jones algorithm [42]. The working of HAAR like feature is based on rectangular regions in neighboring areas at a particular location in the window used for detection. After this, It is used, to sum up the intensities of pixels in every region and then measures the difference among these the values of these windows after sum up. It is defined by an example of detection of a feature, Take an exam- ple of the faces, our cheeks are brighter than our eyes so rectangular areas above the eyes can be known with the help of neighboring areas when they are sum up [19].
It is an approach based on machine learning, and such kind of approaches are pro- viding better results in object detection. As far as detector is the concern which is performing only feature matching are still performing better during matching of shapes, scales and for localization [3]. During training, it is creating a tree of tech- niques in which cascade is created. During this creation of cascade, by joining the weak classifiers, a strong classifier is created this method is called boosting. Actu- ally, these are the methods used for the training of HAAR cascade. It is a cascade which consists of different stages. There is a region of interest through which must pass all the stages of cascade [38]. It is about creating an order of simple classifiers
and it is maintained even during the time of increasing complexity. Classifiers re- quires input after this it is creating a cascade in the shape of a HAAR like feature used to determine the intensities under the rectangular areas, these are computed with help of figures which are given in figure # 2.9:
Figure 2.9: HAAR like features [36]
HAAR cascade classifier:
In more detail about HAAR cascade classifier, It varies a lot of stages where weak learners can be found at each stage of cascade classifier. The sliding window is used in order to check the complete image where it tagged and separates positive and negative. where positive means that under this searching window object is found.
Negative results mean in which required feature is not found an image. After obtain- ing the positive and negative results, classifier pass the result that object is found or not. Classifier labels positive if sliding window found the feature so it will be called a positive result. It is also possible and sometimes called drawback of HAAR cascade in which there is no object in the image but still it labels are positive that is called false positive [42].
By focusing on performance, the negative rate must be considered and reduced, it is a training in which we show an object is not present in negative images. If any of image this object is present it will be considered false negative. In this situation, classification of a particular Branch stops without detecting an object. At every stage of cascade classifier, there will be also false positive rate in which object is not present but still labeled. In this situation, mistakes can be corrected under the subsequent stages in the next stage of cascade classifier. Stages of Classifier can be shown in figure # 2.10:
Figure 2.10: Stages of cascade classifier [42]
The training of HAAR cascade is based on a lot of positive and negative images.
Positive images mean the set of images in which object which we want to detect must be present in them. Negative images means can be anything but an object which we want to detect must not be present in this set of images. These both kind of images are required in order to train the classifier.
Advantages of HAAR Cascade Classifier:
1. Fast and good for real-time operations.
2. Machine learning based.
3. Accurate results.
4. Automatic sample creation function available.
Disadvantages of HAAR Cascade Classifier:
1. Training of a HAAR like feature is time-consuming.
2. Large datasets required for training.
3. Many rules are considered for training.
2.2.2 Scale-invariant feature transform (SIFT)
SIFT is considered as a feature descriptor that was proposed by David Lowe in the year 2004. It is a scale invariant feature descriptor that uses Difference of Gaussian pyramid in order to fetch interesting features at different scales, It helps in detection even the target image is different in size [46]. Extraction of interesting patches can see in Figure # 2.11. In order to know the functionality of SIFT,
It is used for image matching. It is very accurate and provides the functionality which makes SIFT under the challenging environment a scale invariant, Rotation invariant, Illumination invariant and pose invariant. The first under this algorithm is the construction of scale space that is mentioned in the left side of Figure # 2.11.
In which it is mentioned scale space is created with the help of Gaussian filter [9].
L(x, y, σ) = G(x, y, σ) ∗ I(x, y) (2.1) where
G(x, y, σ) = 1 2πσ2e
−(x2+y2)
2σ2 (2.2)
Figure 2.11: Extraction of interesting patches [9]
and Laplacian of Gaussian is:
σ2∆2G (2.3)
Another step is taking the Difference of Gaussian: DoG is the Approximation of Gaussian so,
σ2∇2G = ∂G
∂σ ≈ G(x, y, kσ) − G(x, y, σ)
kσ − σ (2.4)
G(x, y, kσ) − G(x, y, σ) ≈ (k − 1)σ2∇2G (2.5) D(x, y, σ) = (G(x, y, kσ) − G(x, y, σ)) ∗ I(x, y) (2.6)
