In this paper we present a system for real-time on-board obsta- cle avoidance for UAVsbased on an embeddedstereovision ap- proach. It uses imagery from a stereo camera system for a real- time disparity map estimation of the environment in front of the UAV, which is then used for the adaptation of the flight path. The algorithm for image-based disparity estimation, which adopts the semi-global matching approach, is optimized for the deployment on an embedded FPGA. Hereby the strategy for op- timizing the SGM energy function is adapted in order to account for the strengths of FPGAs to process data streams. This is done, among other, by using four aggregation paths rather than the com- monly used eight. It runs on the FPGA, which is operating at 200 MHz and achieves a processing speed of 29 Hz and a latency of 28.5 ms at a frame size of 640×360. This is significantly less powerful than state-of-the-art systems, however, since this work is part for the T ULIPP project, which is aiming, amongst other things, at reducing the gap between application development and hardware optimization, we focus on a user friendly development by implementing the algorithm in C/C++ and porting it with the use of high-level synthesis to the FPGA.
UAVsbased on a stereo camera setup. We implemented a real- time-capable and energy-efficient system based on disparity map estimation for obstacle detection and a reactive approach for col- lision avoidance. The basic functionality has been written in C/C++ and optimized for Xilinx Zynq Ultrascale+ with an ARM Cortex-A53 Quad-Core CPU and a FinFET+ FPGA. We aimed to use High-Level Synthesis for porting parts of out code to FPGA in order to close the gap between application development and hardware optimization and reduce development time and costs. We evaluated our implementation of the disparity estimation on the KITTI Stereo 2015 benchmark. The integrity of the overall real-time collision avoidance system has been evaluated by us- ing Hardware-in-the-Loop testing in conjunction with two flight simulators.
Furthermore, we describe a new visual guiding device called “Seeing Eye Glasses” that meets the requirements we have identified and that utilizes our signal matching algo- rithm. The device is made possible by advances in technol- ogy that allow embedded systems to become both smaller and faster. Two compact color CMOS imagers are used as the input devices, and a custom-designed FPGA-basedboard is used as the visual data processing engine. The system uses a new stereovision algorithm to extract 3D information from the environment in realtime. From that information, the sys- tem detects potential obstacles and computes the approxi- mate distance to these obstacles and the time to impact. The Seeing Eye Glasses use miniature vibrators mounted in the frame near the ears to warn the user about imminent colli- sions. By changing the vibration frequency and controlling the vibrating actuator on each side separately, both direction and distance information can be conveyed.
SURF is a robust local feature detector that can be used in computer vision as object recognition or 3D reconstruction. This technique is inspired by the SIFT descriptor. The standard version of SURF is several times faster than SIFT and claimed by its authors to be more robust against different image transformations than SIFT. SURF is based on sums of 2D Haar wavelet responses and makes an efficient use of integral images. Both feature detector techniques are traditionally useful to find feature in stereo pair. These techniques have high execution time if used in realtime application. Feature detection methods for image registration. Based on the experimental results, it is found that the SIFT has detected more number of features compared to SURF but it is suffered with speed. The SURF is fast and has good performance as the same as SIFT.
Abstract: One of the most challenging problems in the domain of autonomous aerial vehicles is the designing of a robust real-timeobstacle detection and avoidance system. This problem is complex, especially for the micro and small aerial vehicles, that is due to the Size, Weight and Power (SWaP) constraints. Therefore, using lightweight sensors (i.e., Digital camera) can be the best choice comparing with other sensors; such as laser or radar.For real-time applications, different works are based on stereo cameras in order to obtain a 3D model of the obstacles, or to estimate their depth. Instead, in this paper, a method that mimics the human behavior of detecting the collision state of the approaching obstacles using monocular camera is proposed. The key of the proposed algorithm is to analyze the size changes of the detected feature points, combined with the expansion ratios of the convex hull constructed around the detected feature points from consecutive frames. During the Aerial Vehicle (UAV) motion, the detection algorithm estimates the changes in the size of the area of the approaching obstacles. First, the method detects the feature points of the obstacles, then extracts the obstacles that have the probability of getting close toward the UAV. Secondly, by comparing the area ratio of the obstacle and the position of the UAV, the method decides if the detected obstacle may cause a collision. Finally, by estimating the obstacle 2D position in the image and combining with the tracked waypoints, the UAV performs the avoidance maneuver. The proposed algorithm was evaluated by performing real indoor and outdoor flights, and the obtained results show the accuracy of the proposed algorithm compared with other related works.
Autonomous navigation in an unstructured envi- ronment, i.e. an environment that is not modified specifically to suit the robot, is a very challenging task. Current robots that can operate autonomously in an unmodified environment are often large and ex- pensive. Most of today robot navigation algorithms rely on heavy and power-hungry sensors such as laser range finders, high resolution stereo-visions (Thrun et al., 1999; Manduchi et al., 2005; Stentz et al., 2003; Ibanez-Guzman et al., 2004; Batalin et al., 2003). As a consequence, these robots require powerful comput- ing units to be mounted on-board. The size, com- putational power, and energy requirements of these robots limit the range of their applications and opera- tional period. In this work, we have built a small mo- bile robot from cheap off-the-self electronics to per- form obstacleavoidance in an unstructured environ- ment. Obstacleavoidance is the first basic behaviour needed for an autonomous mobile robot. The robot is equipped with a low-power camera and two ultra- sonic sensors. Image processing is done in real-time and on-board. Our robot is small and energy efficient; it is powered by AA batteries.
This paper presents a novel stereo matching algorithm Cyclops2. The algorithm produces a disparity image, provided two rectified grayscale images. The matching is based on the concept of minimising a weight function calculated using the absolute difference of pixel intensities. We present three simple and easily parallelizable weight functions. Each presented function gives a different trade-off between algorithm processing time and reconstructed depth image accuracy. Detailed description of the algorithm implementation in CUDA is provided. The implementation was specifically optimised for embedded NVIDIA Jetson platform. NVIDIA Jetson TK1 and TX1 boards have been used to evaluate the algorithms. We evaluated seven algorithm variations with different parameter values. Each variation results in a different speed accuracy trade-off, demonstrating that our algorithm can be used in various situations. The presented algorithm achieves up to 70 FPS processing time on lower resolution images (750 × 500 pixels) and up to 23 FPS on high-resolution images (1500 × 1000 pixels). The use of optional post-processing stage (median filter) has also been investigated. We conclude that despite its limitations, our algorithm is relevant in the field of real-timeobstacleavoidance.
The stereovision system which can track the object and able to measure the distance from the object in realtime is introduced by I. Kim et al. . The method is used trigonometric measurement and the robot kinematics applied to the cross-visual stereovision system which is fabricated in the mobile robot. Triebel and Burgard  are the person that created 3D models of the environment with the highly accurate in a single 3D laser scan based on the edge features. Davison and Murray  that proposed an idea active vision is another method for localization and mapping. After comparison, vision systems are much better because of the output and high resolution. Moreover, stereo cameras can observe points of the 3D coordinates more clear based on the scene. Therefore, visionbased is highly desirable because of the stable visual landmarks . Instead, a new algorithm for mobile robot obstacleavoidance was implemented by using of cylinder non-circulation circumfluence 
Over the course of this project, the team evaluated different sensor options before choosing to work with computer vision in order to develop an obstacle detection system. The team was able to implement a DIY stereo depth mapping system capable of mounting on a small UAV. Through camera calibration and bench testing, a stereo imaging algorithm was developed. The bench testing results showed that the depth algorithm was highly precise, but more tuning and bias-correction must be performed in order to increase the accuracy. The Raspberry Pi was identified as a capable on-board processor, and a payload and mount were designed and built to allow the IRIS quadcopter to perform flight tests with the stereo imaging system. The results showed that more specialized hardware is required for implementation in the field. Finally, the team created recommendations on how to move forward with this project in the future.
Webcams typically include a lens, an image sensor, and some support electronics. Various lenses are available, the most common being a plastic lens that can be screwed in and out to set the camera's focus. Fixed focus lenses, which have no provision for adjustment, are also available. Image sensors can be CMOS or CCD, the former being dominant for low-cost cameras, but CCD cameras do not necessarily outperform CMOS-based cameras in the low cost price range. Consumer webcams are usually VGA resolution with a frame rate of 30 frames per second. Higher resolutions, in mega pixels, are available and higher frame rates are starting to appear.
In this paper, state-dependent Riccati equation (SDRE) method-based optimal control technique is applied to a robot. In recent years, issues associated with the robotics have become one of the developing fields of research. Accordingly, intelligent robots have been embraced greatly; however, control and navigation of those robots are not easy tasks as collision avoidance of stationary obstacles to doing a safe routing has to be taken care of. A moving robot in a certain time has to reach the specified goals. The robot in each time step needs to identify criteria such as velocity, safety, environment, and distance in respect to defined goals and then calculate the proper control strategy. Moreover, getting information associated with the environment to avoid obstacles, do the optimal routing, and identify the environment is necessary. The robot must intelligently perceive and act using adequate algorithms to manage required control and navigation issues. In this paper, smart navigation of a mobile robot in an environment with certain stationary obstacles (known to the robot) and optimal routing through Riccati equation depending on SDRE is considered. This approach enables the robot to do the optimal path planning in static environments. In the end, the answer SDRE controller with the answer linear quadratic controller will be compared. The results show that the proposed SDRE strategy leads to an efficient control law by which the robot avoids obstacles and moves from an arbitrary initial point × 0 to a target point. The robust performance of SDRE method for a robot to avoid obstacles and reach the target is demonstrated via simulations and experiments. Simulations are done using MATLAB software.
The proposed method is evaluated using a popular benchmark database. The Middlebury dataset is used as a benchmark database for assessing and analyzing the proposed stereovision technique . The proposed method, Census, Rank and Census Sparse have been tested on the Middlebury stereovision dataset (Figure 7) and the results show that this method gained better execution speed compared to other methods. As a result of our experiments, it is clear that the proposed method needs a smaller window size to achieve the best accuracy in comparison to other algorithms. Thus, the executing speed of the algorithm increases and the time to calculate the disparity map reduces. The proposed method also inherits some particular features of Census transform such as being robust to different luminance conditions, and has the ability to reduce effects of camera gain and bias, as shown in Figure 8. From our experiment and as mentioned in ,  the best result of Census transforms are achieved with 16x16 window size, while the proposed algorithm can achieve the best accuracy with 5x5 window. This particular feature can reduce the large window size overloaded on the processor unit and decrease the calculation time. The time consumed by each method and their error rates are demonstrated in Table 1.
Local algorithms reduce ambiguity by aggregating matching costs over a correlation window. The correla- tion window also refers to local support region implicitly implies that the depth is equal for all pixels inside. And this intrinsic assumption will lead to numerous errors especially at the region of depth discontinuities. When doing cost aggregation, the support from a neighboring pixel is valid only if such pixel has same disparity. The way to select appropriate support is a key factor of the correlation method. For this purpose, adaptive support weight (AW) algorithm was proposed to perform aggre- gation on the appropriate support. The idea of adaptive support weight approach originated from a edge-preser- ving image smoothing technique called bilateral filtering . It combines gray levels or colors based on both their geometric closeness and their photometric similar- ity, and prefers near values to distant values in both domain and range. This idea was extended to the cost aggregation in stereo matching. The similarity and proximity are two main visual cues which can help us to pre-estimate the support weight of every pixel. The mechanism relies on the assumption that neighboring pixels with similar color and closer range to the central pixel p are likely from the same object. This is the main idea behind adaptive support weight generation. Based
Our main motivation throughout this work was the belief that teleoperation can be simplified by endowing a robot with autonomy. By enabling the robot to sense its environment and detect objects that could possibly interfere with its trajectory and cause collisions, we have developed a navigation scheme, based on the concept of shared autonomy. In this approach, the function of the autonomous behaviours of the robot is to help the user while not limit their supervision. Moreover, the user must always be aware of the robot’s actions and should be able to stop any undesired behavior. Therefore, our obstacleavoidance algorithm is primarily passive. When the user commands the robot to fly towards an obstacle, the algorithm will try to modify the commanded velocity by changing its direction or magnitude. Only in the presence of uncommanded drift of the platform will the algorithm actively compensate for this motion to minimize the chance of collision. In our approach, we give feedback about the algorithm’s action to the operator through the haptic device. Hence, in the case of unwanted behavior, the user can change its input and halt the avoidance action.
Improving tracking accuracy and speed can improve the writing representation been displayed. However, to achieve a smooth representation some processing must be done to produce equally spaced intermediate points between the sparse pointes tracked by the cameras. This can be accomplished by fitting a Cubic spline to each segment then resample the spline to the desired rate . A further step to enhance the system will be through implementing real-time character and word recognition of the handwritten text utilizing techniques such as Hidden Markov Model, which was proved to have a good recognition performance , .
ABSTRACT: The proposed method would make just right use of latest technology that situated on Embedded Linux board particularly Raspberry Pi and Smartphone android utility. The proposed method works on GPS/GPRS/GSM SIM900A Module which entails all of the three matters specifically GPS GPRS GSM. The GPS present vicinity of the vehicle; GPRS for sending alert message to vehicle’sowner cellular. The proposed approach would situation inside the vehicle whose position is to be determined on the internet web page and monitored at actual time. In the proposed procedure, there is evaluation between the current vehicle route and already designated route into the file method of raspberry pi. Right here within the proposed approach the already particular direction throughout the raspberry pi’s file procedure taken from vehicleowner’s android smartphone using android utility. Means the determination of route from place A to B takes position from car owner’s android application which gives extra safety and secures touring to the visitor. Henceforth the driver drives the vehicle only on the vehicle owner’s specified path. If the driver drives the vehicle on the wrong path then the alert message will be sent from the proposed system to the vehicle’s owner mobile and also speakers alert driven using Raspberry pi’s audio jack.
(Zeng et al., 2016). Brain-inspired methods use a similar technique based on how human understands and detects obstacles. Various studies have been performed on brain- inspired and mono-based techniques (Mori et al., 2013; Al- Kaff et al., 2017; Zeng et al., 2016). One of the key features of obstacle detection algorithms is their functionality in complex environments. As one of the recent techniques, the method proposed by Al- Kaff et al. (2017) is relatively capable of obtaining an obstacle zone in complex environments and is an important research in this field. This technique regards an obstacle as an object that is resizing in consecutive frames. At first, SIFT algorithm extracts some key points with their positions (X, Y) and sizes (S) from consecutive frames, and the matching process is performed between them. Afterwards, the points that are larger in the second frame are compared with those in previous frame. Then, the sum of size-ratio elements of SIFT in the selected points are regarded as the first criterion, and the area-ratio of the convex hulls of the points is considered to be the second criterion for obstacle detection. The conducted test in this study shows that this technique is not efficient to discriminate between the close and far obstacles, and considers farther objects as obstacles, as well. Figure 1 illustrates the separation of far from near objects in the algorithm of Al-Kaff et al. (2017). Additionally, the presence of wrong matched points will greatly affect the quality of the second criterion, i.e. the area ratio of convex hulls. Understanding the above-mentioned problems, this research aims to develop a method of Al-Kaff et al. (2017) by presenting a distance-ratio of matched points as a factor in detecting object size changes. This criterion investigates every point, so that it can discriminate between far and near points, and can be applied on complex environments. Moreover, using the average distance-ratios in matched points, this criterion is not influenced by wrong matched points.
Abstract: Motivation for the Realtimevisionbased security system arises due to increasing crime activity now a days, which includes intrusion in high security areas where the presence of an unauthorized personis unacceptable. The paper carries out research on the low cost implementation of a system for change detection in a closed monitored area. This paper presents a visionbased security system that includes a Linux based target board, web-camera and indicators to alert the concerned user. This work is successfully developed on embedded target platform board comprising of a Broadcom multimedia processor BCM2835 based on ARM11 embedded processor core as the hardware and the software is implemented on the Linux kernel by porting cross-compiled OpenCV, GPU and GUI libraries as well as UVC drivers for USB camera. An image is captured from a web-camera at random intervals and using change-detection technique i.e.image subtraction, thresholding and morphological operations the final image with major blob is produced. The area of the major blob is derived by calculating mean of pixel array elements. These values of cross section area are compared with calculated values. If derived values fall in the range of theoretical values, it can be identified as human intrusion.These ranges of theoretical values (the average cross sectional area of human) are defined in Anthropology theory. This system has been tested for different environments, such asan experimental closed monitored area, and achieves over 86% detection rate at 5-6 frames/s processing speed.
As a results, the system proved its reliability to return the original path after avoiding obstacles, as well as, the accuracy of detecting and avoiding the obstacle comparing to other systems where the system didn’t affect by weather conditions or light condition and didn’t require much processing time to avoid the obstacle.