The UGV Perception System

A simplified UGV perception system employing two sensors to measure the environ- ment is shown in Figure 2.6. Perception is the process of simplifying raw sensor data, typically via a model, into a representation of the environment that is useful for the subsequent higher-level reasoning and decision making processes that are re- quired in the context of the application. Good representations summarise the data strongly, yet provide the necessary input for the widest possible range of downstream decisions from the representation. The representation can contribute towards higher- level strategic goals of the mission through to lower level motion based tasks such as localisation and motion planning (as shown in Figure 2.1). Therefore, a correctly functioning perception system is vital to the success of any UGV mission.

The design of any perception system is dependent on a range of factors including: • The available sensors and subsequent data (the design of a perception system

can choose between a range of sensors of different modalities of sensing1_), • The expected operational environment (e.g. indoor, urban or unstructured

operating conditions will influence perception model design),

1_{Modalities of sensing can be split by the type of data (e.g. range or image) and/or by the}

• The functional objectives of the perception when linked to the requirements of the downstream tasks (e.g. a path planning application will require a represen- tation that allows the traversability of the environment to be calculated while a localisation application, on the other hand, requires a representation that has information about the motion of the environment relative to the vehicle.) Consider the specific examples of the MER perception system (see Section 2.1.2.3 and Fig. 2.4(b)) and the Boss UGV perception system for detecting obstacles (see Section 2.1.2.4 and Fig. 2.5(b)). The MER employs two visual stereo cameras in an unstructured Martian terrain with the requirement to produce a representation that allows the pose of the vehicle over time to be estimated accurately. The Boss UGV employs a number of laser range sensors in semi-urban environments with the requirement to produce a representation that shows the position of obstacles in the environment such that a path planning algorithm can function correctly. In each case, the perception system must function correctly and any failures in the percep- tion system will have a big detrimental effect on the UGV mission. The following Sections 2.1.3.1 and 2.1.3.2 explore in more detail how each of these systems is imple- mented to achieve their specific task. Figures 2.7 and 2.8 show an in depth break-down of the MER and Boss UGV perception systems respectively.

2.1.3.1 The Mars Exploration Rover Perception System

The MER [82] is an example of a perception system feeding a representation of the sensor data to a localisation calculation as shown in Figure 2.7. Two stereo cameras, positioned such that their fields of view overlap, are employed to provide visual- spectrum intensity image data of the environment to the perception system. First, Harris corners [52] are extracted from each image and stored as features. Second, these common features in each image are matched. Third, using standard stereo vision algorithms that take into account the intrinsic camera models and extrinsic calibration between the cameras, the relative position of matched features between paired images allows for the features to be placed into a 3D map relative to the UGV.

Figure 2.7 – Mars Exploration Rover: Perception system interpreting visual camera data to track features to aid localisation

Finally, the matched features are tracked over time in the 3D map. The representation that is passed to the localisation component is a vector of tracked features in the UGV frame that can then be used to estimate the relative change in pose of the vehicle in the world frame. Accurate localisation of the MER is vital for mission success, and so, any failures to produce accurate and correctly tracked features will negatively affect the mission.

2.1.3.2 The Boss UGV Obstacle Detection Perception System

Figure 2.8 – Boss UGV: Perception system interpreting laser range data to produce an obstacle map for path planning

planning application as shown in Figure 2.5(b). A more detailed description of this perception system is shown in Figure 2.8. Numerous onboard laser sensors provide range measurements of the environment surrounding the UGV. These range data are transformed into a world frame using the pre-calibrated position of the sensor on the vehicle and the estimated position of the vehicle. The data from all the sensors are fused into a grid based map and the probability of occupancy for each square in the map is estimated. The vehicle model allows for regions of the map to be interpreted as easy or difficult to traverse and assigned a cost based on the characteristics of each grid square. For example, obstacles are detected by identifying rapid changes in the elevation between neighbouring squares on the map (e.g. caused by a tree, a pole or a wall) that the UGV would be unable to drive over. Other map features are also found with specific characteristics of map elevation data, for example roads are flat with curbs on either side. The result is a 2D cost map that allows the path planning application to decide the best course for the UGV. A failure of the perception system to correctly identify obstacles in the environment can be catastrophic to the success of the mission.

In most situations that are expected by these UGV systems, the perception system provides an accurate representation of the environment that allows the UGV to func- tion as expected. In the case of the Boss UGV, the map will show the position of obstacles and roads while the MER UGV will have a good estimation of the 3D mo- tion. However, these systems have been shown to fail in some particular conditions. The limitations of perception systems will be discussed in the following section.