A human mainly relies on tactile and visual data during the execution of assembly and manip- ulation tasks [50]. Therefore, robots with human-like behaviour robotics approaches are de- sirable in dynamic industrial platforms. Accordingly, industrial robots must be able to acquire knowledge, through human-like perceptions, about the surrounding environment, undergoing processes and their actions.
2.3.1
Vision Perceptual Systems
Vision perceptual systems are much more developed in comparison with haptic perceptual sys- tems. Also, it has been employed in many robotics solutions [51, 52]. The advantages of the visual perception include improving accuracy, object recognition, trajectory planning and iden- tifying process sequence (e.g. assembly sequence). Vision applications generally deal with
finding a part and orienting it for robotic handling or inspection before an application is per- formed. For such an application a vast number of vision-based sensors can be adopted, such as laser sensors, high-resolution cameras, stereo cameras. Usually, captured data through such sensors require further processing to extract useful physical meaning. Then, these physical values can be used to plan/control robots accordingly.
Solvang et al. [53] presented a 2D robot trajectory planning for grinding application based on a 2D camera and a CAD model of a workpiece. In [54], a 2D camera was used to plan weld repair robot trajectory. In this work, an operator specifies the targeted area then a 2D image was used to generate a weld trajectory. The height between the welding torch and the workpiece was measured using a tactile sensor. Gonzalez et al. [55] introduced a trajectory planning approach to track a laser spot in the robot workspace. The tracking process starts by capturing a simple 2D image. Then a laser spot was detected and allocated in the captured image. Finally, the trajectory between the robot current configuration and the laser spot was generated and executed.
Vision systems were also employed in LfD application. In [56], Calinon et al. presented a probabilistic LfD approach to identify and reproduce gestures using a humanoid robot. This work was extended later on to an incremental learning approach as presented in [5]. Also, Schaal [57] introduced an LfD of pole balancing using an industrial manipulator based on 60Hz video-based stereo camera. Most of the exhibited works on vision-based manipulation did not consider a physical contact with the surrounding environment. Hence, several researchers have tackled this limitation by adding a passive compliance element to the robot end-effector. Furthermore, all vision-based approaches required additional processing of the raw image data to extract the task-relevant features.
2.3.2
Force/Torque Perceptions: robot internal forces
It is noticeable from Section 2.2 that force signals play a key role especially when the robots have to interact with the surrounding environment. Nevertheless, the Force/Torque (F/T) sig- nals are noisy and ambiguous to interpret and use [58]. Humans, on the other hand, can robustly perform assembly tasks with tight tolerances [59] because they are very efficient at using F/T information, especially when vision cannot provide the required information. Therefore, captur- ing and utilising how humans use haptic feedback should be explored for performing assembly tasks by robots. This can empower robots to use force and torque with human-like capabili- ties allowing them to learn and adapt according to the variations in the environment and adjust movement for tight tolerance assembly.
erated by the F/T sensor, the end-effector (gripper), workpiece and the unmodelled dynamics [60]. In learning applications, only the contact force patterns are essential. Hence, the internal forces must be filtered out. Noise and disturbances occurring at high frequencies can be sig- nificantly reduced using classical filters such as low-pass filter [61]. Furthermore, disturbances due to robot motion cannot be omitted especially in high-speed applications or when heavy robot tools are used. Approaches reported in the literature that address this problem generally use estimation methods based on dynamic models that consider the internal components of the sensor, gripper and workpiece see for example [62].
The F/T sensor is used to measure both external and internal forces. Measuring external forces is vital in the context of force-based controlling or when a robot needs to interact with the surrounding environment. Based on the dynamic modelling of the robot and the load, many model-based estimation methods have been proposed such as the work presented by Garcia et al. [63] and Li et al. [64]. In [63], a Kalman filter was used to estimate the external forces and torques at the end effector. This model is suitable for high-speed applications. The main draw- backs are the high computational power and the model complexity. Another method proposed a joint based force torque estimation to estimate the contact force at the end effector using a Kalman filter [64]. This method requires an accurate model, a huge computational effort and a sensor fusion algorithm. Many researchers used a Kalman filter to estimate the contact force or external forces (such as object held by the end effector). In [65], contact forces and torques were estimated for multiple cooperative robots. Forces and torques for a fixable payload were estimated using a Kalman filter in [66]. The research mentioned above all use model-based methods that require high computational forces.
Alternatively, machine learning algorithms have been proposed to estimate forces and torques. For example, Locally Weighted Projection Regression (LWPR) was used to adjust a compliance matrix for computed torque controller [67]. However, this requires the skilful tuning of the learning parameters. Also, Gaussian Process Regression (GPR) and LWPR were implemented to achieve an adaptive compliance control [68]. The results showed the advantage of using LWPR over GPR. In the same paper, Nguyen-Tuong and Peters introduced a Locally Gaussian Process (LGP) to learn a computed torque controller and compared it with SVR, GMR and LWPR. The results showed that LGP can achieve higher accuracy and requires lower compu- tational forces compared to GMR and SVR. Also, LGP achieves higher accuracy compared to LWPR. Nonetheless, it requires higher computational effort in comparison with LWPR.