The methods introduced in Section 4.3 have been experimentally evaluated by executing 7 tra- jectories on the robot sub-space with no load attached to the robot end effector. This section exhibits a comparison between LR, RF, SVR and dynamical-based model approaches to cap- ture the undesired internal forces, using joints variables. Moreover, this section discusses the use of different features sets, in addition to the joints positions, to estimate the undesired inter- nal forces. Therefore, relevant features for haptic-based skills can be identified, and determine what modelling approach to use to facilitate learning force-based robotics applications.
LR, RF and SVR models have been trained on data set collected for 7 predefined robot tra- jectories. The joints variables ranges of the collected data are shown in Table 4.1, which have been recorded from the robot controller. The joints’ velocities and accelerations have been numerically differentiated. The three force components (Fx, Fy and Fz) and the three torque
components (Tx, Ty and Tz) have been recorded directly from the sensor. The complete dataset
consists of 13580 samples and has been split into 80% training and 20% testing. The complete dataset contains joint variables that had the statistical properties shown in Table 4.1.
Table 4.1: Joints variable statistics.
Joints q1 q2 q3 q4 q5 q6 q7
max 98.58◦ 106.98◦ 27.21◦ −15.6◦ 179.20◦ 105.46◦ 20.96◦
min −35.56◦ −31.25◦ −169.93◦ −78.00◦ −137.06◦ −102.64◦ −179.90◦
µ 29.00◦ 52.05◦ −23.95◦ −52.24◦ −24.50◦ 15.15◦ −91.09◦
The correspondent predefined trajectories, used to collect the dataset, are shown in Figure 4.3, also they were designed to cover the workspace defined in Table 4.2 while varying velocity and acceleration. The trajectory shown in Figure 4.3 (A) depicts an S-shape raster pattern in X,Y and Z directions, while Figure 4.3 (B) shows multiple plain rasters on the XY plane at different altitude Z. In Figure 4.3 (C) a square spiral trajectory in XY Z is displayed. Close- loop trajectories (return to start point) are shown in Figures 4.3 (D) and (G). Finally, Figures 4.3 (E) and (F) illustrate random trajectories in XY Z directions. Table 4.2 shows the XY Z coordinates of the targeted workspace and their dimensions.
Table 4.2: Workspace on X, Y and Z axis.
X Y Z
Start 37.00 105.00 98.00 End -77.00 -10.00 14.00 Range 114.00 115.00 84.00
(A) Trajectory 1 (B) Trajectory 2
(C) Trajectory 3 (D) Trajectory 4
(E) Trajectory 5 (F) Trajectory 6
(G) Trajectory 7
As stated before, the machine learning approaches were utilised to train models as follows: firstly, only based on the joints’ positions (qi, i ∈ [1, 7]). Secondly, using joints’ positions and
velocities and finally, using joints’ positions, velocities, and accelerations. The performance of the trained models was measured using MSE as shown in Figure 4.4, and NMSE as illustrated in Figure 4.5.
Figure 4.4: MSE for LR, RF and SVR models.
Figure 4.5: NMSE for LR, RF and SVR models.
The MSE shows how accurate the models were in comparison to the actual forces in the testing dataset. On both force and torque components, the RF model trained dataset {q , ˙q, ¨q} have the lowest MSE amongst all other models, which infers that this RF model was the most accurate model. On the other hand, LR models have the highest MSE amongst all other models, due to the nonlinearities in the given problem. Therefore, the RF clearly outperforms the SVR and LR for all F/T components in terms of accuracy. Another important observation that can be taken,
from the MSE bar plot is that RF models performance greatly enhanced by adding velocity and acceleration features. In contrast, the LR and SVR models performances do not improve by adding more features and almost stay constant. Moreover, adding more features to train the LR models, declines the models’ accuracy in the Fz component as shown in Figure 4.4.
The NMSE value indicates how predicted values from different models can fit the true value from the testing dataset, which almost follows the same trend as the MSE on both force and torque components. The low NMSE value for RF model trained on {q , ˙q, ¨q} features indicates a very good model in comparison to other models. However, the high NMSE value does not necessarily conclude a poor performance. To sum up, the RF models trained on (q, ˙q, ¨q) have the best MSE and NMSE on all F/T components.
Table 4.3: Force/Torque statistics.
F/T Fx(N ) Fy(N ) Fz(N ) Tx(N.m) Ty(N.m) Tz(N.m)
max 13.47 13.13 11.84 0.22 0.23 0.02
min −8.66 −11.36 −10.99 −0.23 −0.14 −0.01
µ 1.42 0.80 1.18 −0.01 0.021 −0.001
Figure 4.6 depicts the variance score of the Fx, Fy and Fz models trained using LR, RF and
SVR. Apparently, the variance square of the prediction error from the RF models have the highest values on all F/T components. Also, by including more features (joints’ velocity and acceleration) the variance score of the RF model is increasing, and it is almost 1 for all F/T components. In contrast, the lower values of the variance score for LR and SVR models indicate poor performance.
Based on the aforementioned discussion, it is obvious that RF models trained on {q , ˙q, ¨q} have the lowest MSE and NMSE values. Also, it achieved the highest variance score which indicates better performance in comparison to LR and SVR models. The performance data-driven model fitted on {q , ˙q, ¨q} in comparison to the dynamical-based model is shown in Figure 4.7 and Figure 4.8 for force and torque signals respectively. This shows that dynamical-based model has a comparable performance to the LR models on all F/T components. Figure 4.9 illustrates the internal F/T, during the execution of the test trajectory, based RF trained on {q , ˙q, ¨q} and physical model-based. It can be noticed from this figure that the physical model-based approach is noisy and encompassed spikes and fluctuations. These spikes might have resulted due to jerk movements or changes in direction by the robot.
Figure 4.7: Force NMSE of models trained using RF, LR and SVR on (q, ˙q, ¨q) in comparison to physical- based model.
Figure 4.8: Torque NMSE of models trained using RF, LR and SVR on (q, ˙q, ¨q) in comparison to dynamical-based model.
Figure 4.9: Prediction of undesired F/T using RF trained on (q, ˙q, ¨q) vs physical model-based approach.
Given the relatively low velocities and accelerations of the robot, the forces and torques are mainly generated due to gravity in relation to the current configuration of the robot (joints’ positions). Therefore, in the RF case, it is satisfactory to train models only based on joints’ positions as these models have good MSE, NMSE and variance score. However, trajectories that involved joints’ velocities and accelerations variation must be compensated using models trained on {q , ˙q, ¨q} features. Another advantage of RF models over LR and SVR models is that it can rank features based on their importance and it relies on the most important features for regression as explained in Section 4.3.2. Figure 4.10 shows the important features for the RF models trained on (q , ˙q , ¨q). This figure indicates that the RF regression model relies mostly on the joints variables to predict the undesired F/T signals.