the robots. To overcome this, a mirror was placed in each of the studies, which was pointed towards the robots, as it has been shown to have great importance in eliciting stronger feelings of the body ownership illusion (González-Franco et al., 2010;Preston et al., 2015).
3.3
Motion Remapping Library
As we mentioned earlier when talking about correlating the participant’s movement with the robot to elicit BOIs in Section 2.4.2, an issue that arises with using different body tracking systems with different kinds of humanoid robots is that their tracking data may be represented in different formats. Moreover, since there is no standard based on which these systems are developed, there may be discrepancies in the number of tracked joints or bones, different coordinate systems using either absolute positions or relative positions with respect to a root bone, or even varying bone lengths and their relative rotations. On the other hand, the skeletal structures used by the tracking systems may be completely different from the internal structure of the robots being used. We have already discussed about the huge variability in terms of anatomy, actuator type and dimensions in humanoid robot design (Section2.4.2).
Because of these issues, we have developed a library that enables us to map the skeleton from a variety of tracking systems to many different types of humanoid skeletons (vir- tual avatars or humanoid robots) (Spanlang et al., 2013). The library, which is named
HUMAN, provides specific features useful for mapping different skeletal structures, such
as control over the mapping of each bone, specification of the coordinate system, and providing rotational offsets, if required. Building on the character animation library HALCA (Gillies & Spanlang, 2010) (Hardware Accelerated Library for Character Anima- tion), the library allows us to map the movements from any of the compatible tracking systems to a humanoid robot or a virtual avatar in real-time (See Table3.1for details). The library also allows control over more than one robot simultaneously. Figure 3.6
(a) (b)
Figure 3.6: Examples of combinations of tracking systems for con- trolling humanoid robots, using the HUMAN library. The participant in (a) controls both the Nao and Robothespian, and a RocketBox avatar with the Xsens MVN system. (b) shows an example where two participants physically interact with each other’s humanoid robot representations. One participant controls the Nao via the Optitrack system, while the other participant controls the Robothespian with
the Xsens MVN system.
shows some of the possible combinations of the various tracking systems controlling the humanoid robots that have been discussed above. Although various components of this system have been implemented earlier, such as the immersive teleoperation system
by (Nishio et al., 2012). However, their system lacks a full-body tracking system. On
the other hand, the paper by (Goza et al., 2004) describes a highly specialised system similar to the one we describe, however, to our knowledge our system is the first time a universal comprehensive solution has been provided, using commercially available technologies.
The HUMAN library is built on an XML based orientation file (XOF) and an avatar skele- ton file (XSF) to map the data of the corresponding bones and joints. The XOF file describes the coordinate system transformation from the motion-capture system to the avatar’s coordinate system, and maps the corresponding bones with each other. The
3.3 Motion Remapping Library
library uses the XSF format to describe the skeletal hierarchy along with initial transfor- mations.
Although the library was developed initially only for virtual avatar families (such as RocketBox9and Daz3D10), we incorporated support for mapping humanoid robots specif-
ically for the purposes of carrying out the research presented in the following chapters. However, mapping real-time tracking data to a robot’s limbs is very different from doing the same task with an avatar. As mentioned earlier, various robots are built differently, with different types of motors and actuators controlling their limbs, which in turn define the constraints of the system. The degrees of freedom associated with a robot’s limb will dictate the degree of accuracy and control the user will have while controlling it. The more closely a robot’s limbs have been built to the likeness of a human limb, the more precise the motion can be. Other factors, such as if the robot has the ability to walk, or translate and rotate in a physical space also determine if a real-time mapping of those movements is feasible or not.
The algorithm that has been developed for real-time retargeting modifies orientation and position data for each bone obtained from the tracking system into ‘robot friendly’ angles. The whole process was developed initially for the Nao robot, but was later adapted for the Robothespian. This was fairly straightforward as the core algorithm of converting data was the same as with the Nao, but with the addition of extra rotational offsets to adapt the values according to the Robothespian’s reference frame. Although our solution worked well most of the times, a few movements, especially when the arms were stretched to their extremities would result in discrepancies between the performed movement and the robot’s movement. Furthermore, both the robots have angular limits in each of their motors, and any value supplied that is out of range of acceptable move- ments is ignored. Thus, performing a movement that goes beyond what is physically feasible for the robot also leads to the robot moving all the way to its limit and then stopping, resuming again when the values return into the acceptable range. Although
9 http://www.rocketbox-libraries.com/ 10 http://www.daz3d.com/
we have seen that slight amounts of spatial discrepancies could be ignored, these posi- tions for both the robots were identified while developing the system, and were kept in mind while designing the studies in order to deter participants from making these poses.
Finally, a solution was also developed in order to allow the robots to ‘walk’ around in space. Developing a solution for implementing real-time walking depends heavily on the configuration and physical ability of the robot. While theoretically it seems possible to develop an algorithm that accomplishes it based on the same way as the upper body, practically it is not as straightforward and depends on many factors, such as the walk- ing speed, balance and stability of the robot. Usually, since robots have a low speed, and do not have the same dimensions as the participant controlling it, directly mapping the movement of the legs is not a feasible solution, and also may result in loss of bal- ance, although the solution developed by (Koenemann & Bennewitz, 2012) takes the centre of mass of the robot into account as well, thereby always maintaining the bal- ance. However, even if this were implemented, it would only work for the Nao and not the Robothespian, as the latter does not have programmable legs. Hence, a univer- sal, generic solution for this problem was devised. The algorithm we have developed for translation and rotation in space instead tracks the torso position of the user rather than individually tracking the legs. Initially, the user’s torso position (2D coordinates on the X-Z plane) is obtained and at every frame, it is compared with the updated posi- tion of the torso. Whenever a robot-based threshold is crossed, the walking mechanism is triggered. The advantage of this algorithm is that it is robust and can be used with any robot that has the ability to move, and does not depend on how the robot moves. It could have any mechanism, such as legs, wheels, a movable platform, and the algorithm would still work. This mapping library, including the section with motion retargeting to humanoid robots was presented at the VRST 2013 conference as a systems paper (Spanlang et al., 2013).