Besides deciding on the visual representation of the user’s hands and controllers, several aspects of the hand avatars’ behaviour should still be decided. The first is a similar problem as the unconstrained objects have, as described in section 4.1. This is the question whether the position of the object, its virtual physics or a combination of the two should be prioritized. The difference between the hand avatars and other unconstrained objects is that the hand avatars have a connection with real-world objects at all times, and unconstrained objects
70
in the virtual world only have this connection when the user interacts with them. This makes the solution mapped out for the unconstrained objects less viable in this case, since the break force threshold effectively forces the controller to stop interacting with the unconstrained object, allowing the object to be controlled by the virtual physics.
There was another issue that has been encountered when iterating on different interaction interfaces and also briefly
mentioned in the previous section. This issue is that moving the hand avatar to a position
that no longer corresponds to the user’s actual hand position breaks the user’s presence, which is undesirable. The best solution is therefore to always keep the hand avatars fixed to the location of the controllers, and therefore the user’s hands. The disadvantage of this solution is that the hands will be able to move through other objects, however this is an acceptable compromise considering other options have more downsides.
To compensate for this downside, some kind of feedback could be given to the user, to inform them they were performing an illegal movement. This could be done for example by giving off a tactile pulse with the controller while the user’s hand is within another object. After some deliberation, it was decided not to choose for this feedback, as it was considered to be distracting. The tactile pulse emulates the initial touch of a surface within a reasonable degree, but it does not approximate kinaesthetic feedback (the missing kind of feedback) in any way. It also interferes with the intended effect of the vibration for the fire- safety training that will be described in more detail in section 4.3.
Another way to negate this downside as much as possible, and simultaneously reduce the
Chapter 4 - Concept Detailing
barrier between real world and virtual world, the hand avatars should be able to interact with movable objects without the need to press a button. More specifically, objects should be able to be pushed around if their constraints allow for it. Not only does this make it less noticeable that the hands can move through objects, it also increases the fidelity of the environment, and the presence of the user. It does this by meeting the user’s expectation that for example cabinet doors can be pushed close without grabbing them. This is also a useful mechanic for the fire-safety training. Consider the manner in which a microwave door is closed. This is not done through grabbing the door and closing it, but by merely pushing against the door’s surface. This mechanic therefore allows for a quicker and more natural action of e.g. closing the oven door.
One unresolved topic is how a virtual hand and/ or object should react when they interact with one another. The simplest solution is to have no additional step after an object is grabbed. The hand avatar can simply be present at the location where it should be, and the object will then follow the controller as well. This is an inelegant solution, as there is a high chance the hand will be in an unnatural position in relation to the virtual object. Many VR applications have
also reached this conclusion, and a variety of solutions have been devised in response. Some of these have already been briefly mentioned in previous sections.
Among these previously mentioned options is to have the hand or object snap to a certain matching position and orientation. As discussed before, moving the hands separate from the controller is undesirable, so moving the object remains as an option. This solution leads to several problems. Firstly, most objects can be held in a variety of orientations, and every orientation has to be pre-programmed. The developers may not always account for the intent of the user, causing the simulation to behave in a way the user does not expect. See Fig.47 for an example of one object being
held in three different orientations, requiring three different hand-object poses. Secondly, this requires the object to move to this predefined position. This can be done either instantaneously, or through interpolation. Either option is not ideal, as it interferes with the desired fidelity and the user’s expectation. Lastly, constrained movable objects will be unable to snap to a position outside of their constraints, allowing this concept to only provide a valid solution if the controller is held in one specific orientation. These problems make this solution a less viable option for the fire-safety training.
Another solution mentioned before is to have the controller avatar disappear when grabbing an object (tomato presence). This solution has
71
the advantage that a limited amount of fidelity is sacrificed for several big advantages. Firstly, it is an easy solution to develop, as no separate hand poses have to be modelled, and no
predefined orientation for different objects has to be defined. Secondly, not having any solution in place will also negatively influence fidelity and expectations, and this solution provides better management of the user’s expectations. Considering existing VR applications have proven the merits of this method, and it has the least disadvantages for the fire-safety training, this method is the most viable option for this project. One detail that should be taken into account when implementing this solution is that the grabbed object should maintain its pivot at the position of the controller to ensure the movement complies with the user’s proprioception.
72
4.3 Grounding the
experience
The previous section already explored some additions that will further improve the user experience, such as the movement of the thumb. There are several other additions that provide the user with cues to ground them in the virtual environment, which are also present in the real world.
Sound
One of the most important aspects that will drastically improve immersion and presence is adding sound effects. Foley (named after sound-effects artist Jack Foley [31]) is the reproduction and addition of ambient sound effects to various media, most commonly films. These effects are usually exaggerated versions of certain sounds that would not be heard in real life, such as footsteps or the movement of clothing. Although these sounds are exaggerated, and the user doesn’t actively notice them if they are present, the user will certainly notice if they are absent, making the film feel unnaturally quiet. This concept can be translated to the VR environment, as it helps to compensate the lack of accurate haptic feedback. Considering Foley artist is a profession in and of itself, and as of yet
is mostly applied to film, an entire study could be carried out to how this aspect translates to a VR environment. A reasonable assumption would be that the sounds in a VR environment would also require a certain level of exaggeration, as it will help compensate for the lack of certain sensory input, such as haptic