Xbox Controller Gaze Aim

In first person perspective games played on a two-dimensional display with an Xbox controller, aiming any sort of projectile weapon is most often done with a two-dimensional crosshair fixed at the center of the player’s gaze that marks where the player is aiming. In a two-dimensional space, the crosshair can be rendered as a sprite at the center of the player’s display.

We chose to use the same style of aiming for players using an Xbox controller. In games played on a two-dimensional display, the right analog stick on an Xbox controller is typically used to tilt and pan the direction of the player’s gaze, thus changing where they are aiming with the crosshair. In our game we do not allow the player to tilt their gaze with the right analog stick, because that would result in the player’s head being oriented level with the horizon in real space but not in virtual space, which is disorienting. We do allow for camera panning with the right analog stick, which rotates the player thus adjusting their gaze and their aim. Most of the player’s gaze adjustment is done by looking around with physical panning and tilting of their head. The precision of these movements is the reason we chose to use gaze aiming, where the player is always aiming at a crosshair fixed to the center of their gaze.

However in a three-dimensional space that players experience in VR, there is no two- dimensional display to draw a sprite for the crosshair on. To draw the two-dimensional crosshair in three dimensions, it must be placed on a plane somewhere in three-dimensional space. If the crosshair plane is very far away from the player, then objects in the environment will be able to move in between the crosshair and the player’s viewpoint, making the crosshair useless for aiming. If the crosshair plane is close enough to the player’s viewpoint so that objects are unlikely to occlude it, then the player’s eyes cannot focus on both the crosshair and what they are aiming at. If you hold your finger in front of your nose, and try to look at both

20

your finger and any object a few feet or more in front of you, you will experience the same problem of not being able to focus on both. In addition, a crosshair drawn this way fails to assist in aiming, whether in focus or not. An object so close to one’s viewpoint, if viewed with one eye closed, appears in a different place depending on which eye is looking at it and which eye is closed. The player’s aim is not based on which eye they have open, so the crosshair must remain consistent no matter which of the player’s eyes are open.

We tested two solutions to this problem, which both take the crosshair out of three- dimensional space and attempt to draw it on the player’s head mounted display in a way that is both comfortable to view and in focus for the player.

Figure 11: A diagram of the layers used to render the crosshair to one eye Figure 11 shows our first solution, which was to draw a crosshair directly on top of the center of the image that is displayed in front of the left eye. The left eye was chosen arbitrarily; however we tried the same setup with the right eye and found no different results. With the crosshair drawn directly on top of the environment, even if parts of the environment were closer to the player than the crosshair appeared to be, the crosshair would not be occluded. The reason this was done for only one eye was so that the crosshair could only be seen as over one target instead of two depending which eye was open. If the player had their left eye closed,

Virtual Environment Layer

Crosshair Layer

Virtual Environment Layer

left eye right

21

then they would see no crosshair, which is better than an incorrectly placed crosshair. If the player had their right eye closed, then they saw the crosshair in the correct location, the same as where it was with both eyes open. In practice, this worked as intended, but at the cost of bothersome eye strain to the player. This was likely due to the player’s eyes trying to focus on something that was only visible to one eye.

Figure 12: A diagram of the layers used to render the crosshair to both eyes Figure 12 shows the solution we ended up using in the game. The crosshair is drawn on both eyes this way. The crosshair is drawn as a very large sprite on a plane in three-dimensional space very far away from the player, but in this case, it is drawn last so it occludes the

environment even though objects in the environment are closer to the player’s viewpoint than the crosshair. The environment and crosshair each exist in their own space with their own distance from the player’s viewpoint, and are rendered separately to frames for the left and right eye. The frames that contain the crosshair are placed on top of the frames with the environment, as seen in Figure 12. The resulting effect is a crosshair that appears in the same place in the player’s view no matter if either eye is closed, is always in focus, and is not significantly uncomfortable for the player’s eyes.

Crosshair Layer

Virtual Environment Layer

Crosshair Layer

Virtual Environment Layer

left eye right

22 Figure 13: Dynamic

Crosshair over no surface

Figure 14: Dynamic Crosshair over grapplable

surface in range

Figure 15: Dynamic Crosshair over no surface, and with right

Lighthook attached The first iteration of the crosshair was just an X drawn at the center of the screen. In its current iteration, the crosshair is used to display information about the player’s two Lighthooks. The crosshair itself that is being drawn has a left and right half each with three parts. The left half of the crosshair is for information about the left Lighthook and the right half of the

crosshair is for information about the right Lighthook. In Figure 13 the outermost semicircles of the crosshair are displayed. These semicircles are always present and used just for aiming.

In Figure 14 the crosshair has lined up over a surface which is within range and

grapplable, so the innermost semicircles have become visible to indicate that if the player fires a Lighthook, then that Lighthook is able to attach to the current aiming target. This helps the player know when to shoot as they are trying to line up a shot in situations like falling through the air.

In Figure 15 the middle right semicircle of the crosshair is visible because the right Lighthook is attached to a surface. This is important to know when aiming, since if the player

23

does not know a hook is already attached and they try to fire that hook, the hook will detach instead of firing.