1. Exploring 360 movies and its implications to user embodiment
There is a lot that we do not understand about the uprising popular 360 movies nor the effects of placing ourselves in 360 movie virtual environments. Graphics quality-wise, the constantly changing moving viewpoints in 360 movies raises a lot of challenges to image-based lighting and etc for realistic visualizations. Putting aside these difficulties, we want to understand people’s reactions to seeing them- selves in a constantly changing environment.
We broke down 360 movies into their separate characteristics (genres, view per- spective, etc) and observed its effects on presence and engagement. Based on
Chapter 7. Conclusion and Future Work 106
participants feedback, whether the movie is live or animated mattered the most to them. Though we had no significant differences in sense of presence and user en- gagement, we highly suspect it might be different if the FOV of the visualization is expanded. Also since we were using movies obtained from YouTube and not filmed on our own, we could not control every characteristic of the movie which could influence our investigation.
2. Incorporating a view of the user’s body through visualization into an immersive 360 movie
A lot of work has been done on user embodiment in virtual environments but not with the 360 movie as the virtual world. The fact that it has a story and a deeper emotional connection with users influences differently on how viewers feel present and engaged in the virtual environment. Our research showed a significant difference in the sense of presence and level of user engagement, though some participants felt that seeing their own body was not enough and should come with meaningful purposeful interactions. This has to be explored deeper but is understandable and insightful in how a VE with a story behind it can be so different from a traditional VE.
3. Blending views of both the user’s body and surroundings, through vi- sualization controlled by different designs, into an immersive 360 movie Based on an idea of a dream, fading from the real physical world to an imaginary virtual world, we explored people’s preferences on how much control they would like to have over the transition. There are three types of control: Manual control (by the viewer only), Automated control (by the system only), Shared control (between the viewer and system). Results showed viewers are split between shared and manual control, with automated being the least preferred. Participant feedbacks commonly address the importance of the movie director’s influence, such as if one of the best scenes requires the viewer to see his/her body, then the system should have total control since the director has intentionally done this. Furthermore we believe this type of control can be generalised for any other interactions between a viewer and the system in a cinematic VE.
4. Using hand gestures to control the visualization blending of the user’s body and surroundings, in an immersive 360 movie
Chapter 7. Conclusion and Future Work 107
Participants were mostly neutral to slightly positive in using hand gestures as the interface to interact with the controls over the visualization blending. Participant feedbacks commonly address the type of 360 movie content is played. If the movie has more game elements then more controllability and accuracy is needed such as buttons. But if the movie game is action-packed and requires the usage of hands then certain hand gestures cannot be used. If the movie is a more dwelling and introspective experience, simple controls like the currently implemented hand gestures are sufficient.
5. Explore how visualization and its control can meet the practicalities of a home entertainment system
We found that one could use the controls over the visualization blending as a way to stay or switch connection to the real physical world or virtual world. A few reason why people would want to are 1) to have a drink or eat snacks while watching the movie 2) briefly talk with friends sitting beside you 3) find your smart-phone to answer a phone call 4) most importantly we can do this without taking the HMD off which is known to significantly break immersion.
Participants loved the idea and based on their feedback, the system’s usability can be further improved. The currently implemented blending in and out of the virtual and real worlds is great when needing to do real physical tasks as mentioned above. But within the cinematic VE, a modes approach should be used such as one mode would have only the virtual world viewable, next mode would be with user body visualization, another would have user body and surroundings visualization, last mode would be just the real physical world viewable. The swiping left and right gestures would be the best for swiping between modes so that viewers can easily switch to their desired visualization in one quick instant.
6. Proposal of a home entertainment system with user body visualization capabilities
This system design is one portable lightweight solution to the complex registration and calibration set-ups needed for immersive cinematic experiences. Depth keying is needed for visualization blending between virtual and real world, thus at least one depth camera is needed. RGB-D Camera/s to support a large FOV of the visualization, ideally as large as the HMD used such as Oculus Rift. The usability
Chapter 7. Conclusion and Future Work 108
design for fading/blending while engaged in the cinematic VE, is a swipe-between- modes approach utilising both “SWIPE LEFT” and “SWIPE RIGHT” SoftKinetic iisuTMsupported gestures. Usability design for fading/blending while not engaged in the cinematic VR to do minor real-world tasks without taking HMD off, is a gradual-increasing-decreasing approach exactly how our current system works, utilising the “BIG 5”, “THUMB UP”, AND “THUMB DOWN” SoftKinetic iisuTM supported gestures.