Evaluating the applicability of an immersive
virtual reality-based training device for
dynamic posture control in healthy people
Master Thesis
For attainment of the academic degree of Master of Science in Engineering (MSc)
in the Master Program Digital Healthcare at St. Pölten University of Applied Sciences
by
Jürgen Maureder BSc.
1810756805
First advisor: Dr. Brian Horsak
Wien, 26.05.2020
Declaration
I declare that I have developed and written the enclosed Master Thesis completely by myself and have not used sources or means without declaration in the text. Any thoughts from others or literal quotations are clearly marked. This work was not used in the same or in a similar version to achieve an academic grading or is being published elsewhere.
... ...
Place, Date Signature
Preface
„Veränderungen begünstigen nur den, der darauf vorbereitet ist.“
Louis Pasteur
First of all, I would like to thank my wonderful fiancée who supports me all the time and made this master study initially possible. Big thanks to my mother, who always believes in me. I also like to thank my future parents- in-law for awakening my interest in technologies in the first place.
Further, I like to thank the project team of the Digital Health Lab and the volunteering participants who supported me in the implementation of my project. Last but not least, I would like to thank my supervisor Dr. Brian Horsak, whose constructive manner not only helped me to complete my work, but also helped me to improve my scientific skills.
Abstract
Virtual Reality (VR) promises great potential in health care and rehabilitation. VR is increasingly used for motor rehabilitation. Side effects such as motion sickness have not yet been sufficiently investigated. This thesis deals with the question whether a newly developed, immersive virtual reality-system consisting of a Vicon Motion Capture System with included force plate and an HTC Vive head mounted display is applicable for balance training in rehabilitation regarding the usability and user experience, the physical enjoyment and motion sickness. Furthermore, the dependence of these factors on age, gender, previous VR-experience and the physical status of 49 healthy subjects will be investigated. The participants tested the VR-application for balance training, in which they stood on a force plate wearing the VR-glasses for three rounds of three minutes each to burst virtual soap bubbles. Before and after the application, the participants filled out a questionnaire for this purpose. One trial lasted 45 minutes, there was no repetition date. None of the test persons stated that they had suffered from Motion Sickness. With the help of the Simulator Sickness Questionnaire (SSQ) and the Virtual Reality Sickness Questionnaire (VRSQ) no significant differences between women and men were found.
There is a strong negative correlation between the level of motion sickness and age. The VR-experience and physical status had no influence on motion sickness or the enjoyment of the activity. Enjoyment of activity was measured with the physical activity enjoyment scale (PACES), which showed a high overall joy of VR-application. For assessing the usability and the user experience, the System Usability Score (SUS) and User Experience Questionnaire (UEQ) were used. The usability and user experience ranged from “above average” to “excellent”. In conclusion, the positive aspects outweigh the negative ones. A clear recommendation of the applicability of the VR-system for balance training can be given. Further investigations with "conventional" balance training are desirable. The study referred to healthy volunteers. Further investigations, which include patients, are necessary to determine further aspects of the VR-system.
Kurzfassung
Virtual Reality (VR) verspricht großes Potential im Gesundheitswesen und der Rehabilitation. VR wird immer öfter für die motorische Rehabilitation genutzt. Nebeneffekte wie Motion Sickness sind noch nicht ausreichend untersucht. Diese Arbeit beschäftigt sich mit der Frage, ob sich ein neu entwickeltes, immersives Virtual Reality-System bestehend aus einem Vicon Motion Capture System mit inkludierter Kraftmessplatte und einer HTC Vive für Balancetraining im Rehabilitationsbereich eignet. Dabei wird die Anwendbarkeit und Nutzer-Erfahrung, die Freude an der Aktivität sowie die Motion Sickness untersucht. Weiters wird die Abhängigkeit dieser Faktoren vom Alter, Geschlecht, VR-Vorerfahrung und dem sportlichen Status von 49 gesunden Personen erhoben. Die Proband*innen testeten die VR-Anwendung für Balancetraining, in der sie mit einer VR-Brille auf einer Kraftmessplatte für drei Runden zu je drei Minuten stehen und virtuelle Seifenblasen zerplatzen. Vor und nach der Anwendung wurde ein Fragebogen von den teilnehmenden Personen ausgefüllt. Eine Testung dauerte 45 min, es gab keinen Wiederholungstermin. Keiner der Proband*innen gab an, an Motion Sickness erkrankt zu sein. Mithilfe des Simulator Sickness Questionnaires (SSQ) und des Virtual Reality Sickness Questionnaires (VRSQ) konnten keine signifikanten Unterschiede zwischen Frauen und Männern festgestellt werden. Es besteht eine starke negative Korrelation zwischen dem Maß an Motion Sickness und dem Alter. Die VR- Erfahrung und der physische Status hatten keinen Einfluss auf die Motion Sickness oder der Freude an der Aktivität. Die Freude an der Aktivität wurde mit der Physical Activity Enjoyment Scale (PACES) erhoben, welcher insgesamt eine hohe Freude an der VR-Anwendung ergibt. Die Anwendbarkeit und Nutzer-Erfahrung wurde mit dem System Usability Score (SUS) und dem User Experience Questionnaire (UEQ) erhoben.
Dabei konnte eine Usability und User Experience von „überdurchschnittlich“
bis „exzellent“ festgestellt werden. Schlussfolgernd, überwiegen die positiven Aspekte den negativen. Eine klare Empfehlung der Anwendbarkeit des untersuchten VR-Systems für Balancetraining kann ausgesprochen werden. Weitere Untersuchungen mit „konventionellem“
Gleichgewichtstraining sind wünschenswert. Die Untersuchung bezog sich auf gesunde Personen. Weitere Untersuchungen, welche PatientInnen inkludieren sind erforderlich, um weitere Aspekte des VR-Systems zu erheben.
Table of Content
Declaration ... II Preface ... III Abstract ... IV Kurzfassung ... V Table of Content ... VI
1 Introduction ... 1
1.1 Problem ... 1
1.2 Purpose of this thesis ... 2
1.3 Question of research ... 2
2 Background and Related Work ... 4
2.1 Definition and application areas ... 4
2.1.1 History of Virtual Reality ... 4
2.1.2 What to expect in the future ... 6
2.1.3 Application areas ... 7
2.2 Virtual Reality in Physiotherapy ... 10
2.3 Motion Sickness in Virtual Reality ... 14
2.4 Measurement of Motion Sickness ... 15
2.4.1 Motion Sickness Questionnaire ... 16
2.4.2 Simulator Sickness Questionnaire ... 16
2.4.3 Virtual Reality Sickness Questionnaire ... 17
3 Immersive Virtual Reality ... 19
3.1 Requirements for immersive Virtual Reality ... 19
3.2 Technical aspects of immersive Virtual Reality ... 20
4 Balance Training ... 24
5 Virtual Reality Project ... 26
6 Methodology ... 28
6.1 Participants ... 28
6.2 Study protocol ... 30
6.3 The Virtual Reality System ... 33
6.4 Questionnaires ... 34
7 Evaluation Results ... 37
7.1 Evaluation of the VR-System ... 37
7.2 Evaluation of Physical Activity Enjoyment ... 41
7.3 Evaluation of Motion Sickness ... 42
8 Discussion ... 48
9 Conclusion ... 52
Literature ... 54
List of Figures ... 63
List of Tables ... 65
Appendix ... 66
1 Introduction
In the world of medical and health professionals, progress is constantly influenced by the emerging technological possibilities. Virtual reality (VR) is such a relatively young and emerging technology. While users can interact with virtual objects, they immediately receive visual, acoustic or haptic feedback. Immersive systems offer the most direct experience such as glasses which are worn on the head to change the visual perspective with the head position (Sengupta, Gupta, Khanna, Krishnan & Chakrabarti, 2019). Clinical implementation rapidly follows scientific discovery and technological advancement all over the world. The results are a rapid growth in the number and type of applications of VR for rehabilitation, suggesting that a new promising scientific field is uprising (Garrett et al., 2018).
1.1 Problem
There are several studies that have investigated, that the VR-technology can improve balance and gait in neurological or musculoskeletal patients (Hamzeheinejad, Straka, Gall, Weilbach & Latoschik, 2018). Lack of balance in terms of postural instability is associated with gait impairment and falls (Swee, Wong, You & Kiang, 2017). According to the WHO, falls are the second most common cause of accidental death worldwide with an estimated number of 646 000 victims. Therefore, balance training is one of the most important preventive measures and is often part of rehabilitation programs (World Health Organization, 2008). Studies have shown, that balance training in healthy young adults and in healthy adults older than 65 years, has to be specifically designed, to train certain balance abilities in daily life, for instance standing safe in a tram (Kümmel, Kramer, Giboin &
Gruber, 2016 & Lesinski et al., 2015).
VR can be a good solution for this issue by making balance training more specific, realistic and goal- orientated (Chen et al., 2016). Regardless of the therapeutic benefits, VR has been shown to increase motivation and enjoyment of movement. This ultimately leads to increased self- responsibility, which in turn helps to achieve the therapeutic goal (de Vries, van Dieen, van den Abeele & Verschueren, 2018). However, “Motion Sickness” can be a limitation for some user.” Motion Sickness” is an
aversive state of behaviour that can affect several psychophysiological reaction systems. Therefore, a person can suffer from various symptoms, such as nausea or dizziness. It strongly depends on the context and the used VR-application (Gianaros et al., 2010).
1.2 Purpose of this thesis
The purpose of this thesis is to evaluate the applicability of a virtual reality application for balance training by asking 49 users how physically joyful they evaluate the training and if “Motion Sickness” possibly affects their impression. Therefore, quantitative questionnaires which collect data of physical enjoyment and “Motion Sickness” will be used (Kendzierski &
DeCarlo, 1991; Kennedy, Lane, Berbaum, & Lilienthal, 1993, p. 203; Kim, Park, Choi & Choe, 2018). To be able to answer the questions, all asked users got the chance to test a virtual reality game with an immersive VR- Headset. The data about movement and postural sway during the application was collected with a force plate and a three-dimensional motion capture system. The collection of this data is only intended to realistically replicate the application. This data is not evaluated in this thesis and will be discussed in another paper. A further subject of this examination is to identify if age, sex, current physical status or technical knowledge have influence on the outcome of the evaluation.
In this thesis, the reader will get an overview about virtual reality and its side effects with focus on motion sickness and how this promising technology already influences physiotherapy in different settings with all its challenges.
The second part of this thesis deals with the study that was conducted to answer the research questions listed below. Therefore, the methodology of this study will be explained, and the study results will be presented and discussed. These results are meant to create a foundation for future work with the evaluated system at the St. Pölten University of Applied Sciences in Austria.
1.3 Question of research
This thesis wants to respond to the following questions of research:
• Is an interactive, immersive virtual reality system applicable for balance training considering user experience, usability, physical enjoyment and motion sickness?
Secondary questions of research:
• How do users rate the experienced user experience, usability and physical enjoyment, as well as possible side effects such as "motion sickness" when using the VR-system?
• Are these factors depending on age, gender, physical condition and VR-experience?
2 Background and Related Work
2.1 Definition and application areas
2.1.1 History of Virtual Reality
Virtual reality (VR) is a computer-based technology which creates a virtual environment of an imagined or replicated world or simulates presence in the real world. It is only virtual based and hides the reality entirely. Virtual reality allows the user to manipulate and interact with objects in this virtual world.
This interaction of the user leads mainly to visual or acoustic stimulation.
How and in which intensity depends on the game or virtual setting the user is in and also which system is used. Fundamentally a distinction of immersive, semi-immersive or non-immersive VR-systems can be made.
The most intense VR-experience can be expected with an immersive system where the user wears an opaque head-mounted display (HMD) as shown in Figure 1. Through the head-mounted display the user dives in the virtual world and literally feels like being in another place. Stereoscopic projections which display a three-dimensional picture in a rigid visual perspective can be classified as semi-immersive. Non-immersive VR is known as two-dimensional presentations in established systems such as a computer desktop (Sengputa et al., 2019).
Figure 1: Immersive Oculus Quest VR-Headset (Google, roalty-free picture)
The term „Virtual Reality “, as it is known today, was first used in 1987 by Jaron Lanier (Lanier, 2013, S.202). The first head mounted display was invented years earlier by Ivan Sutherland in 1968. To be precise, he invented the first Augmented Reality (AR) HMD which was meant to be a digital overlay to the visible world. Basically, it was a heavy suspension from the ceiling of a basic wireframe room, and it was called “The Sword of Damocles“ and “The Ultimate Display” (Sutherland,1968). The first attempts of consumer-friendly VR were in the 1990’s and were mainly focused on entertainment. Stephen King’s movie “The lawnmower man” introduced the concept of VR to a wider audience for the first time. Due to a lack of missing technological progress, side effects like dizziness and headache were dominating, therefore VR was a commercial failure at that time and not ready yet (Garrett et al., 2018; American Film Institute Catalog, 2020).
By searching for literature for virtual reality a demarcation is necessary.
Many studies talk about VR but do not clearly describe which kind of VR was used. For example, by searching Song & Park (2015) talk about VR- games for balance training, but in fact used the Xbox Kinect, which is a motion sensing input device for motion tracking.
De Rooij, van de Port and Meijer (2016) published a systematic review and meta-analysis about the effects of VR-training on balance and gait ability in patients with stroke. They included twenty-one studies in their review but did not distinguish between virtual reality which include “2D” methods and immersive systems which use HMD.
Studies using ‘‘3D’’ methods for virtual environments include those with head-mounted displays and cave automatic virtual environments. Examples include a study evaluating a cranial tumor simulator as a tool of planning for surgery (Mazur, Mansour, Mugge & Medhkour, 2018). Wiebrands et al.
(2018) used VR to immerse user into virtual environments for studying biomolecular structures.
According to the Gartner hype curve (Figure 2) new technologies go through different phases including the trigger, the peak of inflated expectations, the trough of disillusionment, the slope of enlightenment, and the plateau of productivity. Virtual reality is now coming out of the bottom of disillusionment, after the first excitement is over and the initial interest in immersive technologies is growing again. VR is currently on the ascending branch; a secondary wave of innovation begins and new chances especially in the health and medicine sector for the technology are existing (Gartner, 2017).
Figure 2: Gartner Hype Cycle for Emerging Technologies 2017 (Gartner, 2017)
2.1.2 What to expect in the future
As mentioned before, VR seems to be on the cusp of a technological breakthrough. A few years ago, VR-glasses were bounded to computers with good performance via cable. The navigation was only possible with two controllers in the user’s hand. Today some companies provide wireless VR- glasses with integrated small computers. The next big step is to get rid of the controllers and push hand tracking forward. These actions have big influence on the user’s freedom, it makes VR possible to use basically everywhere. The current used 4G network will not be able to cope with the amount of bandwidth (=maximum rate of data transfer) and latency (= time delay) requirements of wireless virtual reality-systems. The globally advancing expansion of the 5G network, will provide a much faster wireless data transfer, which allows better quality of games or systems in different application areas. There is no doubt that the future lies in VR, even in networked VR, despite its research and scientific challenges, it will continue to grow in importance in the coming years (Bastug, Bennis, Medard &
Debbah, 2017).
2.1.3 Application areas
Virtual reality was initially used in the entertaining branch and is now gaining more popularity than ever before. But since VR is on the slope of enlightenment different markets are emerging. According to Goldman Sachs Global Investment Research (2016), healthcare will be representing a leading market in virtual reality with an estimated worth of around 5 billion dollars as seen in Figure 3.
Figure 3: Estimated worth of VR (Goldman Sachs Investment Research, 2016)
In healthcare many different medical specialties could benefit from virtual reality. Izard et al. (2018) described two VR-systems for surgical education.
Their used systems consisted of the gear VR (Oculus mobile) and a 360- degree camera for virtualizing a real-world scenario. They concluded that their application was helpful for medical students to maintain the possibility to get familiar with the most important steps of surgical procedure as often as they want. In order to the unlimited repetition, VR-systems are suitable to automate necessary steps in a surgical setting. This chance may be also
useful for internalizing human anatomy or other medical specialties for students in medicine or healthcare professions.
Virtual reality does not just open up the possibilities in medical education, yet the user can also be experience psychosocial aspects such as empathy.
Dyer, Swartzlander and Gugliucci (2018) used an immersive VR-device for training of the workforce for aging service in cooperation with Embodied Labs, who specialized on immersive VR in healthcare. They gave medical and health professional students including physiotherapists and nurses the opportunity to simulate being a patient with age related diseases such as loss of vision and hearing and also cognitive impaired conditions such as Alzheimer’s disease. Due to the positive feedback of the students, the author of this work recommends using VR as a supplement for teaching empathy.
The technical development of virtual reality resulted also in a new field of psychotherapy: virtual reality exposure treatment. Carl et al. (2018) opines that VR suits well for treating anxiety disorders such as phobias, post- traumatic stress disorder and social anxiety disorders. Furthermore, those goal-oriented VR-experiences can easily be transmitted into real life situation and give the anxious patient a useful tool to cope with these kinds of diseases. This conviction is supported by ample evidence and an interesting step towards rising numbers of mental disorders in society (Powers & Emmelkamp, 2008; Carl et al., 2018; Opris et al., 2012).
Hoffman et al. (2011) were exploring analgesia by distracting burn patients who suffer from cruel pain with immersive virtual reality. Especially the wound care of severe burned patients is one of the most painful experiences, where not even opioids provide remedy. Hence the University of Washington invented the so called “SnowWorld” VR-game. The game principles are quite simple: While the health professionals do their necessary work, the patient interacts with snowmen and penguins in a cold, winterly environment. The group of patients who received the standard medication in combination with VR reported a 35-50% reduction in pain level compared to the group with solely standard medication. The explanation of this phenomenon is quite reasonable, since humans have limited attentional capacity. During the application of virtual reality, a substantial amount of this capacity is used. Consequently, the incoming signals from pain receptors are blocked and the patient experiences less pain. What is more, the patient eventually enjoys the VR-application, which also positively effects the pain level (Hoffman et al., 2008; Eccleston & Crombez, 1999).
With this knowledge of acute pain, Rutledge et al. (2019) tested a virtual reality-based mirror therapy with the Oculus VR-HMD for the treatment of chronic phantom limb pain in veteran amputees. Following the principle of mirror therapy, the amputee experienced their limbs intact through the
digital visual sensation and activity in the virtual world. With this equipment, the participants got asked to use a compact modified peddler for exercising in combination with a wireless motion sensor (Figure 4). Among the 14 participants the phantom limb pain and also the unpleasant phantom sensations were significantly lower after the initial VR-treatment. 4 out of 14 participants went through multiple VR-seasons and experienced a much greater relief of chronic pain than the rest of the group.
Figure 4: VR-Hardware, consisting of a motion sensor, bicycle pedaler, computer, Oculus headset, and prosthetic pedal (Rutledge et al., 2019)
In a nonrandomized, controlled trial from Tashjian et al. (2017) of the renown cedars-Sinai medical center, they came to similar results with a cohort of each 50 subjects per group with different pain producing medical conditions. The average pain reduction in the VR-group after experiencing a 15 minutes VR-application (“Pain RelieVR”) was significantly lower compared to the control group which watched a 15 minutes nature video.
There were no significant differences between age, sex or ethnicity.
VR-applications already extend over some special fields and fields of application in the medical field. Scientific papers often use the same software and hardware providers. Although this does not reflect the entire market, it suggests that the providers of medical solutions in the digital healthcare sector of VR are still manageable.
2.2 Virtual Reality in Physiotherapy
You et al. (2005) assigned 10 stroke patients randomly to either the VR- group or the control-group which did not receive treatment. They used the
“IREX “-VR-system which requires a monitor, a video camera, gloves and virtual objects. This system was one of the first VR-systems designed for active rehabilitation. It does not need a HMD, enables the user to manipulate the virtual objects and navigate through the virtual world. The interface and application are developed for specific exercising to improve range of motion, balance, mobility and gait skills as seen in Figure 5.
Figure 5: Interface of “IREX”-VR-System (www.gesturetekhealth.com/products/irex)
The authors could prove a significant reorganization of the motor cortex with a functional MRI. This investigation shows the positive impact of VR in locomotion in the first place, yet no immersive system was used. In relation to the study of this master thesis, only studies and projects which used an immersive VR-system will be included as related work.
Swee et al. (2017) developed a virtual reality-system for balance training.
They used a mobile based VR-headset in combination with the Microsoft Kinect sensor for body tracking in combination with a leap motion sensor as an input device to provide realistic movements while walking in the virtual reality. Also, a personal computer for data processing was used. The
application is designed for post-stroke patients to restore dynamic balance skills and walking ability. The study team used a software called Trinus VR which is able to turn a mobile based VR-system into a PC based one. Also, the well-established Unity software was used for game development. This software allows to deploy applications for PC, gaming consoles, mobile devices and websites. Since it also introduced a built-in VR-support, it is gladly used for VR-systems. For the graphical user interface (GUI) for the VR-rehabilitation system, Microsoft’s Visual Studio was used as the Integrated Development Environment (IDE). It supports different programing languages which are necessary to build such a system, for example C, C++ and C#.
With this system, 6 post stroke-patients trained their balance and gait ability.
The balance training consisted of walking on an even, empty walkway with a minimum length of 3 meters, while virtually the patients were walking over a virtual wooden bar, whilst measuring the required time (Figure 6).
Swee et al. (2017) came to the conclusion, that their system is suitable for restoring and improving balance skills in post stroke patients, although they do not mention the importance of goal-orientated balance training. The importance of goal-orientated balance training is topic in chapter 4 of this work. However, Swee et al. (2017) recommend virtual reality for balance training.
Figure 6: User experience of the balance training application (Swee et al., 2017)
Another immersive VR-system (Figure 7) for neurologically caused gait impairment was tested by Hamzeheinejad et al. (2018). The aim of this user- center design study was to increase the motivation of rehabilitation measurements with healthy participants. The VR-gait training was combined with an exoskeletal robotic assistance. For the VR-experience the
HTC Vive HMD was used. The study design was single blinded and randomized including 21 healthy participants (10 male) at the age of 19 to 35. The inclusion criteria were normal or corrected to normal vision and the absence of musculoskeletal impairments. The participants were instructed to walk in a normal walking pace on a cross-trainer with and without the VR- system for 2 minutes each. Subsequently a questionnaire had to be completed. The results show that no undesirable side effects occurred, and the user experience and acceptance was promising. Therefore, they are eager to conduct further studies with stroke-patients.
Figure 7: Application with and without VR-condition (Hamzeheinejad et al., 2018)
Kim, Darakjian and Finley (2017) recruited thirty-three people to evaluate virtual reality and its adverse effects in elderly adults with Parkinson’s disease. The included participants were eleven young, healthy adults (28±
7 years, 5 male), eleven elderly adults (66 ± 3 years, 3 male) and eleven elderly with Parkinson’s disease (65 ± 7 years, 3 male). All participants were asked to walk for 20 minutes on a treadmill with an individual speed while wearing a HMD (Oculus Rift DK2). The immersive VR-glasses viewed a virtual urban scenario to pretend walking through a city (Figure 8).
The measured side effects were all considered for safety reasons. Hence, the mini-BESTest,a 14-item balance assessment for dynamic balance and gait, the center of pressure (CoP) excursion and motion sickness were investigated with pre- and post-assessment. The static postural sway, also known also CoP, was measured within two trials with each 30 seconds of quiet standing with feet placed shoulder width apart on two force plates; one trial with eyes open and one without visual help. The motion sickness was
investigated by using the Simulator Sickness questionnaire (SSQ). The SSQ was developed for measuring simulation sickness and widely used in aviation, yet it has good acceptance for measuring virtual reality-based motion sickness (Kennedy et al., 1993). Further information of the SSQ can be found in chapter 2.4.2.
Figure 8: Representative images of the virtual urban environment consisting of buildings, avatars and a 800 m pedestrian path (Kim et al., 2017)
In all groups, none of the participants complained of motion sickness after VR-exposure. Also, the center of pressure was not affected after the trial in the Parkinson-group nor in the two control groups. Yet, the dynamic balance and gait were significantly greater in all groups after the VR-experience.
Within the groups the older participants have shown greater improvement compared to the younger, healthy adults. The authors conclude that age and the presence of Parkinson’s disease have negligible influence on the applicability of immersive VR with a HMD like the Oculus Rift.
Due to the fact that this work deals with immersive virtual reality, the main focus of the selection of studies was also placed on immersive VR. The studies in physiotherapy about immersive virtual reality are increasingly higher in the neurological field. The aim of the studies often is to find a difference between conventional physiotherapy and VR applied physiotherapy. Although, many of the above-mentioned studies were able to identify the advantages of VR, the adverse effects and technical challenges have not been sufficiently examined or discussed yet.
2.3 Motion Sickness in Virtual Reality
One of the biggest disadvantages of virtual reality is motion sickness.
“Motion sickness is an aversive behavioral state that affects several psychophysiological response systems. Because multiple response systems may be activated by real or apparent motion, an individual is likely referring to a complex set of symptoms when she or he uses the term “motion sick” (Gianaros et al., 2010).
Motion sickness is manifested by different symptoms such as nausea, dizziness or vomiting and by different contexts. These include visual simulators of vehicles, but also real means of transport like ships or cars can cause a certain symptom. Motion sickness has different subcategories which include sea-, car- or train-sickness, yet any kind of travel sickness.
Due to progress in technologies, newer forms of motion sickness were observed: such as the above-mentioned simulator sickness and VR- or cyber-sickness. Although motion sickness can be indicative of a disease, it is more likely to be a physiological response without any functional disorder while exposed to an unfamiliar motion for a certain duration. Another term of motion sickness which is used in scientific work, is motion maladaptation syndrome (Kennedy & Frank, 1985, p.21).
There is evidence that, age and gender have an effect on the degree of motion sickness. Turner & Griffin (1999) found in their study including 3256 subjects that women are more susceptible to motion sickness than men.
Their work also found that younger people feel ill and show symptoms earlier than older people. It should be mentioned that the study examined people taking public bus transport services.
Munafo, Diedrick and Stoffregen (2016) took this opportunity to investigate the disagreement of VR-sickness between men and women using the HMD Oculus Rift. The current head-mounted display systems like the Oculus Rift are remarkable technological achievements, which allow users to dive in virtual worlds indoor like at home or in a physiotherapy practice. This can quickly lead to a mismatch between real sensation and the VR-experience.
Individual reports show that a high percentage of users, after immersion in the VR-world, quickly turn into a highly aversive feeling of discomfort, disorientation and nausea. This problem persists despite intensive efforts on the part of manufacturers and is recognized as the main limitation for the widespread use of these systems. The number of studies dealing with motion sickness or rather VR-sickness with HMD is currently not high enough to have sufficient knowledge about it (Kim et al., 2018).
The research on motion sickness tries to explain this side effect with the help of various hypotheses and theories. The overstimulation theory was the first model, which was introduced in 1942. This theory is based on the assumption that the vestibular system is responsible for motion sickness.
The higher the vestibular stimulation, the higher the intensity of the motion sickness. Nowadays, it is assumed that overstimulation is caused by different systems in the body that are responsible for motion sickness (McNally & Stuart, 1942). From 1942 until today these theories have been further developed. Especially for the training of balance one theory seems to be worth mentioning: the postural stability theory from 1991. It states that motion sickness is the result of a prolonged instability in posture control.
The authors' description of this phenomenon is, that the person can control not only his own body but also the requirements of the VR-application (Riccio & Stoffregen, 1991, p. 212). This aspect emphasizes the necessity to keep the travel sickness rate low during a VR-balance training.
2.4 Measurement of Motion Sickness
When evaluating any kind of Motion Sickness, test subjects will be knowingly put into situations where motion or Simulator Sickness symptoms will arise. Many researchers use extreme scenarios like rollercoasters (Bruck & Watters, 2009) to force high Simulator Sickness values. Therefore, it is needed to inform subjects about the upcoming symptoms before the start of the experiment. It exists strong evidence that the reported Simulator Sickness is higher when given pre and post experiment questionnaires than performing only a post questionnaire (Young, Adelstein & Ellis, 2006).
Informing subjects at the start of the experiment about Simulator Sickness could cause a similar effect and cause additional sickness. In this chapter, three questionnaires are explained to get a rough impression about the development of the survey from motion sickness to virtual reality sickness.
In this chapter, three questionnaires are explained to get a rough impression of the development of the survey of motion sickness and virtual reality sickness. More questionnaires about motion sickness are existing (Gianaros et al., 2010), yet the selection was made based on the popularity and scientific recognition of the questionnaires and the context of the methodology of the study of this thesis.
2.4.1 Motion Sickness Questionnaire
In 1965 the Pensola Motion Sickness Questionnaire (MSQ) was developed to quantify motion sickness (Kennedy et al., 1993, p. 203). The MSQ consists of 25 to 30 items which are associated with typical symptoms of motion sickness and indicates the presence or degree of severity. After the questionnaire has been filled out, the items will be converted to scores on a scale range from no symptoms to experienced emesis. The resulted score includes type and severity of symptoms. In the recent years, the MSQ was often adapted to its current version due to the variety of motion sickness inducing scenarios. In most of these scenarios, participants were exposed to stimuli to induce motion sickness symptoms such as emesis or till the individuals request to quit from the tests. The reported symptoms of the tests were compared to the severity of outcomes. This has led to the development of clusters that can identify the severity of motion sickness and are displayed with a scale level to allow comparisons to be made.
2.4.2 Simulator Sickness Questionnaire
The Simulator Sickness Questionnaire (SSQ) is an advanced version of the Motion Sickness Questionnaire (MSQ), to evaluate the severity of simulator sickness. It was designed in the 1980s not to put people explicitly under motion sickness induced stress, but to evaluate the effect of simulator sickness in new simulators or computer applications. The questionnaire is used to evaluate newly tested simulators and to improve the diagnostic capability (Kennedy et al., 1993, p. 203). The symptoms of simulator sickness are very similar to those of motion sickness, but they are much less pronounced and occur less frequently. The SSQ has been used originally in the aircraft industry, where pilots are exposed to pseudo- coriolis, visual distortions or motion-related transport delays and asynchronies in flight simulations, although these conditions do not exist in the actual flight situation. The “pseudo-coriolis” illusion is the experience of an abnormal head tilt angle and visual stimulus tilt that occurs when an observer moves his head over the plane of rotation of an optokinetic stimulus that would otherwise produce a vection (Brandt, Dichgans &
Koenig, 1973).
The MSQ does not seem to be the ideal index, since some of the recorded symptoms have not been reported while simulations. Therefore, the SSQ does include less items to distinguish between motion sickness and simulator sickness. The SSQ raises 16 symptoms on a four point-scale from 0-3 (Table 1). To get a better overview, the SSQ was divided into three major categories: Oculomotor, Disorientation, and Nausea, to provide
information about particular symptom cluster (Kennedy et al., 1993). The weighting of the categories is assigned and added together to give an overall score. Although the score is not intended to predict simulator sickness, it provides a good overview of the disease in a particular simulation or simulation environment.
Due to the similarity of cyber sickness or virtual reality sickness, which is described in virtual reality, the SSQ is an accepted tool for the evaluation of VR-applications (Carnegie & Rhee, 2015). According to Google Scholar (2020), the simulator Sickness Questionnaire was cited over 2000 times.
Balk, Bertola and Inman (2013) investigated the relevance of the SSQ after 25 years of existence. They came to the conclusion, that this questionnaire still has an entitlement although technologies have changed.
SSQ Items Nausea Oculomotor Disorientation
General discomfort x x
Fatigue x
Headache x
Eyestrain x
Difficulty focusing x x
Increased salivation x
Sweating x
Nausea x x
Difficulty concentrating x x
Fullness of head x
Blurred vision x x
Dizzy (eyes open) x
Dizzy (eyes closed) x
Vertigo x
Stomach awareness x
Burping x
Table 1: 16 Items of the Simulation Sickness Questionnaire (Kennedy et al., 1993)
2.4.3 Virtual Reality Sickness Questionnaire
With the rapid development in virtual reality, new types and mechanisms of devices have emerged in recent years. Thus, interfaces and interactions often change and new applications in the field of entertainment as well as in the medical setting raise, which also changes the influence of stimuli that cause simulator sickness. Since VR is currently experiencing a revival, voices are also becoming louder to try out new ways of testing the side
effects. Therefore, Kim et al. (2018) have started the attempt to adapt the SSQ in view of the technological progress to better adapt it to the VR- environment. The aim of their study was to develop and derive a VR sickness questionnaire (VRSQ) with the help of 24 subjects (average age:
23 years; 12 female), which can be used as an index specialized in VR- environments to measure virtual sickness.
The VRSQ now consists of only two components, namely oculomotor and disorientation (Table 2). The nausea component plays an insignificant role in VR induced motion sickness according to Kim et al. (2018). With this conclusion, they explain that motion sickness is understood as an inconsistency between what is felt and what is seen. Thus, simulator sickness can be divided into three categories:
1. What is felt but not seen.
2. What is seen but not felt.
3. What is felt but does not fit.
Therefore, these three categories are recorded in the questionnaire. In a VR-environment the 2nd category is especially important, i.e. visually perceived stimuli are experienced without the associated inertial movement (Kennedy, Drexler & Kennedy, 2010).
In summary, virtual reality-sickness is a subcategory of simulator sickness.
This means that some of the characteristics of a simulator sickness do not necessarily apply to a VR-environment.
VRSQ Items Oculomotor Disorientation
General Discomfort x
Fatigue x
Eyestrain x
Difficulty Focusing x
Headache x
fullness of head x
Blurred vision x
Dizzy (eyes closed) x
Vertigo x
Total [1] [2]
Table 2: 9 Items of the virtual reality sickness questionnaire (Kim et al., 2018)
3 Immersive Virtual Reality
Since virtual reality is supposed to improve the medical everyday life and the state of health of the patients, it is essential to understand the technical aspects and to consider the user experience of a VR-application. The experience and the evaluation of technical systems are highly subjective, so it is important to collect a greater amount of data with healthy individuals before testing it with patients. This chapter will give a more detailed technical insight into VR and shows how to possibly evaluate those systems. Technological progress not only offers the consumer a better VR- experience, but also provides scientists with more cost-effective possibilities to make VR-experiments more practical and thus advance the scientific proof of VR (Niehorster, Li & Lappe, 2017). As mentioned before, this work deals with immersive virtual reality. Thus, only immersive VR-glasses are covered in this chapter.
3.1 Requirements for immersive Virtual
Reality
The simplest and cheapest form of immersive virtual reality are VR-glasses for smartphones (Gallagher, Jain & Okhravi, 2016). A prominent representative is the VR One Plus from Zeiss (Figure 9). The design is strongly oriented towards high-end devices but has hardly any technology itself. The software applications are downloaded directly from the mobile phone which is inserted into the glasses. The built-in lens in the VR-glasses enables an immersive VR-experience. This simple handling makes virtual reality possible in every home without a big technical background.
The high-end form of VR offers wired VR-glasses, which are connected to a PC and only work with it. The increased performance of the PC enables a better resolution and frame rate and enables a wider range of VR- applications. These computer-bound VR HMDs are also offered wirelessly, which increases freedom of movement as well as the safety factor by eliminating the danger of tripping over a cable. The latest technological advances are standalone VR-glasses. These have the technical requirements already built into the HMD, making them independent of the computer. No matter what form of immersive HMD is envisaged, they are now affordable to a wider range of private individuals, whereas in the past the circle of customers was smaller due to limited areas of application (Niehorster et al., 2017).
Figure 9: VR One Plus for Smartphones (Google, roalty-free picture)
3.2 Technical aspects of immersive Virtual
Reality
In the study conducted for this master thesis, the HTC Vive VR-glasses were used. These glasses are one of the most renown HMD of the last generation, so they currently offer one of the best technical achievements.
The glasses cover a nominal field of view of about 110° with a resolution of 2160x1200 pixels at a frequency of 90 Hz. Headphones and two infrared laser transmitter units are integrated in the HMD. No external cameras are needed to track the movements, instead two laser emitters, so called Lighthouses, are used. Those lighthouses function in a room flooded with non-visible light as a reference point for the VR-glasses or the VR- controllers to recognize where the VR-device is located in real three- dimensional space. This technology makes cameras obsolete. The light comes from a bundle of stationary LEDs and two rotating active laser beams. The receiver of this light is the HMD, which is covered with small photo sensors. When a light beam is received, the time it takes to hit one of its photosensors is stopped. This time and the location of this photosensor on the headset is related to each other to mathematically calculate its exact position relative to the base stations in the room. The user is able to explore and manipulate real digitized scenarios due to the built-in head-tracking, which enables the estimation of the position in the room-sized environment.
This technology increases the accuracy and therefore the quality of the medical VR-therapy (Niehorster et al., 2017).
The accuracy of the HTC Vive was determined in an experiment. In several tests, the Lighthouse devices were fixed at different distances. In some test runs the units were mounted at a distance of 7.45m. The maximum distance recommended by the manufacturer is 5.5m. The measurement error of the Vive unit was only 3cm. Within the recommended distance, measurements were also taken and a deviation of less than 2cm was found. To make sure that the measured data was not due to a faulty VIVE system, the tests were repeated with a second system (Niehorster et al., 2017).
Apart from the accuracy of the tracking system, an end-to-end latency of less than 22ms could also be measured. The latency is the delay of the movement set by the user until it is virtually available. However, two main problems of the HTC Vive have already been increasingly observed. The reference ground used by the Vive system shows a deviation of the actual ground level. As a result, roll and pitch movements are sometimes measured incorrectly and the height measurements over the tracking area change. Another problem in this context is the inclination of this reference plane, which changes every time the headset has lost and resumed tracking. These deficiencies have been found with different Vive systems and different users. These are major disadvantages of this system, especially in an application, which goal it is to improve the balance of patients (Niehorster et al., 2017).
To get an impression of the accuracy, Borrego, Latorre, Alcaniz and Llorens (2018) compared the HTC Vive with the equally high-quality HMD Occulus Rift, which is often used in studies. The technical differences are shown in Table 3. In the course of the comparison, in addition to the accuracy of the tracking system, the susceptibility to jitter was also examined. Furthermore, the working range of the VR-glasses was determined. A VR-enabled high- end computer equipped with an 8-core Intel Core i7-6700K @ 4.00 GHz, 16 GB RAM and an NVIDIA Geforce GTX 980 with 12 GB GDDR5 and the unity-software was used for the investigation.
Oculus
Rift HTC Vive
Display OLED OLED
Display
size 90 mm x 2456ppi
91.9 mm x 2447
ppi
Refresh
rate 90 Hz 90 Hz
Field of
view
Horizontal 94° 110°
Vertical 93° 113°
Lens type Fresnel Fresnel
Lens
adjust IPD (58-72 mm)
IPD (60,8-74,6
mm)
lens-to-eye
distance
lens-to-eye
distance
Sensors
Accelerometer,
gyroscope
Accelerometer,
gyroscope
magnetometer
Integrated camera No Yes
Audio
Microphone, integrated supra-aural
Microphone, jack for external headphones
3D spatial audio
headphones (removable)
Wireless Bluetooth Bluetooth
Ports Proprietary headset
connector HDMI 1.4, USB 3.0
x 2
(HDMI/USB 3.0)
Materials used Plastic, IR-transparent fabric
Plastic, glass,
foam rubber
glass, foam rubber
Weight (excl. cable) 470 g 563 g Table 3: Comparison of Oculus Rift and HTC Vive (Borrego et al., 2018)
The research group found that both HMD's have a larger application area than stated by the manufacturers. The Oculus Rift made it to 11.75m2 and the HTC Vive even to 24.87m2. However, accuracy and susceptibility to jitter was increased outside of the specified maximum area of 6.25m2.
The height also has an influence on the accuracy of the glasses. While both units showed good values in a seated position, the accuracy and jitter of the HTC Vive was worse in a standing position in a height of 1.70m. This might be due to the larger application area. This should be taken into account considering the body height of the users. Nevertheless, both glasses achieved excellent values in standing and sitting positions. The overall average accuracy of the oculus was 0.6 to 1.2cm (<0.35%) of the working range, whereas the HTC Vive averaged 0.9 to 1.5cm (<0.21%) of its working
range. Thus, the accuracy of both glasses is better than that of other optical cameras and systems such as the Xbox Kinect. In comparison to common motion tracking systems, the accuracy is three times worse, but the jitter is three times better. Yet, the HMD does not use markers and is therefore easier to use and much more affordable (Borrego et al., 2018).
The size of the working fields of both glasses, exceeds the possibilities of semi- or non-immersive VR-applications significantly. The 360° all-round vision and the large working range of the HTC Vive allow for training of rehabilitation measures close to everyday life to enable reintegration into real life. Due to this VR-capability, from a technical point of view external peripheral devices such as a treadmill or force plates and other rehabilitation devices can be connected excellently for the profit of the patients. To put these values and comparisons into context: human perception reaches a jitter 45-90 times higher than that of glasses in daily television. So, it can be said that the glasses themselves are probably not the reason for VR- sickness, but rather the software itself. The results of this experiment recommend the use of immersive VR-glasses for motor rehabilitation measurements (Borrego et al., 2018).
4 Balance Training
The aim of this thesis is to evaluate the applicability of a VR-system for postural control training. It is not the focus of this thesis to deal more concretely with balance training. For a more comprehensive understanding of the functioning of the VR-system, apart from the technical aspects, it is nevertheless essential to address this topic. Postural control is the ability to control the position of the body in space to improve balance and orientation.
A distinction is made between static and dynamic control. With static control, only the center of mass changes, while dynamic control also changes the support base. To improve postural control, the use of balance training is suitable. Regardless of the level of performance, it can be used for a variety of goals. The effects of balance training to prevent injuries, as well as from and after falls and to improve neuromuscular function have already been proven. The proof of an exact dosage and type of balance training is still pending. In order to answer these questions, it is important to increase the knowledge about the effects of balance training. Thus, the question remains open whether the balance is generally improved by the training or whether it only leads to the improvement of specific tasks. Whether the training of one balance task leads to an increase in performance for another balance task is not yet sufficiently clarified. For patients, the balance should not be considered in general terms. A 36-part equilibrium test in six different categories showed that patients with specific pathology-related deficits scored poorly on one task but performed well on other tasks. This indicates that there is no general equilibrium ability, but that it consists of many specific individual aspects. However, a general statement on the healthy overall population could not be made in this study. However, the literature gives no clear answer in this aspect. On the one hand, there are studies that show that improvements have also been achieved in untrained balancing tasks while on the other hand, other studies show that the balancing skills have only been modified for specifically trained tasks and not for other, untrained balancing tasks (Hirase, Inokuchi, Matsusaka &
Okita, 2015; Donath, Roth, Zahner & Faude, 2016). It is therefore necessary to continue research in this area.
Since adherence or self-responsibility, a significant contribution to the therapeutic success of balance training, it is important to promote it. With Virtual Reality it has already been shown that this willingness to take personal responsibility can be increased. De Vries et al. (2018) were able to prove in their study that exercising with games (= “exergaming”) with virtual reality leads to a strong increase in intrinsic motivation. Another important factor in promoting intrinsic motivation is physical enjoyment.
Especially in the rehabilitation process of individuals or in sports in general, the success of the intervention depends on personal responsibility. This
also includes balance training. Therefore, it is important that the VR-system contributes to the improvement of the individual through physical enjoyment. To quantify this enjoyment, the Physical Activity Enjoyment Scale (PACES) was developed (Kendzierski & DeCarlo, 1991). A more detailed explanation is given in chapter 6.4.
Virtual reality thus not only offers an increase in adherence, but also a proven effectiveness in balance training. In the following chapters, the VR- system will be presented, and attempts will be made to answer the scientific questions and integrate the described aspects. Ultimately, not only the research but hopefully also the patients will benefit from this development.
5 Virtual Reality Project
The idea and the goal of this work was born through the Digital Health Lab project at the St. Pölten University of Applied Sciences. The lab serves as an interface for scientists from the health and technology sector.
Competencies from research fields such as virtual and augmented reality, machine learning, visual analytics, motion capturing, and human computer interaction are integrated into project work. The research focuses on the further development of technical possibilities in relation to gait and movement rehabilitation for patients and healthcare professionals. The focus is on practical relevance. Therefore, the researchers work on solutions in the digital health sector in cooperation with external companies.
On this basis, a VR-exergame was designed in the Digital Health Lab. The aim is that the investigators use VR to give users a positive influence on their weight distribution and ultimately on their overall balance. An example of this are prosthesis wearers who may not yet dare to put adequate weight on their new prosthesis. The VR-application in the Digital Health Lab should provide assistance in the future. The VR-exergame, as described in more detail in chapter 6.2, consists of a force plate on which the user stands, an immersive VR-HMD and a motion capture system from Vicon. With this system it is possible to track the movements of the user in real time and to control a weight distribution from outside via the system. This is done by virtual soap bubbles that appear in front of the user from bottom to top. The soap bubbles appear in a random pattern. The goal of the game is to destroy these bubbles. The user is blinded to the fact, that the investigators can control the distribution of the bubbles, which can force a weight distribution to the left or right side.
In order to develop such a system and exergame successfully, the aim should always be to meet the user's requirements. The survey of usability and user experience is suitable for this purpose. This experience can be collected with the help of the user experience questionnaire (UEQ) and provides information about the degree to which a system is accepted and evaluated by the users (Laugwitz, Held, & Schrepp, 2008, p.63-76).
Usability is the extent to which a product can be used by certain users in a certain usage context to achieve certain goals effectively, efficiently and satisfactorily. To measure usability, the System Usability Scale (SUS) has become generally accepted. The usability of a system and the user experience should be tested and taken into account already in the development phase (Brooke, p.189-194). Both questionnaires are described in detail in chapter 6.4. The fact that the VR-system should later support patients in their rehabilitation makes this aspect all the more necessary.
With this information and background of the previous chapters of this work, the methodology and the results of the study are explained and discussed in the next chapters. The aim is to answer whether the described interactive and immersive VR-system is applicable for balance training considering the aspects of user experience, usability, physical enjoyment and motion sickness. In order to answer this question, it will be determined how the test persons evaluate these aspects and whether and to what extent motion sickness occurs. Furthermore, it is investigated whether these factors depend on age, gender, physical status and VR-experience during the use of this VR-system.
6 Methodology
To answer the research question of this thesis, 49 participants from an age between 18 to 65 years were recruited to evaluate an immersive VR-system for balance training. The recruits were healthy individuals with no orthopedic, neurological or any other acute medical diagnosis. The structure of this work and the methodology was based on literature research and was inspired by earlier works with virtual reality (Kim et al., 2017; Hamzeheinejad et al., 2018).
6.1 Participants
The recruitment of participants was done via the online appointment service
"doodle", with personal recruitment and internal invitation at the St. Pölten University of Applied Sciences. 55 candidates registered for the study. 49 adults (mean age = 39.1, 15 male) were included in the study. Due to appointment collisions, six potential participants were not able to participate in this study. The inclusion and exclusion criteria are listed below (Table 4).
Inclusion criteria
- healthy in the sense of an unrestricted locomotor system nor an impairment of the general mental condition with no diagnosed pathology
- Approval for study participation and local presence at the St. Pölten University of Applied Sciences during data acquisition.
Exclusion criteria
- known diseases of the musculoskeletal system which does not allow an independent free standing
- Visual acuity restrictions - Cognitive impairment - Underage
- Geriatric people over 65
Table 4: Criteria for study population
When recruiting contenders, it was taken into account that there was a distribution of age groups between 18-65 (see Figure 10). The distribution of gender is shown in Figure 11. The volunteers were employees and students of the St. Pölten University of Applied Sciences and people from the region of St. Pölten and Vienna. The standard of education ranged from compulsory school to a completed doctoral program. The participants pursued various professions in the medical, artistic, administrative and technical sectors. None of the participants knew the VR-system and all were initially provided with the same information to ensure an identical starting position for all participants. This information is described in more detail in chapters 6.2 to 6.4.
Figure 10: Age distribution of participants
To ensure data protection, the information sheet to be signed was stored separately offline. For anonymization purpose the participants were assigned an ID number and no names were recorded in the questionnaires.
14
11
10
7 7
[18, 28] (28, 38] (38, 48] (48, 58] (58, 68]
Headcount
0 2 4 6 8 10 12 14
Age
Min. = 18
Max.= 63 Mean = 39,14 Std.Dev. = 13,46
Figure 11: Gender Distribution of the 49 participants
6.2 Study protocol
The participants were given an information sheet in advance to burst virtual soap bubbles with their hands by wearing a HTC Vive HMD. They were allowed to decide for themselves whether to use one finger, the whole hand or both hands. The duration of the application was 3x3 minutes. The test person was asked to burst as many soap bubbles as possible per session.
The performance of the user was not evaluated. After reading the information sheet, it was signed by the participant.
The rules of the game were verbally communicated to the test person immediately before the game started and questions were answered without adding any additional information that could influence the test. At the beginning of the game, the virtual hands of the test persons were grey, as were the soap bubbles (Figure 12). After a certain score, the left hand became green and the right hand blue. Also, the soap bubbles turned either blue or green. The test person then had the task to burst the soap bubbles with the corresponding hand. If the wrong hand was used, points were deducted, so that the respondent could possibly get back to the area where the bubbles and hands were grey. Throughout the entire game there were also red soap bubbles, which contained a small toadstool for better differentiation. They should be avoided by the user, but if one was hit, a point deduction was caused. For reasons of enjoyment and gamification, the user experienced audio stimuli in certain events, such as reaching a certain number of points or by hitting the wrong soap bubbles. In addition,
34 15
Female Male
