There is a long history of exploring auditory interfaces and sonification, from the earlier research of Gaver and Blattner to more recent studies by Flowers and Kramer. The audio-centric game system described in this thesis’ research is based on such pioneering work. This game system conveys 2D spatial information through audio cues alone and requires only low-cost consumer-grade equipment. While this research also relates to the work by Golledge, Klatzky, Loomis, and Marston in the field of auditory route guidance it focuses on routes inside a two-dimensional side-view game world and favors non-verbal cues.
This research expands upon Gärdenfor’s guidelines for audio games to allow for more complex, spatially based games. While this effort still does not advance audio games up to the current state of video games, it is hoped that this research helps narrow the entertainment divide by providing more entertainment options for persons with visual impairments and that it may more generally narrow the digital divide by increasing their use of computers.
Chapter 2 explored the basic research in auditory displays and sonficiation that served as the basis for the audio cue design used in this research. The results presented in this paper showed how auditory icons and earcons are used to create an immersive virtual world. This interface allowed blindfolded sighted participants to navigate through the levels with equal adeptness (in terms of no statistically significant difference in the number of restarts) as participants playing the game with graphics.
Chapter 3 explained the results of the participatory design sessions conducted with students at the Indiana School for the Blind. It details the results of a usability study
comparing the results of persons with visual impairments with a group of sighted participants playing the audio-only version of the game and sighted participants playing the graphical version of the game. No difference was found in the performance within the game between
persons who had a visual impairment, were blindfolded, or were able to see while playing the game. However, the results showed that both audio-only groups took three times as long to complete levels as compared to the sighted audio-visual group. While it is unclear why the audio-only groups took longer to complete the levels of the game than those able to see the game’s graphics, it is possible that participants were more cautious using a sense they had less experience using with which to navigate.
Finally, chapter 4 reviewed past work done in the field of spatial route navigation and mental mapping and presented the results of a mental mapping study conducted using the level layouts of our research. This revealed some interesting findings in the differences between the mental maps formed by persons who are visually impaired compared to sighted persons, as sighted individuals playing the audio-only version of the game tended to invent more objects that were not present in the level. While the results for the mapping strategy used by participants were only marginally significant, they did reveal evidence to support Dodds’s findings that persons with visual impairments are more likely to use egocentric mapping strategies.
In general, this research focused on the study of the first audio platform game that maintains all primary components of games within the platform genre. Based on previous research conducted in the fields of auditory displays, and audio games this work serves as an example in the design of an interface to convey spatial relationships via audio cues that allows users to easily interpret and quickly react to objects within the virtual environment. It is hoped that these results are able to increase the understanding of the differences in the spatial abilities of persons with visual impairments and the sighted because this may lead to further awareness of the role auditory perception and visual experience play in human development.
Future Work
Possible future work includes the use of non-verbal information for providing route information to persons with visual impairments (such as while in a mall) and their use in guiding users through a spreadsheet application. The software engine created for the game could be used to develop additional platform games for persons with visual impairments, including educational games that improve educational equality with the education
opportunities already available to people with normal vision. It is also hoped that future studies can be conducted using participants who are blind from birth, who would not have the previous experience of navigating with their visual sense that may provide some evidence for the hypothesis that individuals who are not used to navigating via auditory cues may exhibit a greater level of caution and slow their performance on navigational tasks.
The results of the current study were unable to explain the anomaly of sighted participants in the audio-only group inventing more objects in their mental map of the third game level; this opens up an area of future work where past visual experience is explored as it relates to the interpretation of auditory spatial information.
REFERENCES
Alfonso, Amandine; Katz, Brian FG; Blum, Alan; Jacquemin, Christian; and Denis, Michel (2005, July). "A Study of Spatial Cognition in an Immersive Virtual Audio
Environment: Comparing Blind and Blindfolded Individuals". ICAD ’05. Limerick, Ireland.
American Foundation for the Blind (2002). “Statistics and Sources for Professionals”. Retrieved February 10, 2008 from
http://www.afb.org/Section.asp?SectionID=15&DocumentID=1367.
Andresen, Gavin (2002). “Playing by Ear: Using Audio to Create Blind-Accessible Games”.
Gamasutra. Retrieved February 10, 2008 from
http://www.gamasutra.com/resource_guide/20020520/anersen_01.htm. AudioGames.net. “Super Liam”.
http://www.audiogames.net/db.php?action=view&id=superliam.
Barrass, S. (2003). “Sonification Design Patterns”. Proceedings of the 2003 International Conference on Auditory Displays, pp. 170-175. Boston, MA, USA.
Blattner, M. M.; Sumikawa, D.A.; & Greenberg, R.M. (1989). “Earcons and Icons: Their Structure and Common Design Principles.” Human-Computer Interaction, 4, 11-44: ACM Press.
Blauert, J. (1997). Spatial Hearing (Revised Edition). MIT Press, Cambridge, MA. Boyer, B. (2008). “NPD: 2007 U.S. Game Industry Growth Up 43% To $17.9 Billion.”
Gamasutra.com. Retrieved February 10, 2008 from http://www.gamasutra.com/php- bin/news_index.php?story=17006.
Bregman, A. S. (1990). Auditory Scene Analysis. MIT Press, Cambridge, MA.
Brewster, Stephen; Wright, Peter; and Edwards, Alistair (1993). “An Evalution of Earcons for Use in Auditory Human-Computer Interafaces”. INTERCHI ’93. ACM Press. Brunyé, Tad; Taylor, Holly; and Worboys, Michael (2007). “Level of Detail in Descriptions
and Depictions of Geographic Space”. Spatial Cognition and Computation, 7(3), 227- 266.
Brown, Harry J. (2008). Videogames and Education: Humanistic Approaches to an Emergent Art Form. M E Sharpe Inc: Armonk, NY.
Council for Schools for the Blind. “Council for Schools for the Blind Membership List 2006- 2007.” Retrieved February 20, 2008 from
http://www.cosb1.org/membership/memberlist.php.
Després, Olivier; Candas, Victor; Dufour, André (2005). “Visual Deficit and Auditory Spatial Compensation: Evidence from Self-positioning from Auditory Cues”.
Cognitive Brain Research 23, 444-447.
Després, Olivier; Boudard, D.; Candas, Victor; Dufour, André (2005). “Enhanced Self- localization by Auditory Cues in Blind Humans”. Disability and Rehabilitation
27(13), 753-759.
Dodds, A.G.; Hogwarth, C.I.; and Carter, D.C. (1982, January). “The Mental Maps of the Blind”. Journal of Visual Impairment and Blindness, 76, 5-12.
Eardley, Alison and Pring, Linda (2007). “Spatial Processing, Mental Imagery, and Creativity in Individuals with and without Sight”. European Journal of Cognitive Psychology, 19(1), 37-58.
Epstein, Russell; Higgins, Stephen; Thompson-Schill, Sharon (2005). “Learning Places from Views: Variation in Scene Processing and Navigational Ability”. Journal of
Cognitive Neuroscience 17:1, 73-83: Boston, MA.
Flowers, John (2005, July). “Thirteen Years of Reflection on Auditory Graphing: Promises, Pitfalls, and Potential New Directions”. ICAD ’05. Limerick, Ireland.
Gärdenfors, Dan (2002). Designing Sound-Based Computer Games. Digital Creativity, 14, 111-114.
Gaver, W. (1997). “Auditory Interfaces.” In M. Helander, T .K. Landauer, & P. Prabhu (Eds.) Handbook of human-computer interaction. Pp. 1003-1042, Amsterdam: Elsevier.
Gaver, William W., Norman, Donald A (1988). “Everyday Listening and Auditory Icons.”
Dissertation Abstracts International, 50, 4B.
GMA Games. “Shades of Doom”. Retrieved February 10, 2008 from http://www.gmagames.com/sod.html.
Golledge, Reginald; Marston, James; Loomis, Jack; Klatzky, Roberta (2004). “Stated Preferences for Components of a Personal Guidance System for Nonvisual Navigation”. Journal of Visual Impairment and Blindness.
Gordon, A.D.; Jupp, P.E.; and Byrne, R.W. (1989). "The construction and assessment of mental maps". British Journal of Mathematical and Statistical Psychology, 42, 169- 182.
International Game Developer’s Association (2004). Accessibility in games: Motivation and approach. Retrieved February 25, 2008 from http://igda.org/accessibility.
Klatzky, R. and Golledge, R. (1995, February). “Performance of blind and sighted persons on spatial tasks”. Journal of Visual Impairment and Blindness
Kramer, G., Walker, B., Bonebright, T. Cook, P., Flowers, J., Miner, N., Neuhoff, J., et al. (1999). “Sonification report: Status of the field and research agenda”. Report prepared for the National Science Foundation by members of the International Community for Auditory Display.
Loomis, Jack M.; Klatzky, Roberta L.; Philbeck, John W.; and Golledge, Reginald G. (1998). "Assessing auditory distance perception using perceptually directed action".
Perception & Psychophysics, 60 (6), 966-980.
Loomis, Jack; Marston, James; Golledge, Reginald; Klatzky, Roberta (2005, April). “Personal Guidance System for People with Visual Impairment: A Comparison of Spatial Displays for Route Guidance”. Journal of Visual Impairment and Blindness. Mansur, Douglass; Blattner, Meera; and Joy, Kenneth (1985). “Sound Graphs: A Numerical
Data Analysis Method for the Blind”. Journal of Medical Systems, Vol. 9, No. 3. Marston, James ; Loomis, Jack;;Klatzky, Roberta; Golledge, Reginald (2007, April).
“Nonvisual Route Following with Guidance from a Simple Haptic or Auditory Display”. Journal of Visual Impairment and Blindness.
McCrindle, R.J., Symons, D. (2000). “Audio Space Invaders”. In the Proceedings of the International Conference Disability, Virtual Reality, & Assoc. Technology. University of Reading, UK, 59-65.
National Telecommunications and Information Administration (2000). Falling Through the Net: Toward Digital Inclusion. http://www.ntia.doc.gov/ntiahome/fttn00/Falling.htm.
Oren, Michael (2007, May). "Speed Sonic Across the Span: Building a Platform Audio Game". In the Proceedings of the Association of Computing Machinery's Computer- Human Interaction 2007. San Jose, CA.
Oren, Michael; Harding, Chris; Bonebright, Terri (2007, June). "Speed Sonic Across the Span: A Platform Audio Game". Proceedings of the International Conference on Auditory Displays 2007. Montreal, Canada.
Ossman, Roland (2006). “Guidelines for Developing Accessible Games”. Game Accessibility. http://gameaccess.medialt.no/guide.php.
Parente, Peter and Bishop, Gary (2003, March). “BATS: The Blind Audio Tactile Mapping System”. ACMSE. Savannah, GA.
Perkins Museum. “Founders”. Retrieved February 10, 2008 from http://www.perkinsmuseum.org/museum/section.php?id=186.
Resnikoff, Serge; Pascolini, Danatella; Etya’ale, Daniel; Kocur, Ivo; Pararajasegaram, Ramachandra; Pokharel, Gopal; and Mariotti, Silvio (2004). “Global data on visual impairment in the year 2002”. Bulletin of the World Health Organization. November 2004, 82 (11), 844-852.
Röber, Niklas and Masuch, Maic (2005). "Playing Audio-only Games: A compendium of interacting with virtual, auditory Worlds". Proceedings of the 2nd DIGRA Games Conference: Vancouver, Canada.
Röder, Brigitte; Rösler, Frank (1998, October). "Visual input does not facilitate the scanning of spatial images". Journal of Mental Imagery, 22 (3-4), 165-182.
Röder, Brigitte; Stock, Oliver; Bien, Siegfried; Neville, Hellen; Rösler, Frank (2002). "Speech processing activates visual cortex in congenitally blind humans". European Journal of Neuroscience, Vol. 16, 930-936.
Stevens, R. (1996). Principles for the design of auditory interfaces to present complex information to blind people (Doctoral dissertation, University of York, UK, 1996). Retrieved on February 10, 2008 from:
http://www.cs.man.ac.uk/~stevensr/papers/thesis-rds.pdf.
Stockman, Tony; Hind, Greg; Frauenberger, Christopher (2005, July). "Interactive Sonification of Spreadsheets". ICAD '05. Limerick, Ireland.
Super Liam. Retrieved February 20, 2008 from
http://www.audiogames.net/db.php?action=view&id=superliam.
Targett, Sue and Fernström, Mikael (2003). “Audio Games: Fun for All? All for Fun?”. Proceedings of the 2003 International Conference on Auditory Display: Boston, MA. Tobler, Waldo R. (1976) "The Geometry of Mental Maps" In R. Golledge and G. Ruston
(Eds.) Spatial Choice and Spatial Behavior, Pp. 69-81, Columbus, OH: Ohio State University Press.
United Cerebral Palsy (2001, April). “American Foundation of the Blind Condemns Entertainment Industry’s Greed”. Retrieved February 10, 2008 from
http://www.ucp/ucp_channeldoc.cfm/79/15/65/65-65/1715.
United Nations (1994). Standard rules on the equalization of opportunities for persons with disabilities. Retrieved February 10, 2008 from
US Department of Education (2005). “Thirty Years of Progress in Educating Children with Disabilities Through IDEA.” Retrieved February 10, 2008 from
http://www.ed.gov/policy/speced/leg/idea/history30.html.
US Government Printing Office (2004). “Individuals with Disabilities Education Impovement Act of 2004”. Retrieved February 10, 2008 from
http://frwebgate.access.gpo.gov/cgi-
bin/getdoc.cgi?dbname=108_cong_public_laws&docid=f:publ446.108.
van Tol, R., & Huiberts, S. (2006). What blind gamers want the industry to know. Compiled for the 2006 Game Developer’s Convention. Retrieved February 10, 2008 from http://www.accessibility.nl/games/researchdocs/What blind gamers want the industry to know.doc.
Volbracht, Sabine; Gitta, Domik; Backe-Neuwald, Dorothea; and Rinkens, Has-Dieter (1998). “The ‘City Game’ An Example of Virtual Environment for Teaching Spatial Orientation”. Journal of Universal Computer Science, vol. 4 no. 4, 461-465.
Voss, Patrice; Lassonde, Maryse; Gougoux, Frederic; Fortin, Madeleine; Guillemot, Jean- Paul; and Lepore, Franco (2004, October). "Early- and Late-Onset Blind Individuals Show Supra-Normal Auditory Abilities in Far-Space". Current Biology, 14, 1734- 1738.
Walker, B. N. and Kramer, G. (2005, Oct). “Sonification design and metaphors: Comments on Walker and Kramer, ICAD 1996”. ACM Trans. Appl. Percept. 2, 4, 413-417 Wikipedia. “Platform Games”. Retrieved February 10, 2008 from
Wikipedia. “Super Mario Bros.” Retrieved February 10, 2008 from http://en.wikipedia.org/wiki/Super_Mario_BrosM.R.
Winberg, F. and Hellström, S. O. (2000). “Investigating Auditory Direct Manipulation: Sonifying the Towers of Hanoi.” Proceedings of the 2000 ACM Computer-Human Interaction: Hague, Netherlands.