This paper describes the design process followed for developing FarmerKeeper, a BCI video game to support neurofeedback training for children with autism, controlled by a consumer- grade BCI headset. We found three design implications and eight design considerations a BCI
Fig. 16 Percentage of time paying attention during the 5 min activity phase, per participant. (From video analysis)
video game should incorporate to appropriately support neurofeedback training for children with autism. Moreover, we presented the results of a 4-weeks usability and user experience study of FarmerKeeper with children with autism. The results of the evaluation suggest that FarmerKeeper is easy to use, useful and may have numerous benefits for attention on children with autism.
Although the research reached our goals, we observed some unavoidable limitations. This research was conducted in only one school-clinic of children with autism. Moreover, we conducted the evaluation with a small sample size of participants; therefore, to generalize the results for other populations, the study should be replicated with a much larger population to gather data to get a more realible statistical significance. Additionally, children with autism only played once with the BCI video games. Then, the study also should be replicated in the long term to avoid the bias due to theBnovelty effect^ and to get a more realible knowledge about the acceptance and adoption of the BCI video games –researchers must particularly measure if participants maintain a positive attitude and their engagement over time. However, given that the objective of this study was not to generalize our findings but to get an overall impression of the potential of FarmerKeeper in a particular use case, we feel that our results are valuable for researchers exploring the design space of BCI video games to support neurofeedback training of children with autism. As future work, we plan to conduct a long- term study of the use of FarmerKeeper. We will reflect on the results from this evaluation to re- design our prototype and include collaborative challenges, less monotonous activities, and short objectives into the main objective of the activity. Using this new version of FarmerKeeper we will test the efficacy of such a system in improving the attention impair- ments of children with autism and investigate its potential clinical benefits. In terms of adoption we plan to investigate the acceptance of FarmerKeeper after the Bnovelty effect^ wears off, game engagement, and general usability aspects related to its usage.
Another limitation of this study is that we are only focusing in using neurofeedback training to help children with autism cope with attention issues. However, to truly treat autism, innovative solutions must be focused in targeting the core symptoms of autism by handling issues related to language impairments, social interactions, and behavior. Although, this is a first effort to provide evidence of the potential of BCI video games to support individuals with autism; researchers must explore how such solutions could also be instrumental to be used in a therapeutic and clinical context for autism.
Also, there are some technical limitations on the development of FarmerKeeper. We assumed that the eSense algorithm of the ThinkGear chip on the Brainlink headset provides reliable measures when inferring the attention of its users. But, this inference is really just an Bapproximation^ and as eSense is a proprietary algorithm, its documentation does not provide a detailed explanation of its design. For our particular case, we used the openvibe component instead of the headband API to return the control to the users and enable them to manually change the way to obtain the attention values as input to the BCI video game when needed. So, as future work, research must explore more reliable and robust algorithms to measure attention that could be integrated into FarmerKeeper thought the openvibe component. In addition, during the development of FarmerKeeper, we found out that by controlling the posture of the users might reduce the noise associated with users’ movements. However, we did not infer users’ posture, and more work is required to propose a robust detection method to avoid noise due to user’s movements and bad posture. Researchers must explore both invasive and non- invasive solutions, like using vision-based algorithms to track the users’ posture when playing BCI video games. Finally, when designing FarmerKeeper we found out that enable users to
easily relate to the character of the game is important. And although FarmerKeeper enables some sort of customization, open questions remain to truly investigate how to better achieve this emotional connection between users and the character of the game [77]. As future work, we propose to investigate how the character of the game could mimic users’ behaviors and emotions to achieve this desired level of empathy.
A BCI video game implementing the design considerations proposed could close the gap between accessibility and reliability of neurofeedback training for children with autism. In addition, this project has the potential to transform neurofeedback training towards the use of a more accessible solution that could potentially bring the latest advances in neuroscience to homes and clinics of children with autism. This proposal could potentially reduce costs and barriers associated with the non-pharmacological treatment of autism. And, could help to increase the applications of ubiquitous computation and HCI in the area of neurosciences, allowing the growth in the development of ubiquitous innovative systems to assist neurofeedback training mainly for children with autism.
Acknowledgments We thank all the participants enrolled in this study and the researchers and reviewers who provide helpful comments on previous versions of this document. We also thank CONACYT for the first author fellowship and we thank to the CONACYT project #2209 of the fourth author for their financial support.
Compliance with ethical standards
Conflict of interest Authors declare that they have no conflict of interest.
Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
1. Aamodt S, Wang S (2007) Exercise on the brain. New York Times
2. Abt CC (1970) Serious games: the art and science of games that simulate life. New Yorks Viking, New York, p 6
3. Alves S, Marques A, Queirós C, Orvalho V (2013) Lifeisgame: a serious game about emotions for children with autism spectrum disorders. PsychNology J 11:191–211
4. American Psychiatric Association (2013) Diagnostic and statistical manual of mental disorders, (DSM-5®). American Psychiatric Pub
5. Antshel KM, Zhang-James Y, Wagner KE et al (2016) An update on the comorbidity of ADHD and ASD: a focus on clinical management. Expert Rev Neurother 16:279–293
6. Autism, Investigators DDMNSY 2010 P (2014) Prevalence of autism spectrum disorder among children aged 8 years—autism and developmental disabilities monitoring network, 11 sites, United States, 2010. Morb Mortal Wkly Rep Surveill Summ 63:1–21
7. Bailey R, Wise K, Bolls P (2009) How avatar customizability affects children’ s arousal and subjective presence during junk food – sponsored online video games. Cyberpsychology Behav 12:277–283.
https://doi.org/10.1089/cpb.2008.0292
8. Bakhshayesh AR, Hänsch S, Wyschkon A et al (2011) Neurofeedback in ADHD: a single-blind randomized controlled trial. Eur Child Adolesc Psychiatry 20:481–491.https://doi.org/10.1007/s00787-011-0208-y
9. Bernardini S, Porayska-Pomsta K (2013) Planning-based social partners for children with autism. Twenty- Third Int Conf…, pp 362–370
10. Beyer H, Holtzblatt K (1999) Contextual design. Interactions 6:32–42
11. Birk MV, Atkins C, Bowey JT, Mandryk RL (2016) Fostering intrinsic motivation through avatar identi- fication in digital games. Proc 2016 CHI Conf Hum Factors Comput Syst - CHI’16, pp 2982–2995.
12. Blandón DZ, Munoz JE, Lopez DS, Gallo OH (2016) Influence of a BCI neurofeedback videogame in children with ADHD. Quantifying the brain activity through an EEG signal processing dedicated toolbox. In: Computing Conference (CCC), 2016 IEEE 11th Colombian, pp 1–8
13. Brain S (2016) Attention span statistics – statistic brain.http://www.statisticbrain.com/attention-span- statistics/. Accessed 16 Feb 2018
14. Brooke J (2013) SUS : a retrospective. J Usability Stud 8:29–40
15. Choon Guan Lim B, Lee T-S, Guan C et al (2010) Effectiveness of a brain-computer Interface based Programme for the treatment of ADHD: a pilot study. Psychopharmacol Bull 4343:73–82
16. Cibrian FL, Mercado J, Escobedo L, Tentori M (2018) A step towards identifying the sound preferences of children with autism. In: Proceedings of the 12th EAI international conference on pervasive computing technologies for healthcare, pp 158–167
17. Cmiel V, Janousek O, Kolarova J (2011) EEG biofeedback. Proc 4th Int Symp Appl Sci Biomed Commun Technol - ISABEL’11, pp 1–5.https://doi.org/10.1145/2093698.2093752
18. Coben R, Linden M, Myers TE (2010) Neurofeedback for autistic spectrum disorder: a review of the literature. Appl Psychophysiol Biofeedback 35:83–105.https://doi.org/10.1007/s10484-009-9117-y
19. Dickey MD (2006) Game design narrative for learning: appropriating adventure game design narrative devices and techniques for the design of interactive learning environments. Educ Technol Res Dev 54:245– 263.https://doi.org/10.1007/s11423-006-8806-y
20. Engine U-G (2015) Unity technologies. Available Unity Technol
21. Escobedo L, Tentori M, Quintana E et al (2014) Using augmented reality to help children with autism stay focused. IEEE Pervasive Comput 13:38–46.https://doi.org/10.1109/MPRV.2014.19
22. Friese S (2014) Qualitative data analysis with ATLAS. ti. Sage, London
23. Fuchslocher A, Niesenhaus J, Krämer N (2011) Serious games for health: an empirical study of the game Bbalance^ for teenagers with diabetes mellitus. Entertain Comput 2:97–101
24. Garzotto F, Gelsomini M, Pappalardo A et al (2016) Using brain signals in adaptive smart spaces for disabled children. Proc 2016 CHI Conf Ext Abstr Hum Factors Comput Syst - CHI EA’16, pp 1684–1690.
https://doi.org/10.1145/2851581.2892533
25. Goldstein G, Johnson CR, Minshew NJ (2001) Attentional processes in autism. J Autism Dev Disord 31: 433–440.https://doi.org/10.1023/A:1010620820786
26. Green CS, Bavelier D (2008) Exercising your brain: a review of human brain plasticity and training-induced learning. Psychol Aging 23:692
27. Hallford N, Hallford J (2001) Swords and circuitry: a designer’s guide to computer role-playing games. Premier Press, Incorporated, Boston
28. Hammond DC (2011) What is neurofeedback: an update. J Neurother 15:305–336.https://doi.org/10.1080 /10874208.2011.623090
29. Hardy J, Scanlon M (2009) The science behind lumosity. San Fr CA Lumos Labs, San Francisco 30. Hayes GR, Hirano S, Marcu G et al (2010) Interactive visual supports for children with autism. Pers Ubiquit
Comput 14:663–680.https://doi.org/10.1007/s00779-010-0294-8
31. Hefner D, Klimmt C, Vorderer P (2007) Identification with the player character as determinant of video game enjoyment. In: Entertainment computing–ICEC 2007. Springer, pp 39–48
32. Heinrich H, Gevensleben H, Strehl U (2007) Annotation: neurofeedback - train your brain to train behaviour. J Child Psychol Psychiatry Allied Discip 48:3–16. https://doi.org/10.1111/j.1469- 7610.2006.01665.x
33. Heyvaert M, Saenen L, Campbell JM et al (2014) Efficacy of behavioral interventions for reducing problem behavior in persons with autism: an updated quantitative synthesis of single-subject research. Res Dev Disabil 35:2463–2476
34. Holtzblatt K, Wendell JB, Wood S (2004) Rapid contextual design: a how-to guide to key techniques for user-centered design. Elsevier, San Francisco
35. IJsselsteijn W, Van Den Hoogen W, Klimmt C et al (2008) Measuring the experience of digital game enjoyment. Proceedings of measuring behavior, pp 88–89
36. Keay-Bright W (2007) Can computers create relaxation? Designing ReacTickles© software with children on the autistic spectrum. CoDesign 3:97–110.https://doi.org/10.1080/15710880601143443
37. Keehn B, Müller RA, Townsend J (2013) Atypical attentional networks and the emergence of autism. Neurosci Biobehav Rev 37:164–183.https://doi.org/10.1016/j.neubiorev.2012.11.014
38. Keizer AW, Verment RS, Hommel B (2010) Enhancing cognitive control through neurofeedback: a role of gamma-band activity in managing episodic retrieval. Neuroimage 49:3404–3413.https://doi.org/10.1016/j. neuroimage.2009.11.023
39. Kim SK, Yoo EY, Lee JS et al (2014) The effects of neurofeedback training on concentration in children with attention deficit / hyperactivity disorder. Int J Bio-Science Bio-Technology 6:13–23
40. Kouijzer MEJ, van Schie HT, de Moor JMH et al (2010) Neurofeedback treatment in autism. Preliminary findings in behavioral, cognitive, and neurophysiological functioning. Res Autism Spectr Disord 4:386– 399.https://doi.org/10.1016/j.rasd.2009.10.007
41. Laugwitz B, Held T, Schrepp M (2008) Construction and evaluation of a user experience questionnaire. HCI Usability Educ Work, pp 63–76.https://doi.org/10.1007/978-3-540-89350-9_6
42. Lee K (2009) Evaluation of attention and relaxation levels of archers in shooting process using brain wave signal analysis algorithms. Sci Sensit 12:341–350
43. Lim C-W, Jung H-W (2013) A study on the military serious game. Adv Sci Technol Lett 39:73–77 44. Lim CG, Lee TS, Guan CT et al (2012) A brain-computer Interface based attention training program for
treating attention deficit hyperactivity disorder. PLoS One 7:8. https://doi.org/10.1371/journal. pone.0046692
45. Lubar JF (1995) Neurofeedback for the management of attention-deficit/hyperactivity disorders. In: Chapter in Schwartz MS (ed) Biofeedback: a practitioner’s guide. Guilford Press, New York, pp 493–522 46. Lubar JF, Bahler WW (1976) Behavioral management of epileptic seizures following EEG biofeedback training of the sensorimotor rhythm. Biofeedback Self Regul 1:77–104. https://doi.org/10.1007 /BF00998692
47. Mandryk RL, Kalyn M, Dang Y et al (2012) Turning off-the-shelf games into biofeedback games. Proc 14th Int ACM SIGACCESS Conf Comput Access - ASSETS ’12, pp 199. https://doi.org/10.1145 /2384916.2384952
48. Mandryk RL, Dielschneider S, Kalyn MR et al (2013) Games as neurofeedback training for children with FASD. Proc 12th Int Conf Interact Des Child - IDC ’13, pp 165–172. https://doi.org/10.1145 /2485760.2485762
49. Mandryk RL, Nacke LE, Mandryk RL (2016) Biometrics in gaming and entertainment technologies. In: Biometrics a data driven world trends, Technol challenges, pp 191–224
50. May T, Rinehart N, Wilding J, Cornish K (2013) The role of attention in the academic attainment of children with autism spectrum disorder. J Autism Dev Disord 43:2147–2158.https://doi.org/10.1007/s10803-013- 1766-2
51. Mihajlovic V, Grundlehner B, Vullers R, Penders J (2015) Wearable, wireless EEG solutions in daily life applications: what are we missing? IEEE J Biomed Heal Informatics 19:6–21. https://doi.org/10.1109 /JBHI.2014.2328317
52. Nouchi R, Taki Y, Takeuchi H et al (2012) Brain training game improves executive functions and processing speed in the elderly: a randomized controlled trial. PLoS One 7:e29676.https://doi.org/10.1371/journal. pone.0029676
53. Ochs E, Kremer-Sadlik T, Sirota KG, Solomon O (2004) Autism and the social world: an anthropological perspective. Discourse Stud 6:147–183
54. Owen AM, Hampshire A, Grahn JA et al (2010) Putting brain training to the test. Nature 465:775–778.
https://doi.org/10.1038/nature09042
55. Patsis G, Sahli H, Verhelst W, De Troyer O (2013) Evaluation of attention levels in a Tetris game using a brain computer interface. In: Carberry S, Weibelzahl S, Micarelli A, Semeraro G (eds) User modeling, adaptation, and personalization. Springer Berlin Heidelberg, Berlin, pp 127–138
56. Pineda JA, Brang D, Hecht E et al (2008) Positive behavioral and electrophysiological changes following neurofeedback training in children with autism. Res Autism Spectr Disord 2:557–581.https://doi. org/10.1016/j.rasd.2007.12.003
57. Planchon J, Vacher A, Comblet J et al (2018) Serious game training improves performance in combat life- saving interventions. Injury 49:86–92
58. Pope AT, Palsson OS (2001) Helping video gamesBrewire our minds^. Int J Soc Psychiatry 54:370–382 59. Rauschenberger M, Schrepp M, Perez-Cota M et al (2013) Efficient measurement of the user experience of
interactive products. How to use the user experience questionnaire (UEQ). Example: Spanish language version. Int J Interact Multimed Artif Intell 2:39–45.https://doi.org/10.9781/ijimai.2013.215
60. Read JC, MacFarlane S (2006) Using the fun toolkit and other survey methods to gather opinions in child computer interaction. In: Proceedings of the 2006 conference on Interaction design and children, pp 81–88 61. Rebolledo-Mendez G, Dunwell I, Martínez-Mirón EA et al (2009) Assessing NeuroSky’s usability to detect attention levels in an assessment exercise. In: Jacko JA (ed) Human-computer interaction. New trends. Springer Berlin Heidelberg, Berlin, pp 149–158
62. Rego P, Moreira PM, Reis LP (2010) Serious games for rehabilitation: a survey and a classification towards a taxonomy. In: Information Systems and Technologies (CISTI), 2010 5th Iberian Conference on, pp 1–6 63. Renard Y, Lotte F, Gibert G et al (2010) OpenViBE : an open-source software platform to design , test , and
use brain– computer interfaces in real and virtual environments. Presence 19:35–53.https://doi.org/10.1162 /pres.19.1.35
64. Sawyer B, Smith P (2008) Serious games taxonomy. In: Slides from the serious games summit at the game developers conference
65. Schoneveld EA, Malmberg M, Lichtwarck-Aschoff A et al (2016) A neurofeedback video game (MindLight) to prevent anxiety in children: a randomized controlled trial. Comput Hum Behav 63:321– 333.https://doi.org/10.1016/j.chb.2016.05.005
66. Schwartz MS, Andrasik F (2017) Biofeedback: a practitioner’s guide. Guilford Publications, New York 67. Shaphiro SS, Wilk MB (1965) An analysis of variance test for normality. Biometrika 52:591–611 68. Sitaram R, Ros T, Stoeckel L et al (2017) Closed-loop brain training: the science of neurofeedback. Nat Rev
Neurosci 18:86–100.https://doi.org/10.1038/nrn.2016.164
69. Southam-Gerow MA, Kendall PC (2002) Emotion regulation and understanding: implications for child psychopathology and therapy. Clin Psychol Rev 22:189–222.https://doi.org/10.1016/S0272-7358(01 )00087-3
70. Spence C (2011) Crossmodal correspondences: a tutorial review. Atten Percept Psychophysiol 73:971–995.
https://doi.org/10.3758/s13414-010-0073-7
71. Steiner NJ, Frenette EC, Rene KM et al (2014) Neurofeedback and cognitive attention training for children with attention-deficit hyperactivity disorder in schools. J Dev Behav Pediatr 35:18–27. https://doi. org/10.1097/DBP.0000000000000009
72. Strauss A, Corbin J (1998) Basics of qualitative research techniques. Sage publications, Thousand Oaks 73. Student (1908) The probable error of a mean. Biometrika 6:1–25
74. Tan D, Nijholt A (2010) Brain-computer interfaces and human-computer interaction. In: Brain-computer interfaces. Springer, pp 3–19
75. Tang ST, McCorkle R (2002) Use of family proxies in quality of life research for cancer patients at the end of life: a literature review. Cancer Investig 20:1086–1104
76. Tang YY, Posner MI (2014) Training brain networks and states. Trends Cogn Sci 18:345–350.https://doi. org/10.1016/j.tics.2014.04.002
77. Turkay S, Kinzer CK (2014) The effects of avatar-based customization on player identification. Int J Gaming Comput Simulations 6:1–25.https://doi.org/10.4018/ijgcms.2014010101
78. van Steensel FJ, Bögels SM, Perrin S (2011) Anxiety disorders in children and adolescents with autistic spectrum disorders: a meta-analysis. Clin Child Fam Psychol Rev 14:302–317. https://doi.org/10.1007 /s10567-011-0097-0
79. Vernon D, Egner T, Cooper N et al (2003) The effect of training distinct neurofeedback protocols on aspects of cognitive performance. Int J Psychophysiol 47:75–85.https://doi.org/10.1016/S0167-8760(02)00091-0
80. Wang Q, Sourina O, Nguyen MK (2010) EEG-basedBserious^ games design for medical applications. Proc - 2010 Int Conf Cyberworlds, CW 2010, pp 270–276.https://doi.org/10.1109/CW.2010.56
81. Whyte EM, Smyth JM, Scherf KS (2015) Designing serious game interventions for individuals with autism. J Autism Dev Disord 45:3820–3831.https://doi.org/10.1007/s10803-014-2333-1
82. Wilcoxon F (1946) Individual comparisons of grouped data by ranking methods. J Econ Entomol 39:269.
https://doi.org/10.1093/jee/39.2.269
83. Wilkinson P (2016) A brief history of serious games Phil. Entertain Comput Serious Games 9970, pp 17– 41.https://doi.org/10.1007/978-3-319-46152-6
84. Wilkinson N, Ang RP, Goh DH (2008) Online video game therapy for mental health concerns: a review. Int J Soc Psychiatry 54:370–382
85. Yoon H, Park S-W, Lee Y-K, Jang J-H (2013) Emotion recognition of serious game players using a simple brain computer interface. 2013 Int Conf Inf Commun Technol Converg ICTC 2013, Oct 14, 2013 - Oct 16, 2013, pp 783–786.https://doi.org/10.1109/ICTC.2013.6675478
86. Zalapa R, Tentori M (2013) Movement-based and tangible interactions to offer body awareness to children with autism. In: Urzaiz G, Ochoa SF, Bravo J et al (eds) Ubiquitous computing and ambient intelligence. Context-awareness and context-driven interaction. Springer International Publishing, Cham, pp 127–134 87. Zickefoose S, Hux K, Brown J, Wulf K (2013) Let the games begin: a preliminary study using attention
process training-3 and lumosity™ brain games to remediate attention deficits following traumatic brain injury. Brain Inj 27:707–716.https://doi.org/10.3109/02699052.2013.775484
88. Zoefel B, Huster RJ, Herrmann CS (2011) Neurofeedback training of the upper alpha frequency band in EEG improves cognitive performance. Neuroimage 54:1427–1431. https://doi.org/10.1016/j. neuroimage.2010.08.078
Jose Mercado , is a Ph. D. student in the Computer Science Department at the Center for Scientific Research and Higher Education (CICESE), with a background in Computer System Engineering, his research interest are Human-Computer Interaction, Brain-Computer Interfaces, Ubiquitous Computing, Vision-based Techniques and Pattern Recognition and Classification. Contact him at [email protected]
Ismael Espinosa-Curiel , is an associate professor in CICESE-UT3, Tepic, Nayarit. His research interest are Human- Computer Interaction, Software Engineering, Agile Software Development and IT innovation. Contact him at [email protected]
Lizbeth Escobedo , is an assistant professor in Cetys Universidad, Tijuana, in the School of Engineering where she teaches, and researches how humans interact with their environment to design, implement and evaluate the impact of technology that mainly helps people with cognitive disabilities, their families and caregivers, thus raising their quality of life. Her research interests are Human-Computer Interaction (HCI), Ubiquitous Comput- i n g , C o g n i t i v e A s si s t i v e Te c h n o lo g y ( C AT ) , a n d Mo b i l e Te c h n o l o g i e s . Co n t a c t he r a t [email protected]
Monica Tentori , is an associate professor in CICESE, where she investigates the human experience of ubiquitous computing to inform the design of ubiquitous environments that effectively enhance humans’ interactions with their world. Tentori was the first latin American woman in receiving the Microsoft Faculty Fellowship. Contact her at [email protected]