HAL Id: hal-02173913
https://hal.archives-ouvertes.fr/hal-02173913
Submitted on 4 Jul 2019HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
Brain Invaders Cooperative versus Competitive:
Multi-User P300- based Brain-Computer Interface
Dataset (bi2015b)
Louis Korczowski, Martine Cederhout, Anton Andreev, Grégoire Cattan,
Pedro Luiz Coelho Rodrigues, Violette Gautheret, Marco Congedo
To cite this version:
Louis Korczowski, Martine Cederhout, Anton Andreev, Grégoire Cattan, Pedro Luiz Coelho Ro-drigues, et al.. Brain Invaders Cooperative versus Competitive: Multi-User P300- based Brain-Computer Interface Dataset (bi2015b). [Research Report] GIPSA-lab. 2019. �hal-02173913�
Technical Report
2019-07-04Brain Invaders Cooperative versus Competitive: Multi-User
P300-based Brain-Computer Interface Dataset (bi2015b)
~
L. Korczowski, M. Cederhout, A. Andreev, G. Cattan,
P.L.C. Rodrigues, V. Gautheret, M. Congedo
GIPSA-lab, CNRS, University Grenoble-Alpes, Grenoble INP.
Address: GIPSA-lab, 11 rue des Mathématiques, Grenoble Campus BP46, F-38402, France
Keywords: Electroencephalography (EEG), P300, Brain-Computer Interface (BCI), Experiment, Hyperscanning, Cooperation, Competition.
Abstract - We describe the experimental procedures for a dataset that we have made publicly
available at https://doi.org/10.5281/zenodo.3267307 in mat and csv formats. This dataset contains electroencephalographic (EEG) recordings of 44 subjects playing in pair to the multi-user version of a visual P300 Brain-Computer Interface (BCI) named Brain Invaders. The
interface uses the oddball paradigm on a grid of 36 symbols (1 or 2 Target, 35 or 34 Non-Target) that are flashed pseudo-randomly to elicit the P300 response. EEG data were recorded using 32 active wet electrodes per subjects (total: 64 electrodes) during four randomised conditions (Cooperation 1-Target, Cooperation 2-Targets, Competition 1-Target, Competition 2-Targets). The experiment took place at GIPSA-lab, Grenoble, France, in 2015. Python code for manipulating the data is available at https://github.com/plcrodrigues/py.BI.EEG.2015b-GIPSA. The id of this dataset is bi2015b.
Résumé - Dans ce document, nous décrivons une expérimentation dont les données ont été
publiées sur https://doi.org/10.5281/zenodo.3267307 aux formats mat et csv. Ce jeu de donnée contient les enregistrements électroencéphalographiques (EEG) de 44 sujets jouant par paires à une version multi-utilisateurs du jeu Brain Invaders (2), une interface cerveau-ordinateur de type ‘P300 visuel’. L’interface repose sur le paradigme oddball avec une grille de 36 symboles (1 ou 2 Targets, 35 ou 34 Non-Targets) qui clignotent de façon pseudo-aléatoire afin de produire un P300, un potentiel évoqué apparaissant environ 300 ms après le début d’une stimulation. L'EEG de chaque sujet a été enregistré grâce à 32 électrodes humides réparties sur le scalp (total : 64 électrodes par pair), au cours de quatre sessions expérimentales randomisées (Cooperation 1-Target, Cooperation 2-Targets, Competition 1-Target, Competition 2-Targets). L'expérience a été menée au GIPSA-lab (Université de Grenoble-Alpes, CNRS, Grenoble-INP) en 2015. Nous fournissons également une implémentation python pour manipuler les données disponibles https://github.com/plcrodrigues/py.BI.EEG.2015b-GIPSA. L’identifiant de cette base de données est bi2015b.
Introduction
The experiment was designed to study the inter-brain synchrony of two participants engaged in a cooperative BCI as compared to the same participants playing in competition (1). The visual P300 is an event-related potential (ERP) elicited by an expected but unpredictable target visual stimulation (i.e., oddball paradigm (3)), peaking 240-600 ms after stimulus onset. In this experiment, there were two event-related stimuli: Target (P300 expected) and Non-Target (no P300). The experiment used the multi-player version of the Brain Invaders P300-based BCI open-source software (2), creatively called Brain Invaders 2 (4). During the experiment, the output of a real-time adaptive Riemannian Minimum Distance to Mean (RMDM) classifier was used for assessing the participants’ command (5,6). This classifier allows to have a calibration-free procedure (7). The score of each player was interdependent: In cooperation a bonus was applied when the two players fulfilled their objective simultaneously. In competition, this bonus was applied to a player only if s/he fulfilled his/her objective and the other did not. These objectives could be Coincident (1 Target on screen) or non-Coincident (2 Targets on screen, one for each player), that is, there were four experimental conditions:
Cooperation 1-Target (Coop1T), Cooperation 2-Targets (Coop2T), Competition 1-Target (Comp1T), Competition 2-Targets (Comp2T).
This experiment is part of collective effort to develop plug and play open-source BCI software at GIPSA-lab while assessing their usability for hyperscanning studies (1). An example of application of this dataset can be seen in (8).
Participants
44 subjects with mean (sd) age 23.93 (3.12) were recruited for this experiment and randomly paired (see Table 1). The participants were selected on the basis of their individual score during a preliminary session of Brain Invaders (9) and therefore were not naïve users. At the end of the experiment two tickets of cinema were offered to each subject, for a total value of 15 euros per subject. The best players and the best team in the leaderboard received an additional gift which value was less than ten euros. All participants provided written informed consent confirming the notification of the experimental process, the notification of the data management procedures and the right to withdraw from the experiment at any moment. The study was
approved by the Ethical Committee of the University of Grenoble Alpes (Comité d’Ethique pour la Recherche Non-Interventionnelle).
pair # GENDER AGE1 AGE2 comments 1 MF 21 23
2 FF 18 25
3 MM 23 21
4 MF 22 32 rejected 5 FF 19 27
6 MM 22 22 >28°C in experimental room (too warm) 7 MM 22 21 >28°C in experimental room (too warm) 8 MM 25 23 rejected 9 MM 24 26 10 MM 23 22 rejected 11 MM 25 28 12 MM 25 24 13 MF 24 28 14 FM 28 22
15 MM 20 26 No alpha observed for Player2 during eyes closed. 16 FM 22 23
17 FF 21 28
18 MM 27 32 19 FM 24 28
20 MM 22 23 >30°C in experimental room (too warm) 21 MM 23 20 >30°C in experimental room (too warm) 22 FF 27 22 >30°C in experimental room (too warm)
Table 1. Age and Gender of the participants
Among the 22 pairs of participants, three have been rejected of the study for the following reasons:
Pair 4: one experimental condition missing due to an informatics problem.
Pair 9: due to malfunctioning of the reference electrode of player 1, it was replaced but accidentally switched from the right earlobe (in the first condition) to the left earlobe (in the second, third and fourth condition).
Pair 10: player 1 learnt to blink after a Target flash. The adaptive classifier fitted to these artefacts resulting into almost perfect classification accuracy.
The data of these pairs are still provided as they show common situations that can be faced in BCI and represent interesting cases (e.g., task-dependant artefacts and faulty equipment). However, while assessing offline classification accuracy of the Target/Non-Target task, pair 10 should be removed from the database, and, in condition specific analysis, all three pairs should be removed. We provide additional comments about the recording conditions in Table 1.
Material
EEG signals were acquired by means of a research-grade amplifier (g.USBamp, g.tec, Schiedlberg, Austria) and the g.GAMMAcap (g.tec, Schiedlberg, Austria) equipped with 32 wet Silver/Silver Chloride electrodes, placed according to the 10-10 international system (FP1, FP2, AFZ, F7, F3, F4, F8, FC5, FC1, FC2, FC6, T7, C3, Cz, C4, T8, CP5, CP1, CP2, CP6, P7, P3, PZ, P4, P8, PO7, O1, Oz, O2, PO8, PO9 and PO10) with reference electrode on the right earlobe and ground electrode at the FZ scalp location (total = 64 electrodes) - Figure 1. The g.USBamp amplifier was linked by USB connection to the PC where the data were acquired by means of the open-source software OpenVibe (10,11). The data were stored with no digital filter applied and a sampling frequency of 512 samples per second. In order to reduce the jitter, the experimental tags produced by Brain Invaders 2 were synchronized with the EEG signal using USB digital-to-analog converter connected to the g.USBamp trigger channel. The same tagging procedure is used for all Brain Invaders databases1. This allows comparing the resulting ERP between the experimental conditions thanks to a consistent tagging latency and jitter (12).
Figure 1. In grey the 32 electrodes placed on the scalp to each subjects according to the 10-10 international system. The ground (GRN) was placed at FZ (squared). The reference (REF) was placed on the left earlobe (not shown on the figure).
Procedures
For all subjects, the experiment took place in a small room with a surface of four meters square, containing a screen of length 24’ and all the required hardware materials for acquiring the EEG data. The two subjects were sitting side by side at a distance of approximately 125 cm from of the same screen (Figure 2).
The EEG headset was placed on the two subjects and the integrity of the incoming data was checked by inspecting visible signals such as eye blinks and the posterior occipital dominant rhythm. The experimenter controlled the session from an adjacent room equipped with a one-way glass window.
Brain Invaders Interface
The interface of Brain Invaders is composed of 36 aliens. In the Brain Invaders P300 paradigm, a repetition is composed of 12 flashes of pseudo-random groups of six symbols chosen in such a way that after each repetition each symbol has flashed exactly two times. Thus in each repetition the target symbol flashes twice, whereas the remaining 10 flashes do not concern the target (non-target). The ratio of Target versus non-Target is therefore one-to-five (Figure 3). A detailed description of this paradigm is available in (2,13,14). In this dataset, both players could have the same Target (1-Target conditions, as in Figure 3), or each player could have a different Target (2-Targets conditions, as in Figure 5).
Figure 3.Interface of Brain Invaders at the moment where a group of six non-Target symbols flash (in white). The red symbol is the Target (1-Target condition). The non-Targets which are not flashing are in grey.
A game session was compounded by five levels, consisting in a standardized fixed configuration of 36 symbols of the interface (see Figure 3). Each level consisted of four phases
(Figure 4): the clue (Figure 5), the repetition of flashes (Figure 3), the feedback (Figure 6) and the reward (Figure 7). The players had only one attempt to destroy their Target symbol for each level. If no Target was destroyed, the next level was loaded anyway. As a consequence, the duration for the entire experiment is almost identical for all pairs of participants.
Figure 4. Detailed experimental procedure. Each level consists of four phases: Clue, Flashes, Feedback and Reward. A game session consists of 5 levels in each condition (cooperation/competition and 1-Target/2-Targets). An experiment bloc consists of 4 game session (one for each condition in random order). The entire procedure consists of four blocks.
Figure 5.Clue phase: For each player their Target is emphasized for a few seconds at the beginning of each level by a red marker (Player 1's is surrounded by a red circle, and Player 2's by a red square). If there was only one Target, the red circle and red square were superposed to indicate that both players had to attend to the same Target.
Figure 6.Feedback phase: each player destroys one symbol (left) or both can destroy the same (right). Player 1's destruction is indicated by the red circle, Player 2's destruction is indicated by the red square. If both players destroyed the same Target, both red circle and red square were superposed (right).
Figure 7.Example of reward in cooperation. On the left, only one of the two targets was destroyed, both players obtained +2000 points. On the right: both players missed the only target, no point was gained.
Scoring
In both cooperation and competition, each player had his own score (Table 2). However, in cooperation, each player received a bonus when the two players succeed in destroying their Target(s) at the same time. At the inverse, in competition the bonus was given to a player only if s/he succeeded in destroying his target while the other players did not.
P1 succeeds, P2 fails P1 & P2 succeed P1 & P2 fail Cooperation P1 +2000 +5000 +0
P2 +2000 +5000 +0
Competition P1 +5000 +2000 +0
P2 +0 +2000 +0
Table 2. Score table. Bonus points are given depending on the game mode. In cooperation: if both players destroyed their target. In competition: if the player succeeded while the other failed.
Organization of the Dataset
The EEG recording of each pair for each bloc are provided in mat and csv formats. Each file is a 2D matrix containing the concurrent EEG recording of the two subjects. Columns 2 to 33 contain the recordings on each of the 32 EEG electrodes for the first player. Columns 34 to 65 contains the EEG recordings on each of the 32 EEG electrodes for the second player. The first column of the matrix represents the timestamp of each observation and column 66 contains the experimental events. The rows in column 66 (Events) are filled with zeros, except at the timestamp corresponding to the beginning of an event, when the row gets one of the following values:
- 107 for the beginning of the Coop1T condition. - 108 for the beginning of the Coop2T condition. - 109 for the beginning of the Comp1T condition. - 110 for the beginning of the Comp2T condition.
- 111 when the two players destroy their target(s) simultaneously (bonus point in cooperation)
- 105 when player 1 destroys his Target and player 2 misses (bonus point in competition) - 106 when player 2 destroys his Target and player 1 misses (bonus point in competition) - 112 when no targets are destroyed.
- 102 for the end of a repetition. - 100 for the onset of a new block.
- 20-25 and 40-45 when a group of aliens not containing the target flashes. The twelve groups are separated in six “rows” and six “columns”, in such a way that a symbol is included in exactly one “row” and one “column” (2). Note that the naming of “rows” and “columns” do not refers to the physical rows and columns in the matrix, although it was the case in the first implementation of the protocol (15); those are simply group created pseudo-randomly as we have reported. The first digit of the values indicates whether the group is a “row” (digit 2) or a “column” (digit 4). The second digit indicates the number of the flashed “row” or “column” in the range [0, 5]. Note that the groups are randomized between the repetitions, thus a physical symbol in the matrix does not corresponds to the same “row” or “column”.
- 60-65 and 80-85 when a group of aliens containing the target of the first player flashes. The first digit of the values indicates whether the group is a “row” (digit 6) or a “column” (digit 8). The second digit indicates the number of the flashed “row” or “column” in the range [0, 5].
- 160-65 and 180-85 when a group of aliens containing the target of the second player flashes. The second digit of the values indicates whether the group is a “row” (digit 6) or a “column” (digit 8). The third digit indicates the number of the flashed “row” or “column” in the range [0, 5].
The Header.mat (or Header.csv) file contains the column names, sorted by ascending column number, including the name of the EEG channels. The age and gender of the subjects are provided in Table 1.
Other subjective variables, obtained by means of a questionnaire, such as the preference for the experimental condition and the motivation, will be published later on.
We supply an online code example working with Python (8) using the analysis framework MNE (16,17) and MOABB (18,19), a comprehensive benchmark framework for testing popular BCI classification algorithms. This example shows how to download the data and classify 1s non-Target and non-Target epochs using the Riemannian MDM algorithm (6).
References
1. Korczowski L. Methods for multi-subject electroencephalography and application to brain-computer interfaces [Internet] [Theses]. Université Grenoble Alpes; 2018. Available from: https://hal.archives-ouvertes.fr/tel-01961434
2. Congedo M, Goyat M, Tarrin N, Ionescu G, Varnet L, Rivet B, et al. “Brain Invaders”: a prototype of an open-source P300- based video game working with the OpenViBE platform. In: 5th International Brain-Computer Interface Conference 2011 (BCI 2011) [Internet]. 2011. p. 280–3. Available from: https://hal.archives-ouvertes.fr/hal-00641412/document
3. Squires NK, Squires KC, Hillyard SA. Two varieties of long-latency positive waves evoked by unpredictable auditory stimuli in man. Electroencephalogr Clin Neurophysiol. 1975 Apr;38(4):387–401.
4. Korczowski L, Barachant A, Andreev A, Jutten C, Congedo M. ”Brain Invaders 2” : an open source Plug & Play multi-user BCI videogame. In: 6th International Brain-Computer Interface Meeting (BCI Meeting 2016) [Internet]. Pacific Grove, CA, United States: BCI Society; 2016. p. 10.3217/978-3-85125-467-9. Available from: https://hal.archives-ouvertes.fr/hal-01318726
5. Barachant A, Bonnet S, Congedo M, Jutten C. Multiclass Brain Computer Interface Classification by Riemannian Geometry. IEEE Trans Biomed Eng. 2012 Apr;59(4):920–8. 6. Congedo M, Barachant A, Bhatia R. Riemannian geometry for EEG-based brain-computer
interfaces; a primer and a review. Brain-Comput Interfaces. 2017;4(3):155–74.
7. Barachant A, Congedo M. A Plug&Play P300 BCI Using Information Geometry. ArXiv14090107 Cs Stat [Internet]. 2014 Aug 30; Available from: http://arxiv.org/abs/1409.0107
8. Rodrigues P. Codes for working with the “Brain Invaders 2015b” dataset developed at the
GIPSA-lab [Internet]. 2019. Available from:
https://github.com/plcrodrigues/py.BI.EEG.2015b-GIPSA
9. Korczowski L, Cederhout M, Andreev A, CATTAN G, C. Rodrigues PL, Gautheret V, et al. Brain Invaders calibration-less P300-based BCI with modulation of flash duration Dataset (bi2015a) [Internet]. GIPSA-lab; 2019 Jul [cited 2019 Jul 4]. Available from: https://hal.archives-ouvertes.fr/hal-02172347
10. Renard Y, Lotte F, Gibert G, Congedo M, Maby E, Delannoy V, et al. OpenViBE: An Open-Source Software Platform to Design, Test, and Use Brain–Computer Interfaces in Real and Virtual Environments. Presence Teleoperators Virtual Environ. 2010 Feb 1;19(1):35–53.
11. Arrouët C, Congedo M, Marvie J-E, Lamarche F, Lécuyer A, Arnaldi B. Open-ViBE: A Three Dimensional Platform for Real-Time Neuroscience. J Neurother. 2005 Jul 8;9(1):3– 25.
12. Cattan G, Andreev A, Maureille B, Congedo M. Analysis of tagging latency when comparing event-related potentials [Internet]. Grenoble: Gipsa-Lab ; IHMTEK; 2018 Dec. (GIPSA-VIBS). Available from: https://hal.archives-ouvertes.fr/hal-01947551
13. Andreev A, Barachant A, Lotte F, Congedo M. Recreational Applications of OpenViBE: Brain Invaders and Use-the-Force [Internet]. Vol. chap. 14. John Wiley ; Sons; 2016. Available from: https://hal.archives-ouvertes.fr/hal-01366873/document
14. Van Veen G. Brain invaders-finding the paradox of control in a P300-game through the use of distractions [Internet]. [Netherland]: Twente; 2013. Available from: https://essay.utwente.nl/63413/1/MasterThesisGFPvanVeen.pdf
15. Guger C, Daban S, Sellers E, Holzner C, Krausz G, Carabalona R, et al. How many people are able to control a P300-based brain-computer interface (BCI)? Neurosci Lett. 2009 Oct 2;462(1):94–8.
16. Gramfort A, Luessi M, Larson E, Engemann DA, Strohmeier D, Brodbeck C, et al. MNE software for processing MEG and EEG data. NeuroImage. 2014 Feb 1;86:446–60.
17. Gramfort A, Luessi M, Larson E, Engemann DA, Strohmeier D, Brodbeck C, et al. MEG and EEG data analysis with MNE-Python. Front Neurosci [Internet]. 2013;7. Available from: https://www.frontiersin.org/articles/10.3389/fnins.2013.00267/full
18. Barachant A. Mother of All BCI Benchmarks. [Internet]. NeuroTechX; 2017. Available from: https://github.com/NeuroTechX/moabb
19. Jayaram V, Barachant A. MOABB: trustworthy algorithm benchmarking for BCIs. J Neural Eng. 2018 Sep 25;15(6):066011.