EEG-based Brain Computer Interface (BCI) for Smart Home Control using Raspberry Pi
MA SEET TING
This Report Is Submitted in Partial Fulfillment of Requirements for the Bachelor Degree of Electronic Engineering (Telecommunication)
Fakulti Kejuruteraan Elektronik dan Kejuruteraan Komputer Universiti Teknikal Malaysia Melaka
11
UNTVERSTI TEKNIKAL MALAYSIA MELAKA
9Ltl~~I
FAKULTI KEJURUTERAAN ELEKTRONIK DAN KEJURUTERAAN KOMPUTER UPtlVERSJTI TEKN KAL MALAYStA ~ELAKABORANGPENGESAHANSTATUSLAPORAN PROJEK SARJANA MUDA II
Tajuk Projek
...
J?J;:.Q~~~~~~- ~.r:~~. ~-<?~I?~!~~-~!~!!~~.~.(~~~).~~~-~-I?~~ ...Home Control using Raspberry Pi
Sesi Pengajian 1 5 I 1 6
Say a MASEETTING
(HURUF BESAR)
mengaku membenarkan Laporan Projek Sarjana Muda ini disimpan di Perpustakaan dengan syarat-syarat
kegunaan seperti berikut:
I. Laporan adalah hakmilik Universiti Teknikal Malaysia Melaka.
2. Perpustakaan dibenarkan membuat salinan untuk tujuan pengajian sahaja.
3. Perpustakaan dibenarkan membuat salinan laporan ini sebagai bahan pertukaran antara institusi pengajian tinggi.
4. Sila tandakan ( ...J ) :
D
SULIT*D
TERHAD**TIDAK TERHAD
(T ANDA T ANGAN PENULlS)
.
Cf{b(.)of(-;
Tankh: ... ..
*(Mengandungi maklumat yang berdarjah keselamatan atau kepentingan Malaysia seperti yang termaktub di dalam AKTA RAHSIA RASMI 1972)
**(Mengandungi maklumat terhad yang telah ditentukan oleh organisasi/badan di mana penyelidikan dijalankan)
Disahkan oleh:
~nsya
F9tulti lejufuterHn Elettronik Dan Kejuruterun loml'Utl!r · · · · i Melilkil UT~
ii
76~~urian Tunggat, Meliika Tarikh: ....... ( ..
~.,.
..~
..f.?:::0/
b
111
"I hereby declared that this report entitled EEG-based Brain Computer Interface (BCI)
for Smart Home Control using Raspberry Pi is a result of my own except for notes that
have been cited clearly in the references".
Signature
Name of Student
Date
: ...
~
... .
: Ma Seet Ting
:
a.(bf
~o(~
IV
"I hereby declared that I have read this report and in my opinion, this report is sufficient in term of the scope and quality for the award the Bachelor of Electronics Engineering (Telecommunications) with honours".
Signature
Name of Supervisor Date
Bit. LOW YIN FEN P~nsyerah K•nan
F.icu~ Kejuruterun Elektrenik 9an kejuruterun lt'omputer l niv@rsiti T@kntkil Milaysia Meliki (UTeM)
'.>-0 I I;) Hane Tuah Jiya
7i100 Durian Tunaal, Melaka
v
vi
ACKNOWLEDGEMENT
vii
ABSTRACT
viii
ABSTRAK
ix
TABLE OF CONTENTS
CHAPTER CONTENT PAGE NUMBER
PROJECT TITLE
CONFIRMATION ON REPORT STATUS DECLARATION SUPERVISOR’S CONFIRMATION DEDICATION ACKNOWLEDGEMENT ABSTRACT ABSTRAK
TABLE OF CONTENTS LIST OF TABLE
LIST OF FIGURE
LIST OF ABBREVIATION
I INTRODUCTION 1
1.1 Problem Statement 3
1.2 Objective 3
1.3 Scope of Work 4
x
II LITERATURE REVIEW 5
2.1 Electroencephalography 5 2.1.1 History of Electroencephalography 5
2.1.2 Brain Waves Classification 7
2.1.3 Applications of Electroencephalogram 9 2.1.4 EEG Recording 10
2.1.5 International 10/20 Electrodes Placement System 12
2.1.6 Low Cost EEG System 13
2.2 Brain Computer Interface (BCI) 16
2.3 Smart Home 17
2.4 Raspberry Pi 2 18
III METHODOLOGY 20
3.1 Overview of the System 20 3.2 Project Implementation 21 3.2.1 Load Emotiv EPOC Libraries into MATLAB 24 3.2.2 Perform Training and Experiment with Subject 29
xi
IV RESULT AND DISCUSSION 55
4.1 Properties of Emotiv EEG in MATLAB 55
4.2 2D (two-dimensional) Plotting 56
4.3 3D (three-dimensional) Plotting 57 4.4 Save data in matrix file (.mat) 58 4.5 Load the data stored in matrix file (.mat) 60
4.6 Signal Processing 61
4.7 Complete GUI (Graphic User Interface) 66 4.8 System Development with Raspberry Pi 2 68
V CONCLUSION AND RECOMMENDATION 73
REFERENCES 74
APPENDIX 77
Appendix A: Raspberry Pi 2 77
Appendix B: Coding to Control Smart Home 78 Appendix C: Coding to control GUI of Research Edition 87 Appendix D: Coding to control GUI of Smart Home 91 Appendix E: Silver Medal Award in Innovation and
Technology Competition (INOTEK)2016 95 Appendix F: 2nd Prize Award in Engineering Categories
for Infineon Week FYP Poster / Product
Competition 2016 96
xii
LIST OF TABLE
No. TITLE PAGE
2.1 Frequency Components of Brain Waves 7
xiii
LIST OF FIGURE
No. TITLE PAGE
2.1 Example of EEG Waves at Different Frequencies 8 2.2 Equipment for EEG recording: amplifier unit, electrode
cap, conductive jelly and injection 10
2.3 Adjacent brain areas of human brain 11
2.4 Front view of 10/20 system 12
2.5 Top view of 10/20 system 12
2.6 Emotiv EPOC Headset 14
2.7 The Mind Wave EEG device 15
2.8 Block diagram of BCI system 16
2.9 Smart Home Network 18
2.10 Layout of Raspberry Pi 2 19
3.1 Block Diagram of Smart Home Control Using Brain Wave
with Raspberry Pi 2 22
3.2 Flow chart of Project Implementation 23
3.3 Library files that are added into MATLAB path 25
3.4 Coding to test the libraries 26
3.5 System output shows EDK library already loaded 26
3.6 Assigning channel of electrodes 27
3.7 Electrodes 3 to 16 are chosen to be displayed 27 3.8 Script shows that electrodes 3 to 6 and 13 to 16 are selected
xiv
3.9 Disconnect the Emotiv EPOC form MATLAB 28
3.10 Coding to plot a 2D graph 29
3.11 Coding to plot a 3D graph 30
3.12 EPOC Control Panel 31
3.13 Coding that shows electrodes 3 to 16 are chosen to be displayed 32 3.14 Coding that enable the process of data recording 32 3.15 Filter and Design Analysis (FDA) tool in MATLAB 33
3.16 Generate MATLAB code using FDA tool 34
3.17 Export filter using FDA tool 34
3.18 Filter are exported in workspace and saved as .mat file 35 3.19 Script that display the signals before and after filter 35 3.20 Script that perform remove the mean of the signal 36
3.21 Get Hardware Support Package 37
3.22 Download the support package from internet 37
3.23 Select suitable support package 38
3.24 Download support package 38
3.25 Set the path for the Raspberry Pi support package 39
3.26 Proper board is selected 39
3.27 Create direct network configuration to the host computer 40
3.28 User interface of SDFormatter V4.0 40
3.29 Option setting of SDFormatter V4.0 41
3.30 Last check before the “Format” button is clicked 41
3.31 Proper drive is selected in MATLAB 42
3.32 MATLAB is writing firmware to the microSD card 42 3.33 Connection of Raspberry Pi 2 with laptop 43
3.34 Properties Raspberry Pi 2 44
3.35 Error shows that fail to create connection between Raspberry
Pi 2 and MATLAB 44
3.36 Commands to turn off the power of Raspberry Pi 2 44 3.37 Raspberry Pi 2 is shut down using commands 45
xv
3.39 Create a new blank GUI 46
3.40 An untitled.fig pop out 47
3.41 Adjust the size of GUI 47
3.42 Select push button 48
3.43 Add a push button 48
3.44 Inspector of push button 49
3.45 Create callback function 49
3.46 Save the GUI 50
3.47 Create callback function for push button 50
3.48 Set white colour background 51
3.49 Create button group 51
3.50 Edit the push button group 52
3.51 Add static text in GUI 52
3.52 Add photo using icon axes 53
3.53 Axes with desired size 53
3.54 Coding to display photo in axes 54
3.55 Coding to play music 54
3.56 Callback a GIF 54
3.57 Part of the script in gifplayer.m 55 4.1 Coding to check the properties of Emotiv OPOC 56
4.2 Properties of Emotive EPOC 57
4.3 2D (two-dimensional) Plotting of real time data 58 4.4 3D (three-dimensional) Plotting of real time data 59 4.5 Coding of execution of save function in EmotivEEG.m 59 4.6 .mat file that has been successfully saved 61 4.7 Coding that used to load data in EmotivEEG.m 61 4.8 Data loaded from the saved matrix file (.mat) 62 4.9 Trained data from Channel 4 in 10 seconds 62 4.10 Trained data from Channel 16 in 10 seconds 63
4.11 Detrending the signals 63
xvi
4.13 Magnitude response of Chebyshev type II filter 64 4.14 Script of Chebyshev type II low pass filter with filter order of 100 65 4.15 Script of Chebyshev type II low pass filter with filter order of 30 65
4.16 Original and filtered signals 66
4.17 Research Edition EEG-based BCI for Smart Home Control using
Raspberry Pi 67
4.18 Non-research edition GUI 67
4.19 Smart Home Prototype 69
4.20 Smart Home Prototype before triggered by brain signals 69 4.21 Table lamps and media players are switch on 70
4.22 Coding to control Channel 5 71
4.23 Stand lamps and CCTV are switch on 72
4.24 Frame captured by CCTV 72
[image:16.612.128.516.79.358.2]xvii
LIST OF ABBREVIATION
EEG - Electroencephalogram GUI - Graphical User Interface BCI - Brain Computer Interface
RISC - Reduced Instruction Set Computer
ARM - Advanced RISC Machine
1
CHAPTER I
INTRODUCTION
An electroencephalogram (EEG) is an electrical signal recorded from the scalp surface after it is being picked up by metal electrodes and conductive media known as EEG bio amplifier. The electrodes detect the weak electrical signals and then amplify them before represent them in the form of graph or data in a computer or laptop. This allows users to see how the brain functions over time and it is being implemented in medical industry to evaluate brain disorders. Also, it enables the detection of abnormalities in the brain by investigation on the frequency and waveform of the brain. This can be applied directly to all mankind including patients, adults, adolescents or even children without any risk. This system is totally non-invasive because the EEG signal is directly measured from the cortical surface of the brain.
2
to translate the intent of the users which originally in the form of EEG signals into computer readable commands to execute the task designed.
Recently, many researchers have developed products that can be commercialize to bring convenience to every people in the daily life. This scenario has prompt smart home to become one of the mainstream in research and commercialize areas. Various types of environmental control systems thus are available in market. However, most of the existing BCI control system requires user’s active mental command to control the appliances or devices. For example, it requires the user to raise their arms or turn their head in order to control the devices. This is really inconvenience especially to those people with mobility impairment. Consumers are lack of confidence with this system as it is less ubiquitous to control devices automatically. Hence, BCI with the use of cognitive state only is must in the future and should be implemented as fast as possible to enhance the living quality of every human being.
In this project, two attractive graphical user interfaces (GUI) are developed. The first GUI is used to show users’ real time brain signal activities while the second GUI is used to control the whole smart home with the aid of Raspberry Pi and knowledge in MATLAB. Raspberry Pi is a low cost single board mobile computer equipped with ARM Linux box in credit-card size that enable the users to interact with outside world no matter it is analog or digital. The brain signals are filtered and processed before send to the Raspberry Pi to removes noise and artifacts that will interrupt the execution of designed tasks. Hence, EEG-based brain computer interface (BCI) for smart home control using Raspberry Pi is developed within two semesters.
3
1.1 Problem Statement
Human wishes to get better and more convenience life with the advance of technologies nowadays. Creations nowadays are so smart until things and stuffs can be control wirelessly and digitally by sensors, smartphones, robots, tablets, laptops and so on. Smart Home is one of the example that loves by mankind and it is still being developed. Current trend for smart home nowadays is by using sensors and thumbprint which is troublesome sometimes. It is not ubiquitous with the use of sensors and thumbprint as the task designed can be executed only if the user is within the range of execution of the sensors. Also, the cost for a complete smart home system is very high and thus most of the people is not affordable to own this technology. Apart from that, there are a lot of people in this world suffering malfunction in the motor activities that cause them facing inconvenience in their daily life. They have difficulties in performing their daily activities. Hence, this project is to develop a brand new and attractive GUI with an affordable EEG system to control and monitor home appliances wirelessly using Raspberry Pi with the knowledge in MATLAB software.
1.2 Objectives
The main objectives of this project are:
To develop a prototype of smart home using low-cost EEG system and Raspberry Pi.
To analyze brain signals using MATLAB software.
4
1.3 Scope of Work
5
CHAPTER II
LITERATURE REVIEW
This chapter gives reviews on the terms and information that is essential for the project, this includes Electroencephalogram (EEG), brain wave classification, 10/20 Electrode Placement System, Brain Computer Interface (BCI), Smart Home and Raspberry Pi 2. These are all written based on the published works from related researches.
2.1 Electroencephalography
2.1.1 History of Electroencephalography
6
they can generate electrical voltage in microvolts inside brain cortex. This differences in electrical potential is due to the potential difference between neuron and dendrites [2]. When it comes to the scalp, its voltage will drop to microvolts. These type of signals are massively amplified and then send to the laptop to further process or display. Brain signals can be acquired by attaching metal electrodes onto the scalp surface. This method is known as electroencephalography (EEG) and the use of this technology has expended within these few years as it is very user friendly and the price is affordable.
The discovery of electroencephalography (EEG) by the German psychiatrist Hans Berger in 1929 was a historical breakthrough providing a new neurologic and psychiatric diagnostic tool at the time. Electroencephalography has undergone tremendous changes for more than 100 years and it is still counting. At the very beginning, Richard Caton, an English physician observed the EEG signals of monkeys and rabbits from their exposed brains. Han Berger who was impressed by his great work tend to continue his research and he tried to amplify the brain signal measured on human scalp. Finally, he successfully recorded and reveal the brain signals graphically on paper without opening the skull by using amplifier with ordinary radio equipment. From his experiment, he noticed that the brain waves change accordingly depends on the activities being carried out. For instances, stages like concentrate, sleep and epilepsy will have different waveforms. He was right with his observations and he suggested that general status of a subject changes will cause the brain activities change in a consistent and recognizable way [2]. This great finding had become a great fundamental for many applications of electroencephalography. Also, he was the first person to use the word electroencephalogram to describe brain electric potentials generate by human brain. Later in 1934, a paper verified the concept of “human brain waves’ was published by Adrian and Matthews. They named “alpha rhythm” for brain wave that oscillates around 10 to 12 Hz.
7
technologies. Current EEG unit has rechargeable battery, lightweight with wireless connectivity has stand a place in the market with affordable price. Users can record the EEG signals for days, weeks, or months at once easily [3]. Several of activities can be performed with this advanced of technologies as well.
2.1.2 Brain Waves Classification
EEG signals can be classified into 4 main groups, named Delta, Theta, Alpha and Beta according to the frequencies. The dominant brain signal is Alpha wave. It is mostly can be obtained in relax state where its amplitude ranges 30-50 µV in 8-12Hz. The second dominant wave is Beta wave. In human brain, different frequencies can be acquired and observed easily as via the brain waves. It depends highly on the activities carried out by each mankind [4]. Table 1 below show the frequency components of brain waves:
Brain Wave Frequency (Hz) Amplitude (µV) Cognitive States
Delta 0.1 – 4.0 100 - 200 Deep sleep
Unconscious state
Theta 4.0 – 8.0 <30 Dreaming
Sleep
Alpha 8.0 – 13.0 30 – 50 Relax
Non-sleepy state
Beta > 13 <20
Excited Awake
[image:24.612.127.545.400.625.2] Conscious State Table 2.1: Frequency Components of Brain Waves [4].
![Table 2.1: Frequency Components of Brain Waves [4].](https://thumb-us.123doks.com/thumbv2/123dok_us/77111.7268/24.612.127.545.400.625/table-frequency-components-brain-waves.webp)