ISSN: 2302-9285, DOI: 10.11591/eei.v9i1.1685 238
Deep stair walking detection using wearable inertial sensor via long short-term memory network
Gan Wei Nie, Nurul Fathiah Ghazali, Norazman Shahar, Muhammad Amir As’ari School of Biomedical Engineering and Health Sciences, Universiti Teknologi Malaysia, Malaysia Sport Innovation and Technology Center (SITC), Institute of Human Centered Engineering (iHumEn),
Universiti Teknologi Malaysia, Malaysia
Article Info ABSTRACT
Article history:
Received Jul 18, 2019 Revised Sep 30, 2019 Accepted Nov 12, 2019
This paper proposes a stair walking detection via Long Short-Term Memory (LSTM) network to prevent stair fall event happen by alerting caregiver for assistance as soon as possible. The tri-axial accelerometer and gyroscope data of five activities of daily living (ADLs) including stair walking is collected from 20 subjects with wearable inertial sensors on the left heel, right heel, chest, left wrist and right wrist. Several parameters which are window size, sensor deployment, number of hidden cell unit and LSTM architecture were varied in finding an optimized LSTM model for stair walking detection. As the result, the best model in detecting stair walking event that achieve 95.6% testing accuracy is double layered LSTM with 250 hidden cell units that is fed with data from all sensor locations with window size of 2 seconds. The result also shows that with similar detection model but fed with single sensor data, the model can achieve very good performance which is above 83.2%. It should be possible, therefore, to integrate the proposed detection model for fall prevention especially among patients or elderly in helping to alert the caregiver when stair walking event occur.
Keywords:
Confusion ma trix Deep lea rning LSTM network Sta ir wa lking
Wea ra ble inertia l sensor
This is an open access article under the CC BY-SA license.
Corresponding Author:
Muhammad Amir As’ari,
School of Biomedica l Engineering a nd Hea lth Sciences, Fa culty of Engineering,
Universiti Teknologi Ma la ysia , 81310 Johor Ba hru, Ma la ysia . Ema il: a mir-a sa ri@biomedica l.utm.my
1. INTRODUCTION
Among so ma ny indoor a ctivities of da ily living (ADL), sta ir wa lk is the one tha t ha ve a ma jor potentia l a nd ha za rdous for people to fa lls especia lly elderly. It might ca use significa nt injuries such a s hip fra cture, tra uma tic bra in injuries (TBI) a nd dea ths. The injury Fa cts 2015 [1] of na tiona l sa fety council’s sta tistica l report ha d reports tha t there a re over one million injuries a nd 12000 dea ths a re ca use by sta irwa y a ccidents ea ch yea r. Boyé et.a l [2] a lso ha d investiga ted on fa lls ra te of elderly popula tions in Netherla n ds a nd 409 out of tota l of 5880 fa ll-rela ted Emergency Depa rtment visit is due to wa lking up or down sta irs.
Among the indoor a ctivities, fa lls during sta ir wa lking event ha ve the highest percenta ge to susta in TBI which is 52% for women a nd 61% for men. No t only tha t, the study from Hwa ng, et.a l [3] ha d shown tha t elderly a re 3 times more likely to suffer from TBI a fter sta ir fa lls when compa re to norma l fa ll while wa lking.
Another study in Ma la ysia by Sa zlina et.a l [4] a lso shows tha t 61% of elderly fa lls indoor a nd 57% of them experienced recurrent fa lls. The most common indoor pla ces tha t elderly fa lls a re sta irs a nd ba throom which is 27% both. The fa ctor tha t lea d to sta ir fa lls ca n be cla ssify into two which a re host -rela ted fa ctors a nd environment-rela ted fa ctors. Host-rela ted fa ctor is fa ctor tha t contributes by hea lth condition
of host [5] for exa mple decline in muscle strength [6 -7], a bility of posture control [8], cognitive fa ctor [9 ], visua l condition [10], a nd obesity [11]. An environment-rela ted fa ctor is the fa ctors tha t ca use by environment of host which include sta ir a rchitecture design, a nd sta ir obsta cles such a s a bsence of ha ndra il, irregula r riser height a nd object left on sta irs. These fa ctors will force the sta irca se user to use their m a xima l ca pa bilities to wa lk up or down the sta irs a s grea ter body posture control effort is required.
Since there is high risk to fa ll a nd lea d to serious injuries, huma n a ctivity recognition (HAR) is developed a s a pa rt of a fra mework to a utoma tica lly mo nitor elderly a ctivities a nd reduce the burden of ca regiver. Most of the previous studies [12-14] embedded common machine lea rning model tha t integra ted with sha llow a nd huma n cra fted fea tures extra ction a pproa ches which could only a ble to recognize low level a ctivities. Recently, deep lea rning model ha s been formula ted in HAR rela ted studies [15-19] to overcome the limita tion of common ma chine lea rning a pproa ches. However, to our knowledge, there ha s been la ck of resea rch conducted on detecting sta ir wa lking event from other a ctivities using wea ra ble inertia l sensor via deep lea rning a pproa ch a s well a s detect sta ir fa lls. Therefore, a sta ir wa lking detection is proposed to prevent sta ir fa ll by detect sta ir wa lking a ctivities a s well a s other da ily a ctiv ities using inertia l sensor a nd implement into the LSTM network. This ca n reduce the burden of ca regiver by a lerting ca regiver a s soon a s a ny sta ir wa lk a ctivity is detected before a ny sta ir fa ll ha ppen.
2. LSTM NETWORK OVERVIEW
Genera lly, deep lea rning is a n idea l a pproa ch for HAR a s the property of deep lea rning a ble to solve the limita tions of ma chine lea rning. It a ble to extra ct fea tures a utoma tica lly, recognize complex high -level a ctivities a nd reduce computa tiona l cost. LSTM is a composition from Recurrent Neura l Network (RNN) a nd it is ca pa ble of ca pturing long term dependencies with a lot of memory units ca lled cells [20].
Figure 1 shows LSTM cell.
Figure 1. LSTM cell [21]
LSTM network ha ve lot of memory cell composed in it a nd this la rge sta ck of memory cell property ena ble it to lea rn complex input. Inside a memory cell unit consists a n input ga te , output ga te and a forget ga te as in Figure 1. All these gates unit will regulate the content of memory cell that flow in and out of the cell. A memory cell will connected to a nother cell. The forget ga te in memory cells ma ke LSTM sma rt enough to decide wha t to era se from memory a nd keep only releva nt da ta [22-23]. It removes unnecessa ry da ta memory from previous sta te by multiply with previous cell sta te a s (1) where W is recta ngula r input weight ma trices; b is the bia s vector a nd x is the input vector.
(1) Input ga te is function to a dd new input to present cell sta te. (2) will decide which va lues to be upda ted a nd (3) will crea te vector for new ca ndida te va lues.
(2)
(3)
The present cell sta te is ca lcula ted by using (4)
(4) A sigmoid la yer is run to decide which pa rts of cell sta te is going to output. The cell sta te is ta nh a nd multiplies with sigmoid ga te output by using (5) a nd (6).
(5)
(6)
3. EXISTING WORKS IN WEARABLE SENSOR BASED HAR
There a re ma ny fa ll detector tha t were developed to solve the fa ll problem on elderly. The study of Ozdemir, et.al [24] ha d a lso successfully distinguished fa lls from ADLs tha t ca n ca use high a ccelera tion body pa rt such a s jumping, sitting down suddenly a nd running. 6 ma chine lea rning a pproa ch which a re k-Nea rest Neighbor (k-NN), Lea st Squa re Method (LSM), Ba yesia n Decision Ma king (BDM), Support Vector Ma chine (SVM), Artificia l Neura l Network (ANN) a nd Dyna mic Time Wa rping (DTW) wa s use a s the cla ssifier a nd the performa nce of ea ch cla ssifier is compa re. In terms of the required tra ining time, the cla ssifiers ca n be sorted a s BDM, LSM, DTW, k -NN, SVM, a nd ANN in increa sing order, wherea s in term of the testing time, the order is ANN, SVM, LSM, BDM, k -NN, a nd DTW. The a ccura cy ha s a chieved a bove 95% for a ll cla ssifier.
Steven Eyobu et.al [18] ha d proposed a huma n a ctivities recognition ra nging from wa lking, sitting, fa lling, climbing a nd sta ir wa lking. LSTM neura l network is proposed in the model to solve the issue tha t difficulty in discrimina te a mongst high simila rity fea tures. The proposed a pproa ch is Deep Recurrent Neura l Network (DRNN). This a pproa ch ha s a n a dva nta ge in high throughput which is short recognition time a nd a ble to discrimina te a ctivities tha t ha ve a lmost simila r fea tures.
There is a lso few works tha t a ble to recognize complex instrumenta l a ctivities da ily life (ADL).
For exa mple, the A-Wristocra cy wrist worn sensing recognition [20] tha t proposed by Vepa komma , et.al.
The proposed A-Wristocra cy recognition system a re a ble to recognized fine-gra ined 22 indoor a ctivities by multi-moda l sensors which consists of a ccelerometer, gyroscope, a mbient loca tion context sensing a nd a tmospheric environmental sensors. The 22 complex f ine-gra ined a ctivity is contexts into va rious cla sse s which a re locomotive, sema ntic, tra nsitiona l a nd postura l. All the test a ccura cy for va rious number of neurons in hidden la yers ha d a chieve testing a ccura cy a bove 84%. Pa nwa r et. al [16] a lso ha d proposed a HAR recognition model using Convolutiona l Neura l Network (CNN) to recognize 20 sma ll a ctions in ma king a cup of tea . A single wrist worn tri-a xia l a ccelerometer is used in the study a nd detect with extension a nd flexion of forea rm, rota tion of forea rm a nd rota tion of the wrist a bout long a xis of forea rm. This study ha s a chieved a performa nce a ccura cy of 99.8%.
4. RESEARCH METHOD
The workflow of this study is comprises of five steps which a re da ta a cquisition, da ta pre-processing, LSTM network a rchitecture implementa tion, da ta set tra ining a nd testing, a nd performa nce eva luation. All the steps will expla in further in the following subsections.
4.1. Data acquisition
Ga it Up Physilog 5 Inertia l mea surement sensor unit a s in Figure 2 is used in this study.
This wea ra ble inertia l sensor sensing a bilities includes 3D a ccelerometer, 3D gyroscope a nd ba rometric sensor. However, only 3D a ccelerometer a nd 3D gyroscope a re used in this project. The da ta wa s collected by placing the inertial sensor on subject’s chest, wrists, and heels. The sampling frequency of Physilog 5 sensor is 128 Hz.
20 subjects were involved in this study a nd ea ch subject wa s a sked to perform few da ily living a ctivities. The da ily living a ctivities is including sta ir wa lking, wa lking, sitting, sta nding a nd la ying down 01a s in Figure 3. All the a ctivities were performed continuously a t a subject comforta ble speed. Th e sa me a ctivity set wa s repea ted by ea ch subject for 3 times a t different sta irs.
Figure 2. Ga it up physilog 5 inertia l mea surement sensor unit
Figure 3. Da ily living a ctivities, (a ) Wa lking, (b) Sta nding, (c) Sta ir wa lking, (d) La ying down, (e) Sitting
4.2. Data pre-processing
Before the da ta is used for tra ining, the collected ra w da ta is required to undergo preprocessing.
The collected raw data was first labelled. Walking is represented by “1”; Stair walking is represented by “2”;
sitting is represented by “3”; lying down is represented by “4”; a nd sta nding is represented by “5”. After tha t, a windowing technique is a pplied through the la belled da ta to ta ke sma ll subset through this la rge da ta set.
The window sizes a pplied to the da ta set wa s 0.5 sec, 1 sec, 1.5 sec a nd 2 secs. The da ta a fter the windowing wa s then na med a s X a nd the la beling na med a s Y. Da ta X a nd da ta Y wa s divided into three pa rts which a re 90 % for tra ining set a nd 10% for test set. Tra ining set wa s used to fit the LSTM ne twork model a nd testing set wa s used to eva lua te the fina l LSTM network model.
4.3. LSTM network architecture implementation
LSTM network wa s implemented using deep lea rning toolbox of MATLAB 2018a . Figure 4 shows the fra mework of LSTM network. There will ha ve severa l la yers in the LSTM network which include a n unknown n la yer of LSTM hidden unit la yers, fully connected la yer, softma x la yer a nd cla ssifica tion la yer.
Figure 4. The fra mework of LSTM network
The LSTM network tra ining setting is a s Ta ble 1. The optimizer solver used is Ada m which deca y ra tes ca n be specified. Gra dient threshold is set to 1 so tha t ca n prevent gra dient from exploding. The initia l lea rning ra te is set to 0.01 a nd drop in period o f 20. This is to a llow the network ha ve la rge cha nge in the beginning of the tra ining a nd decrea se the lea rning ra te over the epochs to ha ve sma ller tuning cha nges a t the tra ining la ter. The regula riza tion is set to L2 to control the model ca pa city a nd re duce overfitting of the model. The input signa l dimension is 30 a nd the output dimension is 5.
Ta ble 1. LSTM network tra ining setting
Settings Details
Output Layer Activation Function Softmax
Optimizer Solver Adam
Gradient Threshold 1
Max Epochs 30
Initial learning rate 0.01
Regularization L2
Input signal dimension/ feature dimension 30 Output dimension/ class 5
4.4. Data training and testing
As mention in the subtopic before, there were 5 sensors wear at each subject’s chest, wrists, a nd heels a nd ea ch sensor will ha ve 6 fea tures (tri-a xia l a ccelerometer a nd tri-a xia l gyroscope).
Thus, the tota l fea tures dimension of the da ta set wa s 30. The input da ta sequence will directly loa d into the la yers for tra ining purpose without fea ture extra ction. This is due to the deep lea rning a pproa ch ha s a utomatic fea ture extra ction property.
4.5. Performance evaluation
The performa nce will be eva lua ted using confusion ma trix plot in Ta ble 2 tha t consist of two dimensions which a re a ctua l a nd predicted. True positive, true nega tive, fa lse positive a nd fa lse nega tive ca n know from the confusion ma trix.
Ta ble 2. Confusion ma trix plot Actual
Predicted
Positive Negative
Positive True Positive False Positive Positive Predicted Value = 𝑇𝑃
𝑇𝑃+ 𝐹𝑃
Negative False Negative True Negative Negative Predicted Value = 𝑇𝑁
𝑇𝑁 +𝐹𝑁
Sensitivity = 𝑇𝑃
𝑇𝑃+ 𝐹𝑁 Specificity = 𝑇𝑁
𝑇𝑁 +𝐹𝑃 Accuracy = 𝑇𝑃 +𝑇𝑁
𝑇𝑃 +𝑇𝑁 + 𝐹𝑃+𝐹𝑁
5. RESULT AND DISCUSSION
As mentioned in the previous cha pter, there a re severa l pa ra meters need to be va ried in sea rching a n optimum wa lking detection model which produced highest a ccura cy va lue. The pa ra meters a re sliding window size, sensor deployment, number of hidden cell un it a nd LSTM a rchitecture (either single la yered or double la yered LSTM). The eva lua tion process is divided into severa l sta ges.
5.1. Window size varied (stage 1)
At the first sta ge, the window size is va ried while fixing the sensor deployment which used input da ta from a ll a tta ched sensors on single la yered LSTM networks with 100 hidden cell units.
Ta ble 3 summa rizes the LSTM network models a ccura cy of va rying window size while fixing other pa ra meters. The results obta ined shows tha t window size of 2 sec onds give the best performa nce when compa red to the window size of 0.5 seconds, 1 second a nd 1.5 seconds. Thus, window size of 2 seconds wa s used for the next sta ge.
Ta ble 3. Summa ry of LSTM network models a ccura cy of va rying window size Window Size (seconds) Testing Accuracy (%)
0.5 74.0
1.0 76.1
1.5 77.7
2.0 79.4
5.2. LSTM architecture varied (stage 2)
In the second sta ge, the LSTM network a rchitecture is va ried while fixing the sensor deployment which used input da ta from a ll a tta ched sensors a nd a lso fixing the window size a t 2 seconds.
Ta ble 4 summa rized the LSTM network model a ccura cy of 2 seconds window size for a ll sensors da ta deployed in different a rchitecture. The results show tha t the testing a ccura cy ha s increa se with the increa se number of LSTM hidden cell unit. Also, double la yered LSTM model ha ve higher a ccura cy tha n single la yered LSTM a s demonstra ted in [25]. This is beca use the ca pa city of LSTM network is increa se a nd tra in in a deeper wa y when the number of LSTM la yer is increa se. Among the tra ined LSTM a rchitecture, double la yered LSTM network with 250 hidden cell units per la yer produced the best performa nce.
Thus, double la yered LSTM network with 250 hidden cell units per la yer is used in the next sta ge.
Ta ble 4. Summa ry of LSTM network models a ccura cy of va rying a rchitecture
LSTM Layer Single Double
Architecture Number of LSTM Hidden Cell Unit per Layer 100 250 100 250
Accuracy (%) All Sensors 79.4% 80.6% 92.6% 96.5%
5.3. Varying input sensor data (stage 3)
In the third sta ge, the sensor input da ta used is va ried while fixing the window size a t 2 seconds a nd LSTM network a rchitecture a t double la yered with 250 hidden cell units per la yer. Ta ble 5 summa rized the LSTM network models a ccura cy of different sensor a t 2 seconds window size with double la yered a rchitecture of 250 hidden cell units per la yer. The result a lso shows tha t the LSTM network model tha t feed with a ll sensor da ta ha ve grea ter performa nce tha n single sensor da ta . It ca n be interpreted tha t the performa nce of LSTM network for sta ir wa lking detection is a ffected by the number of sensor da ta tha t fed into network. The more input da ta , the better the LSTM performa nce which is sa me a s sta ted in [26].
It ca n a lso be interpreted tha t sensor da ta from chest, right heel a nd left heel ma jorly contribute in producing grea t a ccura cy for sta ir wa lking detection while right wrist a nd left wrist provide the lea st performa nce. This might due to the ha nd movement is unpredicta ble a nd in ra ndomize direction while performing the a ctivities of da ily living. Thus, the produced sensor da ta from both wrists is very difficult to discrimina te between a n a ctivity to a nother a ctivity.
Ta ble 5. Summa ry of LSTM network models a ccura cy for ea ch single sensor Sensor Chest Right Heel Left Heel Right Wrist Left Wrist Testing Accuracy (%) 87.8% 87.7% 89.7% 86.6% 83.2%
5.4. The best LSTM network
The best LSTM network model is the double la yered LSTM model with 250 hidden units per la yer which showed the best testing a ccura cy for da ta set with 2 seconds window size. The testing a ccura cy obta ined is 95.6% a s in Figure 5 a nd the error ra te is 3.5%. From the sa me confusion ma trix obta in ed, we a lso ca n know tha t the sensitivity to detect sta ir wa lking event (cla ss 1) correctly is 97.9%. Only 1 out of 424 a nd 8 out of 424 is cla ssifying wrongly a s sta nding (cla ss 4) a nd wa lking (cla ss 0) respectively.
The positive predicted va lue is 97% which ha ve 1 out of 188 from sta nding event a nd 12 out of 442 from wa lking event ha d wrongly detected a s sta ir wa lking event.
There is none of the La ying down event (cla ss 3) a nd sitting event (cla ss 2) wrongly cla ssify into it.
In terms of specificity, a ctivities lying down, sitting, sta nding a nd wa lking ha ve 97.3%, 95.1%, 94.1%
a nd 96.6% respectively. This mea ns tha t, there is a lso ha ve a good performa nce in detect a ctivities other tha n sta ir wa lking correctly. For nega tive predicted va lue, a ctivities lying down, sitting, sta nding a nd wa lking ha ve 96.3%, 96.6%, 95.2% a nd 96.6% respectively. All the a ctivities tha t other tha n sta ir wa lking do not ha ve much wrong cla ssify from other cla sses into it.
Figure 5. Confusion ma trix tha t obta ined from the best LSTM network model
6. CONCLUSION
In conclusion, a set of tri-a xia l a ccelerometer a nd tri-a xia l gyroscope da ta wa s collected a nd new a ctivities da ta set wa s crea ted for model tra ining a nd testing. Deep structured LSTM network models wa s implemented to detecting sta ir wa lking event a s well a s other a ctivities of da ily a ctivities. Ba sed on the results obta ined, it shows tha t the window size of 2 seconds gives the best performa nce when compa red to 0.5, 1, a nd 1.5 seconds. In the second sta ge, the result shows tha t the testing a ccura cy increa se with number of hidden units a nd double la yered LSTM give better performa nce tha n single la yered. The best a ccura cy is a t double la yered LSTM with 250 hidden units per la yer which is 96.5%. From the third sta ge, the testing a ccura cy of ea ch single sensor ha ve a chieved a bove 83.2%. It a lso shows tha t the sta ir wa lking event ha s higher dependency on left heel, chest a nd right heel. This LSTM model ca n be further implemented into a n a utoma ted sta ir wa lking detection system tha t ca n detect a nd a lert the ca regiv ers when sta ir wa lking event occurs on elderly or pa tient. The burden of ca regivers ca n be reduced a nd sta ir fa lls on elderly or pa tient ca n be prevented by using this tra ined model.
ACKNOWLEDGEMENTS
The a uthors would like to express their gra titude to Universiti Teknologi Ma la ysia (UTM) a nd the Minister of Educa tion (MOE), Ma la ysia for supporting this resea rch work under FRGS Resea rch Gra nt No. R.J130000.7851.5F108.
REFERENCES
[1] National Safety Council (NSC), Injury Facts 2015 Edition. 2015.
[2] N. D. A. Boyé et al., “Circumstances leading to injurious falls in older men and women in the Netherlands,”
in Injury, vol. 45, no. 8, pp. 1224–1230, 2014.
[3] H. F. Hwang, C. H. Cheng, D. K. Chien, W. Y. Yu, and M. R. Lin, “Risk factors for traumatic brain injuries during falls in older persons,” in J. Head Trauma Rehabil., vol. 30, no. 6, pp. E9–E17. 2015.
[4] S. G. Sazlina, R. Krishnan, A. S. Shamsul, A. Zaiton, and R. Visvanathan, “Prevalence of falls among older people attending a primary care clinic in Kuala Lumpur, Malaysia,” in Journal of Community Health, vol 14, no. 1, pp. 11–16, 2008.
[5] H. A. Ojha, R. W. Kern, C.-H. J. Lin, and C. J. Winstein, “Age affects the attentional demands of stair ambulation:
Evidence from a dual-task approach,” in Phys. Ther., vol. 89, no. 10, pp. 1080–1088, 1 October 2009.
[6] L. Ding and F. Yang, “Muscle weakness is related to slip-initiated falls among community-dwelling older adults,”
in J. Biomech., vol. 49, no. 2, pp. 238-243, 25 January 2016.
[7] M. A. Yusuke Suzuki, “Muscle strength of lower extremities related to incident falls in community-dwelling older adults,” in J. Gerontol. Geriatr. Re, vol. 04, no. 02, pp. 1-5, 2015.
[8] N. M. Kosse, M. H. De Groot, N. Vuillerme, T. Hortobágyi, and C. J. C. Lamoth, “Factors related to the high fall rate in long-term care residents with dementia,” Int. Psychogeriatrics, vol. 27, no. 5, pp. 803-814, May 2015.
[9] D. Schoene, T. Valenzuela, S. R. Lord, and E. D. De Bruin, “The effect of interactive cognitive-motor training in reducing fall risk in older people: A systematic review,” BMC Geriatrics. no. 107, pp. 1-22, 2014.
[10] M. G. Gaerlan, “The role of visual, vestibular, and somatosensory systems in postural balance,” Univ. Nevada Las Vegas, 2010.
[11] Naugle, K. M., Higgins, T. J., Manini, T. M., "Obesity and use of compensatory strategies to perform common daily activities in pre-clinically disable older adults," Archives of Gerontology and Geriatrics, vol. 54, no. 2, pp. e134-e138, 2012.
[12] G. M. Weiss, J. L. Timko, C. M. Gallagher, K. Yoneda and A. J. Schreiber, "Smartwatch-based activity recognition: A machine learning approach," 2016 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), Las Vegas, NV, pp. 426-429, 2016.
[13] X. Su, H. Tong, and P. Ji, “[Read] Activity recognition with smartphone sensors,” Tsinghua Sci. Technol., vol. 19, no. 3, pp. 235–249, 2014.
[14] Attal, F., S. Mohammed, M. Dedabrishvili, F. Chamroukhi, L. Oukhellou, and Y. Amirat, “Physical human activity recognition using wearable sensors,” in Sensors (Basel)., vol. 15, no. 12, pp. 31314-31338, 2015.
[15] A. Ignatov, “Real-time human activity recognition from accelerometer data using Convolutional Neural Networks,”
in Appl. Soft Comput. J., vol. 62, pp. 915–922, 2018.
[16] M. Panwar et al., “CNN based approach for activity recognition using a wrist-worn accelerometer,” in Proc. Annu.
Int. Conf. IEEE Eng. Med. Biol. Soc. EMBS, pp. 2438–2441, 2017.
[17] M. Inoue, S. Inoue, and T. Nishida, “Deep recurrent Neural Network for mobile human ativity recognition with high throughput,” in ArXiv e-prints, 2016.
[18] O. Steven Eyobu and D. Han, “Feature representation and data augmentation for human activity classification based on wearable IMU sensor data using a deep LSTM Neural Network,” in Sensors, vol. 18, no. 9, pp. 2892, 2018.
[19] F. J. Ordóñez and D. Roggen, “Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition,” in Sensors (Switzerland), vol. 16, no. 1, pp. 1-25, 2016.
[20] P. Vepakomma, D. De, S. K. Das and S. Bhansali, "A-Wristocracy: Deep learning on wrist-worn sensing for recognition of user complex activities," 2015 IEEE 12th International Conference on Wearable and Implantable Body Sensor Networks (BSN), Cambridge, MA, pp. 1-6, 2015.
[21] J. Li, H. Xu, X. He, J. Deng, and X. Sun, “Tweet modeling with LSTM recurrent neural networks for hashtag recommendation,” in Proc. Int. Jt. Conf. Neural Networks, vol. 2016–Octob, no. 2, pp. 1570–1577, 2016.
[22] Z. C. Lipton, J. Berkowitz, and C. Elkan, “A critical review of recurrent Neural Networks for Sequence Learning,”
in ArXiv, pp. 1–38, 2015.
[23] Yong Du, W. Wang and L. Wang, "Hierarchical recurrent neural network for skeleton based action recognition,”
in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, pp. 1110-1118, 2015.
[24] A. T. Özdemir and B. Barshan, “Detecting falls with wearable sensors using machine learning techniques,” Sensors (Switzerland), vol. 14, no. 6, pp. 10691–10708, 2014.
[25] S. Li, W. Li, C. Cook, C. Zhu, and Y. Gao, “Independently Recurrent Neural Network (IndRNN): Building a longer and deeper RNN,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., pp. 5457–5466, 2018.
[26] C. C. Aggarwal, “Neural networks and deep learning .” a textbook. 2018.
BIOGRAPHIES OF AUTHORS
Gan Wei Nie currently finished her study in Bachelor of Engineering (Bio-medical) at Universiti Teknologi Malaysia. Her research interest is about human activity recognition using machine learning and deep learning.
Nurul Fathiah Ghazali received Diploma in Electrical Engineering (Power) from Universiti Teknologi Malaysia, Kuala Lumpur (2013) and Bachelor of Engineering in Biomedical from Universiti Teknologi Malaysia, Johor Bahru (2017). Her research interest includes activity recognition studies, signal processing, and machine learning. She is currently working on her PhD in Biomedical Engineering at Universiti Teknologi Malaysia, Johor Bahru and her current research is about activity recognition study in sport using inertial sensor which collaborated with Sport Innovation and Technology Centre (SITC) and Institute of Human Centered Engineering (iHumEn) from UTM JB.
Norazman Shahar received his Bachelor of Engineering (Bio-Medical) from Universiti Teknologi Malaysia, Johor Bahru (2017). His research interest includes accelerometer sensor, activity recognition and signal processing. Currently, he is working on his Doctor of Philosophy (PhD) in Bio-Medical Engineering Universiti Teknologi Malaysia, collaborated with Sport Innovation and Technology Centre (SITC) and Institute of Human Centered Engineering (iHumEn). His current research is concentrating on the inertial sensor, activity recognition, signal processing and machine learning.
Muhammad Amir bin As’ari holds a PHD in Biomedical Engineering from Universiti Teknologi Malaysia. His PhD’s work is in the field of assistive technology, computer vision, and image processing and his work focused on developing a novel 3D shape descriptor for recognizing the activities of daily living (ADLs) based on Kinect-like depth image. He pursued his master and bachelor degree at Faculty of Electrical Engineering in UTM. Currently, he is working on development of signal and image processing with intelligent intervention and human action recognition based on wearable sensor and context-aware modality for assistive and sport technology. Currently, he is a senior lecturer in Universiti Teknologi Malaysia specialized in Biomedical Engineering area. He is also member of Biomedical Instrumentation & Electronics (BMiE) Research Group and Associate fellow researcher of Sport Innovation & Technology Centre (SITC).
