ELECTROMYOGRAPHIC AND ACCELERATION SIGNALS-BASED GAIT PHASE ANALYSIS FOR MULTIPLE TERRAIN CLASSIFICATION USING DEEP LEARNING

Available online athttp://www.iaeme.com/IJARET/issues.asp?JType=IJARET&VType=11&IType=6 ISSN Print: 0976-6480 and ISSN Online: 0976-6499

DOI: 10.34218/IJARET.11.6.2020.060

© IAEME Publication Scopus Indexed

ELECTROMYOGRAPHIC AND

ACCELERATION SIGNALS-BASED GAIT

PHASE ANALYSIS FOR MULTIPLE TERRAIN

CLASSIFICATION USING DEEP LEARNING

School of Biomedical Engineering, Indian Institute of Technology,

(Banaras Hindu University) Varanasi, Uttar Pradesh, India ABSTRACT

The neuromuscular activity emanated from the human leg muscles is of great importance for understanding the human locomotion. This information can be extracted by using surface EMG signals. This paper presents the concept of gait analysis by using a pair of hybrid- EMG and acceleration sensors attached to the Tibialis Anterior and Gastrocnemius muscles of both legs. The response of the EMG signal envelope is analyzed for five locomotion terrains, namely- level ground, ramp ascent, ramp descent, stair ascent, and stair descent. A heuristic algorithm using an acceleration signal is employed to identify heel-strike, toe-off, and mid-swing events. Results show that we identified all these events correctly for five healthy subjects walking on all five terrains. Also, deep learning-based classification is performed for the above five terrains by using both stance and swing phase data separately as a training dataset. We obtained more than 99% accuracy for both stance and swing phase training datasets; and for unknown dataset, classification accuracy is found to be 97.55% from stance phase data and 91.22% from swing phase data.

Key words: Accelerometer, Deep learning, Electromyography, Gait analysis, Gait phase detection.

Cite this Article: Sachin Negi, Pranshu CBS Negi, Shiru Sharma, Neeraj Sharma, Electromyographic and Acceleration Signals-Based Gait Phase Analysis for Multiple Terrain Classification Using Deep Learning, International Journal of Advanced Research in Engineering and Technology, 11(6), 2020, pp. 656-665.

http://www.iaeme.com/IJARET/issues.asp?JType=IJARET&VType=11&IType=6

1. INTRODUCTION

The investigation of human gait analysis is elementary before structuring powered lower limb prostheses. Instrumentation setup required to study human locomotion can be divided into two major categories- Vision (camera) based and wearable sensor-based. Further, the

camera-based setup is limited to the laboratory environments; however, the wearable sensor-camera-based structure can be used in open environments. Wearable sensors include electromyography (EMG), gyroscopes, accelerometers, magnetometers, goniometers, potentiometers, force sensors, etc. [1].

Winter DA [2] and Whittle MW [3] presented the anatomy, muscle physiology, and biomechanics used for understanding the human gait analysis. Ankle joint receives the whole weight of the body; hence the biomechanical analysis of the ankle during gait on different terrain plays a significant role while designing ankle-foot prosthesis. Ankle flexor muscles assume a substantial job in the human walk. Ankle plantar-flexors are active during the stance phase, and it acts to limit the forward pivot of the tibia on the talus bone. It also gives the strength to lower leg, add to knee stability, and conserve energy by limiting the vertical oscillation of the entire body center of mass. Ankle dorsi-flexors, antagonists to plantar-flexors, are active mostly during the swing phase. The main role of dorsi-flexors is to prevent foot slap on the ground in the initial stance phase, to allow the forefoot to clear the land in the initial swing phase, and to hold the ankle in position during initial contact [4].

EMG signal gives an immediate association with the motion intention of the subject [5]. Patla [6] has observed the EMG signals from tibialis anterior (TA), soleus, medial gastrocnemius (MGAS), vastus lateralis, rectus femoris, biceps femoris, and erector spinae muscles of ipsilateral limb for treadmill walk. The approach to acquire the EMG envelope incorporates preprocessing, signal- filtering, rectification, smoothing, and normalization. The recommendation from SENIAM and ISEK suggests the utilization of 10–500 Hz bandpass filters to remove aliasing effect while using a sampling frequency of 1 kHz [7]. Prakash, A. et al. [8] used EMG sensors on amputees for the prosthetic hand control. They used envelope detection for implementing a proportional control scheme that enabled the grasping force of the hand fingers proportional to the EMG signal strength. Nazmi N et al. [9] acquired EMG signals from TA and MGAS muscles to classify stance and swing phases using neural networks (NN). Morbidoni, C. et al. [10] used deep learning methods for the classification of stance and swing phases using EMG signals from TA, gastrocnemius lateralis, hamstrings, and vastus lateralis muscles.

Another wearable sensor that is mostly used by researchers is an inertial measurement unit (IMU). IMU measures angular rate, linear acceleration, and sometimes the orientation of the body, using a combination of gyroscopes, accelerometers, and magnetometers. Gouwanda, D. et al. [11, 12] have presented heuristic and zero-crossing method to detect heel-strike (HS) and toe-off (TO) events using a gyroscopic signal. They performed gait analysis on the ground level and treadmill for the following activities: normal walking, walking with ankle and knee braces. Wang, N. et al. [13] have detected five- human walking patterns by acquiring the triaxial accelerometer data from the waist position (above the iliac spine). They used wavelet packet decomposition for time-frequency domain analysis of the signal and applied multi-layer perceptron neural networks (MLPNN) classifier to achieve 92.05% overall classification accuracy.

In this study, we acquired EMG and accelerometers signals from TA and MGAS muscles using a hybrid wireless sensor for gait analysis. TA muscles are dorsi-flexors, whereas MGAS muscles work as plantar-flexors. The present work is divided into two parts, the objective of the first part is to use acceleration signal for the identification of gait events and gait phase parameters. The second objective is the classification of different terrains by using hybrid- EMG and accelerometer signals.

2. MATERIALS AND METHODOLOGY

This study is approved by the ethical committee of the Institute of Medical Sciences, BHU, Varanasi. We used the Trigno wireless EMG sensor setup that comprises four sensor modules. Each sensor module consists of an EMG sensing unit as well as a 3-DOF (±1.5g range) accelerometer. The sensors and their placement are shown in Figure 1. EMG and acceleration signals were acquired at 1926 and 148 samples/sec, respectively. Both signals were resampled at 1000 samples/sec in Delsys- EMGworks software. Two sensors are affixed on the TA and MGAS muscles site of each leg. The detailed concept regarding the signal acquisition using these sensors are explained in [14], [15].

Figure 1 Trigno Wireless EMG Sensor

To study the muscles pattern, an envelope is obtained from the EMG signal by taking the absolute of the signal and low pass filtering in the python programming language [16]. Figure 2 shows the EMG envelopes acquired from TA and MGAS muscles for five terrains. Also, while walking on different terrains, we understood that the acceleration signal along the vertical axis plays an important role in identifying gait phases and events. The maximum peaks in the acceleration signal represent the mid-swing (MS) phase of a gait cycle, whereas the two local minima following the MS are HS and TO, respectively. Corresponding to these events, the muscle activation pattern can be analyzed using EMG signals from TA and MGAS muscles.

Figure 2(a) shows the EMG signal pattern acquired from both muscles for level ground (LG) walk. Here, the TA muscle activity begins at TO and continues throughout the swing phase; however, the amplitude is decreased during MS. TA muscle is active at HS, and it continues to be active until the foot is entirely in contact with the ground. The MGAS muscle begins its activity during the initial stance and ends its activity at a terminal stance. From Figure 2(d), for stair ascent (SA) walk, we obtained that TA muscle is active during the swing phase; thus, it provides suitable foot clearance from the ground. Also, the MGAS muscle is active for most of the stance phase. For stair descent (SD) walk, as shown in Figure 2(e), the MGAS muscle is active at the end of the swing phase for making foot landing. The TA muscle is active throughout the stance phase, and it is most active during the initial swing phase. All the above responses are in agreement with other literature [17]. For ramp ascent (RA), MGAS muscle is more active during TO, whereas TA muscle is more dominant for ramp descent (RD) walking. The functional roles of different muscles, including TA and MGAS, is explained in detail for sloped terrains in [18]. Also, for a change in the cadence of walking, the EMG patterns change mainly in amplitude; however, the shape of the ensemble remained nearly the same [19]. This study motivated us to use EMG and acceleration- hybrid sensor data for the classification of different locomotion terrains. The length of the LG path,

the slope of the ramp terrain, and the height of stairs used in this study are approximately 4 meters, 3 degrees, and 15 centimeters, respectively.

Figure 2 EMG and acceleration signals pattern for all terrains

3. RESULTS AND DISCUSSION

The first objective is to develop an algorithm to detect MS, HS, and TO events, and also to separate stance and swing phase data from each gait cycles. Figure 2 also illustrates the x-axis acceleration curve acquired from the human shank for the walk on different terrains. For LG, RA, and RD terrains we have chosen x- axis acceleration signal from MGAS muscle, whereas for SA and SD we selected TA muscle. Here the peaks higher than -0.25 correspond to MS, and troughs (or local minima) less than threshold T1 and T2 correspond to HS and TO, respectively. Figure 3 shows the flow chart of the gait phase detection algorithm for a specific subject.

Figure 3 Flow chart for the gait phase detection using accelerometer

Here, the first most step is to calculate the points in the curve where slope sign changes and these are denoted as potential points. Corresponding to these points, the time and magnitude of the signal are stored. Within the loop in python code, the algorithm continues to search for the MS points, followed by TO and HS points by utilizing a set of heuristics. MS points are those potential points that are greater than a predefined threshold value, -0.25 m/sec2. By assuming the gait cycles greater than 1 sec, the algorithm does not allow two MS points within a time interval of 1 sec. After the detection of one MS point, the algorithm search for two local minima as HS and TO points, respectively.

As mentioned in the algorithm, it is based on the value of the threshold that is different for different subjects; therefore, this study deals with subject-specific gait analysis. Once, HS and TO are obtained, the gait parameter can be computed by using the following formula:

𝑡𝑔𝑎𝑖𝑡[𝑖] = 𝑡[ℎ𝑠_𝑝𝑡𝑡[𝑖 + 1]] − 𝑡[ℎ𝑠_𝑝𝑡𝑡[𝑖]] (1)

𝑡_{𝑠𝑤𝑖𝑛𝑔}[𝑖] = 𝑡[ℎ𝑠_𝑝𝑡𝑡[𝑖 + 1]] − 𝑡[𝑡𝑜_𝑝𝑡𝑡[𝑖]] (2)

where, hs_ptt and to_ptt represent HS and TO points respectively, and i is one less than the number of hs_ptt detected in a time frame; 𝑡𝑔𝑎𝑖𝑡 represents one gait cycle, i.e., the duration between two successive HS events for the same leg; 𝑡_{𝑠𝑡𝑎𝑛𝑐𝑒} is the duration for which foot is on the ground, whereas 𝑡_{𝑠𝑤𝑖𝑛𝑔} indicates the duration during that foot has no contact with the ground.

In this section, we first obtained the gait phase parameters using the heuristic algorithm based on accelerometer data. Figure 4 shows the MS, HS, and TO events identified using the above algorithm for all five terrains. In this paper, the results are shown for subject-1 only, whereas the results for other subjects are mentioned in supplementary materials. All the events are detected correctly for all reading trials. After obtaining HS and TO points, the gait cycle, stance phase, and swing phases are calculated for both legs using Equations 1 to 3. Table 1 and 2 shows the gait parameters obtained for both legs. For plain ground walking, the swing and stance phase duration obtained are approximately 42% and 58% respectively, that is close to normal values of 40% and 60%. The swing phase increases for the slope and stair terrains.

Table 1 Gait parameters for right leg

Terrain t_gait (Sec) t_swing (Sec) t_stance (Sec)

LG 1.158 (±0.034) 0.486 (±0.008) 0.672 (±0.031) RA 1.176 (±0.031) 0.483 (±0.009) 0.693 (±0.025) RD 1.121 (±0.024) 0.478 (±0.009) 0.643 (±0.024) SA 1.260 (±0.046) 0.619 (±0.021) 0.641 (±0.035) SD 1.149 (±0.045) 0.616 (±0.027) 0.534 (±0.031)

Table 2 Gait parameters for left leg

Terrain t_gait (Sec) t_swing (Sec) t_stance (Sec)

LG 1.151 (±0.023) 0.497 (±0.007) 0.654 (±0.020) RA 1.185 (±0.036) 0.492 (±0.022) 0.692 (±0.035) RD 1.121 (±0.025) 0.485 (±0.010) 0.636 (±0.021) SA 1.251 (±0.040) 0.632 (±0.025) 0.619 (±0.031) SD 1.148 (±0.045) 0.634 (±0.027) 0.515 (±0.030)

3.2. Deep Learning for Terrain Classification

Another objective of the present paper is to classify five terrains, namely: LG, RA, RD, SA, and SD. In the past, many researchers have applied classifiers in different ways. Pati S. et al. [20] have used linear discriminant analysis (LDA), NN and Naive-Bayes (NB) classifiers for the classification of different locomotion modes: slow walk, normal walk, fast walk, and running using EMG signals. Gupta R. and Agarwal R. [21] have found that support vector machine (SVM), LDA, and NN are the most suitable classifiers for LG, SA, and SD locomotion identification using EMG signal from fibularis longus and biceps femoris muscles. Huang H. et al. [22] have used the SVM classifier for EMG and ground reaction force sensor data to classify five locomotion terrain. All conventional classifiers require feature extraction and feature selection steps before classification, which generates complexity in terms of finding the best feature sets. However, a Deep neural network (DNN) is an advanced network that does not require any feature extraction.

After HS and TO points are detected, we separated the EMG and 3-axes acceleration signals from both muscles into the stance and swing phase sample points. We carried out a bulk of data handling tasks using the NumPy library. For plotting the classification matrix, we used the Matplotlib library. For deep learning, we used Google’s Tensor Flow library. We tested our neural network with varying hyperparameters and model architectures. For optimizer, we used adam optimizer, and for loss function, we used categorical cross-entropy loss as our task is multi-label classification. For each subject, 12 trials data have been recorded for each terrain. Out of these 12 trials, three trials are put aside as an unknown dataset, whereas the remaining nine trials data for each terrain are shuffled and partitioned into 80% training, 10% validation, and 10% testing data. The results of the classification were recorded in a confusion matrix. After the model was trained on a known dataset, we evaluated the model on a previously unseen dataset and recorded results into another classification matrix. Figure 5 and 6 shows the confusion matrix for the prediction of terrains for both known and unknown data using stance and swing phase, respectively.

(a) (b)

Figure 5 Confusion matrix using stance phase samples for prediction with: (a) known data, (b) unknown data

(a) (b)

Figure 6 Confusion matrix using swing phase samples for prediction with: (a) known data (b) unknown data

Table 3 to 6 shows the classification accuracy using DNN classifier for different model structures.

Table 3 DNN classifier results for known dataset using stance phase data Model Architecture Accuracy Precision Recall F1 Score

256/128 99.06 0.9907 0.9906 0.9906 512/256/128 100.00 1.0000 1.0000 1.0000 1024/512/256/128 99.84 0.9984 0.9984 0.9984 1024/512/256/128/64 99.94 0.9994 0.9994 0.9994 1024/512/256/128/64/32 99.84 0.9986 0.9984 0.9985 32/128/512/512/128/32 99.91 0.9991 0.9991 0.9991

Table 4 DNN classifier results for unknown dataset using stance phase data

Model Architecture Accuracy Precision Recall F1 Score

256/128 96.72 ± (1.02) 0.9674 ± (0.0101) 0.9671 ± (0.0101) 0.9673 ± (0.0101) 512/256/128 97.55 ± (0.67) 0.9757 ± (0.0064) 0.9753 ± (0.0070) 0.9755 ± (0.0067) 1024/512/256/128 96.72 ± (2.39) 0.9683 ± (0.0235) 0.9668 ± (0.0239) 0.9675 ± (0.0237) 1024/512/256/128/64 96.58 ± (0.66) 0.9659 ± (0.0065) 0.9657 ± (0.0069) 0.9658 ± (0.0067) 1024/512/256/128/64/32 96.10 ± (1.62) 0.9613 ± (0.0163) 0.9608 ± (0.0163) 0.9610 ± (0.0163) 32/128/512/512/128/32 96.72 ± (1.02) 0.9674 ± (0.0101) 0.9671 ± (0.0101) 0.9673 ± (0.0101)

Table 5 DNN classifier results for known dataset using swing phase data Model Architecture Accuracy Precision Recall F1 Score

256/128 99.37 0.9937 0.9935 0.9936 512/256/128 98.03 0.9809 0.9802 0.9805 1024/512/256/128 98.89 0.9895 0.9888 0.9892 1024/512/256/128/64 99.17 0.9917 0.9915 0.9916 1024/512/256/128/64/32 100.00 1.0000 1.0000 1.0000 32/128/512/512/128/32 99.89 0.9989 0.9989 0.9989

Table 6 DNN classifier results for unknown dataset using swing phase data

Model Architecture Accuracy Precision Recall F1 Score

256/128 91.22 ± (92.45) 0.9128 ± (0.9252) 0.9118 ± (0.9242) 0.9123 ± (0.9247) 512/256/128 88.30 ± (89.46) 0.8839 ± (0.8954) 0.8818 ± (0.8939) 0.8829 ± (0.8947) 1024/512/256/128 89.74 ± (90.09) 0.8981 ± (0.9016) 0.8972 ± (0.9004) 0.8976 ± (0.9010) 1024/512/256/128/64 89.93 ± (90.49) 0.9005 ± (0.9058) 0.8986 ± (0.9045) 0.8996 ± (0.9051) 1024/512/256/128/64/32 91.18 ± (90.99) 0.9120 ± (0.9104) 0.9118 ± (0.9097) 0.9119 ± (0.9100) 32/128/512/512/128/32 89.50 ± (90.84) 0.8963 ± (0.9095) 0.8943 ± (0.9079) 0.8953 ± (0.9087)

4. CONCLUSIONS

In this work, we performed gait analysis using wireless wearable sensors. We first obtained gait phase parameters using an accelerometer signal for five locomotion modes-LG, RA, RD, SA, and SD. Using HS and TO points, we separated the EMG and 3-axes accelerometer signals into stance and swing phase data. We then applied the DNN classifier separately on stance and swing phase data to classify above five locomotion terrains. Different model architectures are used to perform classification. We obtained 97.55% and 91.22% accuracy for predicting the unknown terrains using stance and swing phase data, respectively. This study could be used in future work to generate suitable signals for the control of lower limb prostheses on different terrains.

REFERENCES

[1] Tao, Weijun, et al. (2012) "Gait analysis using wearable sensors." Sensors 12.2: 2255-2283.

[2] Winter, David A. (2009) Biomechanics and motor control of human movement. John Wiley & Sons.

[4] Di Nardo, Francesco, Giacomo Ghetti, and Sandro Fioretti. (2013) "Assessment of the activation modalities of gastrocnemius lateralis and tibialis anterior during gait: a statistical analysis." Journal of Electromyography and Kinesiology 23.6: 1428-1433. [5] Ziegier, Jakob, Hubert Gattringer, and Andreas Mueller. (2018) "Classification of gait

phases based on bilateral emg data using support vector machines." 2018 7th IEEE International Conference on Biomedical Robotics and Biomechatronics (Biorob). IEEE. [6] Patla, Aftab E. (1985) "Some characteristics of EMG patterns during locomotion:

implications for the locomotor control process." Journal of motor behavior 17.4: 443-461. [7] Singh, Rajat Emanuel, et al. (2019) "A Review of EMG Techniques for Detection of Gait

Disorders." Machine Learning in Medicine and Biology. IntechOpen.

[8] Prakash, Alok, Shiru Sharma, and Neeraj Sharma. (2019) "A compact-sized surface EMG sensor for myoelectric hand prosthesis." Biomedical Engineering Letters 9.4: 467-479. [9] Nazmi, Nurhazimah, et al. (2019) "Walking gait event detection based on

electromyography signals using artificial neural network." Biomedical Signal Processing and Control 47: 334-343.

[10] Morbidoni, Christian, et al. (2019) "A deep learning approach to EMG-based classification of gait phases during level ground walking." Electronics 8.8: 894.

[11] Gouwanda, Darwin, and Alpha Agape Gopalai. (2015) "A robust real-time gait event detection using wireless gyroscope and its application on normal and altered gaits."

Medical engineering & physics 37.2: 219-225.

[12] Gouwanda, Darwin, Alpha A. Gopalai, and Boon How Khoo. (2016) "A low cost alternative to monitor human gait temporal parameters–wearable wireless gyroscope."

IEEE Sensors Journal 16.24: 9029-9035.

[13] Wang, Ning, et al. (2007) "Accelerometry based classification of walking patterns using time-frequency analysis." 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE.

[14] De Luca, Carlo J. (2002) "Surface electromyography: Detection and recording." DelSys Incorporated 10.2: 1-10.

[15] De Luca, Gianluca. (2003) "Fundamental concepts in EMG signal acquisition." 12-24. [16] Team, Python Core. (2015) "Python: A dynamic, open source programming

language." Python Software Foundation 78

[17] Benedetti, Maria Grazia, et al. (2012) "Muscle activation patterns during level walking and stair ambulation." Applications of EMG in clinical and sports medicine 8.2: 117-130. [18] Pickle, Nathaniel T., et al. (2016) "The functional roles of muscles during sloped

walking." Journal of biomechanics 49.14: 3244-3251.

[19] Yang, Jaynie F., and David A. Winter. (1985) "Surface EMG profiles during different walking cadences in humans." Electroencephalography and clinical Neurophysiology

60.6: 485-491.

[20] Pati, Sarthak, Deepak Joshi, and Ashutosh Mishra. (2010) "Locomotion classification using EMG signal." 2010 International Conference on Information and Emerging Technologies. IEEE.

[21] Gupta, Rohit, and Ravinder Agarwal. (2018) "Continuous human locomotion identification for lower limb prosthesis control." CSI Transactions on ICT 6.1: 17-31. [22] Huang, He, et al. (2011) "Continuous locomotion-mode identification for prosthetic legs

based on neuromuscular–mechanical fusion." IEEE Transactions on Biomedical Engineering 58.10: 2867-2875.