1
Development of an economic wireless human motion analysis
device for quantitative assessment of human body joint
Corresponding author
Ong Zhi Chao (Z.C. ONG)Mechanical Engineering Department, Engineering Faculty, University of Malaya Mailing address
Mechanical Engineering Department, Engineering Faculty, University of Malaya, 50603 Kuala Lumpur, MALAYSIA.
Phone no. (+603) 79676815; (+6012) 4192911 Fax no. (+603) 79675317 Email address [email protected]; [email protected]
Co-authors
Seet Yuan Chen (Y.C. Seet)
Mechanical Engineering Department, Faculty of Engineering, University of Malaya, 50603 Kuala Lumpur, MALAYSIA
Khoo Shin Yee (S.Y. KHOO)
Mechanical Engineering Department, Faculty of Engineering, University of Malaya, 50603 Kuala Lumpur, MALAYSIA
Siamak Noroozi (S. NOROOZI)
School of Design, Engineering and Computing, Bournemouth University (Talbot Campus), Poole, Dorset, BH12 5BB, UK
2
Development of an economic wireless human motion analysis
device for quantitative assessment of human body joint
Z.C. Ong a *, Y.C. Seet a, S.Y. Khoo a, S. Noroozi b,
a
Mechanical Engineering Department, Faculty of Engineering, University of Malaya, 50603 Kuala Lumpur, MALAYSIA
b
School of Design, Engineering and Computing, Bournemouth University (Talbot Campus), Poole, Dorset, BH12 5BB, UK
ABSTRACT
In recent years, the study of human body dynamics has been attracting a significant amount of attention. Currently there are many camera or active sensor based motion analysis systems available on the market. They have been extensively adopted and used by the film and animation or entertainment industries such as film and video game producers. More recently their potential in studying human dynamics / motion for medical purposes has been realised to the extent that they are now used to study full body human biomechanics in the form of gait analysis systems. Most orthopaedic surgeries are usually about joint repair or implants. According health line, revision surgery is usually due to infection, continued pain, joint stiffness, wear, instability, loosening. Apart from infection, the rest can be linked to the operation itself. Currently, surgical planning and placing implants is performed in a subjective manner, relying on the surgeon’s experience and instinct, current systems to help the surgeon to place implant are also bulky, expensive, slow and not user friendly. The aim of this project is to develop an economic and portable motion assessment system which involves a wireless inertial measurement unit (IMU) dedicated to study and assess body joints. Through the data collected from the IMU, the system is capable real time measurement of relative position and orientation of the human joint. Several tests were conducted to validate the data extracted from gyroscope and accelerometer of the IMU. The joint motion results analysed using the device was compared with the results analysed using commercial video motion analysis software and it shows good correlation. It is found that the gyroscope of the IMU under DMP sensor fusion algorithm and calibration capability is able to give the angular velocity with less than 5% error. This has led to a more accurate orientation data which gives 7% error in average bending angle.
KEYWORDS: Bio-mechanics; Gait analysis; Human dynamics; Kinematics; Orthopaedic
3 INTRODUCTION
Motion analysis of human body parts is an important area in the medical field, where motion analysis is often required for medical diagnosis and physiotherapy. According to National Joint Registry (NJR) 10th annual report as at 31 March 2013, a total of 1,456,756 hip, knee, ankle, elbow, and shoulder replacement procedures had been reported to the NJR. There were 196,403 procedures submitted in 2012/13. Total of 2,225 shoulder replacement procedures were recorded in 2012 including 1,968 primaries and 257 revisions. Osteoarthritis was the primary diagnosis in 61% of cases. 24% of patients were reported to have cuff tear arthropathy. Of the 257 revision procedures, the indications for surgery were variable: 14% out of total needed revisions are aseptic loosening, 30% were conversion of hemi to total and 25% were revision for cuff insufficiency. Knee replacement surgery (arthroplasty) involves replacing a damaged, worn or diseased knee with an artificial joint. In 2012, 90,842 knee replacement procedures were entered into the NJR. Total of 6,009 knee revision procedures were reported in 2012 representing an increase of 17% compared to 2011. Indications for revision surgery were recorded as aseptic loosening in 32% of cases [1]. According to a meta-analysis of worldwide joint registry databases, published in the Journal of Bone & Joint Surgery in 2011, the long term revision rate is 6% after 5 years and 12% after 10 years [2]. Although most of the joint revision can be linked to complications associated with infection, circulation, rejection, etc., some are linked directly to implants mechanical properties, poor locating, poor kinematics, reference able or repeatable positioning, poor materials, design, properties, etc. Problem arises when in many cases identical procedure for different individuals can result in dissimilar outcomes. This indicates that there is existence of other relevant parameters that currently a surgeon is either unaware of or has no means of determining, assessing or measuring before the operation [3].
The traditional approach used to analyse gait parameters during clinical applications are subjective. The specialist usually carried out an observation or survey to evaluate the quality of patients’ gait. These subjective observational measurements are particularly lacking of accuracy and precision, which can have adverse effect on the diagnosis, follow-up and treatment of the pathologies. Progress in new technologies has given rise to creation of new devices and techniques which allow a more objective evaluation of various gait parameters, resulting in more efficient and reliable measurement hence providing specialists with larger amount of reliable and quantitative information about the patients’ gaits. This in turn reduces the error margin caused by traditional and subjective techniques.
In practice, motion analysis of human body parts is dominated by visual based motion analysis systems, where one or multiple cameras are used to track the motion of the subject attached with reflective markers. These images are then processed and analysed using specialised software to derive kinematic data [4-7]. Visual based motion analysis systems have a widespread use and they have remarkable performances on measuring displacements and paths of motion [8]. However, they are weak in measuring high frequency and low amplitude motion because of the lack of resolution. Therefore, a more precise kinematics data is necessary for better assessment of the human joint before and after surgery. High resolution kinematics data is also needed for better assessment on the performance of joint after implant surgery. Besides the lack of resolution in visual based motion analysis system, this type of system usually requires high cost for a certain gait laboratory test. Furthermore, visual based motion analysis systems are usually not portable and the test is limited to a certain area
4 covered by the range of the camera. Thus, the application of the visual based motion analysis system is limited, as it cannot be done outside of the instrumented environment.
Despite the widespread use of visual based motion analysis systems such as the Vicon, Qualysis, Codamotion, Motek, Biometrics, ETB Gaitsmart motion analysis systems, body mounted sensors are sometimes used as an alternative to collect kinematics data from human body parts for the purpose of motion analysis [9-12]. This method has the advantage of identifying human motion in a wide variety of environments without taking into account the camera distance, capture range, resolutions, frame rate and etc. [9]. Body mounted sensors are also suitable to measure high frequency motion with low amplitude, which is hard for visual camera to capture [8]. Some researchers have started to use micro-electro-mechanical system (MEMS) accelerometer, gyroscope and combined unit of them or IMU as an alternative for visual based human motion analysis system [13-19]. Inertial measurement unit, when used together with analysis techniques could be useful in providing an economic and portable solution in locating the implants. It could be used to provide information such as original anatomical differences, magnitude and orientation of body segments, initial muscle tension and etc. The use of this sensor in biomedical applications to study body movements has been frequently discussed. However, the major challenge in this method is to translate data which are in the form of acceleration and angular velocity into three dimensional data such as joint angles, position in 3D space and etc. [9].
Theoretically, performing single integration on angular velocity data and double integration on acceleration data would enable us to obtain the orientation and position of an inertial sensor mounted to the body parts. Tong and Grant have calculated lower limb body segments orientations using integration of measured angular velocity from the gyroscopes attached to the thigh and shank [5]. However, they also reported that noise in the gyroscopes data has caused integration errors. This could be caused by the signal noise in the raw data of acceleration and angular velocity. When integration is performed, the error will accumulate over time and this will cause significant error in the computed results. This is often referred as drift. In order to reduce the signal noise, cyclical properties of human gait have been used for developing signal filters. Liu et al. have developed a gait phase detection algorithm to correct gyro sensor drift by using inclination of lower body segments during mid-stance of gait [18]. Sabatini has also proposed calculation of body segment orientations from angular velocity data of a body mounted gyroscope using quaternions [20]. This proposed method also used the cyclic properties of gait to compensate for the drift in the results. Favre et al. [17] have also used acceleration data for compensation of the drift, but the method could only be used where the only force components measured is the gravitational acceleration. A research has also been done to estimate the acceleration signal output at the centre of rotation of the knee using inertial sensors attached to the thigh and shank separately. This method does not make cyclical properties of gait an assumption in the calculation and is shown useful in obtaining accurate knee joint measurements. This method is then further used in determination of positions of hip, knee and ankle joint centres. Despite its accuracy in measuring flexion-extension movements, internal-external rotation of the joint is not taken into consideration. This problem is not addressed until a sensor system which incorporates magnetometer is developed. Picerno et al. have reported high reliability and accuracy in using the sensors system for gait analysis with specific anatomical calibration [21]. However, Brodie et al. have reported that accuracy exists when the measurements using inertial and magnetic sensing sensors are compared with a camera based motion analysis system [15]. The errors could only
5 be eliminated by recalibrating the sensors regularly and conducting the experiment in a homogeneous magnetic environment, which is not realistic in everyday situations [9]. Tadano et al. have reported a substantially reduce in error caused by noise by implementing sensor attachment error calibration and signal filtering on quaternion calculations. In their experiment, quaternions were implemented to better represent the joint motion while avoiding singularities. Body mounted sensors suffer from many factors such as attachment errors, calibration maintenance, signal noise, filtering errors and integration drift. It is reported that even commercially available motion tracking systems have significant errors [15]. Therefore, it is important to choose a method which could minimise the errors in motion analysis of human body parts.
The use of IMU in medical application provides a high resolution solution to motion analysis of human body parts, which enables better performance in surgery related to human body joints. The use of IMU in the motion analysis of human body parts enables the understanding of the shoulder biomechanics which has resulted in surgical modifications of the implant and eventually reduction of the rate of complications. Motion analysis of knee using IMU could be useful. It could be used to compare the recovery rate against national average to determine the effectiveness of new knee prosthesis. This will involve studying the kinetics and kinematics through quantifying the change in gait and joint forces over time. When possible the kinematics of gait before and after the surgery will be studied to measure the relative contribution of the intervention. This method could also be used to evaluate new knee prosthesis developed. The new implant could be assessed if it meets the functional demands of patients in the aims to improve the life, range of motion, stability, wear resistance, post operation complication such as joint failure, stability, detachment, tension settings and excess motion [19, 21-24].
This eventually leads to the objective of this research, which is to develop a cost effective inertial measurement unit based human motion analysis device. This device is portable to qualitatively and quantitatively assess human body joints with 3D graphical tracking feature and its kinematics data, such as acceleration, angular velocity, orientation (angle), position and bending angle. The work presented here is a pilot study on an approach of extracting the 3-dimensional orientation data using quaternions from the Digital Motion Processor (DMP) of IMU. The DMP acquires data from accelerometers and gyroscopes and provides an integrated motion fusion output which is capable of computing quaternion data from sensor readings and performing device calibration. The DMP generated sensor fusion orientation data with the ability of device calibration is hypothesized to have relatively less errors or drifts compared with orientation data reported in previous studies.
In this study, a wireless human motion analysis device is first developed using Arduino, Xbee Pro IMU consisting of 3-axis accelerometers and 3-axis gyroscopes. A dedicated software is then designed using Processing language to communicate wirelessly with the device and extract raw data, (i.e. acceleration and angular velocity) from the IMU device. The orientation and the position of the device are then computed from the raw data through algorithms developed. 3D visualisation is programmed in the software to visualise the orientation and position of the device. Subsequently, the data from multiple devices are integrated in the software to show the relative motion of the sensors. Testing is then done to test the accuracy of the device in measuring human joint motion.
6 BACKGROUND THEORIES
Quaternions
The quaternions are a 4-dimensional vector space together with a vector multiplication operation that forms a non-commutative associative algebra.
Starting with an axis-angle representation (𝑛�⃗ , 𝜗), where the principal axis of rotation, 𝑛�⃗ is a unit vector and principal angle of rotation, 𝜗 = 𝜔𝜔 is a counter-clockwise rotation about 𝑛�⃗ axis under a constant rate with angular velocity of 𝜔 for a specific time of 𝜔
𝑒0 = 𝑐𝑐𝑐 �𝜗2� 𝑒⃗ = �𝑒𝑒12 𝑒3 � = 𝑐𝑠𝑛 �𝜗 2� 𝑛�⃗ = 𝑐𝑠𝑛 � 𝜗 2� � 𝑛1 𝑛2 𝑛3 �
Then the corresponding quaternion in Hamiltonian form is: 𝑞 = 𝑒0+ 𝑒1𝚤̂ + 𝑒2𝚥̂ + 𝑒3𝑘�
where 𝑒0 is known as the scalar term and 𝑒⃗ with subscripts 1, 2, 3 denoted as vector term of quaternion. The standard basis vectors obey the following rule: 𝚤̂2 = 𝚥̂2 = 𝑘�2 = 𝚤̂𝚥̂𝑘� = −1. In the case of an object is rotating at a constant rate of 𝜔 about the axis 3, 𝜗 = 𝜔𝜔 , (𝑛1, 𝑛2, 𝑛3) = (0,0,1), the quaternion is derived as
𝑞 = 𝑐𝑐𝑐 �𝜔𝜔2 � + 𝑐𝑠𝑛 �𝜔𝜔2 � 𝑘�
In sensor fusion algorithm, gyro bias drift correction component is included. Both acceleration, 𝑎 and angular velocity, 𝜔 are used to compute the quaternion. Difference of quaternion estimate of acceleration, 𝑎𝑞 and acceleration data is computed and known as gradient. It is then used to compute the gyroscope biases. The angular velocity from gyroscope is corrected, 𝜔𝑐 by subtracting the biases and used to compute the quaternion derivative, 𝑞̇ defined as:
𝑞̇ = � 𝑞̇0 𝑞̇1 𝑞̇2 𝑞̇3 � =12 � 𝑒0 𝑒3 −𝑒2 𝑒1 𝑒1 𝑒2 𝑒3 −𝑒0 𝑒2 −𝑒1 𝑒0 𝑒3 𝑒3 −𝑒0 −𝑒1 −𝑒2 � � 𝜔0𝑐 𝜔1𝑐 𝜔2𝑐 𝜔3𝑐 �
The sensor fusion quaternion data, 𝑞, (i.e. 𝑞0, 𝑞1, 𝑞2 and 𝑞3) is then defined as: 𝑞 = �𝑞̇ − �(𝑎𝑞− 𝑎) × 𝛽��𝑑𝜔
where 𝛽=0.04 is the filter gain.
Euler angles
Euler angles are a specification of a rotation (or an orientation) obtained by applying three consecutive principal rotations.
To convert to the Euler angle z-x-y rotation convention, Tait-Bryan angles, (i.e. roll Rz(𝜙), pitch Rx (θ) and yaw Ry(ψ), is computed as follows:
7 𝜙 = 𝜔𝑎𝑛−12((𝑞 1𝑞2+ 𝑞0𝑞3),12 −(𝑞12+ 𝑞32)) 𝜃 = −𝑐𝑠𝑛−1(2(𝑞 2𝑞3− 𝑞0𝑞1)) 𝜓 = 𝜔𝑎𝑛−12((𝑞 1𝑞3+ 𝑞0𝑞2),12 −(𝑞12+ 𝑞22)) where −𝜋 < 𝜙 < 𝜋 , −𝜋 2 < 𝜃 < 𝜋 2 and −𝜋 < 𝜓 < 𝜋. Positions
The positions are calculated based on orientation of IMUs about a fixed frame of reference at point 1 (x0, y0, z0) = (0, 75, 0) mm, as shown in Figure 1. The length of upper part is defined
as l1 and lower part is defined by l2.
The distance of point in x, y and z direction of the length, l is computed as follows: 𝑙𝑦 = 𝑙 𝑐𝑠𝑛(𝜙)
𝑚 = 𝑙 𝑐𝑐𝑐(𝜙) 𝑙𝑥= 𝑚 𝑐𝑐𝑐(𝜓)
𝑙𝑧 = 𝑚 𝑐𝑠𝑛(𝜓)
where 𝑚 is the length in x-z plane
Position of upper part (point 3) with respect to fixed frame of reference is computed as follows:
𝑥1 = 𝑥0+ 𝑙1𝑥
𝑦1 = 𝑦0+ 𝑙1𝑦
𝑧1 = 𝑧0 + 𝑙1𝑧
Position of lower part (point 5) with respect to fixed frame of reference is computed as follows:
𝑥2 = 𝑥1+ 𝑙2𝑥
𝑦2 = 𝑦1+ 𝑙2𝑦
𝑧2 = 𝑧1+ 𝑙2𝑧
8 MATERIALS AND METHODOLOGY
Kinematics Data Extraction and Graphical Visualization
Accelerations and angular velocities
Preliminary test was conducted before the human motion analysis device is fully developed. Program for Arduino to acquire signals from the DMP of the InvenSense MPU 6050 IMU was developed to send the signal to the computer via XBee Pro wireless serial communication. The embedded DMP is located within the IMU and offloads computation of motion processing algorithms from the host processor. The DMP acquires data from accelerometers and gyroscopes and provides an integrated motion fusion output. The resulting data can be read from the DMP’s registers. In addition, the DMP is capable of performing device calibration. When the calibration feature is enabled and if the board is not moving for more than 8 seconds the gyroscope will automatically be calibrated.
In order to display and plot the data received by the XBee, a computer program was developed based on Processing language to read the data using serial communication port. This program is able to receive all of the data, which includes 3-axis acceleration and 3-axis angular velocity simultaneously before plotting them into six graphs. This program aims to give the user a clear view of all the data extracted from each of the accelerometer and gyroscope.
Orientations (Euler angles)
The DMP is a unique hardware feature of the IMU devices which is capable of computing quaternion data from sensor readings. The IMU directly obtains data from auxiliary sensors, allowing the on chip DMP to generate sensor fusion data without intervention from the system applications processor. In this study, a program was developed based on Processing language to show the real-time orientation of the MPU6050 IMU. This program fuses the raw data obtain from accelerometer and gyroscopes through the DMP and represent them in quaternion unit. Euler angle information was then extracted from the quaternion representation to calculate the orientation of the IMU. 3D simulation was done to simulate the yaw, pitch, roll movement of the IMU. The software first gets the orientation values from the XBee Pro. Subsequently, it uses the length of the body part and the orientation values to calculate the relative position of the IMUs.
Positions and bending angle
In order to show the body joint motion, two IMU were used. The relative motions of the IMUs are not calculated directly through double integration of the acceleration data from the IMUs. This is because this displacement data that is solely based on the acceleration value exhibits significant error due to the fact that the raw data of acceleration from the IMU consists of a lot of unfiltered noise and tends to fluctuate for short duration measurement, which make its use impractical in this application. Compared to that, orientation value of the IMU is rather reliable as it is the result of the DMP filter and sensors fusion. In this study, the relative motion of the IMU of the human motion analysis was developed based on the shape of a human body joint. It is assumed that the first IMU is placed on upper part of a body joint which connects a non-moving body joint (Point 1) and a moving body joint (Point 3). It is also assumed that the second IMU is placed on lower part which connects to the moving body joint (Point 5) that is commonly connected with the first body part. By knowing the length of
9 the body part and the location of the IMUs, the relative motion of the IMUs could be calculated using the orientation of the IMUs.
Graphical Representation
With the developed Arduino Uno R3 program, the computer is able to receive the yaw, pitch and roll angles values from both of the IMU. A software was then developed to use the values obtain to simulate the orientation of the IMUs in real time. In the software, cuboid was used to represent the IMU where different sides of the cuboid have different colour so that the user could differentiate the direction. Table 1 shows the colour of the side corresponding to the axis on IMU.
Table 1: Colour of cuboid side corresponding to axis on IMU
Colour of Cuboid Side Axis on IMU
Cyan Purple Positive X Negative X Red Dark Blue Positive Y Negative Y Green Yellow Positive Z Negative Z
The software then represents the IMUs as two cubes in the program and shows the motion of the human body joint using human sticks and points. In this software, the cuboid on the left was used to simulate the orientation of the first IMU while the cuboid on the right was used to simulate the second IMU. The numerical values for the yaw, pitch and rolls are shown in the upper section of the software so that the accurate orientation of the IMUs can be tracked.
The software was improved to show the motion of body joint with the IMUs in 2D before it was furthered improved to show in 3-dimensional space for better representation of the motion of the human body parts and joints in all axes. The software was then developed to represent the motion in isometric view so that the movement in any axis can be viewed clearly. In this software, the orientation of the two IMU was used to calculate the relative motion between them with the distance between them known in advance. The software uses points to simulate joint and line to simulate human body part. There is a fixed joint, one of the IMUs connects the fixed joint with the first moving joint and the other IMU connects the first moving joint with another moving joint. Using this software, the motion of the human body joint was clearly simulated. Besides that, coordinates of the simulated joint were added so that the human motion can be better analysed. In the software, the length of the human body length was first keyed in before the simulation and the software would utilise the value entered to calculate the coordinates of the joint. The value of the coordinates was expressed in millimetre (mm).
10 Validation of Kinematics Data from Human Motion Analysis Device
Three tests were carried out to validate the kinematics data extracted from the human motion analysis device. As the primary measurement data from IMU are the acceleration and angular velocity in all three axes, the ability for the IMU to measure acceleration and angular velocity in each axis accurately was tested. After the acceleration test and the angular velocity test were conducted, the human motion analysis device was tested on human body joint under a controlled experiment to study the accuracy of the human motion analysis device in measuring motion of human joints in terms of the orientations, positions and bending angles.
Acceleration data
To investigate the accuracy of the acceleration measured using the MPU6050 IMU, an experimental setup as shown in Figure 2 which consists of a calibrated vibration test system and a calibrated commercial accelerometer were used. The proposed experiment using calibrated equipment is more controlled, traceable and reliable in validating the acceleration data measured by IMU. The vibration test system TIRAvib 51110 includes power amplifier and shaker. The 100 N permanent magnet shaker, designed for continuous operation, is mounted in a rigid trunnion. The trunnion enables to change the vibration direction from vertical to horizontal (± 90°). The frame is provided with vibration isolators which minimize vibration transfer to the floor. The maximum acceleration the shaker can go is 45g and this wide frequency band shaker will operate in the frequency range from 2 to 7000 Hz with sine excitation. Designed for continuous operation, the shaker provides an excellent lateral and rotational stiffness. The power amplifier is an air cooled amplifier type with a power output of 120 VA. The amplifier is equipped with latest MOSFET power transistors. Asymmetrical input leads to a large bandwidth with high efficiency. The calibrated Integrated Electronic PiezoElectric (IEPE) accelerometer, PCB 352C68 is a shear mode accelerometer using ceramic sensing element and with built in signal conditioner-preamplifier which required constant current excitation of 2-20 mA. It comes with sensitivity of 100 mV/g with accuracy of ±10 %, measurement range of ±50 g Pk with broadband resolution of 0.00016 g r.m.s, frequency range of 0.5 to 10000 Hz with accuracy of ±5%, non-linearity is less than 1 % and transverse sensitivity is less than 5 %. The operating temperature range is -53 to 93 oC.
To validate the acceleration value measured by the IMU, the calibration accelerometer and the IMU were attached side by side on the vibration test system. The acceleration value measured during static condition was first compared to that of a calibrated accelerometer. Subsequently, the shaker was set to vibrate at frequency of 20 Hz, 30 Hz, 40 Hz and 50 Hz with the peak-to-peak amplitude of vibration at 2 g, 4 g and 6 g for each set of frequency. The acceleration value measured by the IMU was recorded and compared with calibrated accelerometer. The testing was repeated for different axis of the IMU by changing the position of the IMU.
11 Figure 2: Experimental setup for IMU accelerometer testing
Angular Velocity Data
An experimental setup consists of a stepper motor was used to validate the angular velocity measured by the IMU. The stepper motor, NEMA 23 YH57BYGH56-401A comes with the following specifications: step angle is 1.8o with the accuracy of ±5 %, rated current is 2.8 A, phase resistance is 0.9 Ω with accuracy of ±10 %, phase inductance is 2.5 mH with accuracy of ±10 % and holding torque is 1.2 Nm. To remain minimal shaft radial and axial play, a rotor disk of less than 450g was attached to the stepper motor. The proposed experiment using a relatively high accuracy stepper motor is therefore more controlled, traceable and reliable in validating the angular velocity data measured by IMU. The IMU was attached to the rotor disk for the test. Arduino Uno R3 was again used to control the speed as well as the number of steps the stepper move during the testing. Figure 3 shows the experimental setup used in the testing. To validate the angular velocity value measured by the IMU, the value measured during the static condition was first recorded. Then, the testing continues with the stepper motor turning in clockwise direction for different speeds. The speed settings used in this testing were 1 rps, 1.5 rps, 2 rps, 2.5 rps in each direction of rotation. The angular velocity values measured during the rotation were then plotted on a graph and compared with the speed of rotation of the stepper motor. The testing was then repeated for all 3 axis of the IMU.
12 Figure 3: Experimental setup for IMU gyroscope accuracy testing
Orientations, Position and Bending Angle
The human motion analysis device is developed to analyse human joint motion and thus its accuracy in measuring human joint movement was tested. One physically active subject (age: 27; height: 168 cm; weight: 65 kg) volunteered to participate in the study. For this testing, the movement of elbow was examined. For the experimental setup, the human motion analysis device was worn on the shoulder and the two IMUs are placed on the upper arm and the lower arm. Figure 4 shows the setup on human and the motion of the subject’s hand during the testing. During the testing, the subject was asked to lower his hand and bend its elbow repetitively for 5 times before straightening his hand to the original position in 2D plane. While the subject was moving, motion data was collected using the human motion analysis device, high resolution videos with effective photo resolution of 12 Mega Pixels and frame rate of 47 fps were also recorded simultaneously in the similar 2D plane using GoPro Hero 3 device. Post processing of the video was then performed using commercial motion analysis software MaxTRAQ 2D to analyse the motion of the elbow. MaxTRAQ 2D is video based motion tracking software. MaxTRAQ 2D is a manual or automatic digitizer that can be used to extract kinematic properties out of standard AVI files. With both manual and auto tracking the user can go through frame by frame to look at angles, distance between points, etc. MaxTRAQ 2D does not discriminate which camera is used. But in this case, high resolution videos in a controlled 2D plane, (i.e. 12 MP, 47 fps) were recorded and used for post processing in kinematics data extraction using MaxTRAQ 2D to improve the reliability of the kinematics data obtained in validating the human joint movement by the human motion analysis device.
13 Figure 4: Movement of subject’s elbow (a) when the hand is straightened
14 RESULTS AND DISCUSSION
The data extracted from the accelerometers and the gyroscopes of the inertial measurement units (IMU) was validated through comparison with calibrated data under controlled conditions. Besides, the ability of the human motion analysis device in measuring human body movement was also tested.
Validation of Acceleration Data
First, the accuracy of the acceleration measured by accelerometers of the IMU at static condition was tested. The error measured during the static condition is tabulated in Table 2.
Table 2: Static error of accelerometer in IMU
Axis Maximum Error (g) Average Error (g)
X 0.07 0.06
Y 0.11 0.10
Z 0.09 0.07
Subsequently, the accuracy of the accelerometer of the IMU was examined under dynamic vibration condition. The test was repeated for vibration frequency of 20 Hz, 30 Hz, 40 Hz and 50 Hz with peak to peak vibration of 2 g, 4 g and 6 g using the calibrated vibration test system. The testing was repeated for all 3 axes for the accelerometer of the IMU. Figure 5, 6 and 7 show the difference between the acceleration measured by the accelerometer of the IMU and the value measured by the calibrated accelerometer under each testing condition for X axis, Y axis and Z axis accelerations respectively.
Figure 5: Peak to peak acceleration value measured by calibrated accelerometer and X axis accelerometer of IMU at different frequency of vibration
15 Figure 6: Peak to peak acceleration value measured by calibrated accelerometer and Y axis
accelerometer of IMU at different frequency of vibration
Figure 7: Peak to peak acceleration value measured by calibrated accelerometer and Z axis accelerometer of IMU at different frequency of vibration
From the test results obtained, the error in acceleration measured by each axis of the accelerometer of the IMU is tabulated in Table 3.
16 Table 3: Overall error in peak to peak acceleration measured by each axis of the
accelerometer of the IMU
Axis Maximum Error in Peak to Peak Acceleration Measurement (%)
Average Error in Peak to Peak Acceleration Measurement (%)
X -22.89 -9.10
Y -16.56 -15.95
Z -17.94 -14.28
From the results obtained, it is observed that the accelerometer of the IMU recorded relatively large average error in peak to peak acceleration measurement (approximately 15%). The deviation is due to the fact that the raw data of acceleration from the IMU consists of a lot of unfiltered noise and tends to fluctuate for short duration measurement. Thus, direct double integration of the acceleration data from the IMUs to calculate the displacement or position is subjected to a lot of errors and it is not recommended in this study. Though the acceleration has no direct relationship on the orientation data used in determining the position and bending angle, however, the data is used by the DMP for computing the sensor fusion quaternion data. The error in measurement of acceleration could have minor effect on the performance of the human motion analysis device in term of accuracy.
Validation of Angular Velocity Data
Test was then conducted on the accuracy of the gyroscopes of the IMU. The test was conducted for different directions and speeds of rotation using high reliability and accuracy stepper motor. The speeds of rotation were set at 1rps, 1.5rps 2rps and 2.5rps for both directions of rotation. The test was repeated for each axis of the gyroscope of the IMU. Figures 8, 9 and 10 show the difference between the angular velocity values measured using gyroscope at various speeds of rotation.
Figure 8: Measured angular velocity of gyroscope at different stepper motor speeds for X axis rotation
17 Figure 9: Measured angular velocity of gyroscope at different stepper motor speeds for Y axis
rotation
Figure 10: Measured angular velocity of gyroscope at different stepper motor speeds for Z axis rotation
From the test results obtained, the error in angular velocity measured by each axis of the gyroscope of the IMU is tabulated in Table 4.
18 Table 4: Overall error in angular velocity measured by each axis of the gyroscope of the IMU
Axis Maximum Error in Angular Velocity Measurement (%)
Average Error in Angular Velocity Measurement (%)
X -3.64 -2.43
Y -3.35 -2.83
Z -2.26 -1.01
In previous studies, it was reported that noise in the angular velocity data accumulated over time and resulted in integration errors. Due to the nature of the mechanism of gyroscope which has great influence by gravitational force, the angular velocity will tend to drift for long duration measurement and cause deviation on the orientation data. The DMP in the IMU used in this study can perform the data fusion on the IMU chip itself and gyroscope calibration. Therefore, it is observed that the angular velocity measurement by the gyroscope of the IMU is quite reliable as the average error is less than 5%. This shows that the gyroscope of the IMU has higher accuracy compared with the accelerometer of the IMU.
Validation of orientations, position and bending angle
To test the accuracy of the human motion analysis device in measuring human joint motion, the values measured by the human motion analysis device were compared with the values obtained from the post-processed video using commercial image processing based software, MaxTRAQ 2D under a controlled experiment. In this test, 5 points were identified along the movement, where point 1, 3 and 5 are the corresponding non-moving joint, first moving joint and second moving joint in the software representation of the human motion analysis device. Point 2 and point 4 represents the IMU. Figure 11 shows the points located in MaxTRAQ 2D for the analysis of the motion of the subject’s elbow. The results obtained using the human motion analysis device was compared to that of the post processed results using MaxTRAQ 2D. Figure 12 shows the corresponding motion simulated by human motion device software. The software displays two positions of the joint. As the position and orientation of the human body part is also shown with numerical value displayed, the motion of the body part and the joints can be analysed.
19 Figure 11: Points identified for analysis of the elbow movement
Figure 12: Simulation of joint motion in the human motion device software
To evaluate the accuracy of the human motion analysis device, the vertical position of the joints which are point 3 and point 5 relative to point 1 analysed by video processing and the human motion analysis device software were compared (Figure 13).
20 Figure 13: Y position of point 3 and point 5 relative to point 1 analysed using post processed
video and human motion analysis device
From the results, it is noted that the position of joint obtained using the human motion analysis device is similar to the post processed video. All the movements of the joint were recorded. The absolute position obtained using the human motion analysis device is found to have errors as compared to the post processed video with the error in maximum position of 23.98% for point 3 and 17.18% for point 5 as tabulated in Table 5. The large deviation is probably due to the algorithm used in calculating the position and the distance moved by the joint which involves the measurement of body length that could be inaccurate at times.
Table 5: Error in position measured by IMU at upper (Point 3) and lower parts of joint (Point 5)
Point Error in Maximum Position (%) Error in Average Position (%) 3 23.98 25.42 5 17.18 9.09
Besides evaluating the position of the joint, the bending angle of the first moving joint was also evaluated. In the testing, the bending angle of the elbow was identified using both human motion analysis device and post processed video. Figure 14 shows the results obtained.
21 Figure 14: Bending angle of elbow analysed using post processed video and the developed
human motion analysis device
The results obtained by both the methods have similar pattern where all the movements of the elbow were displayed and recorded. There is a minimal difference in average angle value of 10o. Table 6 shows that the error in maximum bending angle is only 5%. As discussed in previous section, the DMP is capable of computing sensor fusion quaternion data and performing gyroscope calibration to determine a relatively lesser drift orientation data. This explains the small deviation in bending angle which is mainly calculated by the orientation data.
Table 6: Overall error in position measured by IMU at upper (Point 3) and lower parts of joint (Point 5)
Point Error in Maximum Bending Angle (%)
Error in Average Bending Angle (%)
2-4 5.11 6.99
From the results obtained, it is shown that the human motion analysis device developed is able to show the motion of the human joint movement. In terms of accuracy, there are some differences in terms of the numerical values, (i.e. position and bending angle) obtained when compared to the post processed video. The difference in the values obtained might be due to several experimental errors from both vision and sensor based approaches as follows.
1. Lesser sampling rate and resolution available from the human motion device
2. Body length measurement error as the human motion device uses the measured body part length to calculate the position of the joint. Error in body length measurement could cause error in the analysis.
22 3. Sensor attachment error as human bodies are not flat, error could occur when the IMUs are attached. The slight error in orientation of the IMU could cause greater error in the analysis of positions.
4. Limitation of video resolution as the results obtained using the post processed might also contain error as the analysis using image processing software is limited by the resolution of the video.
5. Video recording angle error. As the post processed video could only process the human motion as a 2-dimensional picture, the video must be recorded perpendicular to the human motion. Slight error in the recording angle could pose great error in the analysis using post processed video.
Should the testing be repeated, the possible errors must be taken care of in order to better measure the ability of the human motion analysis device developed.
CONCLUSION AND RECOMMENDATIONS
The pilot study presented was intended to verify the reliability of the DMP generated sensor fusion data in the assessment of human body joint. In this study, a cost effective and highly portable human motion analysis device which utilised multiple IMUs was successfully developed which is able to show the motion of human body parts in real time with kinematics data such as acceleration, angular velocity, position and orientation extracted and displayed. The joint motion analysed using the device has also been compared to the results analysed using commercial video motion analysis software. It is found that the motion simulated using the device is able to correlate with the results analysed using post processed video. Accuracy tests have successfully been conducted and it is noted that the gyroscope of the IMU under DMP sensor fusion algorithm and device calibration capability is able to give the angular velocity with less than 5% error. This has led to a more accurate orientation data which gives 7% error in average bending angle. However, the accelerometer of the IMU is less reliable, which has an average error of 15% compared to calibrated accelerometer. The correlation of position has 17-24% of error. The accuracy aspect of the device could be further improved and it is suggested to focus on the lack of accuracy in accelerometer of the IMU. This could be done by re-calibrating the accelerometer over a longer measurement time to reduce the bias and noise in the acceleration data. Furthermore, better algorithm could be used in calculating the position and the distance moved by the joint as the algorithm used involve the measurement of body length, which could be inaccurate at times. If necessary, better algorithms such as Kalman filter or complementary filter could be tested in extracting a more reliable and accurate orientation data to calculate the positions. Finally, IMU comes with magnetometer could replace the existing IMU and several algorithms such as Madgwick and Mahony filters could be applied to reduce the gyro drift by utilizing the measured magnetic fields.
ACKNOWLEDGEMENT
The authors wish to acknowledge the financial support and advice given by University of Malaya Research Grant (RP013B-15SUS), Fundamental Research Grant Scheme (FP010-2014A), Newton-Ungku Omar Fund: Mobility Grant (IF006-2016), Advanced Shock and Vibration Research (ASVR) Group of University of Malaya and other project collaborators.
23 REFERENCES
[1] 10th Annual Report of National Joint Registry for England, Wales and Northern Ireland in: N.J.R.N.E. Board (Ed.), National Joint Registry, Northgate Information Solutions (UK) Ltd, 2013.
[2] J.A. Epinette, M.T. Manley, Hydroxyapatite-coated total knee replacement - Clinical experience at 10 to 15 years, J Bone Joint Surg Br, 89b (2007) 34-38.
[3] S. Greengard, Risks and Complications of Total Knee Replacement Surgery, in: Healthline, Healthline Networks, Inc, 2015.
[4] R. Dai, R.B. Stein, B.J. Andrews, K.B. James, M. Wieler, Application of tilt sensors in functional electrical stimulation, Rehabilitation Engineering, IEEE Transactions on, 4 (1996) 63-72.
[5] K.Y. Tong, M.H. Granat, A practical gait analysis system using gyroscopes, Med Eng Phys, 21 (1999) 87-94.
[6] S.B. Thies, P. Tresadern, L. Kenney, D. Howard, J.Y. Goulermas, C. Smith, J. Rigby, Comparison of linear accelerations from three measurement systems during "reach & grasp", Med Eng Phys, 29 (2007) 967-972.
[7] A.J. vandenBogert, L. Read, B.M. Nigg, A method for inverse dynamic analysis using accelerometry, J Biomech, 29 (1996) 949-954.
[8] D.T.W. Fong, J.C.W. Wong, A.H.F. Lam, R.H.W. Lam, W.J. Li, A wireless motion sensing system using ADXL MEMS accelerometers for sports science applications, in: Intelligent Control and Automation, 2004. WCICA 2004. Fifth World Congress on, 2004, pp. 5635-5640 Vol.5636.
[9] S. Tadano, R. Takeda, H. Miyagawa, Three Dimensional Gait Analysis Using Wearable Acceleration and Gyro Sensors Based on Quaternion Calculations, Sensors-Basel, 13 (2013) 9321-9343.
[10] R. Takeda, S. Tadano, M. Todoh, M. Morikawa, M. Nakayasu, S. Yoshinari, Gait
analysis using gravitational acceleration measured by wearable sensors, J Biomech, 42 (2009) 223-233.
[11] R. Takeda, S. Tadano, M. Todoh, S. Yoshinari, Human Gait Analysis using Wearable Sensors of Acceleration and Angular Velocity, Ifmbe Proc, 23 (2009) 1069-1072.
[12] M.J. Yang, H.R. Zheng, H.Y. Wang, S. McClean, D. Newell, iGAIT: An interactive accelerometer based gait analysis system, Comput Meth Prog Bio, 108 (2012) 715-723. [13] M. Alaqtash, H.Y. Yu, R. Brower, A. Abdelgawad, T. Sarkodie-Gyan, Application of wearable sensors for human gait analysis using fuzzy computational algorithm, Eng Appl Artif Intel, 24 (2011) 1018-1025.
[14] M.A. Brodie, A. Walmsley, W. Page, The static accuracy and calibration of inertial measurement units for 3D orientation, Comput Method Biomec, 11 (2008) 641-648.
[15] M.A. Brodie, A. Walmsley, W. Page, Dynamic accuracy of inertial measurement units during simple pendulum motion, Comput Method Biomec, 11 (2008) 235-242.
[16] S. Fantozzi, A. Giovanardi, D. Borra, G. Gatta, Gait Kinematic Analysis in Water Using Wearable Inertial Magnetic Sensors, Plos One, 10 (2015).
[17] J. Favre, B.M. Jolles, O. Siegrist, K. Aminian, Quaternion-based fusion of gyroscopes and accelerometers to improve 3D angle measurement, Electron Lett, 42 (2006) 612-614. [18] T. Liu, Y. Inoue, K. Shibata, Development of a wearable sensor system for quantitative gait analysis, Measurement, 42 (2009) 978-988.
24 [19] R.E. Mayagoitia, A.V. Nene, P.H. Veltink, Accelerometer and rate gyroscope
measurement of kinematics: an inexpensive alternative to optical motion analysis systems, J Biomech, 35 (2002) 537-542.
[20] A.M. Sabatini, Quaternion based attitude estimation algorithm applied to signals from body-mounted gyroscopes, Electron Lett, 40 (2004) 584-586.
[21] P. Picerno, A. Cereatti, A. Cappozzo, Joint kinematics estimate using wearable inertial and magnetic sensing modules, Gait Posture, 28 (2008) 588-595.
[22] H. Kainz, C.P. Carty, L. Modenese, R.N. Boyd, D.G. Lloyd, Estimation of the hip joint centre in human motion analysis: A systematic review, Clin Biomech, 30 (2015) 319-329. [23] A. Peters, B. Galna, M. Sangeux, M. Morris, R. Baker, Quantification of soft tissue artifact in lower limb human motion analysis: A systematic review, Gait Posture, 31 (2010) 1-8.
[24] S.M.T. Reza, N. Ahmad, I.A. Choudhury, R.A.R. Ghazilla, A Fuzzy Controller for Lower Limb Exoskeletons during Sit-to-Stand and Stand-to-Sit Movement Using Wearable Sensors, Sensors-Basel, 14 (2014) 4342-4363.