Data Acquisition and Processing

The datasets obtained in each experiment are presented Table 3.3, including the correspond- ing number of days, number of subjects, DBS state for PD patients, number of data acquisitions per subject and number of gait trials per acquisition (mean and standard deviation for DS2 and DS3). For each acquisition, the Kinect data were acquired at an approximate frame rate of 30 Hz, using our KiT application [78].

Table 3.3. Datasets obtained in each experiment, including the number of days and subjects, DBS state for PD patients, and number of data acquisitions per subject and of gait trials per acquisition.

Datasets Number _{of days}

Number of subjects DBS state for patients

Number of data acquisitions per subject

Number of gait trials per

acquisition PD

patients Healthy All

DS1 1 3 3 6

Stim-on and Stim-off

1 for healthy subjects, and 2 on the same day for patients (one in each DBS

state) 1 DS2 3 (period of 9 months) 11 8 19 8 ± 5 DS3 19 (period of 12 months) 11 ― ― Stim-on 1 to 4 (3 ± 1), on different days 8 ± 2

The first step of data processing consisted of converting the 3-D position of each joint



x y z, ,



into a coordinate system corresponding to a non-tilted camera (tilt angle of 0 degrees), for each body joint data frame. This coordinate conversion facilitates the comparison between the 3-D joint positions acquired using different camera tilt angles, and was performed for all studies presented in this thesis.

Tilting the Kinect by rotating it around its x-axis leads to the effect illustrated in Figure 3.3. Assuming that the camera is placed in such a way that its z-axis is parallel to the horizontal plane (tilt angle of 0 degrees) where a vertical object is standing, two points A and B on this object will have the same value for the z-coordinate (z_A z_B in the figure). However, if the Kinect is tilted around its x-axis, so that its z-axis rotates by an angle  relative to its horizontal position, then the coordinates of points A and B will have different values in the rotated coordinate system ' ' 'x y z (

A B

' '

z z in the figure). In a similar way the 'y -coordinates will be different from the y-coordinates.

Figure 3.3. Coordinate systems of a non-tilted and tilted Kinect, and the effect that the tilting has on the distance between the Kinect and a given object in the z-axis.

To convert a position

x y z, ,



into the coordinate system corresponding to a non-tilted cam- era, the coordinates of the converted position



x y z', ', '



were computed using the following equa- tions: ' x x (3.1)

 

 

' sin cos y  



 z



y (3.2)

 

 

' cos sin z 



 z



y (3.3) z z'  y' O z'A zA zB z'B A B Object being observed

Coordinate system of a tilted Kinect tilt around the x-axis

( of )

Coordinate system of a non-tilted Kinect z-axis paral el to the horizontal plane

( l )

y

3.3 Materials and Methods

The next step of data processing consisted of identifying the gait cycles performed by the subjects during the gait trials. For the first studies, the gait cycles of DS1 and DS2 were manually selected, by identifying the instants corresponding to heel strikes relying on the acquired colour and/or depth data. In the case of DS2, the instants corresponding to heel strikes were saved in our KiMA application [78]. Only the data corresponding to walking towards the camera were considered, since we visually verified that the data acquired while walking away from the camera were much noisier.

For the last study presented below, the gait cycles of DS2 and DS3 were automatically de- tected using the solution described later in this thesis. We again considered only the data correspond- ing to walking towards the camera, since we confirmed in a validation study also presented in this thesis that the data corresponding to walking away from the sensor leads to higher detection errors.

The number of identified gait cycles for each dataset (manually for DS1 and DS2 and auto- matically for DS3) is presented in Table 3.4. In the second day of EXP2, Kv2 data were not acquired since the camera was unavailable due to technical problems. In the third day of EXP2, it was not possible to acquire data from two patients in the Stim-off state. In addition, the battery of the DBS stimulator was low for another patient, so the acquisition carried out in the Stim-on state was not considered in the studies presented below. It was also not possible to acquire Kv2 data from another patient, due to difficulties of the camera in tracking the subject.

Table 3.4. Number of gait cycles included in each dataset.

The version of the used Kinect(s) and the associated number of subjects are also indicated.

Dataset Kinect Number of _subjects

Number of gait cycles PD patients _Healthy

subjects subjects All Stim-on Stim-off DS1 Kv1 3 healthy + 3 PD patients 5 6 6 17 DS2a Kv1 8 healthy + 11 PD patients 109 122 58 289 Kv2b 5 healthy + 8 PD patients 157 143 118 418 DS3 Kv1 11 PD patients 282 ― ― ― Kv2 667

a _{For both Kv1 and Kv2, data were not acquired from two patients while in the Stim-off state.} Additionally, the battery of the DBS stimulator was low for another patient.

For each data frame of a left/right gait cycle, the following 34/43 measures (Kv1/Kv2) were computed:

 Velocity of the head, neck, spine base and spine middle, the right/left shoulder, elbow, wrist and hand, and the left/right foot, ankle, knee and hip, using (3.4);

 Velocity of the left/right hand tip and thumb, and the spine shoulder (Kv2 only), using (3.4);

 Acceleration of the head, neck, spine base and spine middle, the right/left shoulder, el- bow, wrist and hand, and the left/right foot, ankle, knee and hip, using (3.5);

 Acceleration of the left/right hand tip and thumb, and the spine shoulder (Kv2 only), using (3.5);

 Distance between the feet, ankles, knees, hands, wrists and elbows, using (3.6);  Distance between the hand tips and thumbs (Kv2 only), using (3.6);

 Angle at the left/right knee (defined by hip, knee and ankle joints), right/left elbow (de- fined by shoulder, elbow and wrist joints), neck (defined by head, neck and spine mid- dle/shoulder joints) and spine middle (defined by neck/spine shoulder, spine middle and spine base joints), using (3.7).

 Angle at the spine shoulder (defined by neck, spine shoulder and spine middle joints) (Kv2 only), using (3.7). 2 2 2 2 2 2 2 velocity v_x v_y v_z x y z t           (3.4) 2 2 2 2 2 2 2 acceleration _x a_y a_z vx vy vz t a           (3.5) 1 2 distance P P (3.6) 2 1 2 3 angle arccos 2 1 2 3 P P P P P P P P  _     _{ }      (3.7)

In (3.4), vx is the x-axis component of the velocity vector for a given joint, and Δx is the difference between the x-coordinate of the joint position for two consecutive frames. In (3.5), ax is the x-axis component of the acceleration vector for a given joint. Similar notations are used for the y- and z-axis. In (3.4) and (3.5), Δt is the time elapsed between two consecutive frames. In (3.6), P1

3.3 Materials and Methods

position of three different joints. In (3.6) and (3.7), P P_{i j} is the 3-D vector defined by the positions

Pi and Pj.

For each gait cycle, the mean, median, variance and normalized variance (variance divided by the mean) were computed over each measure. Four traditional gait parameters were also com- puted: stride duration and length, gait speed and cadence.

For some of the studies presented below, only a subset of the indicated parameters were considered. Moreover, in the last study on UPDRS estimation, only the following 22 traditional gait parameters were considered:

 Stride, step, stance, swing, single support and double support duration;  Stride and step length;

 Step width;

 Gait speed and gait speed variability;  Foot and arm swing velocity;

 Angle at neck, spine shoulder and spine middle;  Elbow angle minimum and maximum;

 Knee angle minimum and maximum;  Hip and ankle angle range.

The computation of these parameters are described in more detail in the study on Kinect validation presented later in this thesis.

In document Sistema para análise automatizada de movimento durante a marcha usando uma câmara RGB-D (Page 67-71)