Kinect Validation

The agreement between the Kinect and Qualisys (gold standard) was evaluated for each gait parameter by obtaining the Bland-Altman’s mean difference and 95% limits of agreement (LoA), as well as the Pearson’s and concordance correlation coefficients and associated 95% confidence inter- val (CI).

The Bland-Altman’s mean difference (or bias) is the mean of the differences between the measurements obtained by two different methods ( d ) [237, 238]. The lower and upper 95% limits of agreement correspond to d 1.96s and d1.96s respectively, where s is the standard deviation for the differences between paired measurements.

The presence of fixed bias, i.e., a constant difference in measurements between systems, was verified by carrying out a one-sample t-test on the differences between Kinect and Qualisys [239]. There is no fixed bias if the mean differences is indistinguishable from zero (p-value ≤ 0.05). Pro- portional bias is present when the difference in measurements from two methods increases or de- creases in proportion to the mean values [239]. The presence of proportional bias was detected by performing a OLS (ordinary least squares) regression analysis [239, 240]. There is proportional bias if the OLS regression line fitted to the plot of the differences versus the means has a slope that differs significantly from zero (p-value ≤ 0.05).

The Pearson’s correlation coefficient (r) indicates the strength of linear relationship between variables, i.e., their relative agreement [238]. The concordance correlation coefficient (rc) indicates

the absolute agreement between variables, by measuring not only how far each observation deviates from the line fit to the data (precision), but also how far this line deviates from the 45 degrees line through the origin (accuracy) [241]. The value of both r and rc ranges between −1 and 1, where −1

means perfect disagreement, 0 corresponds to an independence situation and 1 indicates perfect agreement [242]. Based on the guidelines given by Portney and Watkins [243], correlation thresholds were set as poor (<0.5), moderate (≥0.5 and <0.75), good (≥0.75 and <0.9) and excellent (≥0.9).

For the mean difference or estimation error, a statistically significant difference between the two Kinect versions, the two walking activities and the three Kinect configurations, was verified by performing a one-way repeated-measures analysis of variance (ANOVA). For configuration compar- ison, if a significant difference was detected (p-value ≤ 0.05), a post-hoc Tukey test was then carried out to find which situations have significantly different errors.

All the analyses described in this section were performed in the R environment (version 3.2.3) [209], using code implemented for performing the validation studies. The stats package [210] was used to compute the Pearson’s correlation coefficient and associated 95% CI and p-value, as

well as for performing the t-test and OLS regression analysis. The concordance correlation coeffi- cient and associated 95% CI were obtained using the cccrm package [244]. The Tukey’s test and ANOVA were performed using the agricolae [245] and car [246] packages, respectively.

6.3 Results

To verify if the two Kinect versions can be used interchangeably for gait analysis, the vali- dation study was firstly carried out for both Kv1 and Kv2, when considering WF data and configu- ration KC1. In this study, we only took into account the WF activity, since the Kinect’s joint tracking algorithm assumes that the subject is always facing the camera.

To investigate if data acquired during both WF and WB activities can be used for gait anal- ysis, a similar validation study was carried out for the two different activities when using the Kinect version that led to the best overall result in the first validation study.

Furthermore, we explored if the physical configuration used for the Kinect has any effect on the computed gait parameter values, by performing the validity study for configurations KC2 and KC3 besides KC1, when taking into account the Kinect version and walking activity that achieved the best overall result in the studies described above.

The complete results of the optimization carried out for the parameters (order and cut-off frequency) of the low-pass zero-lag Butterworth filter, which was used for filtering the measures extracted from Kinect data, are included in Appendix F. The results obtained for each Kinect version and each walking activity are presented in Appendix F.1. The results obtained for the three Kinect configurations are presented in Appendix F.2. These results do not include the spatiotemporal gait parameters for which the computation only depends on time instants and/or the position of a body joint at those instants: stride, step, stance, swing, single support and double support duration, and stride length.

For each one of the remaining parameters, the order and cut-off frequency values were cho- sen for Kv1 and Kv2 separately, by taking into account the mean estimation error for both WF and WB. We chose not to filter the measures used to compute the step width and gait speed, as well as most of the kinematic parameters, since the mean error did not decrease or decreased only slightly when filtering the data. The selected values for the filter’s parameters are indicated in Table 6.5 for each Kinect version.

6.3 Results

Table 6.5. Butterworth filter’s order and cut-off frequency values for each gait parameter, when using Kv1 and Kv2.

Gait parameter

Kv1 Kv2

Order frequency Cut-off

(Hz) Order Cut-off frequency (Hz) Step length 6 5 6 3 Step width Not filtered Gait speed

Gait speed variability 2 1 2 2

Foot swing velocity 4 3 6 4

Arm swing velocity 2 1 4 2

Neck angle Not filtered

Spine shoulder angle — Not filtered

Spine middle angle No filtering

Elbow angle maximum 6 1 Not filtered

Elbow angle minimum

Not filtered Knee angle maximum

Knee angle minimum Hip angle range

Ankle angle range 6 1 6 1

The filter parameters values that led to the best results for each considered Kinect configu- ration were the same as the ones included in Table 6.5, except for the step length. However, the mean error values for step length were similar. Therefore, the values of Table 6.5 were used in all studies for simplicity.

For each study, the results presented below include the practical depth range, number of actual gait cycles and validation results, for each considered situation. The practical depth range is the difference between the largest and smallest distance between the subject and the Kinect for which all joints are tracked (minimum and maximum distance). The depth range value was obtained for each gait trial, by taking into account the WF/WB data selected automatically.

The number of actual gait cycles is the number of gait cycles identified by relying on Qual- isys data, when considering the practical depth range of the Kinect. The results include the number of left, right and both gait cycles, and are indicated for all subject and trials, as well as for each subject and each trial (mean and standard deviation).

The validation results were obtained for each gait parameter and situation. They include the Bland Altman’s mean difference (Mean diff), Pearson's correlation coefficient (r) and concordance correlation coefficient (rc). If there is a statistically significant difference between the considered

situations (one-way repeated-measures ANOVA, p-value ≤ 0.05), when taking into account the esti- mation error for all subjects, the lowest mean difference is indicated in bold. The r and rc values are

indicated using the following colours: green (≥ 0.9), yellow (≥ 0.75 and < 0.9), orange (≥ 0.5 and < 0.75), and red (< 0.5). Moreover, the r values that are statistically significant (p-value ≤ 0.05) are indicated using the * symbol.