Comparison with Previous Linear Approaches

Previous studies have used a number of metrics to evaluate the accuracy of their models. Among other metrics, RMSE (118, 210, 273), correlation (207), SEE (85, 200, 207), maximum percentage error (198), and average percentage accuracy (209) have been used.

120

The present study has calculated several accuracy metrics to allow direct comparison with previous work. However, there are a number of elements to consider which affect how results may be compared between studies. In the present study, three groupings of the participants were considered – the obese group, the normal BMI group, and the combined obese and normal BMI group – whereas previous research has not generally broken down results for specific BMI groups. Previous studies have used treadmill and overground walking in a variety of combinations to train and test their algorithms, as discussed in subsection 2.4.1. The present study calculated separate results based on three walking modes comprising treadmill, overground, and overground and treadmill combined. There are, therefore, nine sets of results for each speed estimation model. Also, the present study aimed to test whether speed estimation models may be applied to a population of mixed BMI individuals without the need to tailor the algorithms to the individual, whereas certain studies have chosen individual calibration.

Schutz et al. (200) investigated the correlation between the RMS of accelerometer output and walking speed. Both linear and quadratic models were applied to a group of fifty healthy women, a large proportion of which were obese (mean BMI was 31.4 ± 5.1kgm-2). Although they observed high correlations between RMS and walking speed for individuals, there was a great amount of inter-subject variance apparent in the group data – for overground walking mean SEE values for the group model were 0.352ms-1 and 0.325ms-1 for the linear and quadratic models respectively, compared with 0.056ms-1 and 0.042ms-1 for the equivalent individually calibrated models. The study by Barnett et al. (207) also compared accuracy between a group model and individual calibration. In this case the relationship between accelerometer counts and walking speed was investigated, and their testing was limited to overground walking. The SEE values returned by Barnett et al. were 0.161ms-1 and 0.053ms-1 for group calibration and individual calibration models respectively, though a SEE score of 0.044ms-1 was also reported when the accelerometer itself was also calibrated to the individual. The present study returned SEE values of around 0.086ms-1 in several of the models tested against the combined walking mode and BMI data, and around 0.09ms-1 in the isolated overground walking for the combined BMI group (see Table 16). The lowest SEE returned by overground walking for the present study was 0.084ms-1 for the isolated normal BMI group, and the lowest SEE for the obese group for overground walking was 0.09ms-1. The SEE values returned by the present study better the group calibration results of Barnett el al. and Schutz et al., and though they fall short of the individually calibrated results they still

121

may be considered to compare well. Although individual calibration leads to higher speed estimation accuracies, it is not desirable when assessing large populations, as each subject is required to perform a calibration procedure under laboratory conditions. It is arguable that, when considering large scale studies or interventions, the expense of the calibration procedure is not worth the additional accuracy it may produce when compared with the present study.

A study by Bonomi et al. (85) investigated a multi-linear speed estimation model based on accelerometer features. The model was developed using fifteen participants and validated against another five. The study returned a SEE value of 0.056ms-1 for outdoor walking data. This result represents good accuracy comparable to the individually calibrated models discussed above. However, five participants in the validation group might be regarded as an insufficient number to reliably test the performance of the algorithm compared with the leave-one-out cross-validation approach used in the present study.

The study by Panagiota et al (199) used hip-mounted accelerometer to estimate walking speed, and also incorporated BMI in the speed estimation algorithm. This study merits detailed discussion, given the similarities with the present study. In the study the walking took place overground, and the participants were split into two groups to perform walking outdoors (n=20) and indoors (n=17). Two walking speed categories were considered: normal walking and brisk walking. Outdoors the participants were free to set the walking pace according to these categories. Indoor speeds were imposed at 1.33 ms-1 and 1.55ms-1 for the two categories; participants were assisted in maintaining these speeds by a regular audible signal. The study used a multi-linear speed estimation model based on ten features, five of which were accelerometer features, and the other five incorporated anthropometric measurements including height, weight and BMI. In this sense their approach could be considered as accounting for obesity. There were, however, an insufficient number of obese participants to allow any conclusions to be drawn about the effects of obesity on the algorithm: there were n=37 with mean BMI of 24.96 ± 3.24 kg/m2, as opposed to the eleven out of twenty-two participants in the present study who exceeded a BMI of 30 kg/m2.

Panagiota et al. applied leave-one-out cross-validation to their linear model, and accuracy was evaluated using the mean and standard deviation of the percentage error, and also the mean and standard deviation of the error in ms-1. It was possible to split the accuracy evaluation between normal and brisk speeds due to the method of data collection, and also the

122

standard deviation of speed was low for both mean walking speeds making overlap of speed categories less likely (mean outdoor speeds were 1.38 ± 0.08ms-1 and 1.78 ± 0.08ms-1, and indoor speeds were 1.34 ± 0.03 ms-1 and 1.55 ± 0.02ms-1 for normal and brisk walking respectively). Most other studies, however, elect to report a single set of accuracy results for the full range of walking speeds. Panagiota et al. achieved an errorof -0.01 ± 0.07ms-1 and a mean percentage error of -0.81 ± 4.90% for the normal walking pace. For the brisk walking pace an error of 0.02 ± 0.08ms-1 and a percentage error of 1.01 ± 4.94% was returned. The best results for overground walking in the present study was from model number 3 which returned a mean error of 0.000 ± 0.089ms-1 (Table 18) and a mean percentage error of 0.46 ± 6.35% for the mixed BMI group (Table 17). The obese group in isolation returned similar standard deviations, but the mean showed slight positive bias with an error of 0.016 ± 0.88ms-1 and a percentage error of 1.99 ± 6.67%. The normal BMI group has a small negative bias with an error of -0.015 ± 0.088ms-1 and a percentage error of -1.11 ± 5.66%.

Although the present study apparently returns marginally lower accuracies for the overall dataset than Panagiota et al., this may be greatly explained by the different approach to categorising speeds for evaluation. The present study has a greater variety of speeds represented, ranging from 0.75ms-1 to 2.46ms-1, (see Table 12) and accuracy is assessed across this full range of speeds. If we consider the results of the present study that were limited by speed bands, model 2 returned RMSE values around 0 ± 0.056ms-1 and mean percentage error of 0.08 ± 3.43% for overground walking in the 1.4ms-1 to 1.7ms-1 range (Table 20). The 1.0ms-1 to 1.3ms-1 range returned mean error values of around 0ms-1 with standard deviation of 0.066ms-1, and mean percentage error of 0.1 ± 5.75% for model 2 (Table 19). Given these results, it is arguable that the estimation model in the current study may be superior to that in Panagiota et al. Also, because Panagiota limited testing to two relatively narrow bands of speeds, it is not certain that their model would perform as well across a broader range of speeds.