Experiment 2: Situation recognition accuracy with ab-

absolute time and time extended evidence

We then included temporal knowledge into the inference process. This was done in two steps so that we could examine the contribution of both type of temporal knowledge 1) inclusion of absolute time and 2) inclusion of both ab- solute time and time extended evidence. Absolute time was incorporated into the inference process using the time patterns listed in table 5.2. Looking at this table, all situations except for ’use toilet’ and ’get drink’ are associated with an absolute time.Figures 6.5 (precision), 6.6 (recall) and 6.7 (f-measure) show the situation recognition results using the evidence decision network with and without absolute time on van Kastersen’s data set.

Precision improves for all but two of the situations, ’toileting’ and ’get drink’. Neither of these situations have an identifying time of occurrence so do not benefit from the inclusion of absolute time. For recall, all situations improve with the exception of ’leave home’. Without the use of absolute time, all sce- narios with no sensor activity were classified as ’leave home’. But with the inclusion of absolute time, ’sleeping’ can now be identified separately from ’leave home’, with a resultant small erosion of ’leave home’ recall. Looking at f-measure, the use of absolute time improves the inference accuracy for the five

0.00 0.20 0.40 0.60 0.80 1.00 Leave house

Toileting Showering Sleeping Breakfast Dinner Drink

No time Absolute Time

Situations

Figure 6.5: Precision for evidence decision network with 1) no time and 2) absolute time using van Kasteren’s data set

Situations leave house use toilet take shower go to bed prep break’t prep dinner get drink leave house 0.832 0.002 0.00 0.165 0.0 0.001 0.0 use toilet 0.185 0.587 0.061 0.168 0.0 0.0 0.0 take shower 0.629 0.076 0.201 0.094 0.0 0.0 0.0 go to bed 0.041 0.017 0.001 0.941 0.0 0.0 0.0 prep breakf’t 0.363 0.039 0.018 0.035 0.324 0.0 0.221 prep dinner 0.589 0.018 0.0 0.0 0.0 0.314 0.079 get drink 0.077 0.059 0.0 0.023 0.0 0.048 0.794

Table 6.4: Confusion Matrix on van Kasteren’s data set using absolute time situations that have an absolute time. ’Leave home’ and ’go to bed’ can now be distinguished because of their time pattern of day and night time occur- rence, respectively. Therefore, the f-measure for ’go to bed’ has jumped from 0 to 0.73. ’Prepare dinner’ and ’prepare breakfast’ are detected from similar sensor events. Both improve when absolute time is used because they occur at mutually exclusive times of the day. Looking at the confusion matrix in table 6.4, these two situations are no longer confused with each other, as was hap- pening when absolute time was not used. All situations continue to have a high proportion of their occurrences confused with the ’leave home’ situation, with some confusion now occurring with the ’go to bed’ situation now that it is recognised as occurring at nighttime.

0.00 0.20 0.40 0.60 0.80 1.00 Leave house

Toileting Showering Sleeping Breakfast Dinner Drink

No time Absolute Time Situations

Figure 6.6: Recall for evidence decision network with 1) no time and 2) abso- lute time using van Kasteren’s data set

0.00 0.20 0.40 0.60 0.80 1.00 leave house

use toilet take

shower go to bed prepare breakfast prepare dinner get drink

No time Absolute Time

Situations

Figure 6.7: F-measure for evidence decision network with 1) no time and 2) absolute time using van Kasteren’s data set

0.00 0.20 0.40 0.60 0.80 1.00

Leave house Toileting Showering Sleeping Breakfast Dinner Drink No time Absolute time Time Extended Situations

Figure 6.8: Precision for 1) no time, 2) absolute time and 3) absolute time and time extended evidence using van Kasteren’s data set

For the second part of the experiment, we then added time extension of evi- dence into the evidence decision network. Five of the situations are derived from transitory evidence: Durations are applicable for ’prepare breakfast’, ’prepare dinner’, ’get drink’, ’use shower’ and ’use toilet’ as each of their con- text events can be spread over time. No sensor is usually fired during ’leave home’ and ’go to bed’ situations so no time extension of evidence is used for these situations. Durations are calculated as the average of the situation dura- tion from the training data folds, as explained in section

4.2.2. Figures 6.8 (precision), 6.9 (recall) and 6.10 (f-measure) compare the in- ference results of using no time, absolute time and absolute time plus extended evidence. Looking at precision, three situations improve, two remain the same, and ’toileting’ situation precision is reduced. Extended time evidence has lim- ited improvement for the ’toileting’ activity because it has the same sensor ac- tivations as ’showering’ situation but does not benefit from an absolute time. Recall for all situation occurrences that use transitory evidence improves be- cause gaps between evidence have been removed. Therefore, annotated times- lices with no sensors triggering can be correctly recognised. Using f-measure, when time extension of transitory evidence is also used, recognition accuracy improves for four out of the five enduring situations. Time extension is not used for ’leave house’ and ’go to bed’ situations, and as expected their infer- ence accuracy is almost identical. For the remaining five time-extended situ- ations, the biggest improvements is shown in ’take shower’, ’prepare break-

0.00 0.20 0.40 0.60 0.80 1.00

Leave house Toileting Showering Sleeping Breakfast Dinner Drink No time Absolute time Time Extended

Situations

Figure 6.9: Recall for 1) no time, 2) absolute time and 3) absolute time and time extended evidence using van Kasteren’s data set

0.00 0.20 0.40 0.60 0.80 1.00

leave house use toilet take shower go to bed prepare breakfast

prepare dinner

get drink

No time Absolute time Time Extended

Situations

Figure 6.10: F-measure for 1) no time, 2) absolute time and 3) absolute time and time extended evidence using van Kasteren’s data set

Situations leave house use toilet take shower go to bed prep break’t prep dinner get drink leave house 0.829 0.003 0.001 0.165 0.0 0.001 0.001 use toilet 0.137 0.673 0.077 0.108 0.0 0.003 0.0 take shower 0.094 0.123 0.701 0.082 0.0 0.0 0.0 go to bed 0.037 0.025 0.002 0.936 0.0 0.0 0.0 prep break’t 0.172 0.091 0.035 0.0 0.557 0.024 0.121 prep dinner 0.245 0.038 0.0 0.0 0.0 0.624 0.093 get drink 0.034 0.069 0.0 0.023 0.0 0.103 0.771

Table 6.5: Confusion matrix on van Kastersen’s data set including absolute time and time extended evidence

No Time Absolute time Time extended and absolute

F-measure 0.40 0.55 0.68

Table 6.6: Comparison of average f-measure for evidence decision network with no time, absolute time and time extended

fast’ and ’prepare dinner’. These activities are longer in duration than the ’get drink’ and ’use toilet’ situations, so their evidence is sparser throughout the duration with longer gaps where nothing happens. Therefore, they benefit more from the extension of their transitory evidence. The ’use toilet’ situa- tion recognition actually decreases very slightly with the use of time-extended evidence. This is because the sensors used in ’use toilet’ overlap with those for ’take shower’ and the two situations were often performed sequentially. Looking at the confusion matrix in Table 6.5, the extent to which the enduring situations are confused with ’leave house’ and ’go to bed’ has reduced. i.e. the static periods where nothing happens is reduced.

The impact of the evidence decision network with temporal extensions is sum- marized in table 6.6. This shows average F-measure for all situations when no time is used in reasoning, when absolute time is used, and when both time ex- tension and absolute time are used. F-measure improves by 70% with the use of both time reasoning techniques. The data set has strong time patterns so we expected the inclusion of this temporal knowledge to improve our results. Our hypothesis is that the evidence decision network with temporal exten- sions improves inference results over the evidence decision network only. To check whether the difference in our results is significant, we construct Mc- Nemar’s contingency table as described in Section 6.2.2. The table, shown in Appendix A, Section 1, contains the classification result totals for the evidence decision network only versus the evidence decision network using absolute

time and time extended time. Using McNemar’s equation 6.4, we find that the difference between the evidence only and temporal evidence classification approaches were statistically significant to the 99.99% level.