5.5 CASL Data set
5.5.5 Situation DAG for CASL data set
The situation DAG, established using the seven steps described, is show in figure 5.5. The situations are derived from the various abstracted contexts of the sensors. There is a particular reliance on the location of the user, as user location is richly informative of their situation, telling us if the user is in the cafe, or at their desk and so on. Both coffee and lunch breaks take place in the cafe. The only detectable difference between coffee and lunch breaks is the time of day that lunch break takes place between 12:00 and 14:00. So we will rely on absolute time of day to differentiate them. In this data set, there is no inference rule frequency used, as each rule is categorical. For example, the user is “always” at their desk when in the situation ’busy at desk’.
5.6
Conclusion
In this chapter, we demonstrated how situation DAGs are established for two real world datasets. This is a critical step in applying the extended Dempster- Shafer reasoning approach because the DAG contains the structure and knowl- edge needed for the evidence decision network. i.e. to process the belief distri- bution and decision stages for an envrionment. We noted that knowledge for
the DAG may come from a variety of sources: system developer knowledge, user interviews, user observation, application stakeholders, localised training data and data mining knowledge. In practice, we anticipate that a hybrid ap- proach will be used, with knowledge from different sources populating differ- ent parts of the situation DAG.
The situation DAGs for our two data sets establish the evidence processing needed for the evaluation in Chapter 6. The smart home data set contains the temporal patterns needed to evaluate our extensions for transitory evidence. As this is a third party data set, we used a portion (one third) for training data in order to establish the DAG in the absence of detailed domain knowledge for the smart home environment and user. The in-house office data set has sensor quality parameters that we can use to evaluate our quality extensions. We have expert domain knowledge about workings and performance of the sensors, and their roles in tracking situations. We established the situation DAGs for both data sets.
In the next chapter, we will use our data sets and situation DAGs for evaluating our evidence decision network.
CHAPTER
SIX
Evaluation
In the previous chapter, we demonstrated the use of our approach by estab- lishing the situation DAGs for two separate data sets. In this chapter, we will use the situation DAGs as input to the evidence decision network for each of the two data sets. We will assess the accuracy of situation recognition in order to evaluate the evidence decision network approach to situation recognition. In Chapter 1, we hypothesised that
(1) the use of temporal knowledge in evidence will improve recognition accuracy (over using evidence only);
(2) the use of sensor quality in evidence will improve recognition accuracy (over using evidence only).
Temporal extensions to evidential theory are evaluated using van Kastersen’s data set. Quality extensions are examined using the CASL data set. The exper- iments conducted test the precision and recall of using evidential reasoning for situation recognition, both with and without our time and quality extensions. In Section 6.1, we explain the belief distribution and decision algorithms used for situation recognition in the evidence decision network. The evaluation methodology is described in Section 6.2. The results of temporal extensions using van Kasteren’s data set are described in Section 6.3. Section 6.4 describes the quality extensions evaluation using the CASL data set. Section 6.5 exam- ined how alternate fusion rules impact situation recognition results for the two data sets. A summary and discussion is provided in Section 6.6.
6.1
Situation recognition using the evidence de-
cision network
Situation recognition using the evidence decision network is split into two parts, as explained in Chapter 3: belief distribution and decision level. In chapters 3 and 4, we explained the evidential operations that support the distribution of belief and decision stages. In this section, we explain the algorithms that we have created to process belief distribution and decision stages.These are re- quired to support the implementation of evidence processing in the evidence decision architecture as shown in figure 3.2.
Belief distribution algorithm: The algorithm for belief distribution is shown in figure 6.1. Belief from sensors is distributed at regular time intervals. Each mass function is executed, populating belief levels for all context values in the DAG. The belief from each context event is then propagated to compatible upper nodes for each context event. Once belief for all context events has been propagated upwards, belief for each upper node is fused and then propagated to the next upper node(s). The propagation and fusion process continues until all nodes in the DAG have been processed.
Time extension of evidence algorithm: If frames exist that are deduced from transitory evidence, the life time of the evidence will be extended to tie in with the duration of the frame. For the duration of the frame, all occurring and extended evidence is considered. The algorithm for extending evidence lifetime assigns the duration or remainder of the duration for the situation to the occurring evidence, as shown in figure 6.2.
Decision algorithm: Once all belief has been processed, the decision algo- rithm is applied. The first step is to distribute belief where belief is assigned to combinations of situations, as explained in Section 3.4.3. The distribution is achieved using Smets modified decision rule in equation 3.12. If only one situation is allowed to occur in the environment at any point in time, the algo- rithm for single situation occurrence is executed, as shown in figure 6.3. This algorithm selects the situation with the highest belief. If two or more situa- tions have the same maximum belief, the algorithm will check which situation exhibits greater certainty.
For environments where more than one situation can occur at the same time, a threshold belief can be used to filter situations, as described by Loke [69] and Clear [16]. In these environments, the decision algorithm for situation co-
input: Sensor readings R from sensors S at time t, situation DAG, frames of
discernment F, valid situation combinations S
output: occurring Situation(s)
foreach sensor S do
execute mass function
if quality used
modify belief distribution
end end
foreach context event foreach upper node
if upper node has duration
execute time extension for context event
end
if inference rule uncertain
execute propagate uncertain belief to compatible node
else
propagate certain belief to compatible node
end end
foreach upper node
process OR (using Max)
fuse AND // including time extended belief
end
Execute decision algorithm Return occurring situation(s)
Figure 6.1: Belief distribution algorithm
// for a situation S that is part of a frame with duration d, with context event evidential of S:
if situation S not in progress
// duration d = remaining duration context event lifetime = d; remaining duration = d;
else
if situation S in progress
if context event has belief > 0
context event lifetime = remaining duration;
end end end
remaining duration: = remaining duration – timegap; // timegap = time between timeslices
input: situations S to be detected output: occurring situation, O.
// distribute combined belief
foreach combined belief allocation
apply modified Smets rule;
end
// apply any absolute times used for situations
foreach situation
if absolute time of situation not= current time
exclude situation from candidate list
end
// determine which situation(s) occurring by finding max belief of remaining situations //
if number of situations with max belief = 1
O = max situation
else
if number of situations with max belief > 1 // tie
find minimum uncertainty of these situations
if number of situation with minimum uncertainty >1
return default of no decision
else
O = minimum uncertainty situation
end else
// there was no max, belief is zero return default or no decision
end end
input: situations S to be detected, invalid situation combinations, belief
threshold b.
output: occurring situations, O
// distribute combined belief
foreach combined belief allocation
apply modified Smets rule;
end
// apply any absolute times used for situations
foreach situation
if absolute time of situation not= current time
exclude situation from candidate list
end
// apply a belief threshold and remove invalid co-occurrences
select situations with belief > t
if invalid situation combination exist in selected situations:
sort selected situations from lowest to highest belief
while (more situations or invalid combinations complete) if situation is part of occurring invalidation combination
remove situation from selection
check if invalid combinations complete
end end end
return situations in selection
Figure 6.4: Decision algorithm for situation co-occurrence
occurrence is executed, as shown in figure 6.4. This algorithm must consider the invalid combinations of situations so that only situations that can occur to- gether are returned. This is done by dropping situations with the lowest belief, if they are part of an invalid combination, where all the situations in the combi- nation have exceeded the threshold. Invalid combinations can be hand crafted, or can be detected using an automated process, based on Ye’s approach [119]. This work defines conflicting context values, which cannot co-occur, such as ’having a shower’ in the bathroom location cannot happen at the same time as ’eating dinner’ in the kitchen location. By checking which context values con- flict, situations that are impossible to occur together can be determined. The setting of the threshold will be environment specific. If applications execute high risk behaviour, the threshold should be set to prevent situations with in- sufficient belief levels [69]. Also, the quality of the sensors and the certainty of the inference rules will determine the level of belief that is possible for sit- uations to be achieved in the first place. Highly uncertain environments will need to set lower thresholds in order to detect situations.