Smart Environment Uncertainty Factors Modelling and Reasoning

The process of developing PR-OWL model is to define an uncertainty variable as priori probability with MEBN fragments (MFrags) and complex MFrags groups to create MEBN theory (MTheory). The joint probability distribution of MTheory and MFrags allow creating situation-specific Bayesian network (SSBN) for each activity. Therefore, upon receiving a piece of evidence from SH, SSBN can be created, and probabilistic queries can be performed to determine the likelihood of an event/activity occurring. For instance, the goal is to determine if a sensor (S1) is sending faulty reading based on S1’s performance attributes. The priori probability of S1’s attributes can be defined in MFrags: battery life can be monitored, duration of sensor active, number of wireless sensors on the same frequency, prone to damage due to human consumption, manufacture sensor error rate. The evidence for S1’s attributes can be added to SSBN and joint probability can be calculated to determine if S1 is faulty.

The common uncertainties caused in HAR are by the use of the everyday object (𝑜𝑏𝑗𝐹𝑎𝑐𝑡𝑜𝑟𝑠), human factors (ℎ𝑢𝑚𝑎𝑛𝐹𝑎𝑐𝑡𝑜𝑟𝑠), technology-based (𝑡𝑒𝑐ℎ𝐹𝑎𝑐𝑡𝑜𝑟𝑠) and environmental factors (𝑒𝑛𝑣𝐹𝑎𝑐𝑡𝑜𝑟𝑠) are described using probabilistic theory, see equation 5-1. The 𝑜𝑏𝑗𝐹𝑎𝑐𝑡𝑜𝑟𝑠 are those that can hinder the functionality of an object, i.e., due to wear and tear (𝑤𝑒𝑎𝑟𝐴𝑛𝑑𝑇𝑒𝑎𝑟𝑠) and manufacture defects (𝑑𝑒𝑓𝑒𝑐𝑡). The ℎ𝑢𝑚𝑎𝑛𝐹𝑎𝑐𝑡𝑜𝑟𝑠 considered when conducting ADLs are accidents such as spillages of content or dropping the object with the content mid-action and missing out key actions. The evidence of spillage or drop is detected using IMU sensors when an object goes into freefall mode. The spillage or drops can occur in individuals with tremor, weak grips due to clumsiness or conditions (i.e., such as arthritis, tendinitis, and repetitive stress injuries). Another ℎ𝑢𝑚𝑎𝑛𝐹𝑎𝑐𝑡𝑜𝑟𝑠 is the individual suffering from memory loss may forget to perform key actions based on the severity of their memory functions. This medical information about an individual can inform the knowledge engineer to predefine belief values and personalised the system. Furthermore, despite strategically positioning the sensor (𝑠𝑃𝑜𝑠ℎ) on the object (𝑂𝑏𝑗𝑐), individual may hold the object in incorrect

orientation or outside the reading range of the sensor (i.e., capacitive touch or fingerprint sensor). Similarly, several 𝑡𝑒𝑐ℎ𝐹𝑎𝑐𝑡𝑜𝑟𝑠 create uncertainties and reliability/trust issues with the data received from the noisy sensor network with the varying communication protocol. The wireless sensors often operate using batteries which can be consumed depending on the frequency of use and may provide false reading with a low battery level. Likewise, the

𝑒𝑛𝑣𝐹𝑎𝑐𝑡𝑜𝑟𝑠 such as fire, flood, room temperature and humidity can have a severe impact on the operating conditions of the sensing devices.

𝜙 = { 𝑜𝑏𝑗𝐹𝑎𝑐𝑡𝑜𝑟𝑠[𝑑𝑒𝑓𝑒𝑐𝑡𝑠, 𝑤𝑒𝑎𝑟𝐴𝑛𝑑𝑇𝑒𝑎𝑟𝑠, . . . ], ℎ𝑢𝑚𝑎𝑛𝐹𝑎𝑐𝑡𝑜𝑟𝑠[𝑂𝑏𝑗𝑐[𝑠𝑃𝑜𝑠ℎ], 𝑎𝑐𝑡𝑖𝑜𝑛[𝑑𝑎𝑚𝑎𝑔𝑒, 𝑎𝑐𝑐𝑖𝑑𝑒𝑛𝑡𝑠, … ], … ], 𝑡𝑒𝑐ℎ𝐹𝑎𝑐𝑡𝑜𝑟𝑠[𝑓𝑎𝑢𝑙𝑡𝑦𝑆𝑒𝑛𝑠𝑜𝑟, 𝑙𝑜𝑤𝐵𝑎𝑡𝑡𝑒𝑟𝑦, 𝑛𝑒𝑡𝑤𝑜𝑟𝑘𝐷𝑒𝑙𝑎𝑦, 𝑛𝑜𝑖𝑠𝑒, … ], 𝑒𝑛𝑣𝐹𝑎𝑐𝑡𝑜𝑟𝑠[ 𝑓𝑖𝑟𝑒, 𝑓𝑙𝑜𝑜𝑑, 𝑡𝑒𝑚𝑝, ℎ𝑢𝑚𝑖𝑑𝑖𝑡𝑦, … ], … } 5-1

5.3.1.1. Probabilistic Ontology Modelling

To model these uncertainties, PR-OWL 2 is leveraged to captures these four types of factors in MEBN. These four types factors described in Table 4.5 are ℎ𝑢𝑚𝑎𝑛𝐹𝑎𝑐𝑡𝑜𝑟𝑠 (A), 𝑜𝑏𝑗𝐹𝑎𝑐𝑡𝑜𝑟𝑠 (B),

𝑡𝑒𝑐ℎ𝐹𝑎𝑐𝑡𝑜𝑟𝑠 (C) and 𝑒𝑛𝑣𝐹𝑎𝑐𝑡𝑜𝑟𝑠 (D).

PR-OWL 2 extends BN with FOL to create a MEBN logic for model complex knowledge. MEBN defines probabilistic knowledge as a set of MFrags to develop a minimum of one MTheory. MFrag contains four types of random variable (RV) nodes: resident, input, ordinary variable and context. MFrag containing RVs and their belief tables with probabilities make up the MTheory. The resident node is a yellow rounded rectangle node that consists of

RVs to form the core subject of the MFrag. The arcs pointing to resident nodes create conditional arcs and signify probabilistic dependence. The input node is a grey trapezoidal- shaped used for building relationships between the resident node from multiple MFrags. The input node can only point to other RVs but not to itself or from other RVs. Similarly, ordinary variable and context node are in green colour with a pentagonal shape. The ordinary variable node contains a variable or an instance of a class described in the ontology. The context holds Boolean RVs representing conditions (defined in first-order-logic (FOL) formulae) which must be fulfilled for the distributions defined in the MFrag to be valid. A context node cannot have any arcs pointing to or from it.

Table 5.1. Overview of Uncertainty Types Considered when Modelling ADL Knowledge Base. Type Uncertainties Description Evidential theory

A Accidents: Spillage /Drop Object free fall detection

A Missing key actions Mandatory/optional events and identify dependencies using Allen’s temporal rules

B Utility device breakdown: caused by wear and tear, leading to incomplete actions.

Device functional status, main power supply status (if applicable), under warranty (durability), fragility in use level

C Sensor failures: manufacturing defect, battery low, maintenance, out of range

Maintenance report: last battery change, estimated power consumption

D Undesirable operating conditions causing sensor damage/failure

Water-related activities, the brute force required, incorrect temperature

D Frequency noise, network Number of radio-frequency devices, the magnetic field D Uncontrolled events: power cuts,

storms, earthquakes

Power cut, storm and earthquake statistics in the area.

Note: Uncertainty types: Human error (A), Object-based (B), Sensor-based (C). Environmental (D)

Figure 5.4. An example of MakeTeaADL MFrag comprising of uncertainties when detecting Kettle pouring action to CeramicCup.

Resident Node Context Node Ordinary Variable Input Node

Figure 5.4 presents an example of MakeTeaADL MFrag consisting of four types of RVs to define human factors and technology-related uncertainties when estimating kettle pouring action to the cup. The ordinary variables or instance of the OWL class are initially created, which can be used as a parameter by the context and resident nodes. In this case, instances of

Kettle, Person, CeramicCup, Parkinson and Alzheimer classes are added to MFrag. The context

nodes are object or data properties (hasParkinsonDisease and hasAlzheimerDisease) defined in another HumanFactor MFrags resident nodes and linked to MakeTeaADL MFrag as context nodes. Similarly, input nodes in MakeTeaADL MFrag are TI Sensor Tag and ESP microcontroller battery level are linked with Technology MFrags. The hasPourAction resident node has arcs from the four input nodes and three ordinary variables as parameters. The local probabilistic distribution values for hasPourAction resident node can be defined with nested if- else conditions of four input nodes as defined in Figure 5.5. This nested if-else condition checks if the BLE TI SensorTag and ESP microcontroller’s battery levels at the first level using

bleBatterySensor and espBatteryLevel ordinary variables. Based on the state of the two sensor’s

battery level variables, i.e., if the battery value is low or empty, second-level nest conditions are executed which checks if the person has Alzheimer or Parkinson diseases. In essence, this nested if-else condition gives a higher probability for the user pouring action successfully if there are no known diseases and the sensor battery levels are not low or empty.

Figure 5.5. Editing hasPourAction probability table based on the known disease of the user and sensor battery levels.

Consequently, MFrags can be created for other four uncertainty factors and ADL of interest. The probabilistic distribution of four types of uncertainty is currently pre-defined. However, with more data over time and online/offline activity learning algorithm, these probabilities can be dynamically updated. In this approach, we propose attributes for abductive reasoning and user feedback mechanism to update the probability distribution table (more details in section 5.3.2). The evidence collated from the SH environment is assessed for uncertainties and provided to the SSBN to create/update belief tables for a given ADL dynamically. The input RV in BN requires a crisp input of an imprecise sensor data (i.e., battery level low, medium and high). Hence, fuzzyDL modelling and reasoning results from fine- grained action level AR (discussed in CHAPTER 4) are used before providing the input to the SSBN model. The effects are propagated with other belief tables in the BN. The propagation results will show the overall estimation of the ADL occurring based on known uncertainties.

5.3.1.2. Progressive Propagating Evidences

As the sensor observations are received, data are filtered and combined with fuzzy reasoning, an SSBN is dynamically created and updated with more evidence for each ADL. The effects are then propagated with all the belief tables in BN in order to calculate the probability of the ADL occurring. There are two types of propagation methods, diagnostic (backward) and prognostic (forward) reasoning. The diagnostic reasoning is traditionally used for decision-making to identify the root cause of the failure based on symptoms or pieces of evidence collected from the SH environment. Therefore, the diagnostic approach is used to enter pieces of evidence collected based on how user’s actions from the SH and AR results to calculate the overall effect in recognising ADLs. In contrast, the prognostic reasoning is concerned with entering evidence about the causes and predicting the likelihood of the future outcome. For example, if the sensor battery is low, the next set of sensor data may be unreliable due to a higher probability of error in data measurements and loss in data packets during transmission. Consequently, the prognostic approach is used to perform offline or online monitoring of not only how technology is responding but also the other factors that enable us to answer or predict the future of a given event occurring. Therefore, the prognostic reasoning process is responsible for updating the probabilistic distribution table for resident nodes in the MEBN knowledge model and diagnostic reasoning to add evidence to SSBN created dynamically.