Imprecise Knowledge Modelling

The core element of fuzzy OWL is Fuzzy Logic. Fuzzy logic is based on the theory of fuzzy sets proposed by L. Zadeh[168] to support imprecise and vague knowledge. The fuzzy set theory enables imprecise sensor value to be assigned as a member of a given set with a membership degree between 0 and 1 for a Type-1 fuzzy set. In comparison to the classical set theory, elements are either part of a given set or not, i.e. 0 (false) or 1 (true). The Type-2 fuzzy set introduces secondary membership functions where upper, and lower membership boundaries are created when it is challenging to recognise simple fuzzy membership value for fuzzy terms/concepts. The region between the upper and lower membership boundaries is called the footprint of uncertainty[172]. The fuzzy ontology allows Type-1 and Type-2 fuzzy sets membership boundaries to be defined using Datatype annotations and minValue/maxValue

attributes[192]. d → left (k1, k2, a, b) right (k1, k2, a, b) triangular (k1, k2, a, b, c) trapezoidal (k1, k2, a, b, c, d) mod (d)

mod → linear (c), triangular (a, b, c) (k1 = minimum, k2 = maximum values) Figure 4.3. Type-1 fuzzy membership functions and modifiers

FKB formally conceptualises imprecise sensor data and fusion of multiple sensors as rules to determine the completion of a given fine-grained action to satisfactory thresholds. There

are three critical steps in developing a fuzzy ontology; (1) fuzzification, (2) rules and inferring system, and (3) defuzzification.

In the fuzzification step, the vague sensor data sets are described as fuzzy concepts (ℱ𝒞𝑏)

with fuzzy membership functions (d) and modifiers (mod) defined in Figure 4.3. The membership functions are trapezoidal, triangular, left(-shoulder), right(-shoulder), crisp interval, and linear. The modifiers are linear and triangular. Table 4.1 presents a fragment of the three types of imprecise sensor data as a Type-I fuzzy concept; temperature, liquid and position (IMU) data. The fusion of multimodal sensor data attached to a given object is considered to increase the accuracy of the fine-grained kettle “pouring” hot water action detection. The temperature sensor values are associated with linguistical concepts such as “hot”, “cold” and “normal” which is often subjective to a given context or person. Similarly, the liquid level sensor enables one to categorise if a container in varying size/dimension is “full”, “half full” or “empty”. Whereas, IMU sensors (i.e., accelerometer, gyroscope) enables one to understand the position of the object and how it has moved in three-dimensional space. Therefore, combining these three parameters, fuzzy rules can be created to define how much one needs to tilt the container to “pour” hot water into another container with respect to the liquid level. Likewise, other fine-grained actions defined in FKB are “filling up”, and “drinking” from the container (i.e., cup or kettle) can be.

The fuzzy rules are constructed mainly with Mamdani and Takagi/Sugeno approaches[166]. The fuzzy rules are constructed with IF (antecedent) and THEN (consequent) statements. Table 4.2 illustrates partial fuzzy rules for a kettle to infer pouring state based on the liquid level, object temperature, and gyroscope Z-axis position. Table 4.2 presents three sets of rules specifying scenarios in which pouring action is incomplete, unsure and complete with respective PouredRed, PouredOrange, PouredGreen flags. The first set of rules are for incomplete pouring action scenarios. The rule, rule_kettle_empty, states that if the kettle’s

liquid level is empty (some liquidLevel kettle_Liquid_Empty_ls) then poured status flag is red (some pouredStatus PouredRed). Likewise, the second set of rules define two scenarios where it is unsure if the pouring action has been completed. First rule, rule_kettle_objTemp_warm, states that if kettle’s object temperature is warm then poured status flag is orange. The second rule, rule_high_veryHot_water_zhigh_midHigh, state if the liquid level is high (some hasLiquidLevel kettle_Liquid_High_rs), object temperature is warm, and tilt threshold is medium to high (some hasAccelerationZ ket tle_gyro_z_pour_thres_liquid_midHigh_tri) then the poured status is orange (some pouredstatus PouredOrange).

Table 4.1. Fragment of Fuzzy Concepts, Roles, and Membership functions for Multimodal Kettle “Pouring” Action Rules in FuzzyDL Syntax

% 1) Membership functions for fuzzy concepts

(define-fuzzy-concept kettle_Liquid_Empty_ls left-shoulder (0.0, 100.0, 14.47, 15.5)) (define-fuzzy-concept kettle_Liquid_Minimum_tri triangular (0.0, 100.0, 15.0, 16.25, 16.5))

(define-fuzzy-concept kettle_Liquid_Low_tri triangular (0.0, 100.0, 16.35, 16.83, 17.0))

(define-fuzzy-concept kettle_Liquid_Medium_tri triangular (0.0, 100.0, 16.85, 17.47, 18.0))

(define-fuzzy-concept kettle_Liquid_MediumHigh_tri triangular (0.0, 100.0, 17.75, 18.12, 18.6))

(define-fuzzy-concept kettle_Liquid_High_rs right-shoulder (0.0, 100.0, 18.5, 18.68)) (define-fuzzy-concept kettle_objTemp_veryhot_rs right-shoulder (-150.0, 150.0, 75.0, 100.0))

(define-fuzzy-concept kettle_objTemp_hot_tri triangular (-150.0, 150.0, 50.0, 70.0, 80.0))

(define-fuzzy-concept kettle_objTemp_warm_tri triangular (-150.0, 150.0, 35.0, 45.0, 60.0))

(define-fuzzy-concept kettle_gyro_z_pour_thres_liquid_ min_tri triangular (-50.0, 50.0, 10.0, 15.0, 17.0))

(define-fuzzy-concept kettle_gyro_z_pour_thres_liquid_ midHigh_tri triangular (-50.0, 50.0, 2.0, 5.0, 8.0))

(define-fuzzy-concept PouredGreen left-shoulder (0,9,1,3)) (define-fuzzy-concept PouredOrange triangular (0,9,3,4.5,6)) (define-fuzzy-concept PouredRed right-shoulder (0,9,8,9)) …

% Fuzzy Relationships between Concepts

(functional hasObjectTemperature)

(range hasObjectTemperature *real* -150 150) (functional hasGyroscopeZ)

(range hasGyroscopeZ *real* -100 100) (functional pouredstatus)

(range pouredstatus *real* 0 9) …

Table 4.2. Partial Kettle “Pouring” description using FuzzyDL Rules % 2) Multimodal sensor rules for kettle pouring action

% 2.1) PouredRed - pouring incomplete rules

(define-concept rule_kettle_empty (g-and (some hasLiquidLevel kettle_Liquid_Empty_ls) (some pouredstatus PouredRed))) …

% 2.3) PouredOrange – pouring potentially completed

(define-concept rule_kettle_objTemp_warm (g-and (some hasObjectTemperature kettle_objTemp_warm_tri) (some pouredstatus PouredOrange)))

(define-concept rule_high_veryHot_water_zhigh_midHigh (g-and (some hasLiquidLevel kettle_Liquid_High_rs) (some hasObjectTemperature kettle_objTemp_veryhot_rs) (some hasGyroscopeZ kettle_gyro_z_pour_thres_liquid_midHigh_tri) (some pouredstatus PouredOrange))) …

% 2.4) PouredGreen – pouring successfully completed

(define-concept rule_midHigh_veryHot_water (g-and (some hasLiquidLevel

kettle_Liquid_MediumHigh_tri) (some hasObjectTemperature kettle_objTemp_veryhot_rs) (some hasGyroscopeZ kettle_gyro_z_pour_thres_liquid_midHigh_tri) (some pouredstatus PouredGreen))) …

% 2.5) Combining all kettle sensor states

(define-concept rulePOURING (g-or rule_kettle_empty rule_kettle_objTemp_warm rule_high_veryHot_water_zhigh_midHigh rule_midHigh_veryHot_water …)

Table 4.3. Example of MOM defuzzification Query results of pouring rules in four scenarios with Multimodal sensors input data from the Kettle

% 3) Input – We consider four scenarios (S1-S4) %S1)liquidLevel=empty, objTemp=hot,

gyroZ=static

(instance run1 (= hasLiquidLevel 10)) (instance run1 (= hasObjectTemperature 80))

(instance run1 (= hasGyroscopeZ 2)) % output ==> pouredstatus = 9.0 (red) %S2)liquidLevel=empty, objTemp=warm, gyroZ=static

(instance run2 (= hasLiquidLevel 10)) (instance run2 (= hasObjectTemperature 45))

(instance run2 (= hasGyroscopeZ 2)) % output ==> pouredstatus = 6.75 (orange)

%S3)liquidLevel=high, objTemp=veryhot, gyroZ=midhigh

(instance run3 (= hasLiquidLevel 20)) (instance run3 (= hasObjectTemperature 75))

(instance run3 (= hasGyroscopeZ 5)) % output ==> pouredstatus = 4.5 (orange) %S4)liquidLevel=midhigh,objTemp=veryhot, gyroZ=midhigh

(instance run4 (= hasLiquidLevel 18)) (instance run4 (= hasObjectTemperature 75))

(instance run4 (= hasGyroscopeZ 5)) % output ==> pouredstatus = 1.0 (green)

Equally, the completion rule, rule_midHigh_veryHot_water, states that if the kettle’s

liquid level is medium to high (some hasLiquidLevel

kettle_Liquid_MediumHigh_tri), object temperature is very hot (some hasObjectTemperature kettle_objTemp_veryhot_rs), and minimum threshold tilt

degree registered (some hasAccelerationZ

kettle_gyro_z_pour_thres_liquid_midHigh_tri), then poured status is green (some pouredstatus PouredGreen). Lastly, all the other possible combinations of the sensor status and scenarios are added to the main pouring rule (rulePOURING) concept to determine which rule is best matched for a given sensor input in the defuzzification step.

The final defuzzification step consists of using the sensor input values and fuzzy rules to query a membership value for a given action. The conventional defuzzification methods available are Centroid Of Area (COA), Bisector Of Area (BOA), Mean Of Maximum (MOM), Smallest Of Maximum (SOM) and Largest Of Maximum (LOM)[166].

Figure 4.4 illustrates MOM defuzzification results of the four scenarios with the multisensory data of a kettle and whether the pouring task is incomplete (S1), unsure if completed (S2-S3) or completed(S4). In the scenario S1, instance of kettle containing values of liquid level 10pF (empty), object temperature 80Celsius (very hot) and gyroscope z-axis value to be 2 (the threshold for pouring when liquid level medium) has been defined with pouredStatus MOM defuzzification output as 9.0 (PouredRed). Equally, in S2, change of object temperature to 45 Celsius resulted in MOM defuzzification value to be 6.75 (PouredOrange). On the other hand, scenario S3 containing liquid level 20 (high), object temperature 75 Celsius (very hot) and gyroscope z-axis value 5 (the threshold for pouring

when the liquid level is medium-high) resulted in 4.5 (PouredOrange). Likewise, the last scenario S4 result in 1.0 (PouredGreen) as the kettle’s liquid level is 18pF (medium-high), object temperature is 75 (very hot), and the gyroscope Z axis is 5 (the threshold for pouring when the liquid level is medium-high).