In order to evaluate to what extent the system is able to predict the user’s ratings of trust based on her ratings of relevant trust factors (transparency, controllability, comfort of use, seriousness, credibility and security), a model of BN was created using the Genie built-in algorithm for learning Bayesian Networks (see http://genie.sis.pitt.edu/). 5
Overall, 4 different networks were constructed from the data received in the online studies, one for each scenario shown in the Table 7.2. The first two scenarios were combined into one network. While the basic structure was shared by all networks and was similar to the example network shown in Fig. 7.3, each had different context nodes and display reactions.
Since the evaluations contained quantitative data (rankings of the users), they enabled us to derive distributions of each trust dimension for each contextual combination. For each trust dimension, the author modeled the probability distribution for all combinations of context and display reaction in the BN after the data taken from the online study. The probability distributions for other node combinations, that were not part of the data inquired in the studies (e.g. how Confidence and Competence influence Trust Disposition) were modeled after the results reported in the study [17].
Although the users’ were asked for the preferred display reaction for each context combination as well as their trust in the display reaction presented in each situation, it should be noted that this information was not used to model the network. Instead, the data on user trust estimations and user preferences was used to validate the decisions which were automatically generated by the BN. For the validation of each created network, decisions were generated for all contextual combinations used in the matching scenario of the web-based study. The resulting decisions of the networks were compared to the results from the web- based study. Moreover, the decisions were compared with the user preferences and the estimation of trust.
The contextual combinations were set by entering appropriate evidence into the context nodes, leaving out one node as uncertain. For example, for a specific situation in the Scenario 1, the evidence would be set to “Privacy of Data → Private”, “Movement → Arriving” and “Others Present → uncertain”. Such combination reproduces a real-life situation: often the sensors fail to recognize the presence of the spectators.
For each combination of context, the display reaction with the highest utility rating was chosen as the system’s decision. It referred directly to the computed value of User Trust.
First, the BN generated reactions were compared with user preferences. The comparison was performed between the rankings of the adaptive actions. In other words, the successful result would be the match of the first user-ranked action and the first action generated by the BN.
For each context combination, we selected the display reaction that received the highest average score in the web-based study. When comparing the display reactions preferred by the users with those generated by the respective network, they matched in 19 out of 22 cases (86.36%). In the three mismatching cases, the generated decisions matched the users’ second preference.
Second, the BN generated reactions were compared with the users’ estimations of their trust. The comparison was performed in the same way as the preferences comparison: the rankings given by the users and the rankings generated by BN were compared. The user rankings were taken as an average of all rankings for the current scenario. The comparison yielded the match in 16 out of 22 cases (72.73%). In two of the mismatching cases, the BN met the second ranking of user trust. In one mismatching case – the third ranking of user trust. Finally, in three cases, the participants’ trust ranked two reactions at the exact same position, while the network assigned them slightly differently values, thus preferring one over the other.
These results show that the BN delivers good accuracy in the generated decisions. As an example of this validation, let us take a look at the BN for Scenarios 1 and 2 (see Table 7.2) and its context combinations. Table 7.3 illustrates the matches between the decisions generated by the BN, user preferences, and user estimations of the highest trust. Note that “Adapt” stands for automatic adaptation, “Show data” for “Arriving” context, and “Remove data” for “Leaving" context.
Context BN Reaction Preference Highest Trust
Private Arriving Alone do nothing ask via mobile do nothing
Private Arriving Spectators ask via mobile ask via mobile do nothing
Private Leaving Alone adapt adapt ask via mobile
Private Leaving Spectators adapt adapt ask via mobile
Table 7.3. Comparison of the BN generated decisions, user preferences, and user estimations of the highest trust. Example of Friend Finder, Scenario 1 and 2.
It is important to emphasize that the highest user trust does not always match the mostly preferred decision. The results of our web-based and live study support this fact: distributions of user preferences did not always reflect distributions of trust. From the comments of the live
study participants, the author found that the feeling of trust often depends on the person’s ability to explain the system reaction and agree with it. For example, when a person comes closer to the display, it seems logical and expected that the display does not show any reaction. We learn this behavior from the everyday life: the objects in the interior usually do not react. Obviously, the option “Do nothing” therefore received highest trust rankings. However, the most understandable reaction might not be the most preferred or the most convenient one. Here, the more intelligent (but less expected) reactions were favored. For example, the users found it smart and convenient that the display noticed them and proposed via a mobile device to show their data on the large screen. Thus, the “Ask via mobile device” option was chosen as a preference.
7.5 Summary
This chapter described the mechanism for automatic decision making for adaptive public displays. The mechanism based on probabilistic model, Bayesian Network, enables to estimate user trust for different contextual situations. In a public display environment with a changing social context, user trust is a valuable resource which is critical for user acceptance and satisfaction of the system.
The constructed Bayesian Network was initialized with empirical data collected in web- based study and controlled life experiments. The chapter illustrated the process of the network construction, data collection, and validation of the network.
The presented work aims to inform designers of adaptive displays in the approach to generate adaptation decisions automatically. Trained with empirically collected data, the network can automatically generate decisions on adaptation, best fitting to the current social context. Moreover, the network is able to handle situations when surrounding context is known only partially. The probabilistic nature of the network enables estimations of the best adaptive actions based on the collected knowledge.