Future Work

Throughout the development of the personalisation approach described in this thesis, a number of issues worthy of further investigation were identied. This section outlines the key areas identied for future work. This work relates to: extending the evaluation, dynamic adaptation of conguration data; utility, temporal, and sequential data; context uncertainty; and user-application scrutiny.

6.2.1 Evaluation Extensions

The evaluation described in the previous chapter highlighted several ndings that are worthy of further investigation. Firstly, it was shown that the extended strategy of the personalisation approach, which adapts information relevance and utility, provides more accurate results for some users than it did for others. An extension of the evaluation would investigate the cause of this, such as an in-depth analysis of the decision making behaviour of individual users to determine when the extended strategy of the personalisation approach is benecial. Secondly, the results of several users across the set of tests did not directly correlate and further tests to assess the accuracy and reliability of the responses from these users about relevance and utility is planned. Other possible evaluation extensions could assess the performance of the personalisation approach for other quality measures as outlined in the previous chapter such as trust, transparency, and eciency/latency of decision making.

6.2.2 Dynamic Adaptation of Conguration Data

The personalisation approach described in this thesis enables developers to explicitly congure the execution of its techniques according to the requirements of the application and the user (Section 4.3). However, these properties are currently required to be statically dened and therefore do not adapt to dierent user problems or context. An interesting direction for future work would be to investigate a means that dynamically adapts this conguration information in order to optimise the eectiveness of the personalisation approach for dierent situations and further minimise the need for explicit human input. As an example, the number of rules generated as part of Implicit Preference Determination could be automatically adjusted depending on the state of the user's device. Also for dierent types of problems, the application may automatically learn an optimal number of rules to compare in order to accurately determine relevant context or automatically customise the number of recommendations presented to the user depending on their context. Further research is necessary to determine if dynamic adaptation of personalisation conguration data would lead to improved recommendation decisions.

6.2.3 Utility, Temporal, and Sequential Data

The personalisation approach described in this thesis, utilises relationships between context and user choices, discovered from past user interactions, to make behaviour recommendations tailored to the user's current problem and context. As discussed in Section 3.2.7, the approach is designed to adjust the utility value inferred for dierent sets of context using information about utility values in past user decisions stored as weights in the user model. However, the current design only the adjustment of utilities based on a pre-dened threshold value (i.e., if the dierence between the inferred utility and utility stored in past cases is greater than this threshold value, then the inferred value is modied so that the dierence no longer exceeds this threshold). An interesting direction for future work would be to investigate techniques that make greater use of utility values stored in the user model with the aim of improving the accuracy of generated recommendations. Specically, a process for mining utility values associated with dierent context types or context sets could be developed using the existing association discovery technique to determine if useful context utility knowledge, that would improve the accuracy of recommendation decisions, can be inferred.

The design of the personalisation approach also does not currently support the identication of temporal or sequential information associated with past user interactions. Temporal and sequential data are seen as useful as they not only capture what action a user takes, but also when those actions are taken and in what order [153, 57, 133]. Therefore, an interesting extension to the personalisation

approach would be to incorporate techniques that support temporal and sequential data, which will facilitate applications to proactively make recommendations (using temporal knowledge about what the users is likely to request next), and improve the accuracy of recommendations that occur in sequence (using sequential data about the order in which users perform tasks). An investigation into suitable techniques for storing and analysing sequential and temporal data (such as sequential data mining [3]) is future work.

6.2.4 Context Uncertainty

The personalisation approach presented in this thesis has been shown to provide accurate personalisa- tion recommendations to users by providing techniques and algorithms that identify a user's preferred behaviour depending on their surrounding environmental context. Context data acquired from the Hermes framework, which are used in the various personalisation techniques and algorithms in the approach, are assumed to be accurate. As a result of this assumption, uncertainties that may exist in acquired context information are not considered as part of the recommendation process. Further research is required to accurately determine the eect of context uncertainty on user preferences and to investigate how the personalisation approach can be extended to mitigate the eect of context uncertainty on recommendation decisions. A number of possible extensions are seen as worthy of future investigation. Firstly, uncertainty associated with context could be incorporated with the de- veloped Information Selection technique so that only information above a specied level of certainty is retained. Secondly, information about uncertainty (e.g., condence values) could be integrated with Utility Assignment so that less certain information is given less weight when ranking candidate choices. A third approach could be to discover patterns between dierent types of context using a similar association determination process as described in Section 3.2.4 to determine those types and values that are likely or unlikely to occur together, with context readings that do not conform to dis- covered patterns considered as unreliable. An investigation into the eectiveness of these approaches may lead to knowledge that would maintain or improve the accuracy of a recommendation in the presence of uncertain data.

6.2.5 User-Application Scrutiny

This thesis provided an approach that supports applications in generating, evaluating, and presenting personalised recommendations to users for dierent problems and context. As part of the current design, user-device interaction is restricted to users making recommendation requests and selecting their preferred choice from a recommended set. However, research has shown that an important

element of user-based applications is the need for users to be able to query or scrutinise the decision making process [27, 180, 16, 34], which facilitates trust and acceptance of recommended behaviour. The extension of the personalisation approach to include techniques that support users to query applications about their decision making behaviour is therefore desirable.

The personalisation approach was designed using techniques that generate and evaluate user pref- erence knowledge in a rules format with the intention that knowledge in rule form would facilitate the integration of developed personalisation techniques and algorithms with new techniques that sup- port user-application scrutiny. Future investigations could focus on possible solutions that analyse and aggregate inferred association rules with the aim of providing an extension to the current design that enables users to query applications for decision making knowledge such as information relevance, utility, and ranking of behaviours.

6.3 Chapter Summary

This chapter summarised the motivation for the research work and the most signicant achievements of the work presented in this thesis. In particular, it outlined how this work contributed to the state of the art in personalised, context-aware computing by providing a personalisation approach that supports the dynamic and implicit determination of user preference, relevant information, information utility, and ranking of candidate behaviour recommendations using information about past context and user interactions. The provisions of techniques and algorithms that minimises the need for explicit user and developer knowledge and input about user preferences eases the deployment of personalised context- aware applications and supports applications with utilising the growing availability of information in context-aware environments. User studies and computer simulations conducted during evaluation showed that the techniques and algorithms developed for this approach are eective in making accurate personalised recommendation to users for dierent problems and context. The chapter concluded with suggestions for future work arising from the research undertaken in relation to this thesis.

Additional Evaluation Results

A.1 Sample Size

We calculate the appropriate sample size using results data gathered from initial users of the context- aware travel recommender system. Using the sample size table [138], the appropriate sample size for our user study evaluation was computed. A combined standard deviation estimate is caculated by averaging the variances in accuracy of two sample responses (1.85 and 2.81). These variances are pooled:

S2= S

2 1+S22

2

giving a pooled value of 2.38. We increase the standard deviation to 4 to give a more conservative estimate of what our sample size should be. A D value to be read from the sample size table is computed using:

D= δ

σ

Using a 5% δvalue representing the yied shift we want to detect, th D value is calculated to be 1.2.

Reference to the sample size table with this value and 0.05 values forαandβ (represnting the risk of

type one and type two errors), a sample size of 20 was calculated.