This thesis considered, for the first time a data-driven simulation approach to reputation management and to establish the validity and practicability of the proposed approach. This chapter gives experimental evidence which demonstrates the usefulness of D3-FRT.
6.4 Discussions and Summary 149
Furthermore, the main research questions that this chapter answers are:
• How useful is the DDDAS paradigm in aiding trusted communication in reputation management systems? To what extent will the framework support dynamism? How dynamic is agent rating?
• How accurate is the predictive capability of D3-FRT? How reliable is the framework as the network size grows?
Ratings are computed after each peer interaction in the network, and the newly computed values are made available across the network for other peers that wish to collaborate with the peer under consideration. Throughout the experiments, RV is computed after every peer interaction in D3-FRT. This approach proactively assigns ratings to peers as they collaborate. Therefore the framework captures the dynamic nature of reputation and trust making it useful in critical and trust-reliant domain. In validating our model, we have shown through the experiments in this chapter that an agents’ RV cannot be outside of the bound [0...5]. In addition, using one or a combination of the attack scenarios and simulation progressed, a good peer exhibiting consistently good behaviour cannot have a RV < 4.0, and a bad agent exhibiting consistently bad behaviour cannot have a RV > 1.0 in the model.
The estimation error rate of the D3-FRT is observed when actual ratings of domain members are compared with the predicted values. Generally, the pass criterion for predictions is largely dependent on the domain of application, the criticality and risk appetite in that domain. Though there were some variances between the actual and predicted values of the framework in the experiments, this is likely to be as a result of the initialisation period, when the simulation does not have any prior knowledge of the
network. However as shown in this chapter, D3-FRT has the capability to converge to make predictions that are closer to reality. Also, varying the value of the trust components using scaling factors have shown to have a great impact on the outcome of the predictions. Hence, making D3-FRT adaptable will allow for better results. The factors should therefore be chosen with care with respect to the intentions of the modeller and the application domain. It is worth noting here that the observations may vary with respect to the modelling approach, the time window of the observations and any other factors that may influence the behaviour of the model.
Scalability in the experiments refers to the extent to which the network can grow before there is degradation in performance. The scalability and reliability of D3-FRT in different network scenarios is examined in this chapter. The framework is tested by gradual increase of domain members and the results indicate that it is able to support increase in the number of network agents both normal and misbehaving agents in different network scenarios. Although the framework eventually detected all misbehaving agents in the different scenarios, it was observed that as the network increased in size the TTD also increased.
Adopting the DDDAS approach to trust management has shown to be useful with the advantage of providing trust and dynamism in the domain. The network is simulated with and without the presence of the predictive DDDAS component. By excluding this component we were able to isolate the individual effect of prediction. The usefulness of anticipating domain events, and making predictions was tested. In its absence, peers suc- cessfully misbehaved (which is unlike the case with the prediction component), preventive measures were taken. The peers RVs were downgraded before successfully carrying out an
6.4 Discussions and Summary 151
attack. This therefore makes D3-FRT proactive, reducing the overall negative impact of misbehaviour in the network.
D3-FRT has shown to be useful as a result of anticipating domain events leading to making predictions that are provided earlier than the actual network. This allows for preventive measures to be taken against agents that have been identified to have a high potential to misbehave. The agents are also grouped to regions depending on their RVs and agents that have ratings below the threshold are considered as high-risk and are excluded from the network. It is worth noting that although the predicted values are not 100% accurate, they differentiate between good agents from misbehaving ones by their relative RV ranking.
The experimental evidence in this chapter shows not only that reputation information encourages good behaviour and helps in the exclusion of misbehaving agents under dynamic scenarios, but a reputation approach that allows domain members to collaborate with trusted agents.
It can therefore be concluded that the DDDAS paradigm and its simulation primitive coupled with agent-based modelling can consequently reduce misbehaviour in a trust- reliant domain and the reliability of reputation management systems increases.
Part IV
We shall not cease from exploration And the end of all our exploring Will be to arrive where we started And know the place for the first time.
Chapter 7
Conclusion and Future Work
7.1
Introduction
The main goal of the work presented in this thesis is a dynamic and predictive approach to reputation management among a network of participants who have no prior knowledge of each other. From our aims and objectives earlier described in this thesis, the issue of trust dynamics has been studied and a novel approach for managing trust has been proposed. In addition to our study, the prediction of futuristic events has shown to be useful in providing timely information about domain events. Through qualitative evaluation, the use of a semi-distributed architecture has been justified.
More specifically, we identified the problems that arise as a result of trust dynamics and proposed a DDDAS-inspired framework using an agent based approach to solving the problems. The framework does not rely on only domain members for making trust decisions, but makes predictions by anticipating possible future states of the system. We demonstrated the effectiveness of our approach through qualitative and quantitative, simulation-based
experiments. The framework introduces flexibility into reputation management by allowing scaling factors for different inputs, and trust components that determine the ratings of domain members. The success of this approach was illustrated in Chapter 6 against a range of reputation management objectives: dynamism, performance and predictive capability. In D3-FRT, the controller is a trusted party. This is a sensible assumption when considering critical domains, where a form of central control is required. This includes domains such as that in the traffic management scenario of the example in Chapter 1 or in military networks where motes (tiny sensors) are deployed in a network to monitor enemy intrusion. Unlike other models, this framework does not only use historical data but also current and anticipated future events for prediction. D3-FRT logically groups the collaborating agents into regions of trust based on their reputation and ratings in the system. Our approach allows for placing more attention on groups of agents that pose a higher risk in the domain.
The D3-FRT approach satisfies some of the properties desirable in reputation and trust based system (as discussed in Section 2.3). Specifically, these properties are listed below:
• The framework provides predictions and ratings that help to distinguish members relative to their behaviours in the system.
• The framework is robust to known forms of misbehaviour and attacks which include intoxication and collusion, by both independent and collective misbehaving domain members.
• Behavioural changes are captured and are reflected in the timely current reputation computations of collaborating domain members. With the help of the DDDAS simulation component ratings converge to reflect the true changes in behaviours.