We compare the performance of automated negotiation systems using different protocols in their configuration to analyze the main effects of the protocol on the outcome measures – proportion of agreements, proportion of Pareto-optimal agreements, utility of the agreement to the buyer and seller, distance of the agreement to the Pareto frontier, and contract imbalance.4 The results of
the multiple pairwise comparisons of these samples – within which the same agent combinations and negotiation problems were used in the simulation runs and between which therefore only the protocol differs – are presented in Tables 6.3 to 6.8 for our six outcome measures.
% protocol 1 protocol 2 protocol 1 100.00
protocol 2 19.85 0.0000
protocol 3 70.39 0.0000 0.0000
Table 6.3: Proportion of agreements in different protocols
% protocol 1 protocol 2 protocol 1 46.47
protocol 2 11.48 0.0000
protocol 3 40.47 0.0000 0.0000
Table 6.4: Proportion of Pareto-optimal agree- ments in different protocols
3The only exception here are the three treatments that use only TFT-agents in the three protocols, which not
only achieve about equal utilities for both sides – around 66 utility points – within a treatment – i.e. for the same protocol –, but also across the treatments – i.e. for the different protocols.
4Proportions of agreements and Pareto-optimal agreements in the samples are compared using Pearson’s χ2
independence tests, differences in all other outcome measures are analyzed by means of Wilcoxon ranked sum tests. Samples sizes for proportion measures are 81 ∗ 3 ∗ 2065 = 501795 as each agent combination is used for three replications of all negotiation problems as input to the systems, for all other measures it is 501795 times the proportion of agreement reached in the protocols provided in Table 6.3. Test statistics are omitted in the subsequent tables to save space and to be consistent with the presentation of the results in subsequent tables, where the provision of the test statistics would consume too much space, and can be requested from the author. All p-values reported are adjusted for multiple comparisons by the Bonferroni-Holm method.
6.2. Comparison of interaction protocols 125
⊘ protocol 1 protocol 2 protocol 1 67.70
protocol 2 77.04 0.0000
protocol 3 71.45 0.0000 0.0000
Table 6.5: Utility of the seller in different pro- tocols
⊘ protocol 1 protocol 2 protocol 1 63.81
protocol 2 75.89 0.0000
protocol 3 69.45 0.0000 0.0000
Table 6.6: Utility of the buyer in different pro- tocols
⊘ protocol 1 protocol 2 protocol 1 5.30
protocol 2 3.28 0.0000
protocol 3 3.34 0.0000 0.0000
Table 6.7: Minimal distance to the Pareto fron- tier in different protocols
⊘ protocol 1 protocol 2 protocol 1 29.43
protocol 2 18.38 0.0000
protocol 3 17.99 0.0000 0.1520
Table 6.8: Contract imbalance in different pro- tocols
The proportion of agreements clearly is highest in protocol 1, which by the mechanisms of this protocol reaches agreements in all simulation runs. That the proportion of reached agreements is 100% in protocol 1, which neither allows software agents to break off negotiations by quit messages nor enables termination due to two subsequent reject messages – as these are not permitted under protocol 1 – is not further surprising but actually adds to the verification of the implementation of the simulation program. The second highest proportion of agreements (70.39%) is reached by systems employing protocol 3, while in only 20% of the simulation runs an agreement is reached when using protocol 2. Differences between the proportions of reached agreements are highly significant as can be seen from Table 6.3 (p < 0.001 for all pairwise comparisons). The sources of these differences are obvious from the protocol descriptions, while protocol 1forces the agents to reach an agreement, in protocol 3 the software agents have the possibility to reject unfavorable offers but keep on negotiating, which could lead to a termination of the negotiation if the opponent also sends a reject message as it was the case in the 30% of the simulation runs that achieved no agreement under protocol 3. Finally, software agents immediately break off the negotiation in case of such unfavorable offers in protocol 2. So the the possibility to reject unfavorable offers of the opponent under protocol 3 allowed to achieve agreements in additional 50% of the simulation runs where the immediate termination by quit messages under protocol 2 prevented such agreements. However, the possibility of rejecting offers also caused 30% of the simulation runs to fail to reach an agreement compared to protocol 1, where this was not possible.
The absolute proportion of Pareto-optimal agreements – as a proportion of the total number of simulation runs in a sample – shows the same tendency (Table 6.4). The proportion of Pareto-optimal agreements is higher for the protocols that reach more agreements and therefore highest in protocol 1 (46.47 %), followed by protocol 3 (40.47 %), and systems that use protocol 2(11.48 %) – again differences between the proportion of Pareto-optimal agreements are highly significant (p < 0.001 for all pairwise comparisons). These numbers, however, indicate that the differences in proportions are smaller for the proportion of Pareto-optimal agreements, than for the overall proportion of agreements. This becomes more evident when looking at the relative proportions of Pareto-optimal agreements – i.e. proportions calculated not on the basis of all simulation runs but on the basis of those that reached an agreement as discussed in the previous chapter – as provided in Table 6.9. The relative proportion of Pareto-optimal agreements is highest for protocol 3, followed by protocol 2, but lowest – in contrast to the
absolute proportion of Pareto-optimal agreements – in simulation runs with systems that use protocol 1. This indicates that the possibility to not engage in further negotiations on the basis of unfavorable offers of the opponent through breaking off the negotiation (protocol 2) or eliciting a new offer from the opponent (protocol 3) moves agreements closer to the Pareto frontier, which consequently match it more often, than not having these options (protocol 1), however, this is achieved at the cost of risking a negotiation break-off, which can be seen from the former results on the proportion of agreements.
⊘ protocol 1 protocol 2 protocol 1 46.47
protocol 2 58.48 0.0000
protocol 3 57.73 0.0000 0.0002
Table 6.9: Relative proportion of Pareto-optimal agreements in different protocols
Results comparable to those for the relative proportion of Pareto-optimal agreements are obtained for the minimal distance to the Pareto frontier (Table 6.7) – the similarity of these measures is not further surprising as the minimal distance to the Pareto frontier is zero when agreements are Pareto-optimal – so this measure covers aspects of the relative proportion of Pareto-optimal agreements well. The distance of reached agreements to the Pareto frontier is smallest when using protocol 2, followed by protocol 3, and both, protocol 2 and 3, achieve agreements closer to the Pareto frontier than protocol 1 does (all p < 0.001).
Similar to the relative proportion of Pareto-optimal agreements and minimal distance to the Pareto frontier, both parties, the seller and the buyer party, achieve – in case of agreement – highest utility of the agreement in protocol 2 followed by protocol 3 and 1 (all p < 0.001 as can be seen from Tables 6.5 and 6.6, respectively). As already mentioned above from Tables 6.5 and 6.6 one can see that the utility of an agreement is lower for the buyer than for the seller in all protocols. Note that the differences in utilities between the parties slightly differs between the protocols used for the system and is highest for protocol 1 (difference of 4 utility points), followed by protocol 3 (2 points) and protocol 2 (around 1.5 points) – so that the difference between the utility of an agreement to the buyer and the seller increases with the proportion of agreements reached and decreases with the relative proportion of the Pareto-optimal agreements. Finally if an agreement is reached, the contract imbalance – as the difference between the utilities of the agreement to the parties – is smallest and therefore agreements are fairest in protocol 3 and protocol 2 without significant differences between these two protocols, which both achieve more balanced agreements than protocol 1 (p < 0.001 in both cases). This partly is explained by the differences between the utility of an agreement to seller and buyer, discussed above, if these differences are low – as in protocol 2 and 3 – also the contract imbalance is low, while it is high otherwise – as found for protocol 1.
The lower contract imbalance found for protocol 2 and protocol 3 also can be explained by the mechanisms of these protocols. While unfavorable offers can be rejected and new offers can be elicited in protocol 3, or negotiations can be broken off in protocol 2, which either leads to a more favorable and balanced agreement or no agreement at all, such messages are not enabled in protocol 1, where strategies have no means to cope with bad offers and interrupt their offering strategies, which could lead to unfavorable agreements for the weaker party – in terms of preferences over the negotiation object – and therefore result in higher contract imbalance and