Dependent variables

For the dependent variables of our simulation study we focus on negotiation outcome measures. Several outcome measures are considered as dependent variables of the study as each of them covers an specific aspect of the negotiation outcome. Prior research on negotiation identified fundamental trade-offs between different outcome measures for negotiations, so that these trade- offs can also be expected for different configurations of systems for automated negotiations – i.e. a system configuration achieving good results in one measure need not perform well for others. As the importance attached to different outcome measures by the users is not obvious and can differ between users we opt for a holistic evaluation of the negotiation outcome in using several outcome measures that complementary evaluate different aspects of the outcome of the negotiation.

5.2.2.1 Proportion of agreements

The outcome of negotiations can be defined in various ways (Tripp and Sondak, 1992). One obvi- ous way, commonly used in empirical studies (e.g. in Coursey, 1982; Neale and Bazerman, 1985; Moore et al., 1999), is to consider whether or not an agreement was reached in the negotiation. For a set of comparable negotiations this results in a proportion of agreements or its inverse the impasse rate (Tripp and Sondak, 1992). In their review of dependent variables of negotiation studies Tripp and Sondak (1992) conclude that the impasse rate is rarely considered as a measure for the negotiation outcome, but that more often negotiations ending with an impasse are deleted from the sample and only of the negotiations that reached an agreement joint payoff – the sum of the parties utilities of the agreement – as the most widely used measure of joint performance is reported. Therefore we report the proportion of agreements agr

n in the treatments as one

outcome measure, which is simply the number of simulation runs that reached an agreement agr divided by the number of simulation runs in a treatment – or group of treatments – n.

While the outcome if the negotiation ends with the acceptance of an offer is an agreement on a specific set of options for the issues under negotiation, the outcome of termination due to quit or two subsequent reject messages is no agreement. For simulation runs without agreement we cannot report more than the proportion of agreements reached in the treatment. However, when an agreement is reached in a simulation run, the evaluation – according to the preferences of the parties – of this agreement provides additional information about the quality of the agreement. A high proportion of agreements does not necessarily indicate that one system configuration is better than an other, that achieves a lower proportion of agreements as not all agreements are equally good.

5.2. Measurement 113

5.2.2.2 Proportion of Pareto-optimal agreements

As not all agreements are equally good, measures evaluating the quality of an agreement are necessary. One such measure evaluating the dyadic performance of the negotiators – or a system for automated negotiation – from an economic perspective is the Pareto-optimality of an agree- ment – or for a set of comparable negotiation the proportion of Pareto-optimal agreements. An agreement is Pareto-optimal if there exist no other possible solutions to the negotiation problem that dominates this focal agreement – i.e. that provides higher utility to one party without making the other party worse off compared to the focal agreement:5

Pareto optimal agreements are those from which no additional joint gains are possible. When negotiators have achieved Pareto optimal agreements, no agreement is possible that would be preferred by both negotiators or would be preferred by one and to which the other would be indifferent. (Tripp and Sondak, 1992, p.279)

0 20 40 60 80 100 0 20 40 60 80 100

Figure 5.4: Pareto-optimal solutions in a negotiation problem

As Pareto-optimality can only be determined for agreements and not for outcomes of simulation runs other than an agreement, the simulation runs reaching Pareto-optimal solutions can only be a subset of those simulation runs that reach an agreement. Pareto-optimality distinguishes agreements between those that are and those that are not Pareto-optimal. So for several com- parable negotiations we again can calculate the proportion of Pareto-optimal agreements for the treatments – or groups of treatments – as the number of simulation runs that reached a Pareto- optimal agreement ef f divided by some basis. Candidates for this basis for calculation of the proportion of Pareto-optimal agreements are on the one hand the total number of simulation runs in the treatment n, or on the other hand the number of simulation runs that achieved an agreement agr. Using the total number of simulation runs in a treatment as divisor – which re- sults in ef f_n as measure of the proportion of Pareto-optimal agreements – compounds the number

5_{Formally – using the notation from the previous section – the set P of Pareto-optimal solutions – illustrated}

as the filled points in Figure 5.4 – is the subset of the set X consisting of the solutions x only, for which holds

of agreements reached with the Pareto-optimality of agreements. Even if most of the agreements reached in a treatment are Pareto-optimal, a low number of agreements reached would result in a low proportion of Pareto-optimal agreements as Pareto-optimality can only be determined for simulation runs that reached an agreement.

Using the the number of agreements reached in the simulation runs agr as basis for determining the proportion of Pareto-optimal agreements (ef f_agr) would avoid the above mentioned deficit. However, the consensus of discussions and previous presentations of the results of this study was that one should not consider the Pareto-optimality of agreements independent of the number of agreements, we therefore accept the bias of this measure and determine the proportion of Pareto-optimal agreements on the basis of the total sample size as ef f_n .6

5.2.2.3 Minimal distance to the Pareto frontier

As said Pareto-optimality discriminates between agreements that are or are not Pareto-optimal. However, just like not all agreements are equally good, not all agreements that are not Pareto- optimal are equally bad. Some of the possible solutions can be closer to the Pareto frontier than others, like solution B is closer to the Pareto frontier than solution A in Figure 5.5(a). Therefore we calculate for an agreement ˆx the minimal Euclidean distance to the Pareto frontier. Using the notation and definitions provided in the previous sections the minimal euclidean distance ED – the length of the shortest straight line between the agreement and one of the solutions of the set of Pareto optimal solutions P i.e. the Pareto frontier – can be determined by (5.4).

ED = minx∈P

q

(uj(x) − uj(ˆx))2+ (u−j(x) − u−j(ˆx))2 (5.4)

5.2.2.4 Contract imbalance

One further measure of joint performance often considered is the fairness of the agreement. Fair- ness could for example be determined by the distance to axiomatic solutions for the bargaining problem provided by cooperative game theory. The axiomatic approach, first proposed by Nash (1950), defines axioms as properties desirable for the solution and then determines solution func- tions that select for a given bargaining problem a certain package as solution to the bargaining problem, which features the properties stated in the axioms. One of these axioms used in many axiomatic solutions is symmetry – e.g. used in the well known Nash solution (Nash, 1950) or the Raiffa solution axiomatized by Kalai and Smorodinsky (1975). Symmetry is argued to be an axiom assuring fairness as it requires that for the same preferences the outcome should also be the same. However, another axiom most axiomatic solutions share is Pareto-optimality. When taking distance measures between actual agreement and axiomatic solutions that have to be both fair and Pareto-optimal the two concepts are compound: Distances between the agreement and the Pareto frontier can be large but the utilities of the agreement to the parties quite equal – and the agreement therefore could be considered fair.

6_{Note that if the proportion of Pareto-optimal agreements on basis of the agreements reached is of interest it}

can easily be derived by multiplying the proportion on total runs ef f_n with the proportion of agreements agr_n –

both provided as outcome measures in this dissertation – as (

ef f

n )

(agr_n ) = ef f agr