Heuristic Methods for Coalition Structure Generation
by
Amir Aatieff Amir Hussin
A Doctoral Thesis
Submitted in partial fulfilment of the requirements for the award of
Doctor of Philosophy of Loughborough University
June 14, 2017
ii
Abstract
The Coalition Structure Generation (CSG) problem requires finding an optimal partition of a set of n agents. An optimal partition means one that maximizes global welfare. Computing an optimal coalition structure is computationally hard especially when there are externalities, i.e., when the worth of a coalition is dependent on the organisation of agents outside the coalition. A number of algorithms were previously proposed to solve the CSG problem but most of these methods were designed for systems without externalities. Very little attention has been paid to finding optimal coalition structures in the presence of externalities, although externalities are a key feature of many real world multiagent systems. Moreover, the existing methods, being non-heuristic, have exponential time complexity which means that they are infeasible for any but systems comprised of a small number of agents.
The aim of this research is to develop effective heuristic methods for finding optimal coalition structures in systems with externalities, where time taken to find a solution is more important than the quality of the solution. To this end, four different heuristics methods namely tabu search, simulated annealing, ant colony search and particle swarm optimisation are explored. In particular, neighbourhood operators were devised for the effective exploration of the search space and a compact representation method was formulated for storing details about the multiagent system. Using these, the heuristic methods were devised and their performance was evaluated extensively for a wide range of input data.
iii
Acknowledgements
After so much hard work I am finally able to submit this thesis which I hope will be at least a particle among the particles of knowledge. All of this would not have been possible without the dedication and persistence of my dear supervisor Shaheen Fatima. I owe you the world and with my deepest sincerity I would like to express a million thanks and gratitude to you, Dr Shaheen Fatima. You helped nurture my research by being supportive and patient with my pace throughout the period of my studies. Thank you for giving me a chance to be an apprentice as a PhD student under your supervision. I am truly proud to be a student of such an outstanding and intelligent academic and utmost expert in multi-agent systems.
Nothing is more important than family, without their support no one can sail through the rough seas of doing a PhD. My dearest wife Syuhada, may Allah bless you always. Your warmth, kind, patient and tender loving care made me cheerful whenever I hit a brick wall with the research. You, Adrianna, Adrian and Aarash sacrificed a lot for me throughout this journey, this leap of faith. Failure would be to fail you all. It was not easy but we did it together. Mama, Dedi, Thank you for believing in me and continuing to support me all throughout this journey, for your endless spiritual, physical and financial support.. To my little brother Aarieff, thank you for helping me apply for this scholarship and my sisters Aalia and Aafaff, you are the best sister a brother could ask for, you literally ‘guaranteed’ my success, without you I would have no scholarship. Mak, Bapak, thank you for always being there for me and for not losing hope in me. Atif Sana Tarar, thank you for keeping our family strong. Thank you Bidah, for filling in for Aarash. Bengah, Bede, Pik, Afiq, Nana, Ira, Aisyah, all of you add colours to my life. A special thanks my former masters’ supervisor, the late Dr. Mohd Syazwan Abdullah who encouraged me to pursue a PhD. Also to Judith Paulton who has tirelessly supported me throughout the course of my studies at Loughborough University, for always being supportive and always proactive in finding solutions when I needed help.
Thank you to all my compatriots from the MML clan. A personal thanks to those families who has contributed so much to me and my family, my most closest of friends in Loughborough Amran Ahmad, Farraen, Shafizal, Nafiss, Rifqi, Muhaimin, Syahibudil, Safwan, Syuib Rambat, Shahrulizan, Nashrudin, Ezhan Johaniff, Faezz, Zack, Zulkifli Omar and all the others which are too many to mention here.
iv Table of Contents Page Abstract ii Acknowledgement iii Table of Contents iv
List of Figures vii
List of Tables x
Chapter 1 Introduction 1.1Background
1.2Coalition Formation in Multiagent Systems 1.2.1 Coalitions and Coalition Structures 1.2.2 Characteristic Function Games 1.2.3 Partition Function Games
1.2.4 Coalition Structure Generation (CSG)
1.2.5 Methods for Optimal Coalition Structure Generation 1.3Research Objectives 1.4Research Contributions 1.5Thesis Structure 1 2 5 6 6 7 8 9 10 11 Chapter 2 Coalitional Games
2.1Characteristic Function Games 2.2Partition Function Games
2.2.1 The General Partition Function Game 2.2.2 A compact representation for PFGs
2.2.2.1Positive Externalities 2.2.2.2Negative Externalities
2.2.2.3A Representation for Mixed Externalities 2.3Chapter Summary 12 13 13 16 17 18 19 29 Chapter 3 Coalition Structure Generation for Characteristic Function
Games: A Review of Literature 3.1Design-to-Time Algorithms
3.1.1 Dynamic Programming
3.1.2 Improved Dynamic Programming
30 30 32
v
3.2Anytime Algorithms
3.2.1 Coalition Structure Graph Search 3.2.2 Integer Partition-Based Search 3.3Heuristic Algorithms
3.3.1 Genetic Algorithms 3.3.2 Simulated Annealing 3.3.3 Greedy-Based Method 3.3.4 Particle Swarm Optimisation 3.3.5 Ant Colony Optimisation 3.4Chapter Summary 35 35 37 40 41 42 43 44 45 46 Chapter 4 Coalition Structure Generation Partition Function Games:
A Review of Literature
4.1Integer Partition-based Algorithm for PFG 4.1.1 𝐼𝑃+/− Algorithm
4.2Distributed CSG with Externalities 4.3PFGs with Mixed Externalities 4.4Chapter Summary 48 49 52 53 54 Chapter 5 Heuristic Methods for Finding Optimal Coalition Structure
5.1Tabu Search for Coalition Structure Generation (TACOS) 5.1.1 Neighbourhood Generation Operators
5.2Simulated Annealing for Optimal Coalition Structure 5.3Ant Colony Search for Optimal Coalition Structure 5.4Particle Swarm Search
5.5Chapter Summary 57 59 62 64 67 69 Chapter 6 Simulation Setup for Performance Evaluation
6.1Evaluation Method
6.2Data Generation for Performance Evaluation 6.2.1 Data for Characteristic Function Games 6.2.2 Data for Partition Function Games 6.3Calculating Bounds
6.3.1 Calculating the Upper Bound for CFGs 6.3.2 Calculating the Upper Bound for PFGs 6.4Chapter Summary 70 73 73 74 75 75 77 83
vi
Chapter 7 Performance Analysis for the Individual Probability Distributions 7.1Performance for Characteristic Function Games
7.1.1 Performance for 25-Agent CFGs 7.1.2 Performance for 27-Agent CFGs 7.2Performance for Partition Function Games
7.2.1 Performance for 10-Agent PFGs 7.2.2 Performance for 27-Agent PFGs
7.3The Effect of Number of Agents on Performance 7.3.1 Performance for CFGs
7.3.2 Performance for PFGs 7.4Statistical Test on Results
7.4.1 Tests on 25-agent CFG 7.4.2 Tests on 27-agent CFG 7.4.3 Tests on 10-agent PFG 7.4.4 Tests on 27-agent PFG 7.5Chapter Summary 85 85 94 103 103 109 116 117 121 125 125 129 133 137 141 Chapter 8 Performance Analysis Across Distributions
8.1Average Performance for CFGs 8.2Average Performance for PFGs
8.3Performance Comparison for Each Method
8.3.1 CFGs: The Effect of Number of Agents on Performance 8.3.2 PFGs: The Effect of Number of Agents on Performance 8.3.3 The Effect of Externalities on Performance
8.4Memory Usage 8.5Chapter Summary 144 147 150 150 151 152 153 153 Chapter 9 Conclusion and Future Work
9.1Conclusions 9.2Future Work 155 161 References 163 Appendix I 171
vii
List of Figures
Page
Figure 2.1 A typical characteristic function game and its input. 12
Figure 2.2 A typical partition function game and its input. 14
Figure 3.1 Example DP movements for 4-agents. 32
Figure 3.2 Multiple paths leading to each CS with more than two coalitions. 33
Figure 3.3 Redundant edges removed as dotted lines. 34
Figure 3.4 Coalition structure graph for 4-agents (Sandholm et al. 1999). 35
Figure 3.5 An example IP search space and sub-spaces given 4 agents. 39
Figure 4.1 Integer Partition for six-agents. 49
Figure 5.1 Ant colony movement by operator strength. 67
Figure 6.1 Space of all coalition structures for 5 agents. 76
Figure 7.1 Performance Comparison for 25-agents (CFG) ≈ 30 seconds running time.
91
Figure 7.2 Performance Comparison for 27-agents (CFG) ≈ 60 seconds running time.
100
Figure 7.3 Performance Comparison for 10-agents (PFG) ≈ 180 seconds running time.
107
Figure 7.4 Performance Comparison for 27-agents (PFG) ≈ 300 seconds running time.
viii
Figure 7.5 Performance for 25-agent and 27-agent CFGs (Uniform Distribution).
117
Figure 7.6 Performance for 25-agent and 27-agent CFGs (Normal Distribution).
118
Figure 7.7 Performance for 25-agent and 27-agent CFGs (Gamma Distribution).
118
Figure 7.8 Performance for 25-agent and 27-agent CFGs (Beta Distribution). 119
Figure 7.9 Performance for 25-agent and 27-agent CFGs (Exponential Distribution).
120
Figure 7.10 Performance for 25-agent and 27-agent CFGs (Triangular Distribution).
120
Figure 7.11 Performance for 10-agent and 27-agent PFGs (Uniform Distribution).
121
Figure 7.12 Performance for 10-agent and 27-agent PFGs (Normal Distribution).
122
Figure 7.13 Performance for 10-agent and 27-agent PFGs (Gamma Distribution).
122
Figure 7.14 Performance for 10-agent and 27-agent PFGs (Beta Distribution). 123
Figure 7.15 Performance for 10-agent and 27-agent PFGs (Exponential Distribution).
124
Figure 7.16 Performance for 10-agent and 27-agent PFGs (Triangular Distribution).
124
Figure 7.17 Confidence Intervals 25-agent CFGs (% of Optimal). 129
ix
Figure 7.19 Confidence Intervals 10-agent PFGs (% of Optimal). 137
Figure 7.20 Confidence Intervals 27-agent PFGs (% of Upper Bound). 141
Figure 8.1 Average performance (25-agent CFGs). 144
Figure 8.2 Average performance (27-agent CFGs). 146
Figure 8.3 Average performance (10-agent PFGs). 147
Figure 8.4 Average performance (27-agent PFGs). 149
Figure 8.5 Effects of the number of agents on performance (CFGs). 151
Figure 8.6 Effects of the number of agents on performance (PFGs). 152
x
List of Tables
Page
Table 1.1 Number of possible coalition structures. 8
Table 2.1 Possible Coalition Types for 3-agents. 20
Table 2.2 Possible Coalition Structure Types. 21
Table 2.3 Externalities for each coalition type in a structure type. 23
Table 2.4 Externalities for singletons. 24
Table 2.5 Formula for calculating positive/negative externalities. 24
Table 2.6 Externalities from Other Coalitions in 𝐶𝑆 on Coalition 𝐶. 25
Table 2.7 Externalities on Coalition 𝐶 in Coalition Structure 𝐶𝑆. 26
Table 3.1 An example showing how 𝑓1 and 𝑓2 are calculated. 31
Table 3.2 Comparison between DP and IDP. 34
Table 3.3 Comparison of the Anytime Algorithms. 40
Table 3.4 A Summary the Heuristic Algorithms. 46
Table 4.1 Comparison of Methods for CSG in PFGs. 55
Table 5.1 Comparison of the Heuristic Methods. 69
Table 6.1 Running Time (rounded to next decimal) for CFGs. 71
xi
Table 6.3 Number of Iterations and the corresponding run time for each method.
72
Table 6.4 Number of Integer Partitions for each 𝑘 for 27-agent games. 79
Table 6.5 Upper bound for partitions containing coalition of size 𝑘. 82
Table 7.1 Average Performance for 25-agent CFGs (Uniform Distribution).
86
Table 7.2 Average Performance for 25-agents CFGs (Normal Distribution).
86
Table 7.3 Average Performance for 25-agents CFGs (Gamma Distribution).
87
Table 7.4 Extended Running Time for 25-agent CFGs (Gamma Distribution).
88
Table 7.5 Average Performance for 25-agents CFGs (Beta Distribution). 88
Table 7.6 Average Performance for 25-agents CFGs (Exponential Distribution).
89
Table 7.7 Extended Running Time for 25-agent CFGs (Exponential Distribution).
90
Table 7.8 Average Performance for 25-agent CFGs (Triangular Distribution).
90
Table 7.9 Performance of Each Method for 25-agent CFGs (1 Best – 4 Worst).
92
Table 7.10 Best and Worst Solutions (25-agent CFGs). 92
Table 7.11 Average Performance for 27-agents CFGs (Uniform Distribution).
xii
Table 7.12 Average Performance for 27-agents CFGs (Normal Distribution).
95
Table 7.13 Average Performance for 27-agents (Gamma Distribution). 96
Table 7.14 Extended Running Time for 27-agent CFGs (Gamma Distribution).
96
Table 7.15 Average Performance for 27-agents CFGs (Beta Distribution). 97
Table 7.16 Average Performance for 27-agent CFGs (Exponential Distribution).
98
Table 7.17 Extended Running Time for 27-agent CFGs (Exponential Distribution).
98
Table 7.18 Average Performance for 27-agent CFGs (Triangular Distribution).
99
Table 7.19 Performance of Each Method for 27-agent CFGs (1 Best – 5 Worst)
100
Table 7.20 Best and Worst Solutions (27-agent CFGs). 101
Table 7.21 Average Performance for 10-agent PFGs (Uniform Distribution).
103
Table 7.22 Average Performance for 10-agent PFGs (Normal Distribution). 104
Table 7.23 Average Performance for 10-agent PFGs (Gamma Distribution). 104
Table 7.24 Average Performance for 10-agent PFGs (Beta Distribution). 105
Table 7.25 Average Performance for 10-agent PFGs (Exponential Distribution).
105
xiii
Table 7.27 Performance of Each Method for 10-agents PFGs (1 Best – 4 Worst)
106
Table 7.28 Best and Worst Solutions (10-agent PFGs) 108
Table 7.29 Average Performance for 27-agent PFGs (Uniform Distribution).
109
Table 7.30 Average Performance for 27-agent PFGs (Normal Distribution). 110
Table 7.31 Average Performance for 27-agents PFGs (Gamma Distribution).
111
Table 7.32 Extended Running Time for 27-agent PFGs (Exponential Distribution).
111
Table 7.33 Average Performance for 27-agent PFGs (Beta Distribution). 112
Table 7.34 Average Performance for 27-agens PFGs (Exponential Distribution).
112
Table 7.35 Extended Running Time 27-agent PFGs (Exponential Distribution).
113
Table 7.36 Average Performance for 27-agent PFGs (Triangular Distribution).
113
Table 7.37 Performance of Each Method for 27-agents PFG (1 Best – 5 Worst).
115
Table 7.38 Best and Worst Solutions (27-agent PFGs). 116
Table 7.39 Confidence Interval 25-Agent CFG (Uniform Distribution). 126
Table 7.40 Confidence Interval 25-Agent CFG (Normal Distribution). 126
xiv
Table 7.42 Confidence Interval 25-Agent CFG (Beta Distribution). 127
Table 7.43 Confidence Interval 25-Agent CFG (Exponential Distribution). 128
Table 7.44 Confidence Interval 25-Agent CFG (Triangular Distribution). 128
Table 7.45 Confidence Interval 27-Agent CFG (Uniform Distribution). 130
Table 7.46 Confidence Interval 27-Agent CFG (Normal Distribution). 130
Table 7.47 Confidence Interval 27-Agent CFG (Gamma Distribution). 131
Table 7.48 Confidence Interval 27-Agent CFG (Beta Distribution). 131
Table 7.49 Confidence Interval 27-Agent CFG (Exponential Distribution). 132
Table 7.50 Confidence Interval 27-Agent CFG (Triangular Distribution). 132
Table 7.51 Confidence Interval 10-Agent PFG (Uniform Distribution). 134
Table 7.52 Confidence Interval 10-Agent PFG (Normal Distribution). 134
Table 7.53 Confidence Interval 10-Agent PFG (Gamma Distribution). 135
Table 7.54 Confidence Interval 10-Agent PFG (Beta Distribution). 135
Table 7.55 Confidence Interval 10-Agent PFG (Exponential Distribution). 136
Table 7.56 Confidence Interval 10-Agent PFG (Triangular Distribution). 136
Table 7.57 Confidence Interval 27-Agent PFG (Uniform Distribution). 138
Table 7.58 Confidence Interval 27-Agent PFG (Normal Distribution). 138
xv
Table 7.60 Confidence Interval 27-Agent PFG (Beta Distribution). 139
Table 7.61 Confidence Interval 27-Agent PFG (Exponential Distribution). 140
Table 7.62 Confidence Interval 27-Agent PFG (Triangular Distribution). 140
Table 8.1 Best and worst performance across distributions (25-agent CFGs).
145
Table 8.2 Average number of neighbours explored (25-agent CFGs). 145
Table 8.3 Best and worst performance across distributions (27-agent CFGs).
146
Table 8.4 Average number of neighbours explored (27-agent CFGs). 147
Table 8.5 Best and worst Performance across distributions (10-agent PFGs).
148
Table 8.6 Average number of neighbours explored (10-agent PFGs). 148
Table 8.7 Best and worst Performance across distributions (27-agent PFGs).
149
Table 8.8 Average number of neighbours explored (27-agent PFGs). 150
Table 9.1 Comparative summary of the performance of all methods (CFGs).
158
Table 9.2 Comparative summary of the performance of all methods (PFGs).
1
Chapter 1 Introduction
This chapter sets the background for this research and lists the main aims and objectives of this research.
1.1Background
In the modern age of computing, complex systems have evolved to become more independent and self-aware. There is an increasing trend towards designing software systems that work mostly autonomously requiring little or no human input. An intelligent agent is an agent that exhibits three main characteristics (Wooldridge, 2009): pro-activeness, reactivity and social ability. An agent is said to be proactive if it exhibits goal-directed behaviour, i.e. is capable of taking initiative to better achieve its intended goal. It is said to be reactive if it can respond to changes in the environment in time for the response to be useful. It is said to be social if it is able to interact with other agents.
A multiagent system (MAS) is comprised of multiple intelligent agents that interact and coordinate with each other. A key characteristic of a MAS is that the individual agents in it have different skills and capabilities and each agent is limited in terms of its capabilities. Cooperation and coordination between agents is therefore necessary in order for them to achieve big and complex goals, i.e., goals that cannot be achieved by an agent individually. Working together in some instances also helps the agents achieve the goals more efficiently (Shehory & Kraus, 1998; Zlotkin & Rosenschein, 1994).
To bring about this cooperation, the individual agents in a MAS must form a group and work together. In this context, the following two terms are useful:
Coalition: It is a subset of the agents that comprise a MAS.
Coalition structure: A typical MAS requires the formation of several coalitions that work simultaneously. In this context, the term coalition structure refers to a partition of the agents in a MAS.
Chapter 1. Introduction 2 The following are some applications that require the formation of coalitions:
1. Autonomous sensors networks where coalitions help to improve the coverage of the surveillance area (Glinton et al., 2008).
2. A coalition of buyers working together to get cheaper prices by purchasing in bulk (Sukstrienwong, 2011).
3. Intrusion detection systems (IDSs) to provide secure and dependable cloud computing service (Liu et al., 2015).
4. Distributed vehicle routing applications where delivery companies need to optimally allocate resources to coalitions within a structure (Sandholm & Lesser, 1997).
5. E-commerce systems where buyers group together to obtain discounts (Tsvetovat & Sycara, 2000).
6. E-business systems that need to optimally allocate resources to market partitions (Norman et al., 2004).
7. Distributed grid computing where virtual organisations must optimally share resources (Foster & Kesselman, 2003; Yong et al., 2003).
8. Information gathering systems where clusters of information servers must process queries together by forming optimal coalitions (Klusch & Shehory, 1996). 9. Multi-sensor surveillance networks where coalitions are used to provide better
coverage over a large area (Dang et al., 2006).
1.2 Coalition Formation in Multiagent Systems
The success of a MAS is frequently measured in terms of the overall system performance of the system. The performance of a MAS depends on what coalitions form. Coalitions can form in many different ways. If formed correctly, coalitions can increase the productivity of a MAS. Likewise, badly formed coalitions can be counter-productive. It is therefore important for the right coalitions to form. But finding the right coalitions is a difficult problem. This thesis is aimed at finding methods for overcoming this difficulty. More precisely, the aim of this research is to devise computational methods for finding optimal coalitions.
Chapter 1. Introduction 3 In order to understand the complexity of the problem, consider a system comprised of 3 agents. Let {a, b, c} be the set of agents. There are 7 different ways in which a coalition can form: 1. {a} 2. {b} 3. {c} 4. {a, b} 5. {b, c} 6. {a, c} 7. {a, b, c}
In general for a system comprised of n agents, there are 2𝑛−1 possible coalitions. Again consider the system comprised of 3 agents {a, b, c}. There are 5 possible coalition structures, i.e., 5 different ways of partitioning the 3 agents:
1. {{a}, {b}, {c}} 2. {{a, b}, {c}} 3. {{a, c}, {b}} 4. {{a}, {b, c}} 5. {{a, b, c}}
In general, for a system of n agents, the number of possible coalition structures is the Bell number (𝐵𝑛) (see section 1.2.4 for details).
Different coalition structures yield different levels of performance. The problem is therefore to determine an optimal coalition structure, i.e., a structure that optimizes system
performance.
In order to find an optimal coalition structure, we must first use a suitable representation for describing the worth of a coalition and the worth of a coalition structure. The most commonly used representation comes from the literature in game theory (Chalkiadakis et al., 2012).
Chapter 1. Introduction 4 A coalition game is defined in terms of the players playing the game (i.e., the agents that comprise a MAS) and the worth of coalitions and coalition structures (Chalkiadakis et al., 2012). There are two types of coalition games characteristic function games (CFG) and partition function games (PFG).
1. Characteristic function games (Di Mauro et al., 2010; Keinänen & Keinänen, 2008; Rahwan et al., 2012; Rahwan et al., 2013; Sen & Dutta, 2000; Yeh, 1986): These are games in which the worth of a coalition depends on its member agents alone.
2. Partition function games (Banerjee & Kraemer, 2010; Epstein & Bazzan, 2013; Michalak et al., 2008; Rahwan et al., 2012; Thrall & Lucas, 1963): These are games in which the worth of a coalition depends not only on its member agents but also on how the external agents form coalitions.
The difference between PFGs and CFGs is that PFGs take into account externalities from forming coalitions. In PFGs the utility that is obtained from forming a coalition could be influenced by other coalitions that are concurrently being formed. This means that the value of a coalition depends on the coalition structure it is embedded with.
Example
Consider a game comprised of three agents {a, b, c}. For a CFG, the value of a coalition, say {a}, is given in terms of the elements of the set {a}. The value of any coalition is a real number. Thus the value of {a} in the structure {{a}, {b, c}} is equal to the value of {a} in the structure {{a}, {b}, {c}}. However, this is not the case for PFGs. For a PFG, the value of {a} in the structure {{a}, {b, c}} need not be equal to the value of {a} in the structure {{a}, {b}, {c}}. Most of the existing literature on determining optimal coalition structures has focussed on CFGs (Rahwan et al., 2015). However, in many cases, such as resource allocation (Dunne, 2005), the performance of a coalition is directly influenced how the external players are organized. In such cases, a coalition that increases its consumption of resources in one part of the system can have adverse effects that could impact the effectiveness of agents in another part of the system. Another example of the occurrence of externalities is conspiracy attempt in oligopolies. In these situations, cooperating organisations explore ways to undermine the competitiveness of other companies in the market (Catilina & Feinberg, 2006).
Chapter 1. Introduction 5 Type of externality
Externalities can be of two types (Rahwan et al., 2009): positive or negative. Externalities are said to be positive (negative) if the merger of two coalitions has a beneficial (harmful) impact on the external players.
An example of positive externalities is environmental policy arrangement among countries (Plasmans et al., 2006). A decision made by a coalition of countries to cut pollution levels will have a beneficial impact on the countries outside the coalition (Finus, 2003). An example of negative externalities is the merger of technological companies; if say a giant like Microsoft were to merge with Facebook, then this decision will typically have a negative impact on other companies such as Google and Oracle.
Negative externalities also arise when major technology corporations decide to cooperate in order to develop a new technology standard; those that are not part of the coalition see a reduction their competitive position. For instance when a group of companies decided to form Blu-ray Disc Association (BDA), the industry consortium that develops and licenses Blu-ray Disc technology, the others that did not join BDA were recipients of negative externalities. The effects of this can clearly be seen in the collapse of HD DVD developed by the DVD Forum (Edwards et al., 2005). PFGs are necessary to model such systems with externalities.
1.2.1 Coalitions and Coalition Structures
In this section, we will introduce notation and formalize terms. The set of agents in a MAS will be denoted 𝐴𝑔 = {𝑎1, … . , 𝑎𝒏}. The term coalition refers to a non-empty subset of 𝐴𝑔. The term coalition structure refers to a set of pairwise-disjoint coalitions. Formally, a coalition structure, 𝐶𝑆 is a set 𝐶𝑆 = {𝐶1… . 𝐶𝑘} of disjoint coalitions such that
⋃𝐶𝑖 = 𝐶, and 𝐶𝑖 ∩ 𝐶𝑗 = for every 𝑖, 𝑗 ∈ {1 … . 𝑘} 𝑎𝑛𝑑 𝑖 ≠ 𝑗.
We will denote the set of all coalition structure over 𝐶 as Π𝐶.
For example, for the four agent set 𝐴𝑔 = {𝑎1, 𝑎2, 𝑎3, 𝑎4}, a possible coalition is 𝐶 =
Chapter 1. Introduction 6 A coalition in a coalition structure will be denoted as 𝐶 combined with a subscript for example
𝐶1 or 𝐶𝑖 and combined with primes either in the form of 𝐶′ or 𝐶′′.
In coalitional games where externalities are considered, an embedded coalition is a pair consisting of a coalition and a coalition structure over 𝐴𝑔. For example, (𝐶, 𝐶𝑆) is an embedded coalition where 𝐶 represents a coalition and 𝐶𝑆 represents a coalition structure which includes 𝐶, i. e.,
𝐶𝑆 ∈ Π𝐴𝑔 and 𝐶 ∈ 𝐶𝑆.
The space of all embedded coalitions will be denoted ℇ𝐶.
1.2.2 Characteristic Function Games
A characteristic function games is a pair (𝐴𝑔, 𝑣) where 𝐴𝑔 denotes a finite set of agents and
𝑣: 2𝐴𝑔 → ℝ is the characteristic function representing the value 𝑣(𝐶) given to coalition 𝐶 ∈
P(𝐴𝑔) where 𝐶 is the powerset of 𝐴𝑔 excluding the empty set. The value 𝑉(𝐶𝑆) of a coalition structure 𝐶𝑆 is the sum of the values of the coalitions in the structure 𝐶𝑆:
𝑉(𝐶𝑆) = ∑ 𝑣(𝐶𝑖)
𝐶𝑖∈𝐶𝑆
(1.1)
1.2.3 Partition Function Games
Games in partition function form were introduced by Thrall and Lucas (Thrall & Lucas, 1963) based on research into cooperative game theory by Neumann and Morgenstern (Neumann & Morgenstern, 1947). For PFGs the value of a coalition is influenced both by the identities of its members together with the ways non-members are partitioned. A PFGs is a pair (𝐴𝑔, 𝑣) where 𝐴𝑔 is the set of agents and 𝑣, represents the value given to an embedded coalition,
(𝐶, 𝐶𝑆). Thus the value of a coalition C in the coalition structure CS is given by 𝑣((𝐶, 𝐶𝑆)). Formally, this is expressed as 𝑣: ℇ𝐶 → ℝ.
Chapter 1. Introduction 7 The value of a coalition structure 𝑉(𝐶𝑆) is given by:
𝑉(𝐶𝑆) = ∑ 𝑣(𝐶𝑖, 𝐶𝑆) 𝐶𝑖∈𝐶𝑆
(1.2)
This means that in a typical CFG, a coalition 𝐶 ⊆ 𝐴𝑔 has only one value, whereas in a PFG C may have as many values as the number of ways to partition the agents in 𝐴𝑔\𝐶.
1.2.4 Coalition Structure Generation (CSG)
The coalition structure generation (CSG) problem for both CFGs and PFGs involves finding an optimal coalition structure𝐶𝑆∗ ∈ Π𝐴𝑔 such that:
𝐶𝑆∗ ∈ 𝑎𝑟𝑔𝑚𝑎𝑥(𝑉(𝐶𝑆)).
𝐶𝑆 ∈ Π𝐴𝑔
Solving the CSG problem is important because there are numerous applications in various fields including bioinformatics, data mining, semantic web, natural language processing and machine learning (Di Mauro et al., 2014). The problem of finding an optimal coalition structure is computationally hard (Aziz & De Keijzer, 2011). While it is solvable using brute force search, its time complexity is exponential in the number of agents in 𝐴𝑔. This is not feasible with the processing capacity of the current state of the art computers.
For n agents, the number of possible coalition structures is given by the Bell number 𝐵𝑛(Graham et al., 1994) or Stirling number of the second kind 𝑆(𝑛, 𝑘) (Knuth, 1992) where the 𝑛th of these numbers counts the number of different ways to partition a set with 𝑛 elements into exactly 𝑘 nonempty subsets. This is calculated by the following recursive formula:
𝐵𝑛+1 = ∑ (𝑛 𝑘)
𝑛
𝑘=0
Chapter 1. Introduction 8 Table 1.1 – Number of possible coalition structures.
Number of elements/agents
𝑛
Bell number 𝐵𝑛
(Possible Number of Coalition Structures)
5 52 10 115,975 15 1,382,958,545 20 51,724,158,235,372 25 4,638,590,332,330,743,949 27 545,717,047,947,902,329,359
Table 1.1 shows how quickly the Bell number grows in 𝑛. Given these numbers, it is clear that a brute force method using present computational technology is impractical for finding an optimal coalition structure. Alternative methods that are practically feasible must be devised. This is the goal of this thesis.
1.2.5 Methods for Optimal Coalition Structure Generation
Existing methods for solving the coalition structure generation problem can be divided into three categories (Service & Adams, 2011):
(i) Anytime algorithms (Changder et al., 2016a; Rahwan et al., 2009; Rahwan et al., 2012; Sandholm et al., 1998; Service & Adams, 2010) – the quality of the solution generated improves with their execution time. Their disadvantage is that, in the worst case, they need to check all coalition structure requiring 𝑂(𝑛𝑛) time complexity.
(ii) Design-to-time algorithms (Yeh, 1986) – These methods guarantee returning an optimal solution, however can only do so if allowed to run to completion. DP (Yeh, 1986) belongs to this category and has a time complexity of 𝑂(3𝑛).
Chapter 1. Introduction 9 (iii) Heuristics algorithms (Di Mauro et al., 2010; Guo & Wang, 2006; Keinänen &
Keinänen, 2008; Sen & Dutta, 2000; Sukstrienwong, 2011) – These algorithm prioritise speed over solution quality. However, when used, it is impossible to provide any form of guarantees.
For CFGs, and more so for PFGs, finding an optimal coalition structure is computationally hard. In the existing literature, a number of methods have been studied for CFGs but the optimal coalition structure determination problem for PFGs has only recently become the focus of attention (see Chapters 3 and 4 for a review of existing methods). A number of deterministic methods have been developed for PFGs but they have exponential time complexity. This presents the need for developing effective heuristic methods for finding a good enough solution as quickly as possible, especially for settings with a large number of agents. Such methods are important for example in mission critical systems where a group of agents representing emergency responders need to partition their resources so the emergency situation is handled optimally. In these systems the agents need to react quickly and time lost looking for the absolute optimal can severely impact on handling the emergency. A quick locally optimal solution would be better than a delayed globally optimal one because the situation may have changed during the time (Di Mauro et al., 2014).
Against this background, the objectives of this research are as follows.
1.3 Research Objectives
The main objective of this research is to develop effective heuristic solutions to the coalition structure generation problem. To this end, four different heuristic search methods will be explored. Specifically, these four methods are as follows:
1. Tabu search method
2. Simulated annealing method 3. Ant colony search method
Chapter 1. Introduction 10 A key consideration in the design of heuristics will be their suitability to CFGs and also to PFGs. Other considerations are their running time memory space requirement, and their scalability.
Given the complexity of the problem, it is especially important and desirable when designing an algorithm, to balance the running time and the amount of memory required to execute a method. This can be achieved, perhaps by using efficient memory management which has been proven successfully adaptable in other combinatorial optimisation problems (Galinier et al., 2008). For example, having a tabu list which is relatively small could potentially be a more efficient way of using memory compared to the memory usage of other algorithms such as Dynamic Programming (Yeh, 1986). It is also worth noting there are no existing heuristic algorithms solving the optimal coalition structure generation problem for PFGs.
Thus, the aim of this thesis is to explore the above list heuristic approaches for solving the coalition structure generation problem and conduct a comparative analysis of their performance. Performance will be measured in terms of the above mentioned desirable characteristics.
1.4 Research Contributions
Heuristic methods have been applied to related optimisation problems such as the travelling salesman problem (Fiechter, 1994), bin packing problem (Lodi et al., 2004) and scheduling problems (Nonobe & Ibaraki, 2002). In solving the CSG problem without externalities i.e. CFGs, there are several heuristic methods such as greedy (Di Mauro et al., 2010), ant colony (Sukstrienwong, 2011), simulated annealing (Keinänen & Keinänen, 2008) and genetic algorithms (Sen & Dutta, 2000) that have been applied with promising results. However, there are no existing heuristic methods for generating optimal coalition structures for PFGs. This research contributes to the state of the art in the following ways:
1) Devising a range of heuristic methods for solving the coalition structure generation problem for CSGs and for PFGs.
2) Devising neighbourhood operators for effective exploration of the search space. 3) Devising compact representations for PFGs.
4) Analysing the performance of each heuristic method.
Chapter 1. Introduction 11 Publications from this research
A. Hussin and S. Fatima (Hussin & Fatima, 2016), Heuristic methods for optimal coalition structure generation, Lecture Notes in AI, 10207, pages 1 – 16, 2017. (DOI: 10.1007/978-3-319-59294-7 11)
1.5 Thesis Structure
The remainder of the thesis is organized as follows. Chapter 2 provides background information on coalitional games. Chapters 3 and 4 review the existing algorithms for solving the coalition structure generation problem. The methods reviewed cover all three types of methods present in literature, namely anytime algorithms, design to time algorithms and heuristic algorithms. Chapter 3 reviews the existing literature for CFGs and Chapter 4 covers existing methods for solving the CSG problem for PFGs. These including centralised and distributed methods.
Chapter 5 describes the four heuristic search methods: tabu search, simulated annealing, ant colony search, and particle swarm search methods.
Chapter 6 is a description of the set-up for conducting simulations for performance evaluation. Chapter 7 provides the result of the simulations. Performance is evaluated both in terms of the time taken to generate a solution and the quality of solution. The heuristic methods are also evaluated in terms of their scalability and how well they handle externalities. This chapter analyses the performance of each of the four heuristic methods for six different probability distribution data.
Chapter 8 presents an analysis of the average performance of each heuristic method across all the data sets. Scalability and the impact of externalities on performance are also analysed. Chapter 9 lists the main conclusions of this research and provides pointers for further research.
12
Chapter 2 Coalitional Games
This chapter is a background on coalitional games. Section 2.1 introduces characteristic function games and provides details regarding the definition of values of coalitions. Section 2.2 does the same for partition function games.
2.1 Characteristic Function Games
In some multi-agent systems, each coalition pursues its own goal with little or no interaction with other coalitions. A lack of interaction between coalitions means that the value generated by a coalition is independent of the external coalitions. Games with no externalities are known as characteristic function games (CFGs).
A CFG is represented as a pair (𝐴𝑔, 𝑣) where 𝐴𝑔 is the set of agents and 𝑣 is the characteristic function that gives the value of any coalition (Neumann & Morgenstern, 1947; Rapoport, 1970).
𝑣: 2𝐴 → ℝ.
Figure 2.1 is an illustration of values for an example game of 3 agents.
Chapter 2. Coalitional Games 13
Definition 1. A coalition structure is an exhaustive partition of a set of agents 𝐴𝑔 into pairwise-disjoint (or non-overlapping) coalitions. For example, for 𝐴𝑔 = {𝑎1, 𝑎2, 𝑎3}, there are exactly 5 possible coalition structures:
{{𝑎1}, {𝑎2}, {𝑎3}}, {{𝑎1, 𝑎2}, {𝑎3}}, {{𝑎1, 𝑎3}, {𝑎2}}, {{𝑎1}, {𝑎2, 𝑎3}} and {{𝑎1, 𝑎2, 𝑎3}}
For n agents, there are Bell (𝐵𝑛) ∼𝑂(𝑛𝑛) possible coalition structures, i.e., the number of coalition structures is exponential in 𝑛.
The value of a coalition structure is the sum of the values of its coalitions:
𝑉(𝐶𝑆) = ∑ 𝑣(𝐶𝑖)
𝐶𝑖∈𝐶𝑆
(2.1)
The precise details about how the value of a coalition is defined will be the subject of Chapter 6.
2.2 Partition Function Games
For partition function games (PFGs), the value of a coalition can be effected by the way the agents external to the coalition are organised (Thrall & Lucas, 1963).
2.2.1 The General Partition Function Game
A coalition structure 𝐶𝑆 is a partition of the set of agents in 𝐴𝑔. Any coalition 𝐶 ⊆ 𝐴𝑔 that is a member of a coalition structure 𝐶𝑆 is embedded in 𝐶𝑆. If the value assigned to a coalition 𝐶 is influenced by other coalitions in that structure, then each coalition may have different values depending on which structure it is embedded in. Let (𝐶; 𝐶𝑆) denote an embedded coalition. Let ℇ𝐶 represent the set of all embedded coalitions, and Π the set of all coalition structures.
Chapter 2. Coalitional Games 14 A PFG is comprised of:
A set of agents, 𝐴𝑔 = {𝑎1, . . , 𝑎𝑛}
A partition function 𝑤 that takes an embedded coalition (𝐶, 𝐶𝑆) ∈ ℇ𝐶 as input and assigns a real number value to coalition 𝐶 in the coalition structure 𝐶𝑆.
𝑤: ℇ𝐶 → ℝ
The value of a coalition structure 𝐶𝑆 is given by:
𝑉(𝐶𝑆) = ∑ 𝑤(𝐶𝑖, 𝐶𝑆)
𝐶𝑖∈𝐶𝑆
(2.2)
This is illustrated in Figure 2.2. A mentioned in Chapter 1, externalities are two main types: positive and negative. However, in some games, externalities may occur in both positive and negative forms. These are called mixed externalities games.
A mixed externalities game consists of a set of agents, 𝐴𝑔 and a partition function which takes, as input, every feasible coalition structure (CS), and for each coalition in each structure, outputs a numerical value that reflects the performance of the coalition in that structure which can either have positive (increase) or negative (decrease) value when moving from one structure to the other.
Chapter 2. Coalitional Games 15 Example
Given two coalition structures 𝐶𝑆 and 𝐶𝑆′ where 𝐶𝑆 = {{𝐶1}, {𝐶2}, {𝐶3}} and 𝐶𝑆′ =
{{𝐶1}, {𝐶2∪ 𝐶3}} the value of {𝐶1} may be different in 𝐶𝑆′ compared to 𝐶𝑆 as a result of the merger between 𝐶2and 𝐶3. This condition effecting 𝐶1is usually attributed as an externality implied on 𝐶1by the formation of coalition {𝐶2∪ 𝐶3} resulting from the merger of 𝐶2and 𝐶3 (Michalak et al., 2008).
Types of PFGs
Three types of Partition Function Games (PFGs) have been studied in literature: Games with positive externalities (Rahwan et al., 2012).
Games with negative externalities (Rahwan et al., 2012). Games with mixed externalities (Banerjee & Kraemer, 2010). A brief description of each is as follows:
1. Games with positive externalities – In games with positive externalities, the merger of some coalitions in a coalition structure adds value to the players who are external to that coalition, making them better off. In other words, the merger of any two coalitions gives a positive impact to other existing coalitions in the system (Rahwan et al., 2012). An example of environmental policy arrangement maybe the decision of a coalition of countries to cut pollution levels. This decision will impact other countries leading to positive externalities (Finus, 2003).
2. Games with negative externalities – For games with negative externalities, the merger of any two coalitions does not benefit the other existing coalitions (Rahwan et al., 2012). For example, collusion in oligopolies where cooperating organisations explore ways to undermine the competitiveness of other companies in the market (Catilina & Feinberg, 2006) gives rise to negative externalities.
3. Games with mixed externalities – In a game with mixed externalities, the formation of a new coalition can either induce a positive or negative externality upon other (non-member) coalitions in the structure. This game is a neither positive only nor negative only externality game as considered by Michalak et al. (2008). A game with mixed
Chapter 2. Coalitional Games 16 externalities can be modelled using agent types where two types of agents will form coalitions that induces either a positive or negative externality on the other type (Banerjee & Kraemer, 2010).
In general, for games with mixed externalities, it would be very difficult to find a solution to the coalition structure generation problem without exploring the entire space and checking every coalition structure (Rahwan et al., 2015). To put in perspective, consider a coalition structure generation problem with externalities where all the structures have been examined except one single structure. It is possible that this single structure has a higher value compared to all the other coalition structures in the system resulting from the externalities imposed on it.
In order to understand the degree of complexity involved in solving the CSG problem for PFGs, we must compare the size of their search space to the size of the search space for CFGs. Storing the values of coalitions by exhaustively listing them in memory is infeasible for all but very small PFGs, i.e., those with very few agents. This is because the number of possible partitions grows very rapidly as the number of agent increases. For example, for 20 agents, there are 4 × 1014 possible structures and exhaustive enumeration requires 394 terabytes of memory (Rahwan et al., 2012). For PFGs, any coalition can have as many different values as there are coalition structures. Storing the value of each coalition in memory is therefore infeasible for any but very small games. Figure 2.2 gives an indication of the size of search space for a PFG of 3 agents.
We therefore need more concise methods for dealing with this problem. The following section provides details regarding our approach for dealing with this problem. Note that Section 2.2.2 is our contribution and does not form part of literature review.
2.2.2 A compact representation for PFGs
In order to define coalition values compactly, we used the following approach. The value of a coalition in a structure is calculated in terms of two components:
an externalities-free value, and an externalities factor.
Chapter 2. Coalitional Games 17 The externalities-free value of a coalition C is defined in the same way as for a CFG, i.e.
𝑣: 2𝐴 → ℝ. The externalities factor is defined in terms of two components:
1. The size of the coalition, i.e., the number of member agents.
2. The size of the structure it is embedded in, i.e., the number of coalitions in the structure.
For a coalition𝐶, this factor is denoted ef and is calculated as follows:
𝑒𝑓 = |𝐶| 𝑑
(2.3)
where 𝑑 is the number of coalitions in the structure in which C is embedded.. As described below, this factor can be used to represent both positive and negative by adding and subtracting the externality free value using the formula given in Table 2.5.
2.2.2.1 Positive Externalities
In a PFG with positive externalities, the merger of any two coalitions decreases their joint value (or keeps it constant), and increases the values of other coalitions in the structure (or keeps them constant). The value 𝑣(𝐶) of any coalition 𝐶 is calculated as follows. Let 𝑅𝑉 denote a randomly drawn value from any probability distribution. This random value is increased by a factor of 𝑒𝑓to obtain 𝑣(𝐶) as follows:
𝑣(𝐶) = 𝑅𝑉 + (𝑅𝑉 × 𝑒𝑓) (2.4)
Therefore, we have the following:
1. If external coalitions merge, the number of coalitions in the structure, i.e. 𝑑 decreases. As a result, 𝑒𝑓 increases.
Chapter 2. Coalitional Games 18 Examples of games where only positive externalities occur includes projects to reduce deforestation in a group of countries benefits other countries environmentally. Another example is the decision by one group of countries to reduce pollution, which has a positive impact on other countries or regions, it induces positive externalities. The sharing of cars between people in a community, if a car is a coalition then merging two coalitions of people into a single car (coalition) benefits the other coalitions as there will be less cars in the structure reducing the traffic resulting in a positive externality on other coalitions in the structure.
2.2.2.2 Negative Externalities
Externalities are said to be negative if the merger of two coalitions reduces the value (or keeps them constant) of the other coalitions in the structure. Again, 𝑅𝑉 denotes a randomly drawn value from any probability distribution. This random value is decreased by a factor of 𝑒𝑓to obtain 𝑣(𝐶) as follows:
𝑣(𝐶) = 𝑅𝑉 − (𝑅𝑉 × 𝑒𝑓) (2.5)
Therefore, we have the following:
1. If external coalitions merge, number of coalitions in the structure, i.e.,𝑑 decreases and 𝑒𝑓 increases, and so will the value to be deducted from 𝑣(𝐶).
2. If 𝑒𝑓 increases, then 𝑣(𝐶) decreases as the original value is subtracted by 𝑅𝑉 × 𝑒𝑓 Situations where negative externalities occur are for example, when high-tech companies decide to cooperate in order to develop a new technology standard, other companies lose some of their competitive position, i.e., they are subject to negative externalities. Research & Development coalitions among pharmaceutical companies, when two companies decide to jointly develop a new drug, the market position of other companies is likely to decrease. Collusion in oligopolies, exogenous coalition formation in e-market places, as well as multi-agent systems with shared resources and/or conflicting goals all invoke negative externalities.
Chapter 2. Coalitional Games 19 A PFG with mixed externalities is represented as follows. Recall that when a coalition is effected by positive externalities, its value is calculated using Equation (2.4) and when it is effected by negative externalities, its value is calculated using Equation (2.5). In order to incorporate mixed externalities, we take the following approach which is similar to the approach taken by Banerjee and Kraemer (Banerjee & Kraemer, 2010):
1. The agents in a game are divided into different types.
2. Then, based on the types of member agents, coalitions are divided into different types. 3. Finally, based on the types of coalitions, coalition structures are divided into different
types.
The underlying idea is that the type of a coalition is correlated to the type of externality it can impose on external players.
Details regarding the types of agents, the types of coalitions and the types of coalition structures are the subject of the following section.
2.2.2.3A Representation for Mixed Externalities
Using this concept, agents are divided into two distinct types with the restriction that any game must contain both types of agents. The two types of agents are:
i) Type A
ii) Type B
Given that there are two type of agents, there can be the following three types of coalitions: i) Type AA – All agents in the coalition are Type A.
ii) Type BB – All agents in the coalition are type B.
Chapter 2. Coalitional Games 20 Example:
In a 3-agent comprised of 2 Type A agents and 1 Type B agent, the following types of coalitions are possible (supposing that agents a1 and a2 are Type A, and agent a3 is Type B):
Table 2.1 – Possible Coalition Types for 3-agents. Coalition Coalition Type
{a1} Type AA
{a2} Type AA
{a1, a2} Type AA
{a3} Type BB
{a1, a3} Type MX
{a2, a3} Type MX
{a1, a2, a3} Type MX
Three types of coalitions give rise to the following 5 types of coalition structures: i) Type AABB – the structure consists of type AA and type BB coalitions ii) Type AAMX – the structure consists of type AA and type MX coalitions. iii) Type BBMX – the structure consists of type BB and type MX coalitions.
iv) Type AABBMX – the structure consists of type AA, type BB and MX coalitions. v) Type MXMX – the structure consists only of type MX coalitions.
Chapter 2. Coalitional Games 21 Example:
In a 3-agent system comprised of 2 Type A agents and 1 Type B agent, the following types (see Table 2.2) of coalition structures are possible (supposing that agents a1 and a2 are Type A, and agent a3 is Type B):
Table 2.2 – Possible Coalition Structure Types. Coalition Structures Coalition Structure Type
{{a1, a2, a3}} Type MXMX (Grand Coalition)
{{a1, a2}, {a3}} Type AABB
{{a1, 3}, {a2}} Type AAMX
{{a1}, {a2, a3}} Type AAMX
{{a1}, {a2}, {a3}} Type AABB
In PFGs with only one agent of a type, certain types of coalition structure will not exist. The grand coalition will always be Type MXMX and the coalition structure of singletons is always type AABB if there is at least one agent of each type.
Example:
Number of Agents by Type Coalition Structure Types that Do Not Exist
1 Type A agent, 𝑛 − 1 Type B agents There will be no Type AAMX coalition structure 1 Type B agent, 𝑛 − 1 Type A agents There will be no Type BBMX coalition structure
Chapter 2. Coalitional Games 22 The effect of externalities
We defined the effect of externalities in terms of the types of coalitions. More precisely, the impact on a coalition when other coalitions form is defined as follows:
1) When a Type MX coalition forms as a result of a merger of a Type AA and a Type BB coalition, it will give positive externalities to Type AA coalitions but negative externalities to Type BB. The collusion will always benefits Type AA coalitions but always harms Type BB coalitions.
2) When a Type MX coalition forms and no Type AA or Type BB are present (i.e. the coalition structure consists only of Type MX coalitions), all coalitions in the structure are effected negatively. The idea is that Type MX coalition is a collusion between Type A and Type B agents. Therefore, if the coalition structure consists only of colluders that are colluding against each other, everyone loses.
3) When a Type AA coalition forms as a result of a merger of two Type AA coalitions, all other coalition types that are not singletons have their value increased due to positive externalities.
4) When a Type BB coalition form as a result of a merger of two Type BB coalitions, all other coalition types that are not singletons have their value increased due to positive externalities.
Special case for singletons (working alone may not be beneficial):
1) The effect of the formation of a structure on a singleton Type AA coalition is defined as follows. If the structure is Type AAMX or AABBMX, then the externality on the singleton is positive. If the structure is Type AABB, then the externality on the singleton is negative.
2) The effect of the formation of a structure on a singleton Type BB coalition is defined as follows. Regardless of the type of the structure, the externality on the singleton is negative.
Chapter 2. Coalitional Games 23
Table 2.3 – Externalities for each coalition type in a structure type. Coalition Structure Type Coalition Structure Membership First Coalition Type (Value) Second Coalition Type (Value) Third Coalition Type (Value)
AABB {{AA}{BB}} {AA} +
(Increased)
{BB} + (Increased)
N/A
AAMX {{AA}{MX}} {AA} +
(Increased) {MX} + (Increased) N/A BBMX {{BB}{MX}} {BB} – (Decreased) {MX} + (Increased) N/A
AABBMX {{AA}{MX}{BB}} {AA} +
(Increased) {MX} + (Increased) {BB} – (Decreased) MXMX {{MX}{MX}} {MX} – (Decreased) {MX} – (Decreased) N/A
Different consideration is given for singletons. Working alone is always bad for Type BB coalitions. However for Type AA coalitions when there is a Type MX coalition in the structure, it benefits from this coalition type (see Table 2.4).
Chapter 2. Coalitional Games 24 Table 2.4 – Externalities for singletons.
Coalition Structure Type Coalition Structure Membership First Coalition Type (Value) Second Coalition Type (Value) Third Coalition Type (Value) Fourth Coalition Type (Value) Fifth Coalition Type (Value)
AABB {{A}{AA}{BB}{B}} [A] – (Decreased) [AA] + (Increased) [BB] + (Increased) [B] – (Decreased) N/A
AAMX {{A}{AA}{MX}} [A] + (Increased) [AA] + (Increased) [MX] + (Increased) N/A N/A BBMX {{B}{BB}{MX}} [B] – (Decreased) [BB] – (Decreased) [MX] + (Increased) N/A N/A
AABBMX {{A}{AA}{MX}{BB}{B}} [A] + (Increased) [AA] + (Increased) [MX] + (Increased) [BB] – (Decreased) [B] – (Decreased)
The calculation of the value of a coalition for each of the two types of externalities is summarized in Table 2.5.
Table 2.5 – Formula for calculating positive/negative externalities.
Equation Externality Formula
(2.4) Positive Externality 𝑣(𝐶) = 𝑅𝑉 + (𝑅𝑉 × 𝑒𝑓)
Chapter 2. Coalitional Games 25 The calculation of the value of a coalition based on the structure it is embedded in is shown in Table 2.6.
Table 2.6 – Externalities from Other Coalitions in 𝐶𝑆 on Coalition 𝐶.
Coalition 𝐶 Other Coalitions in CS Value Calculation
Type AA Single Type (2.4)
Type BB Single Type (2.4)
Type AA At least one Mixed Type (2.4)
Type BB At least one Mixed Type (2.5)
Type MX Single Type (2.5)
Type MX At least one Type AA or BB (2.4)
Type AA Singleton At least one Mixed Type (2.4)
Type AA Singleton Single Type (2.5)
Type BB Singleton At least one Mixed Type (2.5)
Type BB Singleton Single Type (2.5)
The type of externality (i.e., positive or negative) imposed on a coalition in a structure is summarised in Table 2.7.
Chapter 2. Coalitional Games 26 Table 2.7 – Externalities on Coalition 𝐶 in Coalition Structure 𝐶𝑆.
Non-Singleton Coalitions
Coalition (𝐶) Type Coalition Structure (𝐶𝑆) Type Externality
S1. Type AA Type AABB POSITIVE
S2. Type AA Type AAMX POSITIVE
S3. Type AA Type AABBMX POSITIVE
S4. Type BB Type AABB POSITIVE
S5. Type BB Type BBMX NEGATIVE
S6. Type BB Type AABBMX NEGATIVE
S7. Type MX Type MXMX NEGATIVE
S8. Type MX Type AAMX POSITIVE
S9. Type MX Type BBMX POSITIVE
S10. Type MX Type AABBMX POSITIVE
Singleton Coalitions
S11. Type AA Singleton Type AABB NEGATIVE
S12. Type AA Singleton Type AAMX POSITIVE
S13. Type AA Singleton Type AABBMX POSITIVE
S14. Type BB Singleton Type AABB NEGATIVE
S15. Type BB Singleton Type BBMX NEGATIVE
Chapter 2. Coalitional Games 27 This representation is much more compact relative to the representation where a different value is stored for each coalition in each coalition structure. This is because, the externality free value (i.e., RV) is all that is needed to be saved in memory. Externalities are then incorporated on the fly by employing Equation 2.4 for positive and Equation 2.5 for negative. Let’s view the externalities in the context of Producers (Agent Type B) and Consumers (Agent Type A) for each of the scenario in Table 2.7. In a typical setting Producers are traditionally competitors thus the assumption is for each case is:
S1. Suppose a structure of Type AABB has resulted from the merger of two Type AA coalitions. Then the externality imposed on each coalition in the resulting structure is positive. Intuitively, this means that when small consumer coalitions come together to form a bigger coalition of consumers, their purchasing power increases. As a result, trade for products at the top end of the market increases and therefore both producers and consumers benefit (this corresponds to scenarios S1 and S4) except for singleton consumers. Intuitively, this means that, as singletons, they have less purchasing power and are at the bottom end of the market and the prices of products at this end of the market remain unchanged. The singleton consumers correspond to scenario S11.
S2. Suppose a structure of Type AAMX has resulted from the merger of two Type AA coalitions. Then the externality imposed on each coalition in the resulting structure is positive. Intuitively, this means that when small consumer coalitions come together to form a bigger coalition of consumers their purchasing power increases. As a result, trade for products at the top end of the market increases and therefore both producers and consumers (i.e. coalitions of Type AA and MX) benefit everyone (this corresponds to scenarios S2 and S8). Intuitively this means that, even singleton consumers gain because of the existence of Type MX coalitions which may contain just one consumer. Since even single consumers in Type MX coalition benefits, this advantage is passed on to external singleton consumers .The singleton consumers correspond to scenarios S12 and S13.
S3. Suppose a structure of Type AABBMX has resulted from the merger of two Type AA coalitions. Then the externality imposed on each Type AA coalition in the resulting structure is positive. Intuitively, this means that when small consumer coalitions join together to form a bigger coalition of consumer their purchasing
Chapter 2. Coalitional Games 28 power increases. This translates into real benefit for them and Type AA coalitions gets a positive externality. However the coalitions of Type BB get a negative externality (this corresponds to S6 and S16). Intuitively, this means that the needs of the newly merged Type AA coalition of consumers is met by producers in Type MX coalitions. Therefore Type MX coalitions benefit i.e. get a positive externality. Type AA singletons (singleton consumers) benefit and get a positive externality (this corresponds to S13).
S4. Suppose a structure of Type AABB has resulted from the merger of two Type BB coalitions. Then the externality imposed on each coalition in the resulting structure is positive. Intuitively, this means that when small producer coalitions come together to form a bigger coalition of producers their production power increases. As a result, all coalitions except the smallest of producers gain (are effected positively by externalities). What we mean by the smallest of producers is a singleton producers. The singleton producers corresponds to scenario S14 that get a negative externality.
S5. Suppose a structure of Type BBMX has resulted from the merger of two Type BB coalitions. Then the externality imposed on each Type BB coalition in the resulting structure is negative. Intuitively, this means that when small producer coalitions join together to form a bigger coalition of producers their production power increases, but this does not translate to a benefit but rather results in increased cost of creating a merger. This is because of the presence of Type MX coalitions. Intuitively this means that the consumers in the Type MX coalitions being already together in a coalition with producers prefer to work with those producers and not with the newly form merger of producers. This benefits Type MX coalitions and translates to positive externality on it. This corresponds to scenario S9.
S6. Suppose a structure of Type AABBMX has resulted from the merger of two Type BB coalitions. Then the externality imposed on each Type BB coalition in the resulting structure is negative (this corresponds to S6 and S16). Intuitively, this means that when small producer coalitions join together to form a bigger coalition of producers their production power increases, but this does not translate to a benefit but rather results in increased cost of creating a merger. This is because the needs of consumers are met by producers outside the newly formed merger. However, existing coalitions of consumers i.e. Type AA coalitions gain from the
Chapter 2. Coalitional Games 29 merger of producers and this translates into positive externality on Type AA coalitions (this corresponds to S3 and S13). In the same way Type MX coalitions are affected by positive externality.
S7. Suppose a structure of Type MXMX has resulted from the merger of two Type MX coalitions. Then the externality imposed on each Type MX coalition in the resulting structure is negative. Intuitively, this means the merger is unnecessary and counterproductive.
S8. Suppose a structure of Type AAMX has resulted from the merger of two Type MX coalitions. Then the externality imposed on each Type MX coalition in the resulting structure is positive.
S9. Suppose a structure of Type BBMX has resulted from the merger of two Type MX coalitions. Then the externality imposed on each Type MX coalition in the resulting structure is positive. However Type BB coalitions are affected negatively.
S10. Suppose a structure of Type AABBMX has resulted from the merger of two Type MX coalitions. Then the externality imposed on each Type MX and each Type AA coalition in the resulting structure is positive. However Type BB coalitions are affected negatively.
The above discussion is just a possible way of viewing the definition of positive and negative externalities. However our aim is not to consider a specific PFG such as that for producers and consumers. Rather it is to define a general coalitional game with all three externalities i.e. positive, negative and mixed inherent in the game.
2.3 Chapter Summary
This chapter introduced the key concepts that underlie coalitional games. A brief introduction to Characteristic Function Games and Partition Function Games was given. Finally, we introduced our approach for a compact representation of PFGs.
30
Chapter 3 Coalition Structure Generation
for Characteristic Function Games:
A Review of Literature
As outlined Chapter 1, algorithms for finding an optimal coalition structure can be classified into 3 categories (Service & Adams, 2011):
design-to-time algorithms, anytime algorithms, and heuristic algorithms.
This chapter provides a review of the existing methods in each of these three categories. Section 3.1 is about design-to-time algorithms, Section 3.2 about anytime algorithms, and Section 3.3. about heuristic algorithms.
3.1 Design-to-Time Algorithms
These algorithms are guaranteed to provide an optimal solution. However, a solution can only be provided when the algorithm terminates. Since they are not anytime, it is impossible to return any form of solution before completion. These algorithms have mostly been designed using dynamic programming (DP).
3.1.1 Dynamic Programming
Yeh (1986) developed a method based on dynamic programming (Bellman, 1952). Their algorithm maintains two tables 𝑓1 and 𝑓2 that contain an entry for every possible coalition. For every coalition 𝐶 ⊆ 𝑁, 𝑓1[𝐶] and 𝑓2[𝐶] are computed as follows: A best possible split (if any) for 𝐶 is stored in 𝑓1[𝐶] and its evaluation in 𝑓2[𝐶]. If it is best not to split 𝐶 then 𝑓1[𝐶] contains
𝐶 and 𝑓2[𝐶] the value of 𝐶. The value of every splitting 𝐶′, 𝐶′′ of 𝐶 is evaluated as 𝑓2[𝐶′] +
𝑓2[𝐶′′]. This method does not evaluate the splitting of size 𝑠 until it has finished computing
𝑓2 for the coalition of sizes 1 to 𝑠 − 1. Table 3.1 shows an example of how 𝑓1and 𝑓2 are computed for 𝑁={a1,a2,a3,a4}. Once 𝑓1and 𝑓2 are computed for every coalition, the optimal
