Md Masbaul Alam Polash
BSc, MSc
Institute for Integrated and Intelligent Systems School of Information and Communication Technology
Griffith University
Submitted in fulfillment of the requirements of the degree of Doctor of Philosophy
Abstract
Combinatorial problems are believed to be hard in general, most of them are at least NP-complete. Constraint-based approaches employ some convenient and generic techniques to solve these problems. These approaches use basic definitons to model and well-defined constraints to represent a problem. Although these approaches produce good results for smaller instances, for large-sized problems these do not perform so well. In this case, problem specific information can help to increase the scalability of these approaches. Thus in this research, our focus is to find out theoretically proven and heuristically promising properties that goes beyond the basic definition of a problem. These properties help to model the problem efficiently and to reduce the effective search space of a problem. Also, during search these properties can be used as auxiliary or streamlined constraint to boost the efficiency of a search algorithm. These properties along with efficient data structures and modern constraint-based techniques can be used to handle challenging combinatorial problems. The effectiveness of these techniques is shown throughout this thesis by solving several combinatorial problems, such as optimal Golomb rulers, all-interval series and propositional satisfiability.
Finding optimal Golomb rulers is an extremely challenging combinatorial problem. Dif-ferent approaches have been used so far to handle this problem. In this thesis, we provide
tight upper boundsfor Golomb ruler marks and presentsymmetry-based domain reduction
technique. Using these along with tabu and configuration checking meta-heuristics, we then develop a constraint-based multi-point local search algorithm to perform a satisfac-tion search for optimal Golomb rulers of specified length. We then present an algorithm to perform an optimisation search that minimises the length of a Golomb ruler using the satisfaction search repeatedly. Experimental results demonstrate that our algorithms perform significantly better than the existing state-of-the-art algorithms.
All-interval series is a standard benchmark problem for constraint satisfaction search. Different approaches have been used to date to generate all the solutions of this problem but the search space that must be explored still remains huge. In this thesis, we present a constraint-directed backtracking-based tree search algorithm that performs efficient
lazy checkingrather thanimmediate constraint propagation. Moreover, we prove several key properties of all-interval series that help to reduce the search space significantly. The reduced search space essentially results into fewer backtracking. We also present scalable
parallel versions of our algorithm that can exploit the advantages of having multi-core processors and even multiple computer systems. Experimental results show that our new algorithm exhibits better performance than the satisfiability-based state-of-the-art approach for this problem.
The propositional satisfiability (SAT) problem is one of the most studied combinatorial problems in computer science. In recent years, local search approaches have become one of the most effective techniques in solving these SAT problems. In this research, our focus is to exploit the hidden structures of SAT problems in local search. These structures are generated in the form of logic gates. Due to the detection of gates, both the number of independent variables and external gates decreases. Thus the search space becomes narrower than before. But in some cases, the number of external gates or the number of independent variables may become too few to guide the search efficiently. In these cases, detecting only few but not all types of gates will actually perform better. Thus in this research, we investigate the effect of detecting only the basic gates and also all types of gates. Then a dependency lattice is created to propagate the value of the independent variables. However, detection of gates may led to the problem of cycling in the dependency lattice. A new mechanism is proposed to remove those cycles as well. Moreover, we propose a new stagnation recovery technique to handle the cycling problem of local search. The experimental study on structured benchmarks shows that our new approach significantly outperforms the corresponding CNF-based implementations.
Statement of Originality
This work has not previously been submitted for a degree or diploma in any university. To the best of my knowledge and belief, the thesis contains no material previously published or written by another person except where due reference is made in the thesis itself.
List of Publications
The main contributions of this study are either published or in preparation for submission to the reputed journals in the field.
Journal Articles
• Md. Masbaul Alam Polash, M. A. Hakim Newton, Abdul Sattar; “Constraint-based search for optimal Golomb rulers ”, Journal of Heuristics, volume: 23 (6), pp. 501-532, Springer, 2017. ISSN 1572-9397; DOI 10.1007/s10732-017-9353-x.
• Md. Masbaul Alam Polash, M. A. Hakim Newton, Abdul Sattar; “Constraint-directed Search for All-interval Series”, Constraints Journal, volume: 22(3), pp. 403–431, Springer, 2017. ISSN 1383-7133; DOI 10.1007/s10601-016-9261-y.
Conference Paper
• Md. Masbaul Alam Polash, M. A. Hakim Newton, Abdul Sattar; Constraint-based local search for optimal Golomb rulers. In: L. Michel (Ed.): Proc. of CPAIOR, LNCS, vol. 9075, pp. 322–331, 2015. DOI 10.1007/978-3-319-18008-3 22
Papers to be Submitted
• Exploiting the structures of SAT problems in local search
Acknowledgement
This work has benefited greatly from the guidance and support of many people over the past few years. I would like to express my gratitude to everyone who contributed to it in one way or other. But first of all, I would like to thank the Almighty for giving me the strength to complete this work. And also thanks to my family members especially my parents, for their unconditional support and love throughout my life.
I would like to thank my main supervisor Professor Abdul Sattar whose wonderful temperament and canny advice were always appreciated. His constructive discussions and brilliant suggestions helped a lot throughout my PhD candidature. A big thank from the bottom of my heart must go to my associate supervisor Dr. M.A. Hakim Newton. Whenever I faced a difficult challenge, I found him beside me. His continuous guidance and constant support always kept me in the right direction. I owe a great debt of gratitude to him for his supervision as well as greatly assisting in various other ways.
I am also thankful to my lab mates for creating such an wonderful working environment. I would like to acknowledge the generous financial assistance from Griffith University, without which this work would not have been possible. Special thanks also to the Research Computing Services of Griffith University for providing the computing infras-tructure and IT support during my PhD.
And finally, I thank my friends in Brisbane for the fantastic time we have had, and my friends on the other side of the world for keeping in touch despite the distance.
1 Introduction 1
1.1 Motivation . . . 1
1.2 Problems Addressed . . . 3
1.3 Overview of the Contributions . . . 3
1.3.1 Golomb Rulers . . . 4
1.3.2 All-interval Series . . . 5
1.3.3 Propositional Satisfiability . . . 5
1.4 Outline of the thesis . . . 6
2 Preliminaries 7 2.1 Constraint Satisfaction Problems . . . 7
2.1.1 Search Strategies . . . 8
2.2 Local Search . . . 9
2.2.1 Iterative Improvement . . . 11
2.2.2 Tabu Search . . . 11
2.2.3 Configuration Checking . . . 12
2.2.4 Variable Neighborhood Search . . . 12
2.2.5 Constraint Directed Variable Neighborhood Search . . . 13
2.2.6 Software Tools for Constraint-based Local Search . . . 14
2.3 Global Constraints . . . 14 2.4 Evolutionary Algorithm . . . 15 2.4.1 Representation . . . 16 2.4.2 Evaluation Function . . . 17 2.4.3 Initial Population . . . 17 2.4.4 Parent Selection . . . 17 2.4.5 Reproduction . . . 18 2.4.6 Mutation . . . 19
2.4.7 Selection of the New Generation . . . 20
2.4.8 Termination . . . 21
2.5 Variable Selection Heuristics . . . 21
2.6 Conclusion . . . 23
3 Golomb Rulers 24 3.1 Golomb Ruler Problem . . . 24
3.1.1 Applications . . . 25
3.1.2 Challenges . . . 26
3.2 Organisation of this chapter . . . 27
3.3 Related Work . . . 28
3.3.1 A Recent Hybrid Local Search Algorithm . . . 30
3.3.2 A Recent Hybrid Genetic Algorithm . . . 31
3.4 TabuCC Approach . . . 31
3.4.1 Problem Model . . . 31
3.4.2 Avoiding the Cycling Problem . . . 33
3.4.3 Search Algorithm . . . 34
3.4.4 Experimental Setup . . . 35
3.4.5 Experiments and Analyses . . . 35
3.5 The RangeBound Approach . . . 39
3.5.1 New Golomb Ruler Representation . . . 39
3.5.2 New Tight Bounds for Marks . . . 40
3.5.3 Constraint-Based Multi-Point Local Search . . . 44
3.5.4 Constraint-Based Local Search . . . 46
3.5.5 Experimental Results . . . 47
3.6 Symmetry-Based Domain Reduction . . . 49
3.6.1 Experimental Results . . . 53
3.7 Optimisation Search . . . 55
3.7.1 Experimental Results . . . 57
3.8 Parallel Optimisation Search . . . 59
3.8.1 Experimental Results . . . 62
3.9 Conclusion . . . 63
4 All-interval Series 64 4.1 All-interval Series Problem . . . 64
4.1.1 Challenges . . . 65
4.2 Organisation of this chapter . . . 65
4.3 All-Interval Series Preliminaries . . . 66
4.4 Related Work . . . 68
4.4.1 CP-Based Approach. . . 69
4.4.2 SAT-Based Approach. . . 69
4.5 New RAIS Properties . . . 69
4.6 New Auxiliary Constraints . . . 74
4.7 Constraint-Driven Backtracking Search . . . 76
4.8 Immediate Propagation . . . 76 4.8.1 Data Structures . . . 77 4.8.2 Required Procedures . . . 79 4.8.3 Experimental Results . . . 83 4.9 Lazy Checking . . . 85 4.9.1 Data Structures . . . 85 4.9.2 Required Procedures . . . 86 4.9.3 Experimental Results . . . 87 4.10 Parallel Algorithms . . . 88 4.10.1 Multi-Process Version . . . 88 4.10.2 Client-Server Version . . . 90 4.10.3 Experimental Results . . . 90
4.12 Improving Non-Constraint-Based Search . . . 93
4.13 Conclusion . . . 100
5 Propositional Satisfiability 102 5.1 Introduction . . . 102
5.1.1 Propositional Satisfiability Problem . . . 102
5.2 Stochastic Local Search Algorithms for SAT . . . 104
5.2.1 The GSAT Algorithm . . . 104
5.2.2 The GWSAT Algorithm . . . 105
5.2.3 GSAT/Tabu Algorithm . . . 105
5.2.4 HSAT and HWSAT Algorithm . . . 105
5.2.5 WalkSAT Algorithm . . . 106
5.2.6 The Adaptive Noise Mechanism for WalkSAT . . . 107
5.3 Non-CNF SAT Solvers for Structured Instances . . . 108
5.4 Our Approach . . . 111
5.4.1 Gate Detection . . . 112
5.4.2 Handle Multiple Definition . . . 115
5.4.3 Detect and Remove Cycles in Lattice . . . 115
5.4.4 Generic Framework of Our Approach . . . 117
5.4.5 Non-CNF Algorithm Implementation . . . 118
5.5 Experimental Results . . . 121
5.5.1 Effect of Unit Propagation . . . 122
5.5.2 Effect of Tabu Metaheuristics . . . 123
5.5.3 Effect of Gate Detection . . . 125
5.6 Escaping Stagnation in Local Search for SAT . . . 136
5.7 New Stagnation Recovery Technique . . . 138
5.7.1 Impact of New Stagnation Recovery Technique . . . 138
5.8 Conclusion . . . 144
6 Conclusion 145 6.1 Golomb Ruler Problem . . . 145
6.2 All-interval Series Problem . . . 147
6.3 Propositional Satisfiability Problem . . . 149
3.1 Success rates of different algorithms when 25 attempts are made for each
given timeout. . . 37
3.2 Effect of using tabu metaheuristic. . . 38
3.3 Success rates of different algorithms when 25 attempts are made for each given timeout. . . 49
3.4 Success rates of different algorithms when 25 attempts are made for each given timeout . . . 55
3.5 Average time statistics when different length is used as the upper bound for optimisation search . . . 59
4.1 Performance of different CP versions with auxiliary constraints . . . 75
4.2 Backtracking-based tree search for RAIS of size 7 . . . 77
4.3 Maintaining udand ad arrays in immediate propagation technique . . . . 79
4.4 Parallel architectures: multi-process and client-server . . . 89
4.5 Memory statistics for each size of the series . . . 98
5.1 Possible gate structure with two inputs . . . 112
5.2 Clausal representation of Ex-Nor, AND and OR gates . . . 112
5.3 Clausal representation of ExOR, NOR and NAND gates . . . 113
5.4 An example of dependency lattice . . . 120
5.5 Effect of using tabu metaheuristic with tenure 5 and using no tabu. In-stances are arranged in an increasing order of the solution times when tabu is used with tenure 5. . . 124
5.6 Average reduction in the number of variables and clauses . . . 125
5.7 Average success rate for different domain when different timeout are used in AdaptNovelty+ algorithm. . . 130
5.8 Average success rate for all domains when different timeout are used in AdaptNovelty+ algorithm. . . 131
5.9 Average success rate for different domain when different timeout are used in CCAnr algorithm. . . 133
5.10 Average success rate for all domains when different timeout are used in CCAnr algorithm. . . 134
5.11 Comparison between adaptNovelty+ and CCAnr versions. . . 136
5.12 Impact of new stagnation recovery technique in (a) adaptNovelty+ and (b) CCAnr algorithm within different timeout limit . . . 139
5.13 Average success rate for different domain when different timeout are used in AdaptNovelty+ with new stagnation recovery technique. . . 140
5.14 Average success rate for all domains when different timeout are used in AdaptNovelty+ with the new stagnation recovery technique. . . 140
5.15 Average success rate for different domain when different timeout are used in CCAnr algorithm along with the new stagnation recovery technique . . 141 5.16 Average success rate for all domains when different timeout are used in
CCAnr algorithm along with the new stagnation recovery technique. . . . 142 5.17 Comparing all versions of adpatNovelty+ and CCAnr algorithm. . . 143 5.18 Comparing total elapsed time of different approaches for different domains 144
2.1 Software tools for CBLS . . . 14
3.1 A 16-mark optimal Golomb ruler . . . 24
3.2 Performance of our algorithm when compared to GRHEA [1] . . . 36
3.3 Effectiveness of CC heuristics . . . 38
3.4 A 16-mark OGR and its corresponding mirror Golomb ruler . . . 40
3.5 After computing bounds, the range for each mark in a 16-mark OGR . . . 42
3.6 Comparison of various bounds for each mark in a 16-mark OGR . . . 43
3.7 Average domain reduction for OGRs of order 11-17 when different upper bounds are used to determine the domain of a variable . . . 43
3.8 Experimental results of CBLS algorithms . . . 48
3.9 Experimental results of MPLS algorithm . . . 49
3.10 Deviation in number of marks in first one, two and three quarters of known OGRs of order 10-27 . . . 50
3.11 Bounds for a 16-mark OGR after domain reduction . . . 51
3.12 Actual and mirror marks ofm-mark OGR . . . 52
3.13 Actual and mirror marks of a 14-mark OGR . . . 52
3.14 Experimental results of CBLS algorithm . . . 53
3.15 Experimental results of MPLS algorithm . . . 54
3.16 Performance of our algorithm when compared to scatter search [2] . . . . 58
3.17 Finding optimal rulers using different strategies . . . 62
3.18 Experimental results for finding optimal rulers . . . 62
4.1 Execution time comparison . . . 76
4.2 Effect of new properties in immediate propagation . . . 83
4.3 Execution time comparison . . . 84
4.4 Search failure comparison . . . 84
4.5 Effect of new properties in Lazy checking . . . 87
4.6 Execution time comparison . . . 88
4.7 Performance of parallel versions of lazy checking . . . 91
4.8 Varying the task level in client-server version . . . 91
4.9 Varying the number of clients in client-server version . . . 92
4.10 Varying the number of process per client in client-server version . . . 92
4.11 Parameter tuning . . . 97
4.12 Effectiveness of our heuristics in counting solutions . . . 97
4.13 Effectiveness of Lemma 4.19 in generating all the solutions . . . 100
5.1 Effect of unit propagation in different instances . . . 123
5.2 Effect of gate detection in Blocksworld instance. . . 126
5.3 Effect of gate detection in SSA instance . . . 126
5.4 Effect of gate detection in Parity instance . . . 127
5.5 Effect of gate detection in Logistics instance. . . 127
5.6 Effect of gate detection in BMC instance . . . 128
5.7 Effect of gate detection in QG instance . . . 129
5.8 Effect of gate detection in AdaptNovelty+ algorithm . . . 132
Introduction
Combinatorial problems commonly imply finding values to a set of variables. These variable-value assignments are often restricted by a set of constraints. Constraints are sometimes used to optimise a certain function (optimisation) and in some cases only to find a valid solution (satisfaction). Combinatorial search algorithms solve instances of problems that are believed to be hard in general (most of them are at least NP-complete [3]) by exploring the usually large solution space of those instances. These algorithms are being able to achieve this by reducing the effective size of the search space and by exploring it efficiently.
1.1
Motivation
Combinatorial problems are believed to be hard in general, most of them are at least NP-complete. Constraint-based approaches employ some convenient and generic techniques to solve these problems. These approaches use basic definitons to model and well-defined constraints to represent a problem. Although these approaches produce good results for smaller instances, for large-sized problems these do not perform so well. In this case, problem specific information can help to increase the scalability of these approaches. Thus in this research, our focus is to find out theoretically proven and heuristically promising properties that goes beyond the basic definition of a problem. These properties help to model the problem efficiently and to reduce the effective search space of a problem. Also, during search these properties can be used as auxiliary or
streamlined constraint to boost the efficiency of a search algorithm. These properties along with efficient data structures and modern constraint-based techniques can be used to handle challenging combinatorial problems. The effectiveness of these techniques is shown throughout this thesis by solving several combinatorial problems, such as optimal Golomb rulers, all-interval series and propositional satisfiability.
Local Search is known to be a powerful technique especially for dealing with the problems having significantly large search space. Evolutionary algorithms are population based algorithms that mimic biological processes. Memetic algorithms are hybrids that uses local search to yield better results and converge to higher quality solutions.
The first research work presented deals with solving a very hard combinatorial optimi-sation problem known as optimal Golomb Ruler problem. The research developed here focuses on reducing the search space of this problem by using problem-specific knowl-edge. This knowledge is being exploited in reducing the domain of each variable of the OGR problem. Then memetic algorithms are used for modeling and solving optimal and near-optimal Golomb Rulers, providing high quality results that are consistently superior to those presented in the literature.
Systematic algorithms perform well on many large and complex real-world problems. The branching heuristics of a systematic search (i.e tree-based backtracking approach) can benefit from the reduced search space. The research presented here deals with solving Constraint Satisfaction Problems, in particular for a well-known problem named all-interval series problem using systematic search.
Furthermore, problem-specific information is used to reduce the search space of the well-known propositional satisfiability (SAT) problem. For satisfiability, mainly the gate structure information is extracted to decrease the number of choice variables. These extracted gate structure information is used in several local search based SAT solvers to improve the performance efficiency of those solvers.
In the next sections we introduce the problems dealt with throughout the whole research, we establish the boundaries of our work and present its main contributions.
1.2
Problems Addressed
Finding an optimal Golomb ruler (OGR) is an extremely challenging problem. It has interesting applications in many areas that include x-ray crystallography [4], radio as-tronomy [5], information theory [6], pulse phase modulation [7] etc. Moreover, because of its highly combinatorial nature, it has become a standard benchmark to evaluate and compare a variety of search techniques. Different approaches have been proposed to deal with this challenging problem. In particular, genetic algorithms, constraint pro-gramming, local search, and their hybridisations have all been applied to the problem of finding Golomb rulers (e.g., [1, 2, 8–13]).
The all-interval series (AIS) problem was first presented in [14] and was later added to the CSPLIB (www.csplib.org) constraint satisfaction problem library as prob007. The AIS problem is a straightforward example of the graceful graphs problem [15] in which the graph is a path. Since its introduction, it is often used as a standard benchmark problem for constraint satisfaction search (e.g. [16–18]). To deal with this problem, different approaches have been used to date. These include local search for satisfiability [14, 19, 20] and constraint programming [21–24].
In last few decades, the Propositional Satisfiability (SAT) has become a standard for en-coding challenging real world problems. Various research fields like artificial intelligence and constraint programming are now benefited from SAT. These SAT solvers have also been successfully used in solving many practical and industrial applications that include electronic design automation, planning, scheduling, image interpretation, hardware and software verification etc.
1.3
Overview of the Contributions
This research shows that problem specific information can be used to reduce the search space of the combinatorial problems. We also aim to show that different techniques can cooperate in a single algorithm to yield high quality results. Thus, the scope of this research is problem modeling, problem solving, and hybrid algorithms composed with Constraint Satisfaction Problem (CSP), Local Search (LS) and Evolutionary Algorithm (EA) techniques.
This thesis deals mainly with problem solving, and every chapter reports top results in the literature for the various problems addressed. Also, several problem oriented search space reduction techniques are presented as well. The research presented in this thesis contributes to the state of the art in presenting high quality results for the Golomb ruler problem, all-interval series problem and propositional satisfiability problem. Thus the main contributions are:
1.3.1 Golomb Rulers
First of all, the effectiveness of constraint-based local search (CBLS) approach for Op-timal Golomb Ruler (OGR) problem is presented. For the CBLS approach, besides the traditional way of enforcing tabu on recently modified variables for a given number of iterations, a special type of tabu called configuration checking (CC) [25] is also used. The use of CC effectively reduces the number of restarts required during search and thus mitigates thecycling problem of local search for an OGR. This approach that uses tabu
andconfiguration checking to find OGRs has been published in a conference proceeding in [26].
A new representation for Golomb rulers has been introduced. Moreover, tight upper boundsfor each variable of an OGR problem is defined. These upper bounds significantly curtail the search space. Also, an effectivesymmetry-based domain reduction technique is presented here to further reduce the search space. Using these techniques, the CBLS approach performs better than the state-of-the algorithms for this OGR problem.
The scalable parallel versions of the algorithms have been provided that solve large problems using multiple processes on a single or multiple computers. For quite some time, Golomb rulers with 16 or more marks have posed severe hindrance to the progress of constraint-based OGR search. Overall, in this paper, we advance the satisfaction search for OGRs up to 19 marks and the optimisation search up to 17 marks.
This new approach for the satisfaction and optimisation version of the OGR problem has been published as a journal article in [27].
1.3.2 All-interval Series
Several new properties of the reformulated All-interval Series problem has been proved to help reduce the search space. This reduced search space significantly resulting into fewer backtracking for the systematic search. The effectiveness of the new properties has been shown by using them on top of the CP model used in [23]. A constraint-directed backtracking-based tree search algorithm is developed that incorporates these new properties.
We also develop a very efficientlazy checking technique that performs significantly better in finding reformulated AIS than the typical immediate propagation technique used in constraint solvers.
Moreover, we present scalable parallel versions of the tree-search algorithm that solve large problems using multiple processes and multiple computers. These algorithms now generate all AIS solutions of size up to 27 along with significant speed up in problems of size up to 24. We also use the lazy constraint checking in the non-constraint-based breadth-first search of [28] to count the numbers of solutions up to size 48. Moreover, we save significant amount of time in generating all solutions of sizes up to 29 by using a non-constraint-based depth-first search that uses the AIS representation used in [28].
This new methodology to handle the AIS problem has been published in [29].
1.3.3 Propositional Satisfiability
In this thesis, we propose a methodology to exploit structure information in local search for SAT. We detect all the possible types of logic gates from the CNF formula. The idea of gate detection is inspired from [30]. Due to the detection of gates, both the number of independent variables and external gates decreases. Thus the search space becomes narrower than before. The number of satisfied external gates are used as the scoring function of the non CNF search algorithm. When all the external gates are satisfied, the algorithm returns the current assignment as a solution. This scoring function provides limited guidance to greedy moves in cases where there are few constrained output gates or they are far from the input gates. In these cases, detecting only few but not all types of gates will actually perform better. Thus in this research, we investigate the effect of
detecting only the basic gates and also all types of gates. However, detection of gates may led to the problem of cycling in the dependency lattice. Thus a new mechanism is proposed here to remove these cycles as well.
To demonstrate the efficiency of exploiting structure information in SAT, we implement the non CNF version of two better performing local search solver named adaptNov-elty+ [31] and CCAnr [32]. Experimental analysis exhibits that the non CNF versions along with gate structure information outperforms the CNF versions of these algorithms.
Local search techniques are one of the best approaches to solve SAT poblems, but they suffer from thecycling problem. To handle this challenge, different techniques has been proposed so far. In this research, we propose a new stagnation recovery technique. Using this technique, we are able to improve the efficiency of the non CNF version described before.
1.4
Outline of the thesis
Following this introductory chapter, the rest of the thesis is organised in a number of related chapters. In Chapter 2, we review the key concepts of constraint satisfaction problems, constraint-based local search, and evolutionary algorithm. Chapter 3 identifies and discusses the research challenges of the Golomb ruler problem and then outlines the work corresponding to those challenges; Chapter 4 presents the work corresponding to the all-interval series problem; Chapter 5 presents the work corresponding to the propositional satisfiability problem; Finally, Chapter 6 presents a summary of this work, outlines some future directions to extend this work and concludes the thesis.
Preliminaries
This chapter presents the preliminaries of some basic techniques that have been used throughout this thesis. At first, it defines the constraint satisfaction problem and de-scribes different search strategies to handle those problems. Then an overview of the constraint-based local search and evolutionary algorithm is outlined. Also, several vari-able selection heuristics of local search are illustrated here.
2.1
Constraint Satisfaction Problems
A constraint satisfaction problem (CSP) is defined as the problem of finding an as-signment of values to variables so that it satisfies all the specified constraints on those variables, or proving that such an assignment does not exist. In general, many real world problems have constraints on variables: for examples, the tangible constraints (e.g. the constraints on assets such as cost, cash flow, and debt) on budget management problems or the temporal constraints on planning and scheduling problems (e.g.block world problem, sports tournament scheduling etc). These problems can be described as CSPs and solved efficiently using constraint-based solvers. In fact, the constraint-based approach has been successfully applied to solve different types of real-world problems such as bin packing, timetabling, DNA sequencing, circuit layout design etc. [33].
Definition: A CSP is formally defined as a tuple (V, D, C) where
• V ={v1, v2,· · ·, vn} is a finite set of variables
• D={d1, d2,· · · , dn} is a set of finite domains where each domaindi contains the
finite set of values that can be assigned to the variable vi and
• C = {c1, c2,· · · , cm} is a finite set of constraints that restrict the assignment of
values to those variables inV.
Similarly, a constraint optimisation problem (COP) can be defined as the problem of finding an assignment of values to variables so that a pre-defined objective function is maximised (or minimised) under that assignment. In COP there are two types of constraints called hard and soft constraints. Hard constraints are those which must be satisfied by a solution of a problem. And the solution to a COP may or may not be required to satisfy some of the specified constraints on the given variables, those constraints are known assoft constraints. However, a CSP can be viewed and solved as a COP where the objective function is to minimise the number of unsatisfied constraints. In this case, the CSP is satisfiable if there exists an assignment under which the objective function is equal to zero.
2.1.1 Search Strategies
Like other finite-domain artificial intelligence problems, CSPs are usually solved using a search algorithm. In general, there are two main types of search strategy for solving CSPs and COPs: systematic and non-systematic. Typically, a systematic or backtracking algorithm starts with a partial assignment (i.e. only a subset of variables are assigned values and the remainder remains free) and attempts to extend this assignment to a complete one while ensuring all the constraints are satisfied. In other words, it divides the search space hierarchically into a tree structure, where each node represents an assigned variable and each branch coming out of a node represents the value that is assigned to that node. Whenever the search cannot find a domain value to assign to a free variable, it will backtrack to the previous node and attempt to find a new value for the variable at that node. If there is a valid value, the search will continue to travel down the search tree and backtracks if necessary until a solution is found. Otherwise, the search will backtrack further. As a result, systematic algorithms are able to prove that a problem is satisfiable or not. In the optimisation case, systematic algorithms guarantee to find the optimal solution by generating all the solutions to a problem.
Therefore, these algorithms are complete. Unfortunately, despite many improvements in propagation techniques and variables/values ordering heuristics, these algorithms still do not perform well on many large and complex real world problems [33].
In contrast, a non-systematic algorithm starts with a full assignment that may not satisfy all the constraints. It then iteratively tries to improve the current assignment by assigning new values to a subset of variables. The selection of the next move (i.e. the combination of variables and to-be-assigned values) is heuristically guided towards the solution. In the satisfaction case, the objective is to minimise the number of violated constraints. In the optimisation case, the objective also includes optimality criteria to move closer to the optimal solution. These algorithms are known as incomplete because they cannot prove that a CSP is unsatisfiable and cannot guarantee to find the optimal solution for a COP. Despite this incompleteness, these algorithms produce better results within a reasonable time frame than systematic algorithms when solving large and complex problems [34].
2.2
Local Search
Local search is one of the most efficient non-systematic approach. It starts from a random assignment of values to all the variables. Then it randomly moves from one candidate solution to its neighboring solution in the hope of optimising the value of the objective function. Although it sacrifices quality guarantee, it is particularly useful for large-scale problems where (good) solutions must be found within a fixed time limit.
One idea of using constraints to describe and guide local search is currently emerging which is termed as constraint-based local search (CBLS). It is among the most successful techniques for solving computationally hard problems from computing science, opera-tions research and various application areas such as traveling salesman problem, routing and scheduling problems etc. [35].
The key idea is that the CBLS starts from a randomised initial solution and iteratively improves the current candidate solution with small modifications until the termination criteria are met. In general, CBLS is able to find a good solution but cannot guarantee the most optimal one. Despite the fact that it is incomplete, it has the advantage of
using less memory and being able to find acceptable solutions in a reasonable time for large and complex problems.
Given a problem instance, a CBLS algorithm generally has following components [35]:
• A search spacethat is the finite set of all candidate solutions.
• Aneighborhood relationthat determines the criteria to generate the neighbor-ing candidate solutions of an observneighbor-ing solution.
• A finite set of memory state that specifies the stored knowledge during search beyond the candidate solutions (e.g. the tabu tenure in Tabu search [36]) and can be used later as an additional search guidance for selecting a candidate solution from the neighbors.
• An initialisation functionthat indicates an initial search position and memory states.
• Astep functionthat maps each search position and memory states onto a prob-ability distribution over its neighboring positions and memory states.
• The termination criterionthat determines the criteria to terminate the search based on the current search position and memory states.
Among the above components, the neighborhood relation and the step function are particularly important. An intelligent neighborhood relation can restrict the local space surrounding the current solution and a smart step function can speed up the search. The search is typically guided by an evaluation function that is used to heuristically assess or rank candidate solutions. Furthermore, some CBLS methods use more than one evaluation function and some can even adjust the evaluation function while searching.
Most problems can be converted to a search problem in which all candidate solutions create a search space. The requirements for the target are formulated in terms of several constraints and an objective function. There are three issues needed to be defined when formalising a problem for local search. The first one is the initialisation that defines how an initial candidate solution can be generated. The second and the most important one is how to define the neighborhood relationship between two candidate solutions. The
third one is how to evaluate the quality of neighboring candidates, i.e. how to define the objective function.
Diversification and intensification are two important issues that a local search algorithm needs to address. Intensification ensures that CBLS algorithms move quickly toward the target in the local area until the most optimal solution can be found. The best solution found in a local area is called local minimum. Diversification ensures that CBLS algorithms explore widely in an area large enough to reach the global minimum, which is the best solution among all local minima. Therefore, a good CBLS algorithm should balance between diversification and intensification.
2.2.1 Iterative Improvement
The most basic Constraint-based Local Search (CBLS) algorithm, namely iterative de-scent, is based on the idea of iteratively improving a candidate solution of a given problem with respect to an evaluation function,f. As shown in Algorithm 1, the search starts from an initial solution. At each iteration, it replaces the current candidate so-lution s with a neighboring candidate solution s0 such that f(s0) < f(s). The search is terminated when a local minimum is reached, i.e. f(s) < f(s0) for all the neighbors
s0 of s. The drawback of this method is that it does not have a mechanism to escape local minima. The only cure for stagnation is to restart the search from a different position. There are several approaches that allow CBLS to escape from local minima by occasionally accepting worsening search steps or randomly choosing a neighbor.
Algorithm 1:Iterative Descent 01 s← initial solution
02 whiletermination condition are not met AND shas better neighbor
03 choose a neighbor s0 ofs such thatf(s0)< f(s) 04 s←s0
05 return s
2.2.2 Tabu Search
The key idea behind tabu search [36, 37] is to use short-term memory in order to prevent a subsidiary iterative improvement from returning to recently visited search positions.
A tabu search in combination with an iterative improvement algorithm is presented in Algorithm 2. When performing iterative improvement steps, only permissible neighbors which are not tabooed are considered. Subsequently, at each search step, the tabu list is updated and kept in the range of a tabu tenure. Tabu search is considered a simple but powerful way of forcing a CBLS method to explore a larger part of the search space. The performance of a simple tabu search can be very impressive but usually depends strongly on the tabu tenure’s value.
Algorithm 2:Tabu Search 01 s← initial solution
02 whiletermination condition are not met AND shas better neighbor
03 choose a non-tabu neighbors0 ofssuch that f(s0)< f(s) 04 s←s0;
05 return s
2.2.3 Configuration Checking
Recently, a new strategy called configuration checking (CC)[38] is proposed to deal with the cycling problem of CBLS algorithm. The CC strategy reduces the cycling problem by checking the circumstance information of a variable. It remembers each variable’s circumstance information and prevents a variable from being selected if its circumstance has not been changed since its last selection. The intuition behind this idea is to reduce cycles on the whole candidate solution by reducing local structure cycles. The CC strategy can be described as follows: when considering a variable to select, for a variable
vi ∈V, if the configuration of vi has not changed sincevi’s last selection, which means
the circumstance of vi never changes, then it should not be selected. This strategy is
reasonable in terms of avoiding cycles; otherwise, the algorithm is led to a scenario it has recently faced, which is likely to cause a cycle. This strategy is successfully used to solve SAT [39] and minimum vertex cover [38] problem.
2.2.4 Variable Neighborhood Search
The use of multiple neighborhoods in a local search was first formalised in [40] where it is described as Variable Neighborhood Search (VNS). It is a meta-heuristic featuring
an interesting way to escape local minima. It works with a collection of neighborhoods
N1,· · ·, Ni to diversify the current solution. The intuition is that the neighborhoods N1,· · ·, Ni are increasing in size, providing more opportunities for significant
diversifi-cations over time. Algorithm 3 shows the structure of a classic VNS algorithm. Key to the success of VNS algorithm are the following three observations:
• Local optimum is only a local optimum with regard to the current neighborhood; it may not be one in a different neighborhood.
• The global optimal will be a local optimum in every neighborhood
• Local optima in multiple neighborhoods have been empirically shown to be rela-tively close (in terms of shared variable assignments).
Algorithm 3:Variable Neighborhood Search Algorithm
01 s← initial solution
02 i←1
03 whiletermination condition are not met
04 fors0∈ exploreNeighborhood(Ni, s) 05 ifacceptanceFunction(s0) 06 s←s0 07 i←1 08 else 09 i←i+ 1
2.2.5 Constraint Directed Variable Neighborhood Search
A new method called constraint directed variable neighborhood search (CDVNS) is in-troduced in [41] where the selection of the neighborhoods is dynamic and is based upon the violations of the problem constraints. It keeps track of the violations of problem constraints so that appropriate neighborhoods can be chosen to explore. Rather than lin-early exploring all the possible neighborhoods just a subset is chosen which can improve the current constraint violations. Key to the operation of CDVNS is the neighborhood selection matrix. This structure stores which neighborhoods should be searched when attempting to solve any given constraint configuration. It is important to generate spe-cific neighborhoods which are able to reduce the current constraint violations whilst leaving the other constraints unaffected.
2.2.6 Software Tools for Constraint-based Local Search
Constraint programming can express real-world problems in a very flexible way. Histori-cally, most research on modeling and programming tools for combinatorial optimisation has concentrated on systematic search which is at the core of branch-and-bound on constraint satisfaction problems [42]. However, a systematic search cannot be used to solve many highly constrained optimisation problems in a reasonable time. For such problems, local search methods give, in general, near-optimal solutions in a reasonable time. In attempt to gain both advantages of local search and constraint programming, many research works have addressed the combination of these two techniques [43]. In recent years, increased attention has been devoted to the design and implementation of programming tools for local search (see Table 2.1 for more details).
Table 2.1: Software tools for CBLS
Tools Support
Localizer [44] Local Search
EasyLocal++ [45] Local Search
ParadisEO [46] Local Search, Evolutionary algorithm
OpenTS [47] Tabu Search
COMET [42] Systematic search, Local search Kangaroo [48] Constraint-based local search
2.3
Global Constraints
The CSP is a general model for formulating combinatorial search problems. Here we present a set ofglobal constraints1 which can be used to model a given problem as a CSP. Each constraint involves a list of variables [y1,· · · , yp] and possibly other parameters.
LetRC ⊆Dy1 × · · · ×Dyn denotes the set of tuples for which constraintC is satisfied.
• Constraintdifferent ([x, y]): means: s∈RC ↔s(x)6=s(y).
• Constraintdistance ([x, y], D): means: s∈RC ↔ |s(x)−s(y)|> D.
1
• Constraintalldifferent([y1,· · · , yp]) checks that all variablesy1,· · · , yp receive
dif-ferent values. Let Ns represent the number of pairs of variables having the same
value in the tuples, thens∈RC ↔Ns= 0.
• Constraintatmost([y1,· · · , yp], a, P) checks that the number of variablesyi taking
the valueais less or equal toP. LetNs represent the number of variables having
the valueain the tuple s:Ns=|{i/s(yi) =a}|, thens∈RC ↔Ns≤P.
• Constraintatleast([y1,· · · , yp], a, P) checks that the number of variablesyi taking
the value a is greater or equal to P. Let Ns represent the number of variables
having the valueain the tuples:Ns=|{i/s(yi) =a}|, then s∈RC ↔Ns≥P.
These constraints make it possible to represent a number of constraint problems. The constraints presented here are chosen essentially for their generality. However, this set of constraints is far from complete to encompass all the CSPs.
2.4
Evolutionary Algorithm
Evolutionary algorithms represent a large class of problem-solving methodologies, with genetic algorithms (GA) [49] being the most widely known. These algorithms are moti-vated by the way species evolve and adapt to their environment, based on the Darwinian principle of natural selection. Under this paradigm, a population of solutions evolves from one generation to the next through the application of operators that mimic those found in nature, namely, selection of the fittest, crossover and mutation. Through the selection process, which is probabilistically biased toward the best elements in the pop-ulation, only the best solutions are allowed to become parents and to generate offspring. The mating process, called crossover, then takes two selected parent solutions and com-bine their most desirable features to create one or two offspring solutions. This is repeated until a new population of offspring solution is created. Before replacing the old population, each member of the new population is subjected (with a small probability) to small random perturbations via the mutation operator. Starting from a randomly or heuristically generated initial population, this renewal cycle is repeated for a number of iterations, and the best found is returned at the end [50, 51].
Algorithm 4:Generic Evolutionary Algorithm
01 pop← InitialPopulation()
02 Evaluate(pop)
03 whiletermination condition are not met
04 parents ←selectParents(pop)
05 descendants← Combine(parents)
06 descendants← Mutate(descendants)
07 pop← SelectPopulation(pop, descendants)
Algorithm 4 shows the generic structure of evolutionary algorithm. First the population
popis initialised and then evaluated (Lines 01-02). Then, a certain number of iterations is repeated until the termination criteria is reached (Line 3). During these iterations the individuals are selected to be combined (Lines 04-05) and their descendants are mutated (Line 06). Afterwards, a new population is generated from the previous one and the descendants using a replacement strategy (Line 07).
In the next sections we are going to review each one of these steps.
2.4.1 Representation
This is an issue that is prior to the development of the algorithm. Typically, Genetic Algorithm (GA) uses a string of number as a representation, and very often it is only a binary string. However, we should not forget that choosing the right representation of a problem is key to the algorithm’s performance. The issue which arises in some GAs is to link the real problem to the problem representation. This mimics biology where a genotype encodes the information that yields a fenotype which is the natural transformation of that information. Sometimes, this distinction does not appear if the information and the representation are one and the same thing, which happens often. However, it is important to explicitly make this distinction since the genotype is used for individuals interaction, but the fenotype is needed to calculate the real value of the evaluation function.
2.4.2 Evaluation Function
Closely related to the representation, the issue of the evaluation function arises. This function associates a value to every individual in the population, and corresponds to the quality of that individual. Thus, different representations of the same problem may have different evaluation functions, since this is typically calculated from the values of the genes of each individual and through the genotype-to-fenotype mapping. The evaluation function is often referred to as fitnessfunction in theevolutionary computationfield.
2.4.3 Initial Population
Once the representation is fixed, the first issue in developing the algorithm is that of the initial population. This is typically performed by randomly generating individuals so that the population can cover wider areas of the search space. Nonetheless, there are other more specialised methods.
2.4.4 Parent Selection
Selection is the method through which certain elements in the population are chosen to be combined. This selection mechanism tries, in general, to choose parents that are likely to produce a high-quality descendant. Typically, two individuals are chosen to reproduce and yield descendants. Different kinds of selection mechanism are:
• Roulette-wheel Selection
It consists of giving a certain probability to be chosen for every individual. This probability depends directly on the absolute fitness of the individual [49]. The main drawback of this mechanism is that the best candidates are very likely to take over the whole population very quickly. This method is often called fitness proportional selection.
• Ranking Selection
This method is very similar to the previous one. The difference is that, in this case, the individuals are ranked according to their fitness, and then probabilities are given based on the ranking rather than on the fitness itself [52].
• Tournament Selection
This is may be the simplest mechanism, and also the least time-consuming. It consists of choosing k individuals completely at random, and then selecting the two individuals with highest fitness function. Obviously, the complexity of this method depends on the value of k.
There are many other methods, mainly variations of the ones described above.
2.4.5 Reproduction
This operator combines the parents so that a high quality individual is obtained. This mechanism is also known as crossover. In some GAs, crossover is able to generate more than one descendant (usually two), but we will assume that only one descendant is going to be generated. Different typical crossover operators are:
• One point crossover
This is the most popular method. It consists of choosing a point randomly, and copying the genes of a parent, from the beginning until this point, to the descen-dant, and the genes of the other parent from that point till the end. As an example, imagine we have two parents of the form:
ρ1 =h 0 1 0 0 11 1 1 0 i
ρ2 = h0 0 0 1 1 0 1 0 0 i
and k= 5 is the crossover point, the descendant would either be
h 0 1 0 0 1/ 0 1 0 0 i h0 0 0 1 1/1 1 1 0 i
Note that extending this operator to generate two descendants is trivial.
• Multiple point crossover
It is based on the previous operator, and its only difference is that instead of 1 point, several k points are chosen randomly. Then, to generate a descendant it would copy the genes of each parent in turns after each crossover point.
• Uniform crossover
It is slightly different than the previous one. It treats each gene independently and decides from which parent it is going to be inherited.
These methods are the most common ones in the literature. Other more complex ones can also be found.
2.4.6 Mutation
Another operator used in evolutionary algorithm to diversify the search in known as
mutation operator. The most common ones are:
• Random bit modification
It changes the value of some bits with a given probability. The operator changes the value of every bit in the sequence with a certain probability. If the representation is binary, the effect is that offlippinga bit, either from 0 to 1 or from 1 to 0.
• Swap mutation
It simply selects two genes (at random) in the sequence and swaps their values. Imagine the individual:
h 0 [1] 0 [0] 1 0 1 0 0 i
and the swapping genes 1 and 3, the mutated individual would be
h 0 [0] 0 [1] 1 0 1 0 0 i
• Insert mutation
It chooses two genes at random and moves the second one next to the first. Again, if we have the individual
h 0 [1] 0 0 [1] 0 1 0 0 i
and the inserting genes 1 and 4, the mutate individual would be
• Scramble mutation
It selects a region in the sequence and randomly scrambles its values. For example,
h [0 1 0 0] 1 0 1 0 0 i
and the region from 0 to 3, a possible mutate individual would be
h [1 0 0 0] 1 0 1 0 0 i
Note that all these operators can be applied to any kind of representation, even though the illustrations assume a binary representation. Many other complex and specialised mutation operators can be found in the literature, including the ones where the mutation is not random but biased by the subsequent value of the fitness function of the individual.
2.4.7 Selection of the New Generation
This strategy replaces the last population by a new one. In order to do so, some al-gorithms completely replace the previous population for the new set of descendants or offspring. However, this is usually not a very effective technique, and GAs normally implement mechanism to generate the new population from both, the previous one and the offspring. Among these mechanisms, the following distinction can be made:
• Fitness based
It focuses on keeping the individuals with higher fitness for the next generation.
• Generations based
It takes into account the number of generations passed since its creation, and replaces those individuals which have been in the population for a larger amount of generations.
• Replace Worst
It tends to replace the worstnindividuals in each step. The intuition is to always maintain the highest quality individual in the population. This technique is usually referred to as elitism.
2.4.8 Termination
The termination condition indicates when it is time for the algorithm to stop. At this point, the algorithm returns the best individual according to its fitness function. The following two kinds of termination condition can be distinguished:
• Objective reached
When a GA is implemented to reach a certain goal, reaching that goal should be the indication for the algorithm to stop.
• External conditions
If the previous case is not achieved, then a different criteria must be used. Different conditions include:
– Fixed number of generations reached. – Maximum time allowed reached.
– Fitness improvement does not occur for a certain period of time/generations. – Manual inspection.
– A combination of the above.
2.5
Variable Selection Heuristics
Many search procedures choose local moves by a two-step approach, first selecting a variable to re-assign and then a new value for that variable. Typically, the variable selection strategies uses gradients, i.e., information on how much the objective function or the violations may increase or decrease by changing the value of a variable. Since such a variable selection takes place at every iteration of the local search, such gradients are typically maintained incrementally.
The idea of differentiable invariant is introduced in [53]. An invariant declaratively specifies an expression whose value must be maintained incrementally under local moves. In CBLS, objective functions and constraints, which are differentiable objects, not only maintain the value of an expression: they also maintain variable gradient which is to determine how the expression value increases or decreases by changing a variable and
support differentiation which is used to assess the effect of a local move on the expression value. The concept of differentiable invariant is that it automatically lifts an arbitrarily complex expression into a differentiable objective function. Like invariants, the resulting objective function incrementally maintains the value of the expression. Unlike invariants, it also maintains variable gradients and supports the differentiability. The value of the objective function is maintained by invariants. For each decision variable x two methods namelyincrease and decrease are maintained. These methods return gradient information for that decision variable x, i.e., they estimate by how much the objective may increase or decrease by re-assigningx. To implement gradients and differentiations efficiently, one invariant is associated with every sub-expression. Though differentiable invariants are an effective high-level abstraction to bridge the gap between invariants and differentiable objects, they impose a significant overhead of maintaining gradient information for each variable.
Another strategy called hint-based variable selectionis presented in [54]. Each function
f(p1,· · ·, pn) has the parameters pis that are either variables or other functions. A
functionf depends on a variable x, denoted by f →x, if x is itself a parameter off or
f has a parameter p → x. Each function f has a non-negative metric fm denoting its
evaluation. For each x ← f, the function f also has a non-negative hint f(hx) denoting the preference of changing x’s value to improve fm. A constraint f is satisfied when fm = 0 and in that casef(hx)= 0 for any x, which means a constraint’s metric improves
when it is minimised. Given a number of functions in a constraint modelf1,· · · , fn, the hintof a variable x is defined as the following equation:
hint(x) = X
i≤n,fi→x
fih(x) (2.1)
Looking at these hint values, selection of variable is decided in order to minimise the violation of the constraints. If the violations are equal weighted, then the variable with maximumhint value is selected.
2.6
Conclusion
This chapter reviews the basic architecture of different algorithms which are used in this thesis to handle challenging combinatorial problems. The working principle of both systematic and non-systematic approach is described here. Also, the details of constraint-based local search and evolutionary algorithm are illustrated. Finally, the chapter concludes with a discussion of different variable selection strategies. In the sub-sequent chapters, we will discuss how these approaches can be applied to solve different problems like optimal Golomb ruler, all-interval series and propositional satisfiability.
Golomb Rulers
In this chapter, the search space reduction techniques are discussed to determine the tighter bounds for each mark of a Golomb ruler. These reduction techniques significantly curtail the search space and enhances the performance of the search algorithm. However, this chapter starts with an introduction of the Golomb ruler problem and then describes different search space reduction techniques to handle this problem. Finally, a constraint-based multi-point local search algorithm is proposed here to find optimal Golomb rulers.
3.1
Golomb Ruler Problem
Definition 3.1 (Golomb Ruler). A Golomb ruler (GR) G of order m and length n
comprises m integers x1 < x2 < . . . < xm such that each difference dij =xi−xj where i > j is unique. Notice that by definition n= xm−x1 and without loss of generality, one can easily assume x1 = 0, xm=n.
Definition 3.2 (Optimal Golomb Ruler). A Golomb ruler isoptimalif its length is the minimum for the given order. We use G∗m to denote an optimal Golomb ruler (OGR) withm marks and n∗m to denote the length ofG∗m.
Table 3.1: A 16-mark optimal Golomb ruler with marks (xk)1≤k≤m
k 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
xk 0 1 4 11 26 32 56 68 76 115 117 134 150 163 168 177
Definition 3.3 (GR Satisfaction). Given a lengthnand an order m, find the marks of a Golomb ruler as per Definition 3.1.
Definition 3.4 (OGR Satisfaction). Given the optimal length n for a given order m, find the marks of an optimal Golomb ruler.
Definition 3.5 (OGR Optimisation). Given an orderm, find the marks of the optimal Golomb ruler along with its optimal lengthn.
Notice that in GR satisfaction, we just find any Golomb ruler. Moreover, in OGR satisfaction, the optimal length is known beforehand for the given order while in OGR optimisation, the optimal length is not known beforehand.
3.1.1 Applications
Golomb rulers have a wide variety of applications that include radio astronomy [5], x-ray crystallography [4], information theory [6] and pulse phase modulation [7].
Radio Astronomy makes use ofGolomb rulersin the following manner. In locating a dis-tant radio source, it is essential to determine the angle between the incoming wavefront and the antenna array baseline. The antennas are all set to receive the same frequency, and the exact time of reception of the incoming signal can be measured and compared between all pairs of antennas. The direction of the incoming signal can be determined by analysing the phase difference between the pairs of antennas. Maximum accuracy is gained when no two antennas are the same distance apart. By placing the antennas at the marks of a Golomb ruler, this condition is guaranteed. Furthermore, by using optimal or near-optimal Golomb rulers, the total length of the array element is min-imised. This is an important concern when the geographical distances are considered to be utilised in radio astronomy [5].
Another application of Golomb rulers to radio astronomy is in the selection of scanning frequencies. The purpose of this application is not so much to locate distant sources, as it is the location of the antennas themselves. Using a single pair of antennas, and scanning a set of frequencies from a distant point source the location of the antenna elements can be determined to an accuracy of centimeters across continental distances [55]. By assigning the scanning frequencies to correspond to the marks of a Golomb ruler, maximum accuracy is guaranteed for a minimum number of scanning frequencies.
3.1.2 Challenges
Finding an optimal Golomb ruler (OGR) is an extremely challenging problem. For instance, the search for a 19 marks OGR took approximately 36,200 CPU hours on a Sun Sparc workstation (although in year 1998) using a very specialised algorithm [56]. Recently, optimal solutions up to 27 marks were obtained by massive parallelism projects at http://distributed.net, taking several months (even years) for each of those instances. For example, it took five years to find an OGR of 27-mark by 19,919 participants. It has been stated by Cotta et al. (2007) about the distributed.net projects, “although obviously enhanced by clever implementations, this line of attack is ultimately a brute-force approach, and hence is inherently doomed by the curse of dimensionality. It is thus important to explore alternative techniques that might eventually overcome
the limitations of by-force approaches. Indeed, finding optimal Golomb rulers has thus become a standard benchmark to evaluate and compare a variety of search techniques,
in particular, evolutionary algorithms (EAs), constraint programming (CP) and local search (LS). Being such an extremely difficult combinatorial task, the Golomb ruler
problem represents an ideal scenario for deploying the arsenal of search algorithms.”
Although the search space of OGR is bounded [57], the bounds grow geometrically with the number of marks [58]. In contrast, the number of OGRs for a given number of marks does not change much and remains almost the same. To solve this highly combinatorial problem, a number of approaches have already been developed. The most promising results come from a sophisticated scatter search algorithm [2] that combines ideas from greedy randomised adaptive search procedure, tabu search, clustering techniques and constraint programming. The hybrid algorithm finds OGRs for up to 16 marks.
However, it is an important question to ponder why the GR search is so challenging. An analysis of the fitness landscape of OGR presented in [2] shows that high irregularities in the neighbourhood structure introduce a drift force towards low-fitness regions of the search space. For higher order rulers, particularly 16 and beyond, search algorithms thus quickly reach a near-optimal value and then stagnate around it, apparently causing a cycling problem and making the search space less accessible. A restarting mechanism is therefore needed at that stage to get out of that part of the search space.
3.2
Organisation of this chapter
This chapter first presents a constraint-based local search approach to perform satisfac-tion search for OGRs. A satisfacsatisfac-tion search finds an OGR when the optimal length is known for a given order. Instead of a sophisticated hybridisation of a range of tech-niques, we rather rely on simple tabu meta-heuristics and constraint-based variable selection heuristics. Besides the traditional way of enforcing tabu on recently modified variables for a given number of iterations, a special type of tabu called configuration checking (CC) [59] is also used. The CC strategy for OGR prevents a variable, xi from
being selected ifxi’s domain contains only its current value. This might happen because
the marks are all sorted and the domain of xi is dynamically restricted by the values
of its neighbours (xi−1 < xi < xi+1). Consequently, the use of CC effectively reduces
the number of restarts required during search and thus mitigates the cycling problem of local search for an OGR.
This chapter also presents a new representation for Golomb rulers. Moreover, we provide tight upper bounds for each mark of an OGR. These upper bounds significantly curtail the search space without discarding any solution. However, an effective heuristic-based domain reduction technique is presented here that might discard optimal solution from the search space which merely increases search incompleteness, an inherent property of local search technique. Using the domain reduction technique and new tight upper bounds, our constraint-based multi-point local search performs satisfaction search for OGRs. This algorithm finds OGRs of up to 19 marks within 48 hours of CPU time.
Finally, we present algorithms to perform optimisation search for OGRs. An optimisa-tion search that uses the satisfacoptimisa-tion search repeatedly, finds an OGR when the optimal length is not known for the number of marks. The optimisation search therefore starts from an upper bounduand uses GR satisfaction search to repeatedly find GRs of shorter lengths. As soon as a GR of lengthlis found, another search starts with the new length
l−1. For parallel optimisation search, we, however, run a number of parallel processes each performing an optimisation search and as soon as a process finds a solution, it starts its next optimisation search with the best known upper bound at that moment. In this fashion, this parallel optimisation search, given two weeks time, finds OGRs up to 17 marks. For quite some time, Golomb rulers with 16 or more marks have posed severe hindrance to the progress of constraint-based OGR search. Overall, in this chapter, we
advance the satisfaction search for OGRs up to 19 marks and the optimisation search up to 17 marks. We have run our proposed algorithms for size 20 but unfortunately could not find any solution within the given timeout limit. Thus in this thesis, the scope of our work would be limited to rulers up to 19.
3.3
Related Work
Various techniques have been applied to find Golomb rulers. Scientific American algo-rithm was presented in the December 1985 issue of Scientific American [55]. The basic algorithm is composed of two procedures, ruler generation and Golomb verification. The generation procedure (called Exhaust), generates an exhaustive traverse of all possible ruler combinations. Then the ruler verification procedure (called Checker), generates the complete set of distances for all the n marks to check if a series of marks meet the requirement for Golomb rulers. Another algorithm namelytoken passing algorithm was proposed which improves the Scientific American algorithm by manipulating all of the specified marks simultaneously, progressing through all possible mark positions until one is found that meets the criteria of a Golomb ruler. Also,Shift algorithmwas proposed in [60] whose essence is from the Scientific American algorithm, but with major improve-ments in the area of ruler checking. It saves the array of distances between computations so that when a new mark is added only the new distances created by that mark need to be added and checked against the array. The distance array is stored as a bitmap, rather than as an array of boolean values to reduce the checking and generation overhead even more. All of the above three algorithms are described and compared in [61].
Systematic and non-systematic methods
Both non-systematic and systematic methods are proposed to find optimal and near-optimal Golomb ruler. As non-systematic approach the use of geometry tools such as projectile plane construction and affine plane construction [62] are proposed. Using these approaches one can find near-optimal Golomb rulers up-to 158 marks whose lengths can be within several percentage points of the optimum. As systematic methods, one of the most efficient method proposed by Shearer [58] can be mentioned which is based on the utilisation of the branch and bound algorithm along with the depth first search (i.e
backtracking algorithm). It makes use of the upper bounds set equal to the length of the best known solution. This method has been used to generate Golomb rulers up to 16 marks.
Constraint programming have also been used to handle this problem, although with limited success. For example, Smith and Walsh [63] got some promising results in terms of nodes in the branching schema. Afterwards, Galinier [64] proposed an approach based on the combination of constraint programming and sophisticated lower bounds for Golomb ruler. Later a novel hybrid local search and constraint programming approach named constraint local search (CLS) is proposed by Prestwich [11]. For rulers of 14 to 16 marks, this algorithm obtained a best solution with a distance between 9.4% and 12.99% from the optimum.
Evolutionary algorithms
In 1995, Soliday [13] proposed an efficient genetic algorithm based approach for differ-ent solution sizes of the Golomb ruler problem. Here each solution is composed of a permutation of (n−1) integers that represents the sequence of the (n−1) lengths of its segments. They use two evaluation criteria such as the overall length of the ruler and the number of repeated measurements to evaluate the rulers.
Feeney [9] studied three methods for searching Golomb rulers, using GA on its own, with local search and Baldwinian learning and with local search and Lamarckian learning. These hybrid approaches were successful in generating near-optimal ruler and in generat-ing Golomb rulers of a short length. The best results obtained by this approach maintain the distance between 6.8% and 20.3% from the optimum. However, performance of these methods is poor.
A simple hybrid evolutionary algorithm (called GRHEA) is presented in [1] to find an OGR of a specified length. Also, an indirect but effective approach (called GROHEA) is proposed to find near-optimal GRs. For a given order m, GROHEA starts from an upper bound of n and if a GR of length n∗ < n is found, it then tries
