Taxonomy of Algorithms

Before going into details about EC, we give some insight about algorithms division and important notions. Note that we do not give an exhaustive taxonomy of algo-rithms, but rather we concentrate on only those of relevance for this thesis. Further-more, there are several possible divisions of algorithms, but we do not consider all of them here.

Regarding the implementation, algorithms can be divided into deterministic and non-deterministic algorithms. Deterministic algorithms are those that given cer-tain input, will always produce the same output. Non-deterministic algorithms are those that given a certain input can produce various outputs. Non-deterministic algorithms are often used to obtain an approximation of a solution where obtain-ing an exact solution would be too costly. Probabilistic (randomized) algorithms are non-deterministic algorithms whose behavior depends on a random process (i.e.

random number generator). Probabilistic algorithms that always terminate with a correct result with the only variation from one run to another being its running time are known as the Las Vegas algorithms. Probabilistic algorithms that depending on an input may produce incorrect results or fail to produce a result are known as the Monte Carloalgorithms. We show one taxonomy of Monte Carlo algorithms that we follow in Figure 2.6. Notice that we display only a small number of algorithms that are related to this thesis.

When talking about optimization algorithms, we give additional taxonomy that is of relevance for this thesis. Optimization algorithms can be divided to exact and

Figure 2.6: Taxonomy of Monte Carlo algorithms.

approximate methods. Exact algorithms are those that obtain optimal solutions and guarantee their optimality [Tal09]. Approximate algorithms are those that gener-ate high quality solutions in a reasonable amount of time, but without a guarantee that those solutions are global optimal solutions [Tal09]. In the approximate algo-rithms class, we can divide those algoalgo-rithms into approximation algoalgo-rithms and heuristic algorithms. Approximation algorithms are those that provide a provable solution quality and provable run-time bounds [Tal09]. Heuristics are algorithms that find good solutions on a large-size problem instance. In general, they do not have an approximation guarantee on the obtained solutions [Tal09]. Alternatively, heuristics can be defined as parts of an optimization algorithm. There, heuristics use the information currently gathered by the algorithm to help decide which solution candidate should be tested next or how the next solution can be produced [Wei09].

Heuristic algorithms can be further divided into specific heuristics and metaheuris-tics. Specific heuristics are methods that are tailor-made to solve a specific problem.

Metaheuristics, in its original definition represent solution methods that orchestrate an interaction between local improvement procedures and higher level strategies to create a process capable of escaping from local optima and performing a robust search of a solution space [GK03]. Alternatively, metaheuristics are general-purpose algorithms that can be applied to solve almost any optimization problem [Tal09].

Another definition states that metaheuristics are general purpose methods aimed to guide an underlying heuristic [GK03]. To classify metaheuristics, one can follow many criteria, but we divide it into single-solution based metaheuristics and

popula-30

Figure 2.7: Taxonomy of optimization algorithms.

tion based heuristics [Tal09]. Single-solution based metaheuristics manipulate and transform a single solution during the search as in the case of algorithms like local search or simulated annealing. Population based metaheuristics work on a popu-lation of solutions (e.g. EA, Particle Swarm Optimization (PSO), Ant Colony Opti-mization (ACO)). We present a taxonomy of optiOpti-mization algorithms in Figure 2.7.

Similarly to the case of Monte Carlo taxonomy, we display only those algorithm classes that are of relevance for this thesis.

Based on the preceding taxonomies, we see that we are primarily interested in Population based metaheuristicsand EC, in order to conduct the optimization of difficult crypto-related problems.

2.2.2 Origins of EC

Evolutionary Computation (EC) is a research area within computer science that draws inspiration from the process of natural evolution [ES03]. EC is a part of ar-tificial intelligence and more particularly computational intelligence that involves continuous and combinatorial optimization.

Although beginnings of EC can be traced back to 1930s and work of Sewell Wright, only by the emergence of relatively inexpensive computers in 1960s the area started to grow [Jon06]. For a survey of older works, we refer readers to [Fog98].

In 1960s, Rechenberg ans Schwefel started to formulate how evolutionary process can be used to solve difficult optimization processes. From these ideas, they devel-oped ES, a family of algorithms still used today [Rec73,BS02] In that same time, Fogel developed a framework called “Evolutionary Programming (EP)” that was used to evolve better finite state machines [FOW66].

Holland’s work in the early 1960s on a robust self-adaptive systems lead to the development of simple genetic algorithms [Hol92]. In contrast to ES and EP, GA focuses on more application independent algorithms [Jon06].

In 1990s, researchers explored similarities and differences between various algo-rithms which led to Evolutionary Computation (EC) notion that encompasses all those methods [Jon06]. Already these few examples show that in this thesis we will use some methods (granted, in a more advanced form) that were developed more than 40 years ago. This should be also regarded as a proof of the effectiveness of such methods. For more details about the development of EC, we refer readers to [BFM00].