Two types of approaches exist that may be followed in order to evaluate the effectiveness of a packing algorithm. One is to perform a theoretical analysis of the algorithm by comparing an optimal solution to a hypothetical problem with the solution the algorithm provides (the theoretical analysis may also include a time analysis). The second approach is to evaluate implementations of the algorithms on a set of benchmark problem instances.
2.3.1 Theoretical Evaluation Methods
The complexity of an algorithm is the amount of computational resources required by a computer to employ an algorithm. There are typically two measures that define algorithmic complexity: the space complexity and the time complexity. The space complexity measures the amount of memory required to execute an algorithm and the time complexity measures the number of basic operations executed by an algorithm in order to quantify the expected time required to execute
an algorithm. Only time complexity is considered in the remainder of this dissertation1. Let
T(n) denote the number of basic operations required to execute an algorithm for a problem of
size n. If T (n) increases slowly as n increases, then the algorithm may be useful for large values of n. However, if T (n) grows very fast as n increases (such as when T (n) is an exponential or factorial function of n), then the algorithm may not be able to solve moderately large instances of the problem within a realistic time span [143]. If an algorithm is known which can solve a problem within polynomial time (such as when T (n) is a logarithmic, linear or quadratic function), then the problem is called tractable. Otherwise the problem is called intractable [24]. It is difficult to determine the exact number of basic operations that are required by an algorithm as it often depends on input size and conditional statements such as if-statements and while- loops. Instead, the worst-case time complexity is used to indicate the relationship between input size n and the time required to solve the problem in the worst case. Due to the difficulty of calculating exact upper bounds on T (n), asymptotic upper bounds as n → ∞ are preferred as a
description of the worst-case growth behaviour [68]. If g(n) is a function such that T (n) ≤ c1g(n)
for all n larger than some n1∈N (for some c1∈R+), then g(n) is referred to as the asymptotic
upper boundon T (n) as n → ∞. The most common form of expressing this type of bound is by
means of the so-called “Big O” notation, which takes the form T (n) = O(g (n)) [24]. Table 2.6 indicates how algorithmic time complexity may be classified.
Some authors, including Johnson et al. [84, 85], Baker et al. [5, 6], Sleator [148], Coffman et
al. [32], Brown [20], Golan [62], Chung et al. [28] and others, have theoretically evaluated the
quality of solutions produced by packing heuristics. Consider a list L of items to be packed. Let OPT(L) denote the value of some performance measure corresponding to an optimal solution to the problem. This measure is typically the strip height for strip packing problems, or the number of bins required to pack items for single-sized bin packing problems. The same performance measure evaluated for the solution provided by algorithm A is denoted by A(L). An absolute
performance boundfor algorithm A is the worst possible solution for any list L and has the form
A(L) ≤ β OPT (L) ,
where β ∈ R and β ≥ 1. An asymptotic performance bound for algorithm A takes the form
A(L) ≤ β OPT (L) + γ,
1In the current technological environment memory is cheap. Time may be an expensive resource and a common
2.3. Evaluation of Packing Algorithms 23
Rate of Growth Name If n is doubled
O(1) Constant No change
O(log log n) Iterated logarithmic Small time increments O(log n) Logarithmic Constant time increments
O(n) Linear Time is doubled
O(n log n) Linearithmic Slightly more than double time O n2
Quadratic Time increases 4-fold O n3
Cubic Time increases 8-fold O(nc) Polynomial Time increases 2c-fold
O(cn) Exponential Time required is squared
O(n!) Factorial Time increases (2n)!/n!-fold O(nn) Super-exponential Time increases extremely quickly
Table 2.6: Classifications of algorithmic time complexity [143]. The constant c ∈R+.
where γ ∈ R. Coffman et al. [32, p. 809] remark that an asymptotic bound is of greater interest than an absolute performance bound, because the latter often applies only to small, very specialised examples of items. Instead, an asymptotic performance bound characterises the performance of the algorithm as the ratio of the height of an optimal solution OPT(L) to the height of the tallest item tends to infinity. Such a bound is often established for an item set where the maximum item height is 1. However, any height may be used (see Sleator [148] for an example), as it only affects the additive constant γ; the multiplicative bound β remains unchanged [32].
2.3.2 Computational Evaluation Methods
Dowsland and Dowsland [42, p. 8] comment that while average or worst-case performance bounds are useful guidelines, it is best to determine an algorithm’s usefulness by testing it on data sets typical to the intended problem. Repositories are available on the internet where benchmark data sets for packing problems are stored for the purpose of evaluating algorithms. These include the benchmarks from the EURO Special Interest Group on Cutting and Packing
(ESICUP) [46], the instances at PackLib2 [50] or from the online repository by Van Vuuren and
Ortmann [154].
In order to measure the solution quality of a strip packing algorithm, called Algorithm A, the packing height achieved by the algorithm (denoted by A(I), where I denotes a set of items) may be divided by the packing height associated with an optimal solution (denoted by OPT(I)).
This result is called the strip packing accuracy αSP, and is defined as
αSPA (I) = A(I)
OPT(I).
However, optimal solutions are not known for all benchmark instances. In order to compare algorithms by means of benchmark instances for which optimal solutions are not known, the packing height of the algorithms may be divided by valid lower bounds (such as those by Martello et al. [110]). In order to compare algorithms with respect to both solution quality and
the speed at which the solution is found, the strip packing efficiency ΓSP is defined as
ΓSPA (I) = OPT(I)
A(I) ×
τI
tAI
!1`
where ` ∈ Z+, tA
I denotes the time required by algorithm A to find a solution for the items in
I and where τI denotes the time required by the fastest algorithm in the comparison group
to find a solution to the same problem. The influence of time on ΓSP decreases as the value
of ` increases. An algorithm may be labelled as more efficient than a second algorithm for a given value of ` if its efficiency is larger. If no optimal solution is known for a strip packing benchmark instance, the packing height corresponding to an optimal solution may be replaced with a valid lower bound (see Martello et al. [110]).
Two measures appear in the literature (see [75]) for the evaluation of packing solutions to the multiple bin size bin packing problem. The utilisation µ of a packing is the total area of the items in I (denoted by A(I)) divided by the area of the bins that contain items (denoted by
A(BI), where B denotes the set of bins), that is
µA(P) =
A(I)
A(BI).
The objective in the multiple bin size bin packing problem is to maximise µ.
The fitness ν of a solution to the multiple bin size bin packing problem is a measure that aims to reward algorithms for dense packing of bins. This allows for the separation of algorithms when their utilisations are equal for all solutions. A solution in which most bins are densely packed and one bin is not, would typically achieve a higher fitness score than an algorithm that packs bins less densely. Figure 2.6 illustrates how fitness may be used to differentiate between two solutions where utilisation cannot. The fitness of a solution is defined as
νA(P) = PM i=1 A(IBi) A(Bi) k M ,
where A IBi denotes the total area of the items packed into bin B
i, M is the number of bins
that contain items in the solution and typically k = 2 (as in Hopper [75]).
(a) Solution SB (b) Solution SG
Figure 2.6: A comparison of two solutions to a packing problem. The bins have height and width 40. Both solutions have a utilisation of 0.479, while SB has a fitness of 0.220 and SG has a fitness of 0.317.
A higher fitness is more desirable as it indicates a higher probability of the wasted space being of such a size that it may be used for further packing in the future. This may be important in, for example, trim loss or stock cutting problems, where the offcuts (wasted space) may be re-employed as raw material at a later stage.
In order to compare the algorithms in terms of both solution quality and time, the multiple bin
size bin packing efficiencyΓMS is defined as
ΓMSA (P) = µA(P) × τP
tA
P