Partition Sort

After having discussed two advanced sorting methods based on the principles of insertion and selection, we introduce a third improved method based on the principle of exchange. In view of the fact that Bubblesort was on the average the least effective of the three straight sorting algorithms, a relatively significant improvement factor should be expected. Still, it comes as a surprise that the improvement based on exchanges to be discussed subsequently yields the best sorting method on arrays known so far. Its performance is so spectacular that its inventor, C.A.R. Hoare, called it Quicksort [2.5 and 2.6].

Quicksort is based on the recognition that exchanges should preferably be performed over large distances in order to be most effective. Assume that n items are given in reverse order of their keys. It is possible to sort them by performing only n/2 exchanges, first taking the leftmost and the rightmost and gradually progressing inward from both sides. Naturally, this is possible only if we know that their order is exactly inverse. But something might still be learned from this example.

Let us try the following algorithm: Pick any item at random (and call it x); scan the array from the left until an item ai > x is found and then scan from the right until an item aj < x is found. Now exchange the two items and continue this scan and swap process until the two scans meet somewhere in the middle of the array. The result is that the array is now partitioned into a left part with keys less than (or equal to) x, and a right part with keys greater than (or equal to) x. This partitioning process is now formulated in the form of a procedure. Note that the relations > and < have been replaced by ≥ and ≤ , whose negations in the while clause are < and >. With this change x acts as a sentinel for both scans.

PROCEDURE partition;

VAR i, j: INTEGER; w, x: Item; BEGIN i := 0; j := n-1;

select an item x at random; REPEAT

WHILE a[i] < x DO i := i+1 END ; WHILE x < a[j] DO j := j-1 END ; IF i <= j THEN

w := a[i]; a[i] := a[j]; a[j] := w; i := i+1; j := j-1 END

UNTIL i > j END partition

As an example, if the middle key 42 is selected as comparand x, then the array of keys 44 55 12 42 94 06 18 67

requires the two exchanges 18 ↔ 44 and 6 ↔ 55 to yield the partitioned array 18 06 12 42 94 55 44 67

and the final index values i = 5 and j = 3. Keys a1 ... ai-1 are less or equal to key x = 42, and keys aj+1 ... an are greater or equal to key x. Consequently, there are three parts, namely

Ak : 1 < k < i : ak≤ x Ak : i ≤ k ≤ j: ak ? x Ak : j < k < n : x ≤ ak

The goal is to increase I and decrease j, so that the middle part vanishes. This algorithm is very straightforward and efficient because the essential comparands i, j, and x can be kept in fast registers throughout the scan. However, it can also be cumbersome, as witnessed by the case with n identical keys, which result in n/2 exchanges. These unnecessary exchanges might easily be eliminated by changing the scanning statements to

WHILE a[i] <= x DO i := i+1 END ; WHILE x <= a[j] DO j := j-1 END

In this case, however, the choice element x, which is present as a member of the array, no longer acts as a sentinel for the two scans. The array with all identical keys would cause the scans to go beyond the bounds

of the array unless more complicated termination conditions were used. The simplicity of the conditions is well worth the extra exchanges that occur relatively rarely in the average random case. A slight saving, however, may be achieved by changing the clause controlling the exchange step to i < j instead of i ≤ j. But this change must not be extended over the two statements

INC(i); DEC(j)

which therefore require a separate conditional clause. Confidence in the correctness of the partition algorithm can be gained by verifying that the ordering relations are invariants of the repeat statement. Initially, with i = 1 and j = n, they are trivially true, and upon exit with i > j, they imply the desired result.

We now recall that our goal is not only to find partitions of the original array of items, but also to sort it. However, it is only a small step from partitioning to sorting: after partitioning the array, apply the same process to both partitions, then to the partitions of the partitions, and so on, until every partition consists of a single item only. This recipe is described as follows. (Note that sort should actually be declared local to

Quicksort).

PROCEDURE sort(L, R: INTEGER); VAR i, j: INTEGER; w, x: Item; BEGIN i := L; j := R;

x := a[(L+R) DIV 2]; REPEAT

WHILE a[i] < x DO INC(i) END ; WHILE x < a[j] DO DEC(j) END ; IF i <= j THEN

w := a[i]; a[i] := a[j]; a[j] := w; i := i+1; j := j-1 END

UNTIL i > j;

IF L < j THEN sort(L, j) END ; IF i < R THEN sort(i, R) END END sort;

PROCEDURE QuickSort; BEGIN sort(0, n-1) END QuickSort

Procedure sort activates itself recursively. Such use of recursion in algorithms is a very powerful tool and will be discussed further in Chap. 3. In some programming languages of older provenience, recursion is disallowed for certain technical reasons. We will now show how this same algorithm can be expressed as a non-recursive procedure. Obviously, the solution is to express recursion as an iteration, whereby a certain amount of additional bookkeeping operations become necessary.

The key to an iterative solution lies in maintaining a list of partitioning requests that have yet to be performed. After each step, two partitioning tasks arise. Only one of them can be attacked directly by the subsequent iteration; the other one is stacked away on that list. It is, of course, essential that the list of requests is obeyed in a specific sequence, namely, in reverse sequence. This implies that the first request listed is the last one to be obeyed, and vice versa; the list is treated as a pulsating stack. In the following nonrecursive version of Quicksort, each request is represented simply by a left and a right index specifying the bounds of the partition to be further partitioned. Thus, we introduce two array variables low, high, used as stacks with index s. The appropriate choice of the stack size M will be discussed during the analysis of Quicksort.

PROCEDURE NonRecursiveQuickSort; CONST M = 12;

VAR i, j, L, R, s: INTEGER; x, w: Item;

low, high: ARRAY M OF INTEGER; (*index stack*) BEGIN s := 0; low[0] := 0; high[0] := n-1;

REPEAT (*take top request from stack*) L := low[s]; R := high[s]; DEC(s); REPEAT (*partition a[L] ... a[R]*)

i := L; j := R; x := a[(L+R) DIV 2]; REPEAT

WHILE a[i] < x DO INC(i) END ; WHILE x < a[j] DO DEC(j) END ; IF i <= j THEN

w := a[i]; a[i] := a[j]; a[j] := w; i := i+1; j := j-1 END

UNTIL i > j;

IF i < R THEN (*stack request to sort right partition*) INC(s); low[s] := i; high[s] := R

END ;

R := j (*now L and R delimit the left partition*) UNTIL L >= R

UNTIL s = 0

END NonRecursiveQuickSort

Analysis of Quicksort. In order to analyze the performance of Quicksort, we need to investigate the behavior

of the partitioning process first. After having selected a bound x, it sweeps the entire array. Hence, exactly n comparisons are performed. The number of exchanges can be determind by the following probabilistic argument.

With a fixed bound x, the expected number of exchange operations is equal to the number of elements in the left part of the partition, namely x-1, multiplied by the probability that such an element reached its place by an exchange. An exchange had taken place if the element had previously been part of the right partition; the probablity for this is (n-(x-1))/n. The expected number of exchanges is therefore the average of these expected values over all possible bounds x.

M = [Sx: 1 ≤ x ≤ n: (x-1)*(n-(x-1))/n]/n =

[Su: 0 ≤ u ≤ n-1: u*(n-u)]/n2

= n*(n-1)/2nù - (2n2 - 3n + 1)/6n = (n - 1/n)/6

Assuming that we are very lucky and always happen to select the median as the bound, then each partitioning process splits the array in two halves, and the number of necessary passes to sort is log n. The resulting total number of comparisons is then n*log n, and the total number of exchanges is n * log(n)/6.

Of course, one cannot expect to hit the median all the time. In fact, the chance of doing so is only 1/n. Surprisingly, however, the average performance of Quicksort is inferior to the optimal case by a factor of only 2*ln(2), if the bound is chosen at random.

But Quicksort does have its pitfalls. First of all, it performs moderately well for small values of n, as do all advanced methods. Its advantage over the other advanced methods lies in the ease with which a straight sorting method can be incorporated to handle small partitions. This is particularly advantageous when considering the recursive version of the program.

Still, there remains the question of the worst case. How does Quicksort perform then? The answer is unfortunately disappointing and it unveils the one weakness of Quicksort. Consider, for instance, the unlucky case in which each time the largest value of a partition happens to be picked as comparand x. Then each step splits a segment of n items into a left partition with n-1 items and a right partition with a single element. The result is that n (instead of log n) splits become necessary, and that the worst-case performance is of the order n2.

Apparently, the crucial step is the selection of the comparand x. In our example program it is chosen as the middle element. Note that one might almost as well select either the first or the last element. In these cases, the worst case is the initially sorted array; Quicksort then shows a definite dislike for the trivial job and a preference for disordered arrays. In choosing the middle element, the strange characteristic of Quicksort is less obvious because the initially sorted array becomes the optimal case. In fact, also the average performance is slightly better, if the middle element is selected. Hoare suggests that the choice of x be made at random, or by selecting it as the median of a small sample of, say, three keys [2.12 and 2.13]. Such a judicious choice hardly influences the average performance of Quicksort, but it improves the worst-case

performance considerably. It becomes evident that sorting on the basis of Quicksort is somewhat like a gamble in which one should be aware of how much one may afford to lose if bad luck were to strike. There is one important lesson to be learned from this experience; it concerns the programmer directly. What are the consequences of the worst case behavior mentioned above to the performance Quicksort? We have realized that each split results in a right partition of only a single element; the request to sort this partition is stacked for later execution. Consequently, the maximum number of requests, and therefore the total required stack size, is n. This is, of course, totally unacceptable. (Note that we fare no better -- in fact even worse -- with the recursive version because a system allowing recursive activation of procedures will have to store the values of local variables and parameters of all procedure activations automatically, and it will use an implicit stack for this purpose.)

The remedy lies in stacking the sort request for the longer partition and in continuing directly with the further partitioning of the smaller section. In this case, the size of the stack M can be limited to log n.

The change necessary is localized in the section setting up new requests. It now reads IF j - L < R - i THEN

IF i < R THEN (*stack request for sorting right partition*) INC(s); low[s] := i; high[s] := R

END ;

R := j (*continue sorting left partition*) ELSE

IF L < j THEN (*stack request for sorting left parition*) INC(s); low[s] := L; high[s] := j

END;

L := i (*continue sorting right partition*) END