1) Let M be an arbitrary Turing machine.
a) Suppose that timereq(M) = 3n3(n+5)(n-4). Circle all of the following statements that are true:
i) timereq(M) O(n). False
ii) timereq(M) O(n6). True
iii) timereq(M) O(n5/50). True
iv) timereq(M) (n6). False
b) Suppose that timereq(M) = 5n3n3. Circle all of the following statements that are true:
i) timereq(M) O(n5). False
ii) timereq(M) O(2n). False
iii) timereq(M) O(n!). True
2) Show a function f, from the natural numbers to the reals, that is O(1) but that is not constant.
f(n) = 0 if n = 0, else 1.
3) Assume the definitions of the variables given in the statement of Theorem 27.1. Prove that if s > 1 then:
O(nt2n) O(
2
(ns)).From [4], raising to the power of 2, we get:
[5]
2
tlog2n 2
n 2
ns.But
2
tlog2n n
t. So, substituting into [5], we get:[6] (nt2n)
2
(ns).4) Prove that, if 0 < a < b, then nb O(na).
Suppose, to the contrary, that it were. Then there would exist constants k and c such that, for all n k:
nb cna. Since a < b,
c
ba1 exists and is positive. Choose:
1 ,
max(
1
k cba
n ).
Then we have n k and n >
c
ba1 . So nb-a > c and nb > cna. But this is a contradiction. So nb O(na).
5) Let M be the Turing machine shown in Example 17.9. M accepts the language WcW = {wcw : w {a, b}*}.
Analyze timereq(M).
M is:
1. Loop:
1.1. Move right to the first character. If it is c, exit the loop. Otherwise, overwrite it with and remember what it is.
1.2. Move right to the c. Then continue right to the first unmarked character. If it is , halt and reject.
(This will happen if the string to the right of c is shorter than the string to the left.) If it is anything else, check to see whether it matches the remembered character from the previous step. If it does not, halt and reject. If it does, mark it off with #.
1.3. Move back leftward to the first .
2. There are no characters remaining before the c. Make one last sweep left to right checking that there are no unmarked characters after the c and before the first blank. If there are, halt and reject. Otherwise, halt and accept.
In our macro language, M is;
>R x a,b Rc,
c c R# n y x Ry#
y = x y #L
Let n be the length of M’s input string w. We must determine the number of steps that M executes in the worst case. So we will not consider cases in which it executes the loop of statement 1 prematurely. So we will consider only strings of the form xcxy, where both x and y {a, b}*. Note that xcx WcW.
In such cases, M will execute the complete loop of statement 1 |x| times. On each iteration, it visits |x| + 2 squares as it moves to the right in statements 1.1 and 1.2. Then it visits |x| + 1 squares when it moves back to the left in statement 1.3. Then it enters the loop one last time and reads the c. The total number of steps required for this process is |x|(2|x| + 3) + 1 = 2|x|2 + 3|x| + 1. After exiting the statement 1 loop, M must
So we can place an upper bound on the number of steps M executes as a function of n:
2|n/2|2 + 2|n/2| + 1 + n = n2/2 + n + 1.
6) Assume a computer that executes 1010 operations/second. Make the simplifying assumption that each operation of a program requires exactly one machine instruction. For each of the following programs P, defined by its time requirement, what is the largest size input on which P would be guaranteed to halt within a week?
a) timereq(P) = 5243n+649.
b) timereq(P) = 5n2. c) timereq(P) = 5n.
There are 6060247 = 604,800 = 6.048105 seconds in a week. So our computer executes 6.0481015 operations in a week.
a) We need to find the largest n such that 5243n+649 < 6.0481015. When n = 1,153,538,050,734, 5243n+649
= 6,047,999,999,999,011. When n = 1,153,538,050,735, 5243n+649 = 6,048,000,000,004,254 >
6.0481015. So the largest size input on which P would be guaranteed to halt within a week is 1,153,538,050,734.
b) We need to find the largest n such that 5n2 < 6.0481015. When n = 34,779,304, 5n2 = 6,047,999,933,622,080. When n = 34,779,305, 5243n+649 = 6,048,000,281,415,125. So the largest size input on which P would be guaranteed to halt within a week is 34,779,304.
c) We need to find the largest n such that 5n < 6.0481015. When n = 22, 5n = 2,384,185,791,015,625. When n = 23, 5n = 11,920,928,955,078,125. So the largest size input on which P would be guaranteed to halt within a week is 22.
7) Let each line of the following table correspond to a problem for which two algorithms, A and B, exist. The table entries correspond to timereq for each of those algorithms. Determine, for each problem, the smallest value of n (the length of the input) such that algorithm B runs faster than algorithm A.
A B
Row 1: When n = 579, n2 = 335,241 and 572n + 4171 = 335,359. When n = 580, n2 = 336,400 and 572n + 4171 = 335,931. So the smallest value of n such that algorithm B runs faster than algorithm A is 580.
Row 2: When n = 13,746, n2 = 188,952,516 and 1000n log2 n = 188,962,471.5. When n = 13,747, n2 = 188,980,009 and 1000n log2 n = 188,977,660.9. So the smallest value of n such that algorithm B runs faster than algorithm A is 13,747.
Row 3: When n = 7, n! = 5,040 and 450n2 = 22,050. When n = 8, n! = 40,320 and 450n2 = 28,800. So the smallest value of n such that algorithm B runs faster than algorithm A is 8.
Row 4: When n = 6, n! = 720 and 3n + 2 = 731. When n = 7, n! = 5,040 and 3n + 2 = 2,189. So the smallest value of n such that algorithm B runs faster than algorithm A is 7.
8) Show that L = {<M>: M is a Turing machine and timereq(M) O(n2)} is not in SD.
The basic idea is that timereq(M) is only defined if M halts on all inputs. We prove that L is not in SD by reduction from H. Let R be a mapping reduction from H to L defined as follows:
R(<M, w>) =
1. Construct the description <M#> of a new Turing machine M#(x) that, on input x, operates as follows:
1.1. Run M on w for |x| steps or until it halts naturally.
1.2. If M would have halted naturally, then loop.
1.3. Else halt.
2. Return <M#>.
If Oracle exists and semidecides L, then C = Oracle(R(<M, w>)) semidecides H:
If <M, w> H: M does not halt on w, so M# will never discover that M would have halted. So, on all inputs, M# halts in O(n) steps (since the number of simulation steps it runs is n). So timereq(M#) O(n2) and Oracle(<M#>) accepts.
If <M, w> H: M halts on w. So, on some (long) inputs, M# will notice the halting. On those inputs, it fails to halt. So timereq(M#) is undefined and thus not in O(n2). Oracle(<M#>) does not accept.
But no machine to semidecide H can exist, so neither does Oracle.
9) Consider the problem of multiplying two n n matrices. The straightforward algorithm multiply computes C = AB by computing the value for each element of C using the formula:
Multiply uses n multiplications and n-1 additions to compute each of the n2 elements of C. So it uses a total of n3 multiplications and n3-n2 additions. Thus timereq(multiply) (n3).
We observe that any algorithm that performs at least one operation for each element of C must take at least n2 steps. So we have an n2 lower bound and an n3 upper bound on the complexity of matrix multiplication.
Because matrix multiplication plays an important role in many kinds of applications, the question naturally arose, “Can we narrow that gap?” In particular, does there exist a better than (n3) matrix multiplication algorithm? In [Strassen 1969], Volker Strassen showed that the answer to that question is yes.
Strassen’s algorithm exploits a divide and conquer strategy in which it computes products and sums of smaller submatrices. Assume that n = 2k, for some k 1. (If it is not, then we can make it so by expanding the original matrix with rows and columns of zeros, or we can modify the algorithm presented here and divide the original matrix up differently.) We begin by dividing A, B, and C into 2 2 blocks. So we have:
With this decomposition, we can state the following equations that define the values for each element of C:
C1,1 = A1,1B1,1 + A1,2B2,1
C1,2 = A1,1B1,2 + A1,2B2,2
C2,1 = A2,1B1,1 + A2,2B2,1
C2,2 = A2,1B1,2 + A2,2B2,2
So far, decomposition hasn’t bought us anything. We must still do eight multiplications and four additions, each of which must be done on matrices of size 2k-1. Strassen’s insight was to define the following seven equations:
Q1 = (A1,1 + A2,2)(B1,1 + B2,2)
These equations can then be used to define the values for each element of C as follows:
C1,1 = Q1 + Q4 - Q5 + Q7
C1,2 = Q3 + Q5
C2,1 = Q2 + Q4
C2,2 = Q1 - Q2 + Q3 + Q6
Now, instead of eight matrix multiplications and four matrix additions, we do only seven matrix multiplications, but we must also do eighteen matrix additions (where a subtraction counts as an addition). We’ve replaced twelve matrix operations with 25. But matrix addition can be done in O(n2) time, while matrix multiplication remains more expensive.
Strassen’s algorithm applies these formulas recursively, each time dividing each matrix of size 2k into four matrices of size 2k-1. The process halts when k = 1. (Efficient implementations of the algorithm actually stop the recursion sooner and use the simpler multiply procedure on small submatrices. We’ll see why in part (e) of this problem.) We can summarize the algorithm as follows:
Strassen(A, B, k: where A and B are matrices of size 2k) =
C1,1 =
… /* Compute all the C matrices as described above.
C2,2 = Return C.
In the years following Strassen’s publication of his algorithm, newer ones that use even fewer operations have been discovered . The fastest known technique is the Coppersmith-Winograd algorithm, whose time complexity is O(n2.376). But it too complex to be practically useful. There do exist algorithms with better performance than Strassen, but it opened up this entire line of inquiry, so we should understand its complexity.
In this problem, we will analyze timereq of Strassen and compare it to timereq of the standard algorithm multiply. We shold issue one caveat before we start, however: The analysis that we are about to do just counts scalar multiplies and adds. It does not worry about such things as the behavior of caches and the use of pipelining. In practice, it turns out that the crossover point for Strassen relative to multiply is lower than our results suggest.
a) We begin by defining mult(k) to be the number of scalar multiplications that will be performed by Strassen when it multiplies two 2k 2k matrices. Similarly, let add(k) be the number of scalar additions. Describe both mult(k) and add(k) inductively by stating their value for the base case (when k = 1) and then describing their value for k > 1 as a function of their value for k-1.
mult(k) = if k = 1 then 7.
if k > 1 then 7mult(k-1)
To describe add(k), we consider the additions that are done when when Strassen is called the first time, plus the additions that are done when it is called recursively to do the multiplies. Adding two matrices of size n take n2 additions. So we have:
add(k) = if k = 1 then 18.
if k > 1 then 18(2k-1)2 + 7add(k-1)
b) To find closed form expressions for mult(k) and add requires solving the recurrence relations that were given as answers in part (a). Solving the one for mult(k) is easy. Solving the one for add(k) is harder.
Prove that the following are correct:
mult(k) = 7k. add(k) = 6(7k-4k).
We first show how we derived the claim about add(k):
For k > 1, we have:
=
c) We’d like to define the time requirement of Strassen, when multiplying two n n matrices, as a function of n, rather than as a function of log2 n, as we have been doing. So define mult(n) to be the number of multiplications that will be performed by Strassen when it multiplies two n n matrices. Similarly, let add(n) be the number of additions. Using the fact that k = log2 n, state mult(n) and add(n) as functions of n.
mult(n) = 7log2 n.
e) Let ops(n) = mult(n) + add(n) be the total number of scalar multiplications and additions that Strassen performs to multiply two n n matrices. Recall that, for the standard algorithm multiply, this total operation count is 2n3 – n2. We’d like to find the crossover point, i.e., the point at which Strassen performs fewer scalar operations than multiply does. So find the smallest value of k such that n = 2k and ops(n) <
2n3 – n2. (Hint: Once you have an equation that describes the relationship between the operation counts of the two algorithms, just start trying candidates for k, starting at 1.)
ops(k) = mult(k) + add(k) = 7k + 6(7k- 4k)
Chapter 27 164
Stated in terms of k, the number of operations executed by multiply is:
2(2k)3 – (2k)2 = 28k - 4k
So we need to find the smallest k such that:
7k + 6(7k- 4k) 28k - 4k 7k + 6(7k- 4k) - 28k + 4k 0 77k - 54k - 28k 0
Trying candidates for k, starting at 1, we find the smallest value that satisfies the inequality is 10.
10) In this problem, we will explore the operation of the Knuth-Morris-Pratt string search algorithm that we described in Example 27.5. Let p be the pattern cbacbcc.
a) Trace the execution of buildoverlap and show the table T that it builds.
We show T, as well as the kernels that correspond to each of its elements:
j 0 1 2 3 4 5 6
T[j] -1 0 0 0 1 2 1
the kernel b ba bac bcab bacbc
b) Using T, trace the execution of Knuth-Morris-Pratt(cbaccbacbcc, cbacbcc).
cbaccbacbcc cbacbcc
cbaccbacbcc
cbacbcc
cbaccbacbcc
cbacbcc