We call a functionf :Z+→Z+fully time-constructibleif there is a multitape Turing machine
that for each input of lengthn using exactlyf(n) time steps. The meaning of this strange definition is that with fully time-constructable functions, it is easy to bound the running time of Turing machines: If there is a Turing machine making exactlyf(n) steps on each input of lengthnthen we can build this into any other Turing machine as a clock: their tapes, except the work tapes, are different, and the combined Turing machine carries out in each step the work of both machines.
Obviously, every fully time-constructible function is recursive. On the other hands, it is easy to see thatn2, 2n, n! and every “reasonable” function is fully time-constructible. The
lemma below guarantees the existence of many completely time-constructable functions. Let us call a functionf :Z+→Z+well-computableif there is a Turing machine computing
f(n) in timeO(f(n)). (Here, we writenandf(n) in unary notation: the number nis given by a sequence 1. . .1 of length n and we want as output a sequence 1. . .1 of length f(n). The results would not be changed, however, if n and f(n) were represented e.g. in binary notation.) Now the following lemma is easy to prove:
Lemma 5.2.1 (a)To every well-computable functionf(n), there is a fully time-constructible function g(n)such that f(n)≤g(n)≤const·f(n).
(b) For every fully time-constructible function g(n) there is a well-computable function
f(n)withg(n)≤f(n)≤const·g(n).
(c)For every recursive functionf there is a fully time-constructible functiongwithf ≤g. This lemma allows us to use, in most cases, fully time-constructible and well-computable functions interchangeably. Following the custom, we will use the former.
5.2. TIME-CONSTRUCTIBLE AND WELL-COMPUTABLE FUNCTIONS 81 Theorem 5.2.2 If f(n)is fully time-constructible and g(n) logg(n) =o(f(n))then there is a language in DTIME(f(n))that does not belong to DTIME(g(n)).
This says that the time complexities of recursive languages are “sufficiently dense”. Anal- ogous, but easier, results hold for storage complexities.
Exercise 5.2.1 Prove the above theorem, and the following, closely related statement: Let
t0(n) logt0(n) = o(t(n)). Then the t(n)-bounded halting problem cannot be decided on a two-tape Turing machine in time t0(n).
Exercise 5.2.2 Show that ifS(n) is any function andS0(n) =o(S(n)) then theS(n) space- bounded halting problem cannot be solved in timeS0(n).
The full time-constructibility of the function f plays very important role in the last theorem. If we drop it then there can be an arbitrarily large “gap” belowf(n) which contains the time-complexity of no language at all.
Theorem 5.2.3 (Gap Theorem) For every recursive function ϕ(n)≥n there is a recur- sive function f(n)such that
DTIME(ϕ(f(n))) = DTIME(f(n)).
Thus, there is a recursive functionf with DTIME(f(n)2) = DTIME(f(n)),
moreover, there is even one with DTIME(22f(n)) = DTIME(f(n)).
Proof. Let us fix a 2-tape universal Turing machine. Denoteτ(x, y) the time needed forT
compute from inputxon the first tape andyon the second tape. (This can also be infinite.) Claim 5.2.4 There is a recursive function hsuch that for all n > 0 and all x, y ∈ Σ∗
0, if
|x|,|y| ≤nthen either τ(x, y)≤h(n)or τ(x, y)≥(ϕ(h(n)))3.
If the function
ψ(n) = max{τ(x, y) :|x|,|y| ≤n, τ(x, y) is finite}
was recursive this would satisfy the conditions trivially. This function is, however, not recur- sive (exercise: prove it!). We introduce therefore the following “constructive version”: for a
givenn, let us start from the time boundt=n+ 1. Let us arrange all pairs (x, y)∈(Σ∗
0)2,
|x|,|y| ≤n in a queue. Take the first element (x, y) of the queue and run the machine with this input. If it stops within timetthen throw out the pair (x, y). If it stops inssteps where
t < s≤ϕ(t)3 then lett:=sand throw out the pair (x, y) again. (Here, we used that ϕ(n)
is recursive.) If the machine does not stop even afterϕ(t)3 steps then stop it and place the
pair (x, y) to the end of the queue. If we have passed the queue without throwing out any pair then let us stop, withh(n) :=t. This function clearly has the property formulated in the Claim.
We will show that with the functionh(n) defined above, DTIME(h(n)) = DTIME(ϕ(h(n))).
For this, consider an arbitrary language L ∈ DTIME(ϕ(h(n))) (containment in the other direction is trivial). To this, a Turing machine can thus be given that decides L in time
ϕ(h(n)). Therefore a one-tape Turing machine can be given that decidesLin timeϕ(h(n))2.
This latter Turing machine can be simulated on the given universal Turing machine T with some programpon its second tape, in time,|p| ·ϕ(h(n)). Thus, ifnis large enough then T
works on all inputs (y, p) (|y| ≤n) for at mostϕ(h(n))3steps. But then, due to the definition
of h(n), it works on each such input at most h(n) steps. Thus, this machine decides, with the given program (which we can also put into the control unit, if we want) the languageL
in timeh(n), i.e.L ∈DTIME(h(n)). ¤
As a consequence of the theorem, we see that there is a recursive function f(n) with DTIME((m+ 1)f(n)) = DTIME(f(n)),
and thus
DTIME(f(n)) = DSPACE(f(n)).
For a given problem, there is often no “best” algorithm: the following surprising theorem is true.
Theorem 5.2.5 (Speed-up Theorem) For every recursive functiong(n)there is a recur- sive languageLsuch that for every Turing machineT decidingL there is a Turing machine
S decidingL with g(timeS(n))<timeT(n).
The Linear Speedup Theorem applies to every language; this theorem states only the existence of an arbitrarily “speedable” language. In general, for an arbitrary language, better than linear speed-up cannot be expected.
5.2. TIME-CONSTRUCTIBLE AND WELL-COMPUTABLE FUNCTIONS 83 Proof. The essence of the proof is that as we allow more complicated machines we can “hard-wire” more information into the control unit. Thus, the machine needs to work only with longer inputs “on their own merit”, and we want to construct the language in such a way that this should be easier and easier. It will not be enough, however, to hard-wire only the membership or non-membership of “short” words in L, we will need more information about them.
Without loss of generality, we can assume that g(n) > n and that g is a fully time- constructable function. Let us define a functionhwith the recursion
h(0) = 1, h(n) = (g(h(n−1)))3.
It is easy to see thath(n) is a monotonically increasing (in fact, very fast increasing), fully time-constructable function. Fix a universal Turing machine T0 with, say, two tapes. Let
τ(x, y) denote the time spent by T0 working on input (x, y) (this can also be infinite). Let
us call the pair (x, y) “fast” if|y| ≤ |x|andτ(x, y)≤h(|x| − |y|).
Let (x1, x2, . . .) be an ordering of the words e.g. in increasing order; we will select a word
yifor certain indicesias follows. For each indexi= 1,2, . . .in turn, we check whether there
is a wordynot selected yet that makes (xi, y) fast; if there are such words letyibe a shortest
one among these. LetLconsist of all wordsxifor whichyiexists and the Turing machineT0
halts on input (xi, yi) with the word “0” on its first tape. (These are the words not accepted
byT0with programyi.)
First we convince ourselves thatL is recursive, moreover, for all natural numbersk the question x∈ L is decidable inh(n−k) steps (where n=|x|) ifn is large enough. We can decide the membership ofxi if we decide whetheryi exists, findyi(if it exists), and run the
Turing machineT0 on input (xi, yi) for timeh(|xi| − |yi|).
This last step itself is already too much if |yi| ≤k; therefore we make a list of all pairs
(xi, yi) with |yi| ≤ k (this is a finite list), and put this into the control unit. This begins
therefore by checking whether the given wordxis in this list as the first element of a pair, and if it is, it accepts x (beyond the reading of x, this is only one step!). Suppose that
xi is not in the list. Then yi, if it exists, is longer than k. We can try all inputs (x, y)
with k <|y| ≤ |x|for “fastness” and this needs only (m2n+ 1)h(n−k−1) (including the
computation ofh(|x| − |y|)). The function h(n) grows so fast that this is less thanh(n−k). Now we haveyi and also see whetherT0 accepts the pair (xi, yi).
Second, we show that if a program y accepts the language L on on the machine T0
(i.e. stops for all Σ∗
0writing 1 or 0 on its first tape according to whetherxis in the language
L) thenycannot be equal to any of the selected wordsyi. This follows by the usual “diagonal”
reasoning: ifyi=y then let us see whetherxiis in the languageL. If yes thenT0must give
ofL, we did not putxiinto it. Conversely, ifxi6∈ Lthen it was left out sinceT0answers “1”
on input (xi, yi); but thenxi∈ Lsince the programy=yidecidesL. We get a contradiction
in both cases.
Third, we convince ourselves that if program y decidesLon the machine T0 then (x, y)
can be “fast” only for finitely many words x. Let namely (x, y) be “fast”, where x = xi.
Since y was available at the selection of yi (it was not selected earlier) therefore we would
have had to choose some yi for this iand the actually selectedyi could not be longer than
y. Thus, ifxdiffers from all wordsxj with|yj| ≤ |y|then (x, y) is not “fast”.
Finally, consider an arbitrary Turing machine T deciding L. To this, we can make a one-tape Turing machineT1 which also decides L and has timeT1(n)≤(timeT(n))
2. Since
the machineT0 is universal, T0 simulates T1 by some program y in such a way that (let us
be generous)τ(x, y)≤(timeT(|x|))3 for all sufficiently long wordsx. According to what was
proved above, however, we haveτ(x, y)≥h(|x| − |y|) for all but finitely many x, and thus timeT(n)≥(h(n− |y|))1/3.
Thus, for the above constructed Turing machineS decidingLin h(n− |y| −1) steps, we have
timeT(n)≥(h(n− |y|))1/3≥g(h(n− |y| −1))≥g(timeS(n)).
¤ The most important conclusion to be drawn from the speed-up theorem is that even though it is convenient to talk about the computational complexity of a certain languageL, rigorous statements concerning complexity generally don’t refer to a single function t(n) as the complexity, but only giveupper boundst0(n) (by constructing a Turing machine deciding the language in timet0(n)) orlower boundst00(n) (showing that no Turing machine can make the decision in timet00(n) for alln).