This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Chapter 5 Space and Time Complexity Were are many models of Turing Machines which are all equivalent in their ability to recognize languages; however, their space and time requirements vary signiﬁcantly. In order to discuss the space or time complexity of TMs, we need to deﬁne standard models. 5.1 Space Complexity R/W Tapes Figure 5.1: Space Complexity Model Figure 5.1 shows the standard TM model used for space complexity. The input tape is readonly. Were are a ﬁnite number of readwrite work tapes. If the input head is 1—way, then the machine is called online and if the input head is 2way, then the machine is called ofﬂine. Deﬁnition 18 An S(n) spacebounded TM uses no more than S(n) cells on each work tape for
any input of length n. A TM is weakly S spacebounded if it uses no more than 8' cells on each work tape for any input of length n that is accepted by the TM. 473 ©O.H.Ibarra 50 For space—bounded computation, a kworktape S space—bounded TM M can be converted to an equivalent lworktape S spacebounded TM M ’ (Ml uses 2k; tracks to simulate the tapes of M, two tracks for each tape of M. See similar result for time—bounded computation below for details.) Thus for spacebounded computation, we need not specify the number of worktapes.
Space Compression: Let M be an S(n) space—bounded TM and k; a positive integer. We can construct an S /l~€ space—bounded TM M’ equivalent to M. (It is understood that S /l~€ denotes the ceiling of S/ M ' uses a larger worktape alphabet  it “packs” k; symbols of M into one symbo . Thus for spacebounded computation we need not specify constants of proportionality the base when taking logarithms, etc. For space bounds that are greater than linear, i.e., S 2 nVn, the ofﬂine and online versions are equivalent. This is true because if we are using at least it space, then the online TM can ﬁrst copy the input to one of its work tapes before doing anything else and then read the input from
that tape. Deﬁnition 19 A function S(n) is fully space constructible (“nice” if there exists an S(n) space—bounded TM that uses exactly S(n) cells for every input of length n. Most common functions such as log n, n2, 2” are fully space constructible. An example of a function which is not fully space constructible is log log n. Deﬁnition 20 A function S(n) is space constructible if there exists an S(n) spacebounded TM such that for every ii, there is at least one input a: of length n that requires the machine to use exactly Si ni cells. If S is fully constructible then a weakly S spacebounded TM is also an S space—bounded
TM Claim: Let L be the language consisting of all strings of the form x#x, where at is a binary string. Suppose M is an on—line S spacebounded TM accepting L. Then S is at least linear
for inﬁnitely many n’s.
Proof: (Cutand—paste argument): Let M be an S spacebounded on—line TM accepting L. Consider the computation of M on strings of the form $79,931, where r = n. The partial ID (PID) of M when it reaches # is given by (q,j, w), where q is the state, in is the contents of the worktape, and j is the position of the R/W head within 111. Let s and t be the number of states and worktape alphabet size of M (including
the blank symbol). The number of possible PID7s when M reaches # is at most sS(n)tSl”) g 05W) for some 0. If S grows slower than linear for all but a ﬁnite number of n’s, then 050” < 2" for all but a ﬁnite number of n’s. Then for some no, 050” < 2” for all n 2 n0. Since there are 2”
strings of the form x#x, where = n, there are distinct strings ac and m’ of length n such that on
input x#x, the PID of M when it reaches # is equal to the PID when it reaches # on input $’#$’. ©O.H.Ibarra 51 Since both ac#ac and x’#x’ are accepted by M , it follows that M will also accept ac#ac’. This is a contradiction, since x#x’ is not in L. Similar yZ Claim: Any 3 spacebounded on—line TM accepting L' = Ozlzlt > 0 requires logn space for inﬁnitely many n7s.
It is easy to see that L and L’ in the above claims can be accepted by on—line TM’s in linear space and logn space7 respectively. We bound logn is necessary and sufﬁcient for accepting a nonregular language. Sufﬁciency follows from the fact that L’ above can be accepted by an on—line log n space—bounded TM. The necessity is given by the following theorem. Theorem 6 Let L be a nonregular language accepted by an onltne 8(a) spacebounded TM. Then S'(n) is at least logn for inﬁnitely many n’s. Proof: Assume that no symbol is overwritten by the blank symbol7 A (otherwise7 use a special symbol X and treat it like /\), and there is only one work tape (otherwise, merge all worktapes into one worktape ). o The partial ID of a TM gives the current status of the computation. It consists of the current state of the ﬁnite control7 the contents of the work tape and the position of the worktape
head. Let k; be a positive integer. Then the number of different Ple of the TM that uses
less than k; cells of the work tape is given by H mm];
2 satZ < skt E1 0 Let a be the input alphabet size. 0 Let p be the smallest integer such that 19’“ 2 asktk. o For any given positive integer 71, let mm 2 $1M    m be the shortest string such that pro— cessing of wn by M causes it to use at least logpn work tape cells. Let k; = logp n. Such a string must exist. (Otherwise : no such that all strings of any length use less than logp n0 cells which is constant; this is impossible because the language is nonregular). Claim: No PID is repeated when the TM works on this string wn. If some PID is repeated then wn cannot be the shortest such string which is a contradiction. Therefore we see that 7“ S asktk 3 pk = plogP" Hence , 7" < n. Therefore for every n, there exists a string of length n, i.e., 901262  nmryyny (padded so that the length is n)7 that uses exactly logp n cells. ©O.H.Ibarra 52 Thus for online computation, log n space is necessary and sufﬁcient to accept a nonregular set. (As noted earlier, sufﬁciency is shown by the language L = 2 1}  using a binary counter to count the number of OS or 1s).
Claim: Let L be the language consisting of strings of the form :13#33, z binary. Any off—line S spacebounded TM accepting L must use logn space for inﬁnitely many n7s. The proof of this claim uses the Kolmogorov complexity argument.
Let E be a ﬁxed alphabet. For every string r 6 2+, let M be a TM with an initially blank two—way R/ W worktape and an initially blank one—way writeonly output tape7 which when started in its initial state writes (generates) an on the output tape and halts. (initially blank) + 2—way RTW tape _> _> _> ? q0 halt ﬁr —l (initially blank) X l—way writeonly tape Let M denote the size of M , i.e., length of the binary encoding of the description (program)
m
Elxamp es: 1. a: = 1011 We can construct a TM M whose size is greater than or equal to n = :1: . (M writes the bits of as one at at time, using approx n states.) 2. 101010 10 (repeated n times) We can construct a TM M whose size is polylog n (2 polynomial in log (M writes n in binary (as a counter) on the worktape and then generates 10 n times using the counter.) 3. Any unary string of length n can be generated by a TM of size polylog n. Deﬁnition 21 For every string :10, let : Kolmogorov complexity ofzv = [M generates I.e., is the length of the encoding of the smallest TM that prints m and halts. (Note: we don’t care about the space and time complerity of Lemma 1 Let E = {0,1}. For every n, there emits a binary string as” such that = n and KWMZn. Proof: For every n > 1, there are 2” distinct strings of length n. Suppose that each of these strings can be generated by a smallest TM whose binary encoding length is less than n. Then there ©O.H.Ibarra 53 are less than 2” such machines. Hence at least 2 strings of lenth n can be generated by the same machine. This is 1mposs1ble. It follows from the lemma that for every n7 there is at least one string an of length n such that any TM generating a has binary encoding 2 n7 i.e.7 2 n. Deﬁnition 22 Any binary string r of length n such that 2 n is called random the sense that the pattern in a: is “not regular” so as to be generable by a TM with size < Claim: Let L = {:13#:13 r E {07 1}+}. Then L can be accepted by an offline TM in space log n. Proof: The TM compares corresponding bit positions using a binary counter to record the posi tions. Claim: Let L = {$#$ :1: E {0, 1}+}. Any offline TM accepting L uses log n space for inﬁnitely many strings of length n.
Proof: (Kolmogorov complexity argument): Let M be an S space—bounded off—line TM accepting L. We show that for every n7 there exists an input of length n that requires M to use at least logn space on this input. Let n be a positive integer. Pick a random string :16” of length n. Hence 2 n. Consider the computation of M on input xn#$n: xn # xn $ X1 We assume w.l.o.g. that M accepts with the input head on the left delimiter. Recall that a PID is a triple of the form (q, z,j)7 where q is the state, 2 is the contents of the worktape, and j is
the position of the R/W head within 2. In the figure7 o Xi = PID the i—th time M reaches # from the left. o Yi = PID the i—th time M reaches # from the right. ©O.H.Ibarra 54 Call (X1,Y1, ...,Xk, Yk) the crossing sequence (on 7%). Observations: 1. A PID requires dS(2n + 1) space to store for some constant d. 2. The number of PID’s on inputs of length 2n + 1 is 65(2”+1) for some constant 6. Given the rules (program) of M and the crossing sequence, i.e, given the information (M , X 1, Y1, ..., X k, Yk), we can construct a TM N to generate an” as follows: N uses its worktape as a 2—track tape: o N ﬁrst writes on the ﬁrst track the information (M, X1,Y1, ...,Xk, Yk) and then writes 71 in binary (thus using logn cells) to the right. o N then systematically generates on the second track, strings of the form $#$ with n = z : E worktape l (M, X1,Y1, Xk,Yk)
x#x For each string $#$ on its second track, N uses (M, X1, Y1, ..., XK, Yk) to check ifz = 3:71. To check if z = z”, N simuates M ’s computation on $#$ and see if the computation is consistent with the crossing sequence (X1, Y1, ..., Xk, Yk). If the computation is consistent, then N knows that at = :16”, and outputs 30(2 an”) on its output tape. If the computation is not consistent, M tries another :16. The size of the binary encoding of N depends only on the information (M, X1,Y1, ...,Xk, Yk) and the rules to write n in binary: 0 size of M is a constant. 0 size of each X, or Yi is dS(2n + 1) for some constant d.
. k is at Jmcst 3S(2n+1) fa]: Emma CCmstamt 3 0 writing n in binary requires f logn rules for some contant f. Hence the size ofM is O(dS(2n+ 1) (65(2n+1))+f log n) g 95(2”+1) for some constant 9, assuming Sim is at least log log n. Since an” is random, 95(27H1) 2 n. It follows that S(2n + 1) Z logg n. It follows that there is a constant it such that for every n, there is an input of length n that requires M to use at least h log 71 space. ©O.H.Ibarra 55 Theorem 7 Let L be a nonregulai" language accepted by an oﬂline S(n) spacebounded TM. Then S(n) is at least log logn for inﬁnitely many n’s. Proof: (Omitted for now.) Claim: The log log n bound is sufﬁcient to accept a nonregular language, e.g., L = {x1#...#$klh >
0, ac, is binary representation of i with LSB l}. CONVENTION: Unless otherwise noted, TM will mean Deterministic Turing Machine. Also, from now on unless otherwise noted, we will deal only with oﬁIine TM’S. Deﬁnition 23 DSPACE(S(n)) = {L L can be accepted by a TM within S(n) space Deﬁnition 24 PSPACE(S(n)) = {L L can be accepted by a TM within S'(n) space and S'(n) is a polynomial space PSPAOE = U DSPAOEW‘)
{71 If M is an S(n) spacebounded TM, with S(n) 2 log n, then there exists another TM, M’ which is equivalent to M, is also S space bounded and always halts. For example, M’ could just keep a counter of size S for the execution and check for overﬂow. For an 8' space bounded TM, the maximum number of distinct IDs is limited (s(n + 2)tS(")S(n) < cSW for some constant c); therefore one can incorporate a counter that shuts off the computation if the TM runs more than this bound. Note that this is non trivial because it is hard to compute 3 One can get around this by working 8' out as the simulation proceeds  begin with an estimate of 1 cell, then if the worktape uses more cells than the estimate we increase the estimate for the counter and restart the computation (the counter counts in base 0, the constant mentioned above). 5.2 Time Complexity Deﬁnition 25 A TM is T(n) timebounded for all inputs of length n, it runs no more than Tl ni steps. The time complexity must be at least linear since the TM must read all the input symbols for a nontrivial computation. Deﬁnition 26 DTIME(T(n)) = {LlL can be accepted by a TM within T(n) steps Deﬁnition 27 PTIME = {L L can be accepted by a TM within polynomial time }, i.e., PTIME = U DTIME(ni)
{71 ©O.H.Ibarra 56 Theorem 8 PTIME g PSPAC'E Proof: This obvious since a TM cannot use more cells than the number of steps that it takes. The guestion of whether the containment is proper is open. Theorem 9 IfL is accepted by a k—tape T(n) time—bounded TM, then L can be accepted by a I—tape
0(T2l”)) timebounded TM. Proof: iIdeal: M’ simulates M by creating 2k: tracks on its tape, two tracks for each tape of M. One track contains the contents of the tape being simulated while the second track contains a marker indicating the position of the read/write head on that tape. To simulate a move of M, M’ ﬁrst makes a sweep between the leftmost marker and rightmost marker to “pick up” (read) the symbols under the k: heads of M. Then M’ applies the move rule by rewriting the symbols under the heads on the k: tracks and moving the markers appropriately. Since the leftmost marker and the rightmost marker can be T(n) cells apart, it would take about 2T(n) steps to simulate one move of M. Thus, M’ is
0(T2(")) timebounded. Claim: The quadratic simulation time in the theorem above is best possible. Proof: Let L be the language consisting of all strings of the form x# X, where X is a binary string. Clearly, L can be accepted by a 2—tape TM in linear time (i.e., O(n) time). We show below that any single—tape TM accepting L requires quadratic time for inﬁnitely many 11 s. Figure 5.2: Computation of M Let M be a singletape TM (i.e., there is only one read / write tape which initially contains the input. Consider the computation of M on input a1    an. We may assume that M has no stationary moves, i.e., if 6(q, a) = (p, b, d), then d = +1 or d = —1. Also, assume that the “change” in state from q to p occurs while M is moving its head to +d (i.e. to the left or right of the current position). The ©O.H.Ibarra 57 computation of M has the form shown in Figure 5.2. Without loss of generality, we assume that M accepts only at the right end of the cells initially occupied by the input. Between any 2 tape cells, the sequence of states the boundary is crossed is called the crossing sequence (CS) at that boundary.
The following lemmas are easily veriﬁed. Lemma 2 If a1an is accepted by M and the CS between a, and ai+1 is the same as the CS between aj and aj+1, where i < j, then a1aiaj+1an is also accepted by M. Lemma 3 If a1    an and b1    bm are accepted by M and the (33 between ai and aiﬁl is the same as the CS between bj and bj+1, then a1a,~bj+1bm and b1bja,~+1an are also accepted by M Lemma 4 The sum of the lengths of the nonempty 08’s is at most T(n), the time complexity of M. (Note that if a boundary is never crossed, the the CS at that boundary is empty.) Theorem 10 Let L = w is a binary string IfM is a T(n) time—bounded single—tape TM accepting L, then T(n) must grow at least as fast as n2 for inﬁnitely many n’s. Proof: Suppose there are two strings x1$2#$1x2 and $3x4#x3$4 that are accepted by M with $1 75 :133 but length(:1:1) = length($3) = i for some i. Then by Lemma 3, the CS between :131 and 3:2 is different from the CSLetween :133 and $4. . Over all strings of length n in L (n odd), let be the average length of the CS between the ith and i + lst symbols. . The number of strings of length n in L whose CS between the ith and i + lst symbols is < 4. 17.71 17.7171  1 .—_
2p(’t)182§'2 2 —2 2 The number of distinct CS’S of length 3 2p(i) is < s2p(l)+1, where s is the number of states
m 17.71 2T—1
$2P(i)+1 ' Thus, the number of strings with same CS between the ith and i + 1st symbols is > 5. The number of strings a1aiai+1    a(n_1)/2   with distinct strings in positions i+ 1, i+2, 6. gn— 1gp is 2’34”. 1 If is > 2nT—l’i, then there will be two strings with the same CS between the ith and i + lst symbols, have the same strings in positions i + l, i + 2, (n — l)/2, but distinct
21174—1
$2P(i)+1 n71 strings in positions 1, 2, i. This is impossible. Hence, 3 277i. ©O.H.Ibarra 58 Therefore, (2p(i) +1) 2 — 1)/log2 s, and > — 1)/210g2 —%. Summing from i = 1 to i = (n — 1) / 2 gives a lower bound on the average sum of the CS7s over all positions. Clearly, this
sum grows at least as fast as n2 (for odd Hence by the Lemma 47 T(n) must grow at least as fast as n2 for inﬁnitely many n’s. Theorem 11 Let L be accepted by a ttape TM with (t Z 2) in T(n) time. Let k be a positive
integer. Then L can be accepted by a t—tape TM in n +T(n)/l<: time. Proof: As in the case of Space Compression one can pack the input symbols into fewer symbols using a larger alphabet and thereby reduce the time complex1ty. Note however that we need to have at least time n to read the complete input in order to encode it into the new alphabet. Corollary 5 UL is accepted by a t—tape TM in T(n) time, and T(n) grows faster than linear, then
for any positive integer k, L can also be accepted by a ttape TM in time. Corollary 6 Let L be accepted by a t—tape TM with (t Z 2) in O(n) time (i.e. in linear time). Then for any 6 > 07 we can construct an equivalent t—tape TM that runs in (1 + 6)n time. For singletape TM’s (i.e.7 TM7s with only one read/write tape which initially contains the inputi, we can prove: Theorem 12 Let L be a nonregular language accepted by a singletape T(n) timebounded TM. Then T(n) must grow at least as fast as nlogn for inﬁnitely many n’s. Proof: iOmittedi Claim: The n log n bound above can be achieved7 eg.7 L = {02? i > 0} is a nonregular language that can be accepted by an nlogn time—bounded singletape TM Theorem 13 Let L be a language accepted by a T(n) time—bounded multitape TM. Then L can be
accepted by a Q—tape T(n) log T(n) time—bounded TM. Proof: (Omitted! 5.3 NonDeterminism A nondeterministic TM (NTM) is one which may have more than one choice of next move. The
classes of languages NSPAC'E7 NPS'PACE7 NTIME7 NPTIME are the nondeterministic coun—
terparts of the languages DSPACE, PS'PACE7 DTIME7 PTIME. An S space bounded NTM can be simulated by a DTM. The execution of an NTM can be viewed as a tree of possible paths — if any of these leads to acceptance, then the input is accepted, ©O.H.Ibarra 59 otherwise it is rejected. Simulating the NTM cannot be done using a depth—ﬁrst search of this tree because some of the paths may never terminate. (However, we can use breadthﬁrst search.) But as seen before, we have an upperbound on the number of distinct IDS for a space bounded machine. We can therefore limit the depth to which we search. In order to do this, we need a counter that
Sm) can count up to 0 since there are at most 050“ possible IDS. Searching the tree would require exponential space. In 1969, Savitch showed the following result: Theorem 14 [Savitch’s Theorem] For S(n) 2 log n, NSPACE(S(n)) g DSPACE(32(n)) Proof: Assume that $01) is fully space constructible. The TM is initially in ID1 = (q0,1,)\,1) which represents the state, the input head position, work tape contents and the work tape head position, respectively (we assume that there is only one work tape). For an accepted string, there
ex1sts an accepting path, ID1I—ID2IDf=(f,1,)\,1)
:
<cSn The symbol ID,~  [DZ+1 means ID, can reach IDi+11n one step. Note that we assume that the
TM accepts with both heads at the beginning of the tapes and that the work tape is blank  this is easy to achieve by adding additional states to move the heads back to the beginning after erasing the work tape. We also assume that there is exactly one accepting state, f. The length of this accepting path can also be assumed to be exactly CS by repeating the accepting ID. I D1 can reach I Df in 059“ steps if and only if there exists I D3 that lies halfway and reachable
from I D1 in half—the steps and I D f is reachable from I D5 in halIThe steps, i.e. ID1 ism) IDf <=> :IDS s.t. ID1 ’[Cs(n>J IDS ’fcs(n)] IDf Thus, we can check whether a given
‘ 2 ID is reachable from another using thezfollowing recursive procedure: procedure TEST(ID1, IDg, // check if ID1 * IDg in at most 2' steps M
m
if (ID1 = ID2) or (ID1 l IDg) then
return l‘true’l
8% return i ‘false’ 5 else begin
for each ID3 using at most S(n) space
if (TEST(ID1, 103, L j) and (TEST(ID3, IDQ, §) then i
2 ©O.H.Ibarra 60 returnt ‘true7l
end ret urni ‘false’ 6
end Thus we have a recursive procedure which can simulate the nondeterministic Turing machine. The length of each ID is given by ID = (q,t,w,j) =c+logn+S(n)+logS(n)=O(S(n)) Thus each ID requires O(S(n)) space S(N) 2 logn  because if S(n) < logn then the 71 ﬁeld of the ID cannot be stored in 0(8' space). This recursive procedure can be “unrolled” and implemented on a dterministic TM. The amount of space needed in the implementation is:
The size of the stack frame 2 O(S(n))
The number of frames 2 log(cSl”)) = O(S(n)) Therefore, the space requirement is 0(S Corollary 7 PSPACE E NPSPAC'E Proof: It is clear to see that
PSPAC'E g NPSPAC'E. To Show that we use Savitcﬁ’s Theorem. Since all polynomial functions are at least as large as log n, and the square of a polynomial function is also a polynomial function, by Savitch’s theorem, any language that is accepted by NTM in polynomial (say space will also be accepted by some DTM in polynomial space (p2 In Savitch’s theorem, 3 need not be fully space constructible. One need only do the simu lation for S = 1, 2,   . The proof of Savitch’s theorem actually proves the following: Theorem 15 Let L be accepted by an NTM in S(n)—5pace and T(n) time (simultaneously). Then
L can be accepted by a DTM in space S'(n) log Open Problem: Is Savitch’s result optimal? I.e., is NSPACE(S(n)) g DSPACE(S’(n)) for
some S’(n) less than S201)? In particular, is NSPACE(S(n)) = DSPACE(S(n))? Actually, a T(n)time bounded NTM can be simulated by a DTM in T(n)—space. Claim: Let L be accepted by an NTM in T(n)—time. If T(n) is fully space constructible, then L
can be accepted by a DTM in T(n)—space. Proof: The T(n)space DTM performs a depthﬁrst search of the computation of the NTM. ©O.H.Ibarra 61 5.4 Complementation Suppose S 2 log n. Since an S space—bounded DTM can be made halting, the complement of a language accepted by such a machine can also be accepted by an S spacebounded DTM. (We simply call the “accept” states “reject” states7 and vice—versa.) What about the languages accepted by nondeterministic machines? Suppose we have an S spacebounded NTM and assume it always halts on any input (otherwise, it can be made halting). Can we simply call the “accept” states ‘reject’ states and viceversa and obtain a machine that will accept the complement language?
The answer is no. Why? Clearly7 by Savitch’s theorem7 there is an S (n)2 spacebounded DTM (and hence an NTM) that can accept the complement. An interesting question is whether S space is sufﬁcient for an NTM to accept the complement language. The answer is a surprising yes.
WW We assume that S 2 logn and that it is fully space constructible. In the following7 M is an
S space—bounded NTM with a single readwrite work tape. START represents the initial ID. For a given input at of length n, we assume a lexicographical ordering of all S space—bounded ID s. First and Last represent the ﬁrst and last ID’s in the ordering. Theorem 16 Let M be an S(n) spacebounded NTM. We can construct an S(n) spacebounded
NTM M’ such that L{M’) : complement of Proof: We ﬁrst prove the theorem under the assumption that when M’ is given an input r of length n, it is also given on another tape, a number N (in binary) = the exact number of lD’s reachable from START on input :16. Note that N is at most 05W) for some constant c and therefore takes only O(S(n)) space to store. We NTM M is an implementation of the foﬂowing nondeterministic algorithm: CW
TARGET <— First
1 Goto 2 or 3
2 if TARGET 7E Last then [TARGET <— TARGET+1; goto 1]
if TARGET 2 Last then output(‘reject’); halt
3 Guess a path from START to TARGET (Le.7 guess that TARGET is reachable) if successful then
case: TAWET is an accepting ID
case: TARGET is a rejecting ID
COUNTER <— COUNTER + 1
casezCOUw
if TARGET 75 Last then TARGET <— TARGET+1; goto 1 ©O.H.Ibarra 62 else [output(‘reject7); halt]
case: UNTER = N Note that in the algorithm above7 guessing a path can lead to success, lead to an inﬁnite loop7 or the procedure can get stuck. We are only interested in the ﬁrst case. Clearly, M7 accepts 1E if and only if it has found exactly N reachable ID7s, none of which is accepting. Moreover7 M’ is S spacebounded. To complete the proof7 we need to show how M’ can compute N. We do th1s 1n the next lemma. Lemma 5 Given 1E and START, M’ can compute N : the exact number of [D’s reachable from
START on input a“. Proof: Let Nd be the total number of ID7s reachable from START in at most d steps. We show,
by induction, how M7 can compute Nd,,1 from Nd. Note that the bases N0 and N1 are obvious. The nondeterministic algorithm below computes Nd+1 given Nd. HQUNTEB2 t1!
T ET2 Fir
0 UNTERl <— 0
TARGETl <— First
1 Goto 2 or 3
2 if TARGETl # Last then [TARGETl <— TARGETl—H; goto 1]
if TARGETl 2 Last then output(‘reject’); haltl
3 Guess a path from START to TARGETl reachable in at most d steps
if successful then
COUNTERl <— COUNTERl—l—l
case: TARGET2 is reachable from TARGETl in 0 or 1 step
UNTER2 <— UNTER2—1
if TARGET2 # Last then [TARGET2 <— TARGET2+1; goto 0]
if TARGET2 2 Last then goto 4
case: TARGET2 is not reachable from TARGETl in 0 or 1 step
if COUNTERl < Nd and TARGETl 75 Last then
[TARGETl <— TARGET1+1; goto 1]
if COUNTERl < Nd and TARGETl 2 LAST then
norm
[TARGET2 <— TARGET2+1; goto 0] ©O.H.Ibarra 63 4 ( continue I In the above algorithm, the value of N614 is computed in COUNTER2. COUNTER2 is incre
mented by one every time a TARGET2 is found that is reachable from a TmETl in 0 or 1 step
provided TARGETl is reachable from START in at most d steps. Clearly, when the algorithm exits to statement labeled 4, cm
&m The algorithm is iterated until Nd,,1 = Nd for some d. When this happens, N = Nd. Note that the requirement that S 2 logn is not necessary. Why? We give an application to graphs. For a connected directed graph G with vertices 1, ..., n, let
E (G ) be a string which represents G, i.e., E (G) is a list of edges (in binary) separated by delimiters.
Note that for a dense graph, the length of E(G) is 0072). Let L = { E(G) l G is a connected directed graph with n vertices, and there is a path from vertex 1 to Clearly, L can be accepted by a logn spacebounded NTM. From the theorem avove, L’s complement can also accepted by a
logn spacebounded NTM. Note that both languages are in WIME. It is not known if NTIM is closed under complementation. In fact, it is not known if the complements of the languages in NP are in P. Clearly, the complement of every language in
NP is in DTIME(2p(n)) for some polynomial p(n). 5.5 SpaceTime Tradeoffs Claim If L e NSPACE(S(n)), then L e DTIME(dS(”))).
Proof Let M be an $01) space—bounded (onetape offline) NTM. An ID of M is given by a 4 tuple, (q,i,w,j), where q is the state, i is the input head position, 111 is the worktape content, and j is the position of the worktape head. The number of distinct ID’s M can enter is at most
s(n+2)S(n)tS(”) 3 0501) time for some constant c. (The +2 takes into account the input delimiters.) We construct a 3—tape DTM, M’ which when given input 1E on Tape 1, constructs the list of all reachable IDS from the initial ID using Tapes 2 and 3. M uses the following dynamic programming algorithm: [DO 2 q0,1,)\,1
Tape 2 <— Q
Tape <— {IDO}
while Tape 2 7E Tape 3 d0
egln
Tape 2 <— Tape 3
Tape 3 <— { All IDs in Tape 2 } U {IDE ID in Tape 2, ID  ID } ©O.H.Ibarra 64 m if Tape 2 contains an accepting ID then
M l accepts
else
M’ rejects The while loop is executed at most cSW times. Constructing the list in Tape 3 takes at most
O(cS(")) time. Thus, M’ has time complexity O(cS(").cSl”)) 3 ask”) for some constant d. Corollary 8 NSPACE(log n) g PTIME Proof dlog" S 71’“ for some k.
We state the following without proof:
Claim: Let L be accepted by a T(n) time—bounded DTM. Then L can be accepted by a log T(n) space oun e T .
It is not known if the above result is optimal. Corollary 9 Let L be accepted by a linear time NTM. Then L can be accepted by a linear space
DIM For singletape TM’s (i.e., TM7s with only one read/write tape which initially contains the input), there is a stronger result. Again, we state it without proof:
Claim: Let L be accepted by a T(n) timebounded DTM, where T(n) 2 n2. Then L can be
accepted by a T(n)1 2 spacebounded DTM. 5 .6 Hierarchy Theorems Binary encoding of offline oneworktape DTMs: Choose a binary encoding, E(M), ofeach TM M, M =< Q, E, T‘, 6, q1, qs >, where Q = {q1,    ,qs},
E = {0,1}, T‘ = {A1,    ,At}. Encode each state by a binary string of length log2 5 and each
worktape symbol by a binary string of length [logg t]. The input symbols, 0,1 and input delimiters are encoded as 0,1,00,01. The moves +1 and —l are encoded as 0 and 1 respectively. A rule R
of the form 6(q,a,A) = (p, d1,B,d2) is encoded as E(R) = E(q)E(a)E(A)E(p)E(d1)E(B)E(d2).
If M has rules R1,R2,,Rk, M is encoded as, = 1101501t0E(R1)E(R2)~~~E(Rk). Note
that the sizes of the state set (3) and tape alphabet (t) are given as part of the encoding for easy deciphering of the rules. For reasons to become apparent later, we allow the encoding to have the ‘padding’ preﬁx liO for any positive integer i. Thus, a TM has an inﬁnite number of encodings. Note that if M on an input of length it uses S(n) space, then this space in binary is log2 tS(n). ©O.H.Ibarra 65 Theorem 17 (Space Hierarchy): Let 8'1 and 82(71) be space bounds with 81(n), 82(71) 2 logn C
and 52(n) fully space constructible. Iflimnnoo = 0, then DSPACE(Sl(n)) 75 DSPACE(52(H)). Proof: We need to show that DSPACE(8'1 g DSPACE(82(H)) and SPACE(Sg(n)) — DSPACE(81 75 gb. The ﬁrst is obvious. Deﬁne the language: L = E {0, 1}",90 = for some off—line one—worktape DTM M with t worktape symbols,
n = :1: , and M on input $, halts and rejects :1: within 32(n)/log2 t space First we construct an 32(77.) space—bounded DTM H to accept L as follows: Given input $ of length
n, H does the following: 1. H marks exactly 5501) cells on its worktapes. (H can do this because 52(77.) is fully space
constructible.) It also marks 82(71) / [logg t] cells of this space. From now on , H is restricted to use no more than 82(71) space. This guarantees that H is 82(71) space bounded. 2. H Checks that :1: = E(M) for some offline oneworktape TM M. If not, reject and halt. If okay, proceed to next step. 3. H uses 90 (which is the binary encoding of M) to simulate the computation of M on :10: a If M halts and accepts within Sg(n)/[log2 if] space, H halts and rejects.
b (c
(d If M goes into an inﬁnite loop within 82(n)/[log2 t] space, H halts and rejects. (Note (
( If M halts and rejects within 82(n)/ [logZ t] space, H halts and accepts. If M attempts to use more than 52(n)/log2 t space, H halts and rejects. )
)
)
) that H can detect this by incorporating a counter which uses 3 52(77.) space.) We now show that L cannot be accepted by any 8'1 spacebounded DTM M. Suppose such an M exists. Let :1: be an encoding of M. We may assume that M halts on all inputs (otherwise, we can incorporate a counter to M Let M have tape alphabet size t. Since M has an inﬁnite number of encodings, we choose an encoding lb of M such that = 7?. satisﬁes: 32 n 2 lOg2t 51 n This is possible since limnnoO gi—Ennﬁ = 0. Now consider the behavior of H and M on input :10: o If M halts and accepts 1E within 81 cells (which in binary is [log 75] 81 H halts and
rejects. Note that H has enough space to do the simulation since 8501) 2 log2 tSl(n). o If M halts and rejects, H halts and accepts. Note that cases c) and (1) above do not occur. Thus M accepts 1E <=> H rejects :10. This is a 1.. ©O.H.Ibarra 66 The proof of the space hierarchy theorem does not work for nondeterministic machines (why?). C
We can prove weaker results using Savitch’s theorem. For example, NSPACE(n) Q DSPACE(n2) 75 C
DSPACE(n2 log n) g NSPACE(n2 log n). Thus, NSPACE(n) 7A NSPACEm2 log n). To prove stronger results, we need the following translation (or padding) lemma. Lemma 6 (Translation). Let 51(77.) 2 logn,Sg(n) 2 logn and f(n) 2 n. Assume 51(77.) and
Sg(f(n)) are fully space constructible. Then NSPACE(S1(n)) Q NSPACE(Sg(n)) => NSPACE(Sl(f(n)) C NSPAOEgs2 (ﬂog). Proof: Let L1 be in NSPACE(Sl(f(n))). Then L1 is accepted by an 81(f(n)) space bounded
NTM, M1. Let # be a new symbol. Deﬁne the language L2 = {$#iM1 accepts $ within space Sl(:1: + We can construct an NTM M2 which accepts L2 as follows: On input 3:769, M2 marks off 8'1( :1: +t') cells on its worktapes (it can do this since 81 is fully space constructible). Then M2 simulates M1 on an (ignoring the sufﬁx #6). M2 accepts if and only of M1 accepts. Clearly, M2 is 81(n) space bounded. (Note that the length of the input to M2, which is used to measure its space bound,
includes #i.) Note that :1: is in L1 if and only if $#f(”)_” is in L2 (of course, other $79,975 < f(n) — 71
may also be in L2). Now since L2 is in NSPACE(Sl(n)), L2 is also in NSPACE(Sg(n)) by
hypothesis. Hence, there is an 82(71) space—bounded NTM, M3 accepting L2. We now construct an Sg( space—bounded NTM M4 accepting L1 (the original language) from M3. On input :1: of length 71, M4 marks off 32(f cells on its worktapes. (It can do this since
$2( is fully space constructible.) M4 is restricted to use only the marked cells. M4 simulates M3 on 907%“. However, Since it does not know t, it Will systematically try 2' = 0,1,2, M4 uses a counter of size logt' to simulate M3’s computation on my. (Note that M4 has only :1: as its input; thus, it needs the counter to simulate the “input head movements” of M3 on the segment #i.) If for some 1', M3 accepts, M4 also accepts. Clearly, M4 accepts L1. To see that $2( is sufficient to do the simulation, consider a string 90 of length n in L1. Then M1 accepts 90 within 81(f(n)) space. Then $#f(”)_” is accepted by M2 within space 5401 + f(n) — n) 2 8'1 space. Since
NSPACE(81 Q NSPACE(Sg(n)), M3 accepts $#f(")_”; hence M4 accepts :10. Note also that
the counter for the #i segment uses at most log'i S log(f(n) — n) space. Since 2 n and
8501) 2 log n, Sg(f(n)) 2 log(f(n) — Thus, M4 has enough space for the counter. We translation lemma can be used to prove tighter hierarchies for nondeterministic space. For
m C
Claim: NSPACE(n) # NSPACE(n2).
Proof: Suppose not. Then NSPACE(n2) Q NSPACE(n). Choosing f(n) 2 712, we get
NSPACE(n4) g NSPACEW). The two containment relations imply NSPACE(n4) g NSPACE(n). This is impossible by Savitch7s theorem and the space hierarchy theorem. There is also a time hierarchy theorem. However, the hierarchy is not as tight as for space. ©O.H.Ibarra 67 Theorem 18 Let T1 and T2(n) be time functions. Subject to some “ntcencss” conditions, if hm T1 n 10g T1 n : 0
n—>oo T2(n) then DTIME(T1 (71)) ipTIMEwgm». Proof: Omitted. Tie padding technique can also be used to prove tlght nondetermlnistlc time hlerarchies. Here is another example of the use of padding: Claim: If every language accepted by an 0(n) timebounded NTM can be accepted by a polyno
mial tlmebounded WM then P = NP. Proof: (Sketch). Similar to the padding lemma. Let L1 be accepted by an nk timebounded
NTM M1. We deﬁne a new language (padded version of L1), L2, that can be accepted by an 0(n)
timebounded NTM M2. By hypothesis, L2 can be accepted by an 0(n7") timebounded DTM M3.
Finally, we can construct from M3 a DTM M4 accepting the original language L1 in 0(n5) time
M Relationships Among Important Complexity Classes 1. DSPAC'E(10g n) g NSPAC'E(10g n) g P g NP g NSPACE = PSPAC'E(bySavitch). If C is a complexity class, deﬁne co—C 2 {EL 6 C}. 2. co—PS’PACE = PSPAC'E = NPS'PACE = co—NPSPAC'E
. co— = 4. Open problem: co—NP NP
Let Lsat = {F F is a Boolean formula, F is satisﬁable },
Lsat is in NP, but does not seem to be in co—NP. 5. For certain time classes and restricted models, nondeterminisrn makes a difference, e.g., let L = {x#yl:v,y binary and 90 7E y}.
L can be accepted by a onetape NTM (i.e., there is only one read—write tape which initially contains the input) in nlogn time. It can be shown that any nondeterministic one—tape TM accepting 3 requires n2 time. ©O.H.Ibarra 68 6. Let 8(a) 2 log n. Then co—NSPACE(S(n)) = NSPACE(S(n)). as we showed in the section
on Complementation. 7. For any space bound 3(a) (i.e. can be less than log 71). any 3(a) spacebounded DTM can
be made halting. Claim: Let M be an offline S(n)—space TM with a single R/W worktape. We can effectively construct an equivalent off—line S(n)—space TM N which always halts. Proof: Fix the input to the TM M. If k is a positive integer, an ID = (q.i.w.j) is a kspace ID if the length of the worktape content 111 is at most k. We assume that the starting ID is start = (go. 1. A. 1). and the unique halting accepting ID is accept = (f. 1. A. 1). Thus. we assume that the machine erases the the worktape and moves it’s input head and worktape head to their initial positions before halting in a unique accepting state. The kspace I D7s of M can be viewed as forming a ﬁnite directed graph with the I D’s as nodes. Were is an edge from a k—space I D1 to a kspace I D2 if I D1 can reach I D2 in one step.
We deﬁne a procedure called REACH(5tart. ID. REACH returns ‘yes’ if start can reach
ID gomg only thru kspace ID’s; else it returns ‘nor. In the following. we assume that ID is a kspace ID and either (a) it is halting (i.e.. it is in a halting state). or (b) its direct succesor uses (k: + 1) space.
Because M is deterministic. given ID. the subgraph which contains ID is a tree rooted at ID. Thus to determine whether start can reach ID. we need only determine whether start is a node of the tree.
We describe an offline TM N which implements the procedure REACH (5tart.I D.k). For convenience. we can assume that N has several R/W worktapes (which can later be merged into one tape). One worktape is used to represent M 7s worktape. Assume N has been conﬁgured to “represent” ID. (By this. we mean that the input and one worktape of N are conﬁgured to represent the ID. The other tapes are used as auxiliary scratch tapes.) To determine whether start can reach ID. N performs a depth—ﬁrst search of the tree. When N visits a node. it conﬁgures itself to represent the visited node. Forward edges (to successor nodes) are traversed easily by simulating one forward step of M. Backward edges (to the predecessors) are traversed by running M backwards one step. There may be several valid predecessors when running M backwards; however. there are only at most a ﬁxed number of possible predecessors (independent of the input). and N can select them one at a time in some predtermined order. A leaf is reached if the current node has no predecessor or the predecessors are all (k: + 1)space I D7s. When a leaf is reached. the search moves forward in the usual depthﬁrst search manner. The procedure above assumes advance knowledge of k: (i.e.. the space M uses). Since k; is not
known in advance. N can run procedure REACH(sta7"t. accept. k) for k; = 1. 2.   until it ﬁnds a
k (if it exists. i.e.. M accepts) such that REACH returns ‘yes7. However. if It does not exist. N
will run forever. Thus. if with space k. REACH returns ‘no’. N must ﬁrst determine whether M ©O.H.Ibarra 69 uses k + 1 space before running REACH on space k + 1; if M does not use k; + 1 space, N rejects the In ut.
Here is the complete algorithm, where MORETHAN is a procedure that determines if M uses k + 1 Space: procedure COMPLETE(sta7't, accept) egln
m
do forever
egln
if REACH(start,accept, k) then [ return (‘yes’); halt ]
8%
if not MORETHAN(/<:)  return (‘no’); halt 
k: <— k —i
eTld
eTld The procedure MORETHAN will systematically look at all kspace I D7s (in some ordering)
that can lead, in one step, to an ID that uses (k + 1) space, and checks if one of these is reachable M procedure MORETHAN(k)
eg1n
ID <— ﬁrst kspace ID in the ordering do until all k—space I D7s have been examined eg1n
CongureW
if ID can lead in one step to some ID’ using (/6 + 1) space
and REACH(3tart,ID,/~€) then return (‘yes7)
ID <— next k—space ID in the ordering
end return l‘no’)
end Corollary 10 DSPACE(S(n)) is closed under union, intersection, and complementation. Claim: Let M be an n—state 2DFA. We can effectively construct a halting 0(n2)—state 2DFA N egulvalent to M.
Proof: Assume w.l.o,g. that M accepts in a unique state with its input head at the left end of the tape. N performs a single depthﬁrst (reverse) search to determine whether the initial conﬁguration ©O.H.Ibarra 70 can reach the accepting conﬁguration. Since in the “backward” treversal, a node can have 0(n) predecessors, N in its computation must keep track of pairs of conﬁgurations. Thus N needs 0(n2)
states
Claim: Let M be a khead 2DFA (i.e., a 2DA with k: independent read heads). We can effectively
construct a halting k—head ZDFA N equivalent to M. Claim: (k + 2)—head 2DFA’s are strictly more powerful than khead ZFDFA’s. Proof: We show that there is a language accepted by a (k:+2)head 2DFA that cannot be accepted
by a k—head 2DFA. Fixed an E encoding of khead 2DFA’s, as follows. In a rule: states, symbols
scanned by the k; heads, directions of moves of the k; heads are represented in unary (‘1’) with the
items separated by ‘0’. = 150E(R1)...0E(Rm), where s is the number of states of M, and is the encoding of rule 2'. Let L = {$ :1: = for some khead 2DFA, M rejects :1: and
halts We can construct a (k + 2)head 2DFA N accepting L as follows: Given input $, N ﬁrst
checks that lb 2 E(M) for some k—head ZDFA. Then N simulates the computation of M by using
16 heads to simulate the k; heads of M one head to keep track of the current state of M, and one head to extract (from :13) the appropriate rule to use for the next move of M. (Note that the head for the state is used in conjunction with head for the rule extraction to determine the appropriate rule to use. 1 To see that L cannot be accepted by a k—head ZDFA M , assume that such an M exists. We
may assume by the previous claim that M always halts. Now let :1: = E(M Consider running M on 3:. If M accepts $ then by deﬁnition of L $ is not in L. If M rejects and halts on 3:, then $ is not in L. (These are the only cases. The case M goes into an inﬁnite loop on 1E cannot happen.) Hence M accepts an if and only if 1E is not in L. This is a contradiction.
Claim: multihead 2DFA’s are equivalent to logn space—bounded DTM’s.
Proof: Clearly any khead 2DFA can be simulated by a logn space—bounded DTM. The DTM simply uses one worktape for each read head to record its position. For the converse, suppose that M is a logn space—bounded DTM With only one worktape. First convert the DTM to an equivalent DTM M’ whose worktape alphabet is binary (not including the blank symbol). This new machine will have space bound k; log2 n for some k. We may assume without loss of generality that the non— blank content of the worktape starts and ends with a ‘1’ (why?) The binary string starting with the leftmost ‘1’ and ending at the bit position of the R/ W head represents an integer 7". Similarly, the reverse of the binary string starting at the bit position directly to the right of the R/ W head up to the rightmost ‘1’ represents another integer 5. Clearly, 7' and s have values no more than 71’“. We can construct a multihead 2DFA M” to simulate M’. The values of 7“ and s can be represented by k heads (for each) on an input of length 77.. Updating the values (i.e., simulating a move of M’) can be accomplished using some auxiliary heads. Hence, a multihead 2DFA M” can be constructed
trLsimulatrLML ©O.H.Ibarra 71 The proof above applies to the nondeterrninistic varieties as well, i.e., the class of languages ac
cepted by rnultihead ZNFA’s = NSPACE(log Since it is not known wheter NSPAC'E(log n) = DSPACE(log n)7 it is not known whether rnul
tihead 2NFA7s are equivalent to rnultihead 2DFA’s. Tie situation is different for multihead pushdown automata. We state the following7 omitting the proof: o rnultihead 2NPDA’s are equivalent to rnultihead 2DPDA7s. 0 Every language accepted by a k—head ZDPDA (and, hence also, by a khead 2NPDA) for some k can be accepted by a polynomial tune—bounded DTM, and conversely. 0 Every language accepted by a lhead ZDPDA can be accepted in linear time by a random
accesimachine ...
View
Full Document
 Winter '08
 Ibarra,O
 Space, Computational complexity theory, TM, Ntm

Click to edit the document details