chapter5

chapter5 - Chapter 5 Space and Time Complexity Were are...

Info iconThis preview shows pages 1–23. Sign up to view the full content.

View Full Document Right Arrow Icon
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 2
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 4
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 6
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 8
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 10
Background image of page 11

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 12
Background image of page 13

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 14
Background image of page 15

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 16
Background image of page 17

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 18
Background image of page 19

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 20
Background image of page 21

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 22
Background image of page 23
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Chapter 5 Space and Time Complexity Were are many models of Turing Machines which are all equivalent in their ability to recognize languages; however, their space and time requirements vary significantly. In order to discuss the space or time complexity of TMs, we need to define standard models. 5.1 Space Complexity R/W Tapes Figure 5.1: Space Complexity Model Figure 5.1 shows the standard TM model used for space complexity. The input tape is read-only. Were are a finite number of read-write work tapes. If the input head is 1—way, then the machine is called online and if the input head is 2-way, then the machine is called offline. Definition 18 An S(n) space-bounded TM uses no more than S(n) cells on each work tape for any input of length n. A TM is weakly S space-bounded if it uses no more than 8' cells on each work tape for any input of length n that is accepted by the TM. 473 ©O.H.Ibarra 50 For space—bounded computation, a k-worktape S space—bounded TM M can be converted to an equivalent l-worktape S space-bounded TM M ’ (Ml uses 2k; tracks to simulate the tapes of M, two tracks for each tape of M. See similar result for time—bounded computation below for details.) Thus for space-bounded computation, we need not specify the number of worktapes. Space Compression: Let M be an S(n) space—bounded TM and k; a positive integer. We can construct an S /l~€ space—bounded TM M’ equivalent to M. (It is understood that S /l~€ denotes the ceiling of S/ M ' uses a larger worktape alphabet - it “packs” k; symbols of M into one symbo . Thus for space-bounded computation we need not specify constants of proportionality the base when taking logarithms, etc. For space bounds that are greater than linear, i.e., S 2 nVn, the offline and online versions are equivalent. This is true because if we are using at least it space, then the online TM can first copy the input to one of its work tapes before doing anything else and then read the input from that tape. Definition 19 A function S(n) is fully space constructible (“nice” if there exists an S(n) space—bounded TM that uses exactly S(n) cells for every input of length n. Most common functions such as log n, n2, 2” are fully space constructible. An example of a function which is not fully space constructible is log log n. Definition 20 A function S(n) is space constructible if there exists an S(n) space-bounded TM such that for every ii, there is at least one input a: of length n that requires the machine to use exactly Si ni cells. If S is fully constructible then a weakly S space-bounded TM is also an S space—bounded TM Claim: Let L be the language consisting of all strings of the form x#x, where at is a binary string. Suppose M is an on—line S space-bounded TM accepting L. Then S is at least linear for infinitely many n’s. Proof: (Cut-and—paste argument): Let M be an S space-bounded on—line TM accepting L. Consider the computation of M on strings of the form $79,931, where r = n. The partial ID (PID) of M when it reaches # is given by (q,j, w), where q is the state, in is the contents of the worktape, and j is the position of the R/W head within 111. Let s and t be the number of states and worktape alphabet size of M (including the blank symbol). The number of possible PID7s when M reaches # is at most sS(n)tSl”) g 05W) for some 0. If S grows slower than linear for all but a finite number of n’s, then 050” < 2" for all but a finite number of n’s. Then for some no, 050” < 2” for all n 2 n0. Since there are 2” strings of the form x#x, where = n, there are distinct strings ac and m’ of length n such that on input x#x, the PID of M when it reaches # is equal to the PID when it reaches # on input $’#$’. ©O.H.Ibarra 51 Since both ac#ac and x’#x’ are accepted by M , it follows that M will also accept ac#ac’. This is a contradiction, since x#x’ is not in L. Similar yZ Claim: Any 3 space-bounded on—line TM accepting L' = Ozlzlt > 0 requires logn space for infinitely many n7s. It is easy to see that L and L’ in the above claims can be accepted by on—line TM’s in linear space and logn space7 respectively. We bound logn is necessary and sufficient for accepting a nonregular language. Sufficiency follows from the fact that L’ above can be accepted by an on—line log n space—bounded TM. The necessity is given by the following theorem. Theorem 6 Let L be a nonregular language accepted by an on-ltne 8(a) space-bounded TM. Then S'(n) is at least logn for infinitely many n’s. Proof: Assume that no symbol is overwritten by the blank symbol7 A (otherwise7 use a special symbol X and treat it like /\), and there is only one work tape (otherwise, merge all worktapes into one worktape ). o The partial ID of a TM gives the current status of the computation. It consists of the current state of the finite control7 the contents of the work tape and the position of the worktape head. Let k; be a positive integer. Then the number of different Ple of the TM that uses less than k; cells of the work tape is given by H mm]; 2 satZ < skt E1 0 Let a be the input alphabet size. 0 Let p be the smallest integer such that 19’“ 2 asktk. o For any given positive integer 71, let mm 2 $1M - - - m be the shortest string such that pro— cessing of wn by M causes it to use at least logpn work tape cells. Let k; = logp n. Such a string must exist. (Otherwise : no such that all strings of any length use less than logp n0 cells which is constant; this is impossible because the language is nonregular). Claim: No PID is repeated when the TM works on this string wn. If some PID is repeated then wn cannot be the shortest such string which is a contradiction. Therefore we see that 7“ S asktk 3 pk = plogP" Hence , 7" < n. Therefore for every n, there exists a string of length n, i.e., 901262 - nmryy-ny (padded so that the length is n)7 that uses exactly logp n cells. ©O.H.Ibarra 52 Thus for online computation, log n space is necessary and sufficient to accept a nonregular set. (As noted earlier, sufficiency is shown by the language L = 2 1} - using a binary counter to count the number of OS or 1s). Claim: Let L be the language consisting of strings of the form :13#33, z binary. Any off—line S space-bounded TM accepting L must use logn space for infinitely many n7s. The proof of this claim uses the Kolmogorov complexity argument. Let E be a fixed alphabet. For every string r 6 2+, let M be a TM with an initially blank two—way R/ W worktape and an initially blank one—way write-only output tape7 which when started in its initial state writes (generates) an on the output tape and halts. (initially blank) + 2—way RTW tape _> _> _> ? q0 halt fir —l (initially blank) X l—way write-only tape Let M denote the size of M , i.e., length of the binary encoding of the description (program) m Elxamp es: 1. a: = 1011 We can construct a TM M whose size is greater than or equal to n = :1: . (M writes the bits of as one at at time, using approx n states.) 2. 101010 10 (repeated n times) We can construct a TM M whose size is polylog n (2 polynomial in log (M writes n in binary (as a counter) on the worktape and then generates 10 n times using the counter.) 3. Any unary string of length n can be generated by a TM of size polylog n. Definition 21 For every string :10, let : Kolmogorov complexity ofzv = [M generates I.e., is the length of the encoding of the smallest TM that prints m and halts. (Note: we don’t care about the space and time complerity of Lemma 1 Let E = {0,1}. For every n, there emits a binary string as” such that = n and KWMZn. Proof: For every n > 1, there are 2” distinct strings of length n. Suppose that each of these strings can be generated by a smallest TM whose binary encoding length is less than n. Then there ©O.H.Ibarra 53 are less than 2” such machines. Hence at least 2 strings of lenth n can be generated by the same machine. This is 1mposs1ble. It follows from the lemma that for every n7 there is at least one string an of length n such that any TM generating a has binary encoding 2 n7 i.e.7 2 n. Definition 22 Any binary string r of length n such that 2 n is called random the sense that the pattern in a: is “not regular” so as to be generable by a TM with size < Claim: Let L = {:13#:13 r E {07 1}+}. Then L can be accepted by an off-line TM in space log n. Proof: The TM compares corresponding bit positions using a binary counter to record the posi- tions. Claim: Let L = {$#$ :1: E {0, 1}+}. Any off-line TM accepting L uses log n space for infinitely many strings of length n. Proof: (Kolmogorov complexity argument): Let M be an S space—bounded off—line TM accepting L. We show that for every n7 there exists an input of length n that requires M to use at least logn space on this input. Let n be a positive integer. Pick a random string :16” of length n. Hence 2 n. Consider the computation of M on input xn#$n: xn # xn $ X1 We assume w.l.o.g. that M accepts with the input head on the left delimiter. Recall that a PID is a triple of the form (q, z,j)7 where q is the state, 2 is the contents of the worktape, and j is the position of the R/W head within 2. In the figure7 o Xi = PID the i—th time M reaches # from the left. o Yi = PID the i—th time M reaches # from the right. ©O.H.Ibarra 54 Call (X1,Y1, ...,Xk, Yk) the crossing sequence (on 7%). Observations: 1. A PID requires dS(2n + 1) space to store for some constant d. 2. The number of PID’s on inputs of length 2n + 1 is 65(2”+1) for some constant 6. Given the rules (program) of M and the crossing sequence, i.e, given the information (M , X 1, Y1, ..., X k, Yk), we can construct a TM N to generate an” as follows: N uses its worktape as a 2—track tape: o N first writes on the first track the information (M, X1,Y1, ...,Xk, Yk) and then writes 71 in binary (thus using logn cells) to the right. o N then systematically generates on the second track, strings of the form $#$ with n = z : E worktape l (M, X1,Y1, Xk,Yk) x#x For each string $#$ on its second track, N uses (M, X1, Y1, ..., XK, Yk) to check ifz = 3:71. To check if z = z”, N simuates M ’s computation on $#$ and see if the computation is consistent with the crossing sequence (X1, Y1, ..., Xk, Yk). If the computation is consistent, then N knows that at = :16”, and outputs 30(2 an”) on its output tape. If the computation is not consistent, M tries another :16. The size of the binary encoding of N depends only on the information (M, X1,Y1, ...,Xk, Yk) and the rules to write n in binary: 0 size of M is a constant. 0 size of each X, or Yi is dS(2n + 1) for some constant d. . k is at Jmcst 3S(2n+1) fa]: Emma CCmstamt 3 0 writing n in binary requires f logn rules for some contant f. Hence the size ofM is O(dS(2n+ 1) (65(2n+1))+f log n) g 95(2”+1) for some constant 9, assuming Sim is at least log log n. Since an” is random, 95(27H1) 2 n. It follows that S(2n + 1) Z logg n. It follows that there is a constant it such that for every n, there is an input of length n that requires M to use at least h log 71 space. ©O.H.Ibarra 55 Theorem 7 Let L be a nonregulai" language accepted by an ofl-line S(n) space-bounded TM. Then S(n) is at least log logn for infinitely many n’s. Proof: (Omitted for now.) Claim: The log log n bound is sufficient to accept a nonregular language, e.g., L = {x1#...#$klh > 0, ac,- is binary representation of i with LSB l}. CONVENTION: Unless otherwise noted, TM will mean Deterministic Turing Machine. Also, from now on unless otherwise noted, we will deal only with ofi-Iine TM’S. Definition 23 DSPACE(S(n)) = {L L can be accepted by a TM within S(n) space Definition 24 PSPACE(S(n)) = {L L can be accepted by a TM within S'(n) space and S'(n) is a polynomial space PSPAOE = U DSPAOEW‘) {71 If M is an S(n) space-bounded TM, with S(n) 2 log n, then there exists another TM, M’ which is equivalent to M, is also S space bounded and always halts. For example, M’ could just keep a counter of size S for the execution and check for overflow. For an 8' space bounded TM, the maximum number of distinct IDs is limited (s(n + 2)tS(")S(n) < cSW for some constant c); therefore one can incorporate a counter that shuts off the computation if the TM runs more than this bound. Note that this is non trivial because it is hard to compute 3 One can get around this by working 8' out as the simulation proceeds - begin with an estimate of 1 cell, then if the worktape uses more cells than the estimate we increase the estimate for the counter and restart the computation (the counter counts in base 0, the constant mentioned above). 5.2 Time Complexity Definition 25 A TM is T(n) time-bounded for all inputs of length n, it runs no more than Tl ni steps. The time complexity must be at least linear since the TM must read all the input symbols for a nontrivial computation. Definition 26 DTIME(T(n)) = {LlL can be accepted by a TM within T(n) steps Definition 27 PTIME = {L L can be accepted by a TM within polynomial time }, i.e., PTIME = U DTIME(ni) {71 ©O.H.Ibarra 56 Theorem 8 PTIME g PSPAC'E Proof: This obvious since a TM cannot use more cells than the number of steps that it takes. The guestion of whether the containment is proper is open. Theorem 9 IfL is accepted by a k—tape T(n) time—bounded TM, then L can be accepted by a I—tape 0(T2l”)) time-bounded TM. Proof: iIdeal: M’ simulates M by creating 2k: tracks on its tape, two tracks for each tape of M. One track contains the contents of the tape being simulated while the second track contains a marker indicating the position of the read/write head on that tape. To simulate a move of M, M’ first makes a sweep between the leftmost marker and rightmost marker to “pick up” (read) the symbols under the k: heads of M. Then M’ applies the move rule by rewriting the symbols under the heads on the k: tracks and moving the markers appropriately. Since the leftmost marker and the rightmost marker can be T(n) cells apart, it would take about 2T(n) steps to simulate one move of M. Thus, M’ is 0(T2(")) time-bounded. Claim: The quadratic simulation time in the theorem above is best possible. Proof: Let L be the language consisting of all strings of the form x# X, where X is a binary string. Clearly, L can be accepted by a 2—tape TM in linear time (i.e., O(n) time). We show below that any single—tape TM accepting L requires quadratic time for infinitely many 11 s. Figure 5.2: Computation of M Let M be a single-tape TM (i.e., there is only one read / write tape which initially contains the input. Consider the computation of M on input a1 - - - an. We may assume that M has no stationary moves, i.e., if 6(q, a) = (p, b, d), then d = +1 or d = —1. Also, assume that the “change” in state from q to p occurs while M is moving its head to +d (i.e. to the left or right of the current position). The ©O.H.Ibarra 57 computation of M has the form shown in Figure 5.2. Without loss of generality, we assume that M accepts only at the right end of the cells initially occupied by the input. Between any 2 tape cells, the sequence of states the boundary is crossed is called the crossing sequence (CS) at that boundary. The following lemmas are easily verified. Lemma 2 If a1---an is accepted by M and the CS between a, and ai+1 is the same as the CS between aj and aj+1, where i < j, then a1---aiaj+1---an is also accepted by M. Lemma 3 If a1 - - - an and b1 - - - bm are accepted by M and the (33 between ai and aifil is the same as the CS between bj and bj+1, then a1---a,~bj+1---bm and b1---bja,~+1---an are also accepted by M Lemma 4 The sum of the lengths of the nonempty 08’s is at most T(n), the time complexity of M. (Note that if a boundary is never crossed, the the CS at that boundary is empty.) Theorem 10 Let L = w is a binary string IfM is a T(n) time—bounded single—tape TM accepting L, then T(n) must grow at least as fast as n2 for infinitely many n’s. Proof: Suppose there are two strings x1$2#$1x2 and $3x4#x3$4 that are accepted by M with $1 75 :133 but length(:1:1) = length($3) = i for some i. Then by Lemma 3, the CS between :131 and 3:2 is different from the CSLetween :133 and $4. . Over all strings of length n in L (n odd), let be the average length of the CS between the ith and i + lst symbols. . The number of strings of length n in L whose CS between the ith and i + lst symbols is < 4. 17.71 17.7171 -- 1 .—_ 2p(’t)182§'2 2 —2 2 The number of distinct CS’S of length 3 2p(i) is < s2p(l)+1, where s is the number of states m 17.71 2T—1 $2P(i)+1 ' Thus, the number of strings with same CS between the ith and i + 1st symbols is > 5. The number of strings a1---aiai+1 - - - a(n_1)/2 - -- with distinct strings in positions i+ 1, i+2, 6. gn— 1gp is 2’34”. 1 If is > 2nT—l’i, then there will be two strings with the same CS between the ith and i + lst symbols, have the same strings in positions i + l, i + 2, (n — l)/2, but distinct 21174—1 $2P(i)+1 n71 strings in positions 1, 2, i. This is impossible. Hence, 3 277i. ©O.H.Ibarra 58 Therefore, (2p(i) +1) 2 — 1)/log2 s, and > — 1)/210g2 —%. Summing from i = 1 to i = (n — 1) / 2 gives a lower bound on the average sum of the CS7s over all positions. Clearly, this sum grows at least as fast as n2 (for odd Hence by the Lemma 47 T(n) must grow at least as fast as n2 for infinitely many n’s. Theorem 11 Let L be accepted by a t-tape TM with (t Z 2) in T(n) time. Let k be a positive integer. Then L can be accepted by a t—tape TM in n +T(n)/l<: time. Proof: As in the case of Space Compression one can pack the input symbols into fewer symbols using a larger alphabet and thereby reduce the time complex1ty. Note however that we need to have at least time n to read the complete input in order to encode it into the new alphabet. Corollary 5 UL is accepted by a t—tape TM in T(n) time, and T(n) grows faster than linear, then for any positive integer k, L can also be accepted by a t-tape TM in time. Corollary 6 Let L be accepted by a t—tape TM with (t Z 2) in O(n) time (i.e. in linear time). Then for any 6 > 07 we can construct an equivalent t—tape TM that runs in (1 + 6)n time. For single-tape TM’s (i.e.7 TM7s with only one read/write tape which initially contains the inputi, we can prove: Theorem 12 Let L be a nonregular language accepted by a single-tape T(n) time-bounded TM. Then T(n) must grow at least as fast as nlogn for infinitely many n’s. Proof: iOmittedi Claim: The n log n bound above can be achieved7 eg.7 L = {02? i > 0} is a nonregular language that can be accepted by an nlogn time—bounded single-tape TM Theorem 13 Let L be a language accepted by a T(n) time—bounded multitape TM. Then L can be accepted by a Q—tape T(n) log T(n) time—bounded TM. Proof: (Omitted! 5.3 Non-Determinism A nondeterministic TM (NTM) is one which may have more than one choice of next move. The classes of languages NSPAC'E7 NPS'PACE7 NTIME7 NPTIME are the nondeterministic coun— terparts of the languages DSPACE, PS'PACE7 DTIME7 PTIME. An S space bounded NTM can be simulated by a DTM. The execution of an NTM can be viewed as a tree of possible paths — if any of these leads to acceptance, then the input is accepted, ©O.H.Ibarra 59 otherwise it is rejected. Simulating the NTM cannot be done using a depth—first search of this tree because some of the paths may never terminate. (However, we can use breadth-first search.) But as seen before, we have an upperbound on the number of distinct IDS for a space bounded machine. We can therefore limit the depth to which we search. In order to do this, we need a counter that Sm) can count up to 0 since there are at most 050“ possible IDS. Searching the tree would require exponential space. In 1969, Savitch showed the following result: Theorem 14 [Savitch’s Theorem] For S(n) 2 log n, NSPACE(S(n)) g DSPACE(32(n)) Proof: Assume that $01) is fully space constructible. The TM is initially in ID1 = (q0,1,)\,1) which represents the state, the input head position, work tape contents and the work tape head position, respectively (we assume that there is only one work tape). For an accepted string, there ex1sts an accepting path, ID1I—ID2|----|-IDf=(f,1,)\,1) : <cSn The symbol ID,~ - [DZ-+1 means ID,- can reach IDi+11n one step. Note that we assume that the TM accepts with both heads at the beginning of the tapes and that the work tape is blank - this is easy to achieve by adding additional states to move the heads back to the beginning after erasing the work tape. We also assume that there is exactly one accepting state, f. The length of this accepting path can also be assumed to be exactly CS by repeating the accepting ID. I D1 can reach I Df in 059“ steps if and only if there exists I D3 that lies halfway and reachable from I D1 in half—the steps and I D f is reachable from I D5 in halIThe steps, i.e. ID1 ism) IDf <=> :IDS s.t. ID1 -’[Cs(n>J IDS -’|fcs(n)] IDf Thus, we can check whether a given ‘ 2 ID is reachable from another using thezfollowing recursive procedure: procedure TEST(ID1, IDg, // check if ID1 |-* IDg in at most 2' steps M m if (ID1 = ID2) or (ID1 l- IDg) then return l‘true’l 8% return i ‘false’ 5 else begin for each ID3 using at most S(n) space if (TEST(ID1, 103, L j) and (TEST(ID3, IDQ, |§|) then i 2 ©O.H.Ibarra 60 returnt ‘true7l end ret urni ‘false’ 6 end Thus we have a recursive procedure which can simulate the nondeterministic Turing machine. The length of each ID is given by ID = (q,t,w,j) =c+logn+S(n)+logS(n)=O(S(n)) Thus each ID requires O(S(n)) space S(N) 2 logn - because if S(n) < logn then the 71 field of the ID cannot be stored in 0(8' space). This recursive procedure can be “unrolled” and implemented on a dterministic TM. The amount of space needed in the implementation is: The size of the stack frame 2 O(S(n)) The number of frames 2 log(cSl”)) = O(S(n)) Therefore, the space requirement is 0(S Corollary 7 PSPACE E NPSPAC'E Proof: It is clear to see that PSPAC'E g NPSPAC'E. To Show that we use Savitcfi’s Theorem. Since all polynomial functions are at least as large as log n, and the square of a polynomial function is also a polynomial function, by Savitch’s theorem, any language that is accepted by NTM in polynomial (say space will also be accepted by some DTM in polynomial space (p2 In Savitch’s theorem, 3 need not be fully space constructible. One need only do the simu- lation for S = 1, 2, - - -. The proof of Savitch’s theorem actually proves the following: Theorem 15 Let L be accepted by an NTM in S(n)—5pace and T(n) time (simultaneously). Then L can be accepted by a DTM in space S'(n) log Open Problem: Is Savitch’s result optimal? I.e., is NSPACE(S(n)) g DSPACE(S’(n)) for some S’(n) less than S201)? In particular, is NSPACE(S(n)) = DSPACE(S(n))? Actually, a T(n)-time bounded NTM can be simulated by a DTM in T(n)—space. Claim: Let L be accepted by an NTM in T(n)—time. If T(n) is fully space constructible, then L can be accepted by a DTM in T(n)—space. Proof: The T(n)-space DTM performs a depth-first search of the computation of the NTM. ©O.H.Ibarra 61 5.4 Complementation Suppose S 2 log n. Since an S space—bounded DTM can be made halting, the complement of a language accepted by such a machine can also be accepted by an S space-bounded DTM. (We simply call the “accept” states “reject” states7 and vice—versa.) What about the languages accepted by nondeterministic machines? Suppose we have an S space-bounded NTM and assume it always halts on any input (otherwise, it can be made halting). Can we simply call the “accept” states ‘reject’ states and vice-versa and obtain a machine that will accept the complement language? The answer is no. Why? Clearly7 by Savitch’s theorem7 there is an S (n)2 space-bounded DTM (and hence an NTM) that can accept the complement. An interesting question is whether S space is sufficient for an NTM to accept the complement language. The answer is a surprising yes. WW We assume that S 2 logn and that it is fully space constructible. In the following7 M is an S space—bounded NTM with a single read-write work tape. START represents the initial ID. For a given input at of length n, we assume a lexicographical ordering of all S space—bounded ID s. First and Last represent the first and last ID’s in the ordering. Theorem 16 Let M be an S(n) space-bounded NTM. We can construct an S(n) space-bounded NTM M’ such that L{M’) : complement of Proof: We first prove the theorem under the assumption that when M’ is given an input r of length n, it is also given on another tape, a number N (in binary) = the exact number of lD’s reachable from START on input :16. Note that N is at most 05W) for some constant c and therefore takes only O(S(n)) space to store. We NTM M is an implementation of the foflowing nondeterministic algorithm: CW TARGET <— First 1 Goto 2 or 3 2 if TARGET 7E Last then [TARGET <— TARGET+1; goto 1] if TARGET 2 Last then |output(‘reject’); halt| 3 Guess a path from START to TARGET (Le.7 guess that TARGET is reachable) if successful then case: TAWET is an accepting ID case: TARGET is a rejecting ID COUNTER <— COUNTER + 1 casezCOUw if TARGET 75 Last then |TARGET <— TARGET+1; goto 1| ©O.H.Ibarra 62 else [output(‘reject7); halt] case: UNTER = N Note that in the algorithm above7 guessing a path can lead to success, lead to an infinite loop7 or the procedure can get stuck. We are only interested in the first case. Clearly, M7 accepts 1E if and only if it has found exactly N reachable ID7s, none of which is accepting. Moreover7 M’ is S space-bounded. To complete the proof7 we need to show how M’ can compute N. We do th1s 1n the next lemma. Lemma 5 Given 1E and START, M’ can compute N : the exact number of [D’s reachable from START on input a“. Proof: Let Nd be the total number of ID7s reachable from START in at most d steps. We show, by induction, how M7 can compute Nd,,1 from Nd. Note that the bases N0 and N1 are obvious. The nondeterministic algorithm below computes Nd+1 given Nd. HQUNTEB2 t1! T ET2 Fir 0 UNTERl <— 0 TARGETl <— First 1 Goto 2 or 3 2 if TARGETl # Last then [TARGETl <— TARGETl—H; goto 1] if TARGETl 2 Last then |output(‘reject’); haltl 3 Guess a path from START to TARGETl reachable in at most d steps if successful then COUNTERl <— COUNTERl—l—l case: TARGET2 is reachable from TARGETl in 0 or 1 step UNTER2 <— UNTER2—1 if TARGET2 # Last then [TARGET2 <— TARGET2+1; goto 0] if TARGET2 2 Last then goto 4 case: TARGET2 is not reachable from TARGETl in 0 or 1 step if COUNTERl < Nd and TARGETl 75 Last then [TARGETl <— TARGET1+1; goto 1] if COUNTERl < Nd and TARGETl 2 LAST then norm [TARGET2 <— TARGET2+1; goto 0] ©O.H.Ibarra 63 4 ( continue I In the above algorithm, the value of N614 is computed in COUNTER2. COUNTER2 is incre- mented by one every time a TARGET2 is found that is reachable from a TmETl in 0 or 1 step provided TARGETl is reachable from START in at most d steps. Clearly, when the algorithm exits to statement labeled 4, cm &m The algorithm is iterated until Nd,,1 = Nd for some d. When this happens, N = Nd. Note that the requirement that S 2 logn is not necessary. Why? We give an application to graphs. For a connected directed graph G with vertices 1, ..., n, let E (G ) be a string which represents G, i.e., E (G) is a list of edges (in binary) separated by delimiters. Note that for a dense graph, the length of E(G) is 0072). Let L = { E(G) l G is a connected directed graph with n vertices, and there is a path from vertex 1 to Clearly, L can be accepted by a logn space-bounded NTM. From the theorem avove, L’s complement can also accepted by a logn space-bounded NTM. Note that both languages are in WIME. It is not known if NTIM is closed under complementation. In fact, it is not known if the complements of the languages in NP are in P. Clearly, the complement of every language in NP is in DTIME(2p(n)) for some polynomial p(n). 5.5 Space-Time Tradeoffs Claim If L e NSPACE(S(n)), then L e DTIME(dS(”))). Proof Let M be an $01) space—bounded (one-tape off-line) NTM. An ID of M is given by a 4- tuple, (q,i,w,j), where q is the state, i is the input head position, 111 is the worktape content, and j is the position of the worktape head. The number of distinct ID’s M can enter is at most s(n+2)S(n)tS(”) 3 0501) time for some constant c. (The +2 takes into account the input delimiters.) We construct a 3—tape DTM, M’ which when given input 1E on Tape 1, constructs the list of all reachable IDS from the initial ID using Tapes 2 and 3. M uses the following dynamic programming algorithm: [DO 2 q0,1,)\,1 Tape 2 <— Q Tape <— {IDO} while Tape 2 7E Tape 3 d0 egln Tape 2 <— Tape 3 Tape 3 <— { All IDs in Tape 2 } U {IDE ID in Tape 2, ID |- ID } ©O.H.Ibarra 64 m if Tape 2 contains an accepting ID then M l accepts else M’ rejects The while loop is executed at most cSW times. Constructing the list in Tape 3 takes at most O(cS(")) time. Thus, M’ has time complexity O(cS(").cSl”)) 3 ask”) for some constant d. Corollary 8 NSPACE(log n) g PTIME Proof dlog" S 71’“ for some k. We state the following without proof: Claim: Let L be accepted by a T(n) time—bounded DTM. Then L can be accepted by a log T(n) space oun e T . It is not known if the above result is optimal. Corollary 9 Let L be accepted by a linear time NTM. Then L can be accepted by a linear space DIM For single-tape TM’s (i.e., TM7s with only one read/write tape which initially contains the input), there is a stronger result. Again, we state it without proof: Claim: Let L be accepted by a T(n) time-bounded DTM, where T(n) 2 n2. Then L can be accepted by a T(n)1 2 space-bounded DTM. 5 .6 Hierarchy Theorems Binary encoding of off-line one-worktape DTMs: Choose a binary encoding, E(M), ofeach TM M, M =< Q, E, T‘, 6, q1, qs >, where Q = {q1, - - - ,qs}, E = {0,1}, T‘ = {A1, - - - ,At}. Encode each state by a binary string of length |log2 5| and each worktape symbol by a binary string of length [logg t]. The input symbols, 0,1 and input delimiters are encoded as 0,1,00,01. The moves +1 and —l are encoded as 0 and 1 respectively. A rule R of the form 6(q,a,A) = (p, d1,B,d2) is encoded as E(R) = E(q)E(a)E(A)E(p)E(d1)E(B)E(d2). If M has rules R1,R2,---,Rk, M is encoded as, = 1101501t0E(R1)E(R2)~~~E(Rk). Note that the sizes of the state set (3) and tape alphabet (t) are given as part of the encoding for easy deciphering of the rules. For reasons to become apparent later, we allow the encoding to have the ‘padding’ prefix liO for any positive integer i. Thus, a TM has an infinite number of encodings. Note that if M on an input of length it uses S(n) space, then this space in binary is |log2 t|S(n). ©O.H.Ibarra 65 Theorem 17 (Space Hierarchy): Let 8'1 and 82(71) be space bounds with 81(n), 82(71) 2 logn C and 52(n) fully space constructible. Iflimnnoo = 0, then DSPACE(Sl(n)) 75 DSPACE(52(H)). Proof: We need to show that DSPACE(8'1 g DSPACE(82(H)) and SPACE(Sg(n)) — DSPACE(81 75 gb. The first is obvious. Define the language: L = E {0, 1}",90 = for some off—line one—worktape DTM M with t worktape symbols, n = :1: , and M on input $, halts and rejects :1: within 32(n)/|log2 t| space First we construct an 32(77.) space—bounded DTM H to accept L as follows: Given input $ of length n, H does the following: 1. H marks exactly 5501) cells on its worktapes. (H can do this because 52(77.) is fully space constructible.) It also marks 82(71) / [logg t] cells of this space. From now on , H is restricted to use no more than 82(71) space. This guarantees that H is 82(71) space- bounded. 2. H Checks that :1: = E(M) for some off-line one-worktape TM M. If not, reject and halt. If okay, proceed to next step. 3. H uses 90 (which is the binary encoding of M) to simulate the computation of M on :10: a If M halts and accepts within Sg(n)/[log2 if] space, H halts and rejects. b (c (d If M goes into an infinite loop within 82(n)/[log2 t] space, H halts and rejects. (Note ( ( If M halts and rejects within 82(n)/ [logZ t] space, H halts and accepts. If M attempts to use more than 52(n)/|log2 t| space, H halts and rejects. ) ) ) ) that H can detect this by incorporating a counter which uses 3 52(77.) space.) We now show that L cannot be accepted by any 8'1 space-bounded DTM M. Suppose such an M exists. Let :1: be an encoding of M. We may assume that M halts on all inputs (otherwise, we can incorporate a counter to M Let M have tape alphabet size t. Since M has an infinite number of encodings, we choose an encoding lb of M such that = 7?. satisfies: 32 n 2 lOg2t 51 n This is possible since limnnoO gi—Ennfi = 0. Now consider the behavior of H and M on input :10: o If M halts and accepts 1E within 81 cells (which in binary is [log 75] 81 H halts and rejects. Note that H has enough space to do the simulation since 8501) 2 |log2 t|Sl(n). o If M halts and rejects, H halts and accepts. Note that cases c) and (1) above do not occur. Thus M accepts 1E <=> H rejects :10. This is a 1.. ©O.H.Ibarra 66 The proof of the space hierarchy theorem does not work for nondeterministic machines (why?). C We can prove weaker results using Savitch’s theorem. For example, NSPACE(n) Q DSPACE(n2) 75 C DSPACE(n2 log n) g NSPACE(n2 log n). Thus, NSPACE(n) 7A NSPACEm2 log n). To prove stronger results, we need the following translation (or padding) lemma. Lemma 6 (Translation). Let 51(77.) 2 logn,Sg(n) 2 logn and f(n) 2 n. Assume 51(77.) and Sg(f(n)) are fully space constructible. Then NSPACE(S1(n)) Q NSPACE(Sg(n)) => NSPACE(Sl(f(n)) C NSPAOEgs2 (flog). Proof: Let L1 be in NSPACE(Sl(f(n))). Then L1 is accepted by an 81(f(n)) space bounded NTM, M1. Let # be a new symbol. Define the language L2 = {$#i|M1 accepts $ within space Sl(|:1:| + We can construct an NTM M2 which accepts L2 as follows: On input 3:769, M2 marks off 8'1( :1: +t') cells on its worktapes (it can do this since 81 is fully space constructible). Then M2 simulates M1 on an (ignoring the suffix #6). M2 accepts if and only of M1 accepts. Clearly, M2 is 81(n) space- bounded. (Note that the length of the input to M2, which is used to measure its space bound, includes #i.) Note that :1: is in L1 if and only if $#f(”)_” is in L2 (of course, other $79,975 < f(n) — 71 may also be in L2). Now since L2 is in NSPACE(Sl(n)), L2 is also in NSPACE(Sg(n)) by hypothesis. Hence, there is an 82(71) space—bounded NTM, M3 accepting L2. We now construct an Sg( space—bounded NTM M4 accepting L1 (the original language) from M3. On input :1: of length 71, M4 marks off 32(f cells on its worktapes. (It can do this since $2( is fully space constructible.) M4 is restricted to use only the marked cells. M4 simulates M3 on 907%“. However, Since it does not know t, it Will systematically try 2' = 0,1,2,--- M4 uses a counter of size logt' to simulate M3’s computation on my. (Note that M4 has only :1: as its input; thus, it needs the counter to simulate the “input head movements” of M3 on the segment #i.) If for some 1', M3 accepts, M4 also accepts. Clearly, M4 accepts L1. To see that $2( is sufficient to do the simulation, consider a string 90 of length n in L1. Then M1 accepts 90 within 81(f(n)) space. Then $#f(”)_” is accepted by M2 within space 5401 + f(n) — n) 2 8'1 space. Since NSPACE(81 Q NSPACE(Sg(n)), M3 accepts $#f(")_”; hence M4 accepts :10. Note also that the counter for the #i segment uses at most log'i S log(f(n) — n) space. Since 2 n and 8501) 2 log n, Sg(f(n)) 2 log(f(n) — Thus, M4 has enough space for the counter. We translation lemma can be used to prove tighter hierarchies for nondeterministic space. For m C Claim: NSPACE(n) # NSPACE(n2). Proof: Suppose not. Then NSPACE(n2) Q NSPACE(n). Choosing f(n) 2 712, we get NSPACE(n4) g NSPACEW). The two containment relations imply NSPACE(n4) g NSPACE(n). This is impossible by Savitch7s theorem and the space hierarchy theorem. There is also a time hierarchy theorem. However, the hierarchy is not as tight as for space. ©O.H.Ibarra 67 Theorem 18 Let T1 and T2(n) be time functions. Subject to some “ntcencss” conditions, if hm T1 n 10g T1 n : 0 n—>oo T2(n) then DTIME(T1 (71)) ipTIMEwgm». Proof: Omitted. Tie padding technique can also be used to prove tlght nondetermlnistlc time hlerarchies. Here is another example of the use of padding: Claim: If every language accepted by an 0(n) time-bounded NTM can be accepted by a polyno- mial tlme-bounded WM then P = NP. Proof: (Sketch). Similar to the padding lemma. Let L1 be accepted by an nk time-bounded NTM M1. We define a new language (padded version of L1), L2, that can be accepted by an 0(n) time-bounded NTM M2. By hypothesis, L2 can be accepted by an 0(n7") time-bounded DTM M3. Finally, we can construct from M3 a DTM M4 accepting the original language L1 in 0(n5) time M Relationships Among Important Complexity Classes 1. DSPAC'E(10g n) g NSPAC'E(10g n) g P g NP g NSPACE = PSPAC'E(bySavitch). If C is a complexity class, define co—C 2 {EL 6 C}. 2. co—PS’PACE = PSPAC'E = NPS'PACE = co—NPSPAC'E . co— = 4. Open problem: co—NP NP Let Lsat = {F F is a Boolean formula, F is satisfiable }, Lsat is in NP, but does not seem to be in co—NP. 5. For certain time classes and restricted models, nondeterminisrn makes a difference, e.g., let L = {x#yl:v,y binary and 90 7E y}. L can be accepted by a one-tape NTM (i.e., there is only one read—write tape which initially contains the input) in nlogn time. It can be shown that any nondeterministic one—tape TM accepting 3 requires n2 time. ©O.H.Ibarra 68 6. Let 8(a) 2 log n. Then co—NSPACE(S(n)) = NSPACE(S(n)). as we showed in the section on Complementation. 7. For any space bound 3(a) (i.e. can be less than log 71). any 3(a) space-bounded DTM can be made halting. Claim: Let M be an off-line S(n)—space TM with a single R/W worktape. We can effectively construct an equivalent off—line S(n)—space TM N which always halts. Proof: Fix the input to the TM M. If k is a positive integer, an ID = (q.i.w.j) is a k-space ID if the length of the worktape content 111 is at most k. We assume that the starting ID is start = (go. 1. A. 1). and the unique halting accepting ID is accept = (f. 1. A. 1). Thus. we assume that the machine erases the the worktape and moves it’s input head and worktape head to their initial positions before halting in a unique accepting state. The k-space I D7s of M can be viewed as forming a finite directed graph with the I D’s as nodes. Were is an edge from a k—space I D1 to a k-space I D2 if I D1 can reach I D2 in one step. We define a procedure called REACH(5tart. ID. REACH returns ‘yes’ if start can reach ID gomg only thru k-space ID’s; else it returns ‘nor. In the following. we assume that ID is a k-space ID and either (a) it is halting (i.e.. it is in a halting state). or (b) its direct succesor uses (k: + 1) space. Because M is deterministic. given ID. the subgraph which contains ID is a tree rooted at ID. Thus to determine whether start can reach ID. we need only determine whether start is a node of the tree. We describe an off-line TM N which implements the procedure REACH (5tart.I D.k). For convenience. we can assume that N has several R/W worktapes (which can later be merged into one tape). One worktape is used to represent M 7s worktape. Assume N has been configured to “represent” ID. (By this. we mean that the input and one worktape of N are configured to represent the ID. The other tapes are used as auxiliary scratch tapes.) To determine whether start can reach ID. N performs a depth—first search of the tree. When N visits a node. it configures itself to represent the visited node. Forward edges (to successor nodes) are traversed easily by simulating one forward step of M. Backward edges (to the predecessors) are traversed by running M backwards one step. There may be several valid predecessors when running M backwards; however. there are only at most a fixed number of possible predecessors (independent of the input). and N can select them one at a time in some predtermined order. A leaf is reached if the current node has no predecessor or the predecessors are all (k: + 1)-space I D7s. When a leaf is reached. the search moves forward in the usual depth-first search manner. The procedure above assumes advance knowledge of k: (i.e.. the space M uses). Since k; is not known in advance. N can run procedure REACH(sta7"t. accept. k) for k; = 1. 2. - -- until it finds a k (if it exists. i.e.. M accepts) such that REACH returns ‘yes7. However. if It does not exist. N will run forever. Thus. if with space k. REACH returns ‘no’. N must first determine whether M ©O.H.Ibarra 69 uses k + 1 space before running REACH on space k + 1; if M does not use k; + 1 space, N rejects the In ut. Here is the complete algorithm, where MORETHAN is a procedure that determines if M uses k + 1 Space: procedure COMPLETE(sta7't, accept) egln m do forever egln if REACH(start,accept, k) then [ return (‘yes’); halt ] 8% if not MORETHAN(/<:) | return (‘no’); halt | k: <— k —i eTld eTld The procedure MORETHAN will systematically look at all k-space I D7s (in some ordering) that can lead, in one step, to an ID that uses (k + 1) space, and checks if one of these is reachable M procedure MORETHAN(k) eg1n ID <— first k-space ID in the ordering do until all k—space I D7s have been examined eg1n CongureW if ID can lead in one step to some ID’ using (/6 + 1) space and REACH(3tart,ID,/~€) then return (‘yes7) ID <— next k—space ID in the ordering end return l‘no’) end Corollary 10 DSPACE(S(n)) is closed under union, intersection, and complementation. Claim: Let M be an n—state 2DFA. We can effectively construct a halting 0(n2)—state 2DFA N egulvalent to M. Proof: Assume w.l.o,g. that M accepts in a unique state with its input head at the left end of the tape. N performs a single depth-first (reverse) search to determine whether the initial configuration ©O.H.Ibarra 70 can reach the accepting configuration. Since in the “backward” treversal, a node can have 0(n) predecessors, N in its computation must keep track of pairs of configurations. Thus N needs 0(n2) states Claim: Let M be a k-head 2DFA (i.e., a 2DA with k: independent read heads). We can effectively construct a halting k—head ZDFA N equivalent to M. Claim: (k + 2)—head 2DFA’s are strictly more powerful than k-head ZFDFA’s. Proof: We show that there is a language accepted by a (k:+2)-head 2DFA that cannot be accepted by a k—head 2DFA. Fixed an E encoding of k-head 2DFA’s, as follows. In a rule: states, symbols scanned by the k; heads, directions of moves of the k; heads are represented in unary (‘1’) with the items separated by ‘0’. = 150E(R1)...0E(Rm), where s is the number of states of M, and is the encoding of rule 2'. Let L = {$ :1: = for some k-head 2DFA, M rejects :1: and halts We can construct a (k + 2)-head 2DFA N accepting L as follows: Given input $, N first checks that lb 2 E(M) for some k—head ZDFA. Then N simulates the computation of M by using 16 heads to simulate the k; heads of M one head to keep track of the current state of M, and one head to extract (from :13) the appropriate rule to use for the next move of M. (Note that the head for the state is used in conjunction with head for the rule extraction to determine the appropriate rule to use. 1 To see that L cannot be accepted by a k—head ZDFA M , assume that such an M exists. We may assume by the previous claim that M always halts. Now let :1: = E(M Consider running M on 3:. If M accepts $ then by definition of L $ is not in L. If M rejects and halts on 3:, then $ is not in L. (These are the only cases. The case M goes into an infinite loop on 1E cannot happen.) Hence M accepts an if and only if 1E is not in L. This is a contradiction. Claim: multihead 2DFA’s are equivalent to logn space—bounded DTM’s. Proof: Clearly any k-head 2DFA can be simulated by a logn space—bounded DTM. The DTM simply uses one worktape for each read head to record its position. For the converse, suppose that M is a logn space—bounded DTM With only one worktape. First convert the DTM to an equivalent DTM M’ whose worktape alphabet is binary (not including the blank symbol). This new machine will have space bound k; log2 n for some k. We may assume without loss of generality that the non— blank content of the worktape starts and ends with a ‘1’ (why?) The binary string starting with the leftmost ‘1’ and ending at the bit position of the R/ W head represents an integer 7". Similarly, the reverse of the binary string starting at the bit position directly to the right of the R/ W head up to the rightmost ‘1’ represents another integer 5. Clearly, 7' and s have values no more than 71’“. We can construct a multihead 2DFA M” to simulate M’. The values of 7“ and s can be represented by k heads (for each) on an input of length 77.. Updating the values (i.e., simulating a move of M’) can be accomplished using some auxiliary heads. Hence, a multihead 2DFA M” can be constructed trLsimulatrLML ©O.H.Ibarra 71 The proof above applies to the nondeterrninistic varieties as well, i.e., the class of languages ac- cepted by rnultihead ZNFA’s = NSPACE(log Since it is not known wheter NSPAC'E(log n) = DSPACE(log n)7 it is not known whether rnul- tihead 2NFA7s are equivalent to rnultihead 2DFA’s. Tie situation is different for multihead pushdown automata. We state the following7 omitting the proof: o rnultihead 2NPDA’s are equivalent to rnultihead 2DPDA7s. 0 Every language accepted by a k—head ZDPDA (and, hence also, by a k-head 2NPDA) for some k can be accepted by a polynomial tune—bounded DTM, and conversely. 0 Every language accepted by a l-head ZDPDA can be accepted in linear time by a random accesimachine ...
View Full Document

This note was uploaded on 01/28/2012 for the course CS 220 taught by Professor Ibarra,o during the Winter '08 term at UCSB.

Page1 / 23

chapter5 - Chapter 5 Space and Time Complexity Were are...

This preview shows document pages 1 - 23. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online