chapter1

chapter1 - Chapter 1 Introduction Theory of computation is...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Chapter 1 Introduction Theory of computation is an attempt to formalize the notion of computation, i.e., the notion of an algorithmic process. Computational Complexity aims to determine the solvability/unsolvability of computational problems, to develop techniques for discovering e cient algorithms, and to explain why some problems are inherently di cult in terms of the time and/or space they take to solve. For example, the problem of sorting numbers by comparison, can be solved in O(n log n) time by using merge sort. Since this bound has been achieved, it can be viewed as an upper bound the question that remains is - can we do better? It has been shown that if n numbers are sorted using comparisons, then we need at least O(n log n) steps. Hence, for this problem the lower bound has been established and realized, and merge sort is an asymptotically optimal comparison-based algorithm. Consider the problem of multiplying two n n matrices. A naive algorithm requires O(n3 ) operations. Is this the best we can do? Other more sophisticated algorithms are known which take less time: there is an algorithm to multiply two n n matrices using O(n2:376:::) operations. Can we do better? The answer to this question is not known since no non-trivial lower bound has been established for matrix multiplication. A trivial lower bound is O(n2 ). The partition problem is de ned as follows. Given a set of positive integers, c1 c2 cn , is it possible to partition them into two sets, P1 P2 , such that the sum of the integers in each set is the same i.e., X X ci = ci ? ci 2P1 ci 2P2 Although this looks like a simple problem, all known algorithms require exponential running time. This is an example of a what are called NP-complete problems. Finally let us consider Hilbert's Tenth Problem (HTP) and its application to the equivalence problem for programs. De nition 1 Hilbert's Tenth Problem (HTP): Given: A polynomial p(x1 x2 xn) of n variables and integer coe cients (e.g. p(x1 x2 x3 ) = 1 c 2 O.H.Ibarra 45x2 x2 ; 3x1 x3 + x2 ; 5). 1 2 Question: Does p(x1 x2 that p(a1 an ) = 0)? xn) have an integral root (i.e. are there integer values a1 an such Is HTM decidable? I.e. does there exist an algorithm which when given an arbitrary polynomial p will answer Yes if p has an integral root and No otherwise? Yes (p has an integral solution) p ALGO No (p does not have an integral solution) If p is over a single variable then it is decidable. Why? The problem was posed in 1900 and in 1970 it was shown to be undecidable, i.e., it was shown that no algorithm exists. It should also be noted that the result holds for solutions over the set of non-negative integers, f0 1 2 g. The proof of this is left to the reader. Hint: To show that an algorithm (i.e. decidability) for nonnegative integers case implies an algorithm (decidability) for the integers case, look at all possible combinations of positive and negative values for the variables. For the converse, you'll need the triangular integer hint: Any non-negative integer can be represented as the sum of 3 triangular numbers. n is a triangular number if n = i(i+1) , for some integer i (+,-, or 0). 2 Two applications of the undecidability of HTP are now presented. 1. Given: two non-looping programs (i.e., programs without loops in some programming language) P1 and P2 . Question: Are they equivalent, i.e., is P1 P2? This is the problem of Equivalence of Programs. We show that the equivalence of programs is undecidable (over the integral domain). Proof: Suppose the equivalence of programs is decidable, i.e., 9 an algorithm for it. Then we will show that there is an algorithm for HTP, which is a contradiction since HTP is known to be undecidable. Suppose p(x1 xn) is a polynomial. We de ne the following two programs: P1 : input(x1 xn) y p(x1 xn ) /* use +,-,* to compute the value of p and store in y */ if y 6= 0 then output(`0') end else output(`1') Hence P1 will always output `0' if p(x1 xn) has no solution. c 3 O.H.Ibarra P2 : input(x1 xn ) output(`0') end Thus we see that P1 P2 if and only if p(x1 xn ) has no solution. Corollary 1 6 9 an algorithm (i.e. it is unsolvable) for program optimization. Proof: Suppose 9 an algorithm for program optimization. Given a polynomial p(x1 xn), we construct the program P1 as described above. We give this program as input to the optimization algorithm which gives as output the program P1optimal , the optimized version of P1 . Now P1optimal will be the same as P2 if and only if p(x1 xn) has no solution. Hence using the optimization algorithm, we can solve HTP - which is a contradiction. Hence no algorithm for optimization exists. 2. The second application is in Number Theory. A logician named Alonzo Church showed that there is no algorithm (i.e., it is undecidable) that can decide, in general, whether statements in number theory are true or false. To be more precise: De nition 2 Let the universe (or domain) be the set of non-negative integers, N = f0 1 2 3 g. Statements (or formulas) are built using operations + , quanti ers 9 8, connectives _ ^ : over the universe N and variables x1 x2 . Examples: 8x9y x + x = y] - this is a True statement. 9y8x x + x = y] - this is a False statement. 8q9p8x y |p > q ^ (x y {z 1 ) xy = p}] - this is a True statement (it states that there > 6 can be rewritten using only_^ and : are an in nite number of primes). The formulas can be written such that all quanti ers appear in front of the formula such as: F = Q1 x1 Q2 x2 Qn xn ] where Qi is either 9 or 8, is a formula without quanti ers over variables x1 xn . Theory(N + ) is the set of all true formulas over N using only the operators + and . Theory(N +) is the set of all true formulas over N using only the operator +. Claim: Theory(N + ) is undecidable, i.e., 9 no algorithm to decide given a formula, F , in Theory(N + ) whether F is true or false. (Thus some theorems or true statements in Theory(N + ) are not provable.) In other words, theorem proving cannot be \mechanized". c 4 O.H.Ibarra Proof: Given a polynomial p(x1 xn), let p1 (x1 xn) be the positive terms and p2(x1 be the negative terms. Then de ne the formula F to be: F = 9x1 9x2 9xn p1(x1 xn ) = p2 (x1 xn)] E.g. For p = 45x2 x2 ; 3x1 x3 + x2 ; 5, p1 = 45x2 x2 + x2 and p2 = 3x1 x3 +5. The corresponding 1 2 1 2 formula would be: 9x19x29x3 45x2 x2 + x2 = 3x1 x3 + 5] 1 2 which can be simpli ed by introducing more variables. For example: 9x19x29x39z 45x2 x2 + x2 = z ^ z = 3x1x3 + 5] 1 2 9x19x29x39z9w 45x2 x2 + w = z ^ z = 3x1 x3 + 5 ^ w = x2 x2 ] 1 Now F is true if and only if p has a solution. Thus if there exists an algorithm to determine whether F is true or false, then we can use it to determine whether p has a solution, which is a contradiction. Hence there is no algorithm to decide the validity of formulas in Theory(N,+,*). Claim: Theory(N +) is decidable. Proof: (Sketch) Consider the formula F = 9x8y9w9zE , where E is the expression (x + x = y) ^ (x + y = z ) _ :(z = w)] Suppose we want to check whether F is true. Denote by: E1 the subexpression x + x = y E2 the subexpression x + y = z E3 the subexpression z = w De ne the following languages: L1 = f(x y z w)jx y z w in N x + x = yg L2 = f(x y z w)jx y z w in N x + y = z g L3 = f(x y z w)jx y z w in N z = wg x y z w will be represented in binary with leading ( rst) bit the LSB. So, for example, x = 5 is 1010 y = 7 is 1110 z = 3 is 1100 w = 12 is 0011 xn ) c 5 O.H.Ibarra Note that we can assume that the binary numbers have the same length. Thus, the tuple (5 7 3 12) in binary can be represented as (1 1 1 0)(0 1 1 0)(1 1 0 1)(0 0 0 1). Thus the binary representation of (5 7 3 12) is a string over the alphabet, 4 = f(s1 s2 s3 s4 )jsi 2 f0 1gg The \binary" representation (version) of L1 is the language over the alphabet 4: f(a0 b0 c0 d0 )(a1 b1 c1 d1 ) (an bn cn dn )jx = a0 an y = b0 bn z = c0 cn w = d0 dn are binary strings and x + x = yg. For notational convenience we will also use L1 to denote its binary representation. We can construct a DFA A1 to accept L1 . A1 simply reads the input string and performs the binary addition (in this case adding x to itself) and checking that the binary sum is equal to y. A1 only looks at the rst and second tracks and ignores the third and fourth tracks. Similarly, we can construct DFA's A2 and A3 to accept L2 and L3 , respectively. Now let E4 be the expression :E3 , then the language L4 corresponding to E4 is the complement of L3 . We can construct from A3 a DFA A4 accepting L4 . Let E5 be the expression E2 _ E4 . Clearly, L5 = L2 L4 . We can construct from A2 and A4 a DFA A5 to accept L5 . Finally, let E6 (this is E ) be the expression E1 ^ E5 . Then L6 = L1 \ L5 . Again, we can construct from A1 and A5 a DFA A6 to accept L6 . Now consider the expression 9zE6 . Call this expression E7 . Then L7 = f(x y w)j9z such that(x y w z) 2 L6 g + Note that now L7 3 , where alphabet 3 = f(s1 s2 s3 )jsi 2 f0 1gg. We can construct an NFA B7 to accept L7 . Note that the input to B7 has only three tracks (corresponding to x y w) - the track corresponding to z is not part of the input. B7 is constructed from A6 . B7 \simulates" the computation of A6 , but \guesses" (using non-determinism) the bits of z as it reads the input. B7 is then converted to a DFA A7 . We can do the same thing for the expression E8 = 9w9zE . We can construct from A7 a DFA A8 to accept L8 +, where now 2 = f(s1 s2)jsi 2 f0 1gg. For E9 = 8y9w9zE , we note 2 + that this expression is equivalent to E9 = :9y:(9w9zE ). The language L9 1 , where 1 = f(s1 )js1 2 f0 1gg. We can construct a DFA A9 to accept L9 as follows: Construct a DFA B8 from A8 that accepts the complement of L8 . Thus B8 accepts the language corresponding to :(9w9zE ). Next construct from B8 a DFA C8 to accept the language corresponding to 9y:(9w9zE ) (we already know how to do this!). c 6 O.H.Ibarra Then construct from C8 a DFA A9 to accept the complement of the language accepted by C8 . Thus A9 accepts the language corresponding to the expression :9y:(9w9zE ). Finally, we construct from A9 a DFA A10 to accept the language L10 corresponding to the + formula F = 9x8y9w9zE . Note that L10 0 , where 0 = f()g, where () is a symbol corresponding to an empty tuple. It is easily veri ed that A10 accepts any input if and only if F is true. So all we need to check is whether A10 accepts the null string . The constructions in the example above can be formalized to prove the claim. 1.1 Machine Models We can classify algorithms in terms of thier resource (memory) requirements: A. Fixed memory procedures. Some procedures use only a xed amount of memory, independent of the size of the input (i.e. the device carrying out such a procedure only needs a xed amount of memory). For example, the addition of two integers represented in binary. (a1 ,b1 ), (a2 ,b2 ),....,(an,bn ) - Device doing Addition - s1,s2 ,...,sn This device needs only 1 bit of memory for the carry bit. B. Unbounded memory procedures. Some procedures use memory that grows with the input size (i.e. the device carrying out such a procedure needs an unbounded memory). For example, the multiplication of two integers represented in binary. (a1 ,b1 ), (a2 ,b2 ),....,(an,bn ) - Device doing Multiplication - p1,p2,...,pn Figure 1.1: This device needs memory that is a function of n. Access to memory may be restricted or unrestricted. Some examples of restricted memory access machines are pushdown stacks, queues, and dequeues. Some examples of unrestricted memory access machines are disks, drums, tape units, and RAMs. c 7 O.H.Ibarra C. A general machine. We now introduce a general model of computation. By restricting this machine in various ways, we get machines such as the Turing machine, pushdown automaton, and nite automaton. For a general model, we need to have the following components: { { { { Input/Output Potentially unbounded memory CPU Program Input tape (2-way read only tape) c a1 a2 a3 ... an $ 2-way read only head Finite State Control {q1 , q 2, ..., q k} 1-way write only head Output tape( 1-way write only tape) Auxiliary Memory Figure 1.2: A General Model of a Computing Machine The above gure shows the components of such a model. The tapes are divided into cells which the control can read one at a time. The nite state control represents the CPU and program components. This control has access to the input tape which holds the input to the machine. This tape is a read only two way tape. In other words, the head can move in both directions - left and right or remain at the same cell, but it cannot modify the contents of any cell. The nite control also has access to the an auxillary memory which models the unbounded memory of the machine. This memory may be organized in various ways such as a pushdown stack or a queue or in nite length tapes depending upon the machine. The control also has access to an output tape onto which it writes the output. This output tape is one way, i.e. the control cannot move back on this tape once it has written a cell - it only moves forward or remains at the same cell. The machine shown above can be described as M =< Q ; / $ q0 >. Where c Q is the set of states of the nite control, c 8 O.H.Ibarra is the input alphabet, is the output alphabet, ; is the memory or worktape alphabet, c / is the left end marker for the input, $ is the right end marker for the input, q0 is the initial state in which the machine begins the computation, is the program of the machine consisting of a nite set of rules. Programming this machine is done by using rules of the following form. (q a ) ! (q0 d 0 b) q 2 fq1 q2 ::: qk g is the current state. a 2 is the current symbol under the ROH. is a \local portion" of the auxiliary memory. q0 2 fq1 q2 ::: qk g is the next state. d is the direction of the ROH movement (;1, 0, or +1). 0 is the change in the local portion of the auxiliary memory. b is the output symbol written on the WOH (could be ). For example, if the auxiliary memory is a pushdown stack then machine is a pushdown automaton, and a possible rule is (q5 0 a) ! (q3 +1 pop ). 1.2 Examples of Machines A simpli ed version of the general machine is shown in the gure below. We have combined the input tape and the memory to form an in nite tape in one direction which initially holds the input. The input tape is now a 2-way read/write tape. Let us now consider some speci c examples of machines. Example 1 If the machine has no memory and a 1-way input, then it is called a 1-way Finite Automaton or 1FA. Finite automata are simple machines that can recognize Regualar Expressions. The working of Finite Automata can also be described by state diagrams. Consider the following example of a binary adder which adds two binary numbers. The numbers are represented in binary and are given to machine as input. The machine works correctly only if the digits are in least signi cant c 9 O.H.Ibarra c / a1 a2 a3 ... $ 2-way read/write head Finite State Control {q1 , q 2, ..., q k} 1-way write only head Output tape( 1-way write only tape) Figure 1.3: A Simpler Model of a Computing Machine c / a1 a2 a3 ... $ 1-way read-only head Finite State Control {q1 , q 2, ..., q k} 1-way write only head Output tape( 1-way write only tape) Figure 1.4: A Model of a 1-way Finite Automaton. End markers are not needed for one-way FA's. c 10 O.H.Ibarra bit rst order. The input alphabet consists of pairs of bits, one for each number: P = f(0 0) (0 1) (1 0) (1 1)g The output alphabet consists of bits: = f0 1g. Finite automata can be described by state diagrams such as the one shown in Figure 1.5 for the binary adder. (1,1)/0 (0,0)/0 (0,1)/1 (1,0)/1 q0 (1,0)/0 (0,1)/0 (1,1)/1 q1 (0,0)/1 Start Figure 1.5: State Diagram for a Binary Adder Some FA have no output, and are called pattern recognizers. Given an input string the machine will enter an accept or a reject state after processing the input string, depending on whether the input string is part of the language that the machine recognizes. An example is a machine which is given a sequence of Heads (H ) and Tails, (T ) and it accepts the string only if every Head is followed by at least 2 Tails. Figure 1.6 shows the state diagram for this machine. T T q0 T q1 H q2 T q3 H H Start dead T,H Figure 1.6: State Diagram of a Pattern Recognizer P The machine can be described as A =< Q q0 F >, where the input alphabet of the P machine, = fH T g, Q = fq0 q1 q2 q3 deadg is the set of states, q0 is the initial state, F = fq1g is the set of nal or accepting states and the transition function, is de ned by the diagram. c O.H.Ibarra 11 1.3 Non-determinism Non-determinism is the mathematical model for parallel computation. In terms of machines, nondeterminism introduces choices. The translation rules, , or program of the machine describes the next state of the machine given the current state. In a deterministic machine there is no choice of the next state. For a non-deterministic machine, in contrast, the machine can have more than one choice of the next state. Due to this choice, it is not clear which of the multiple paths available will be followed in any given execution of the machine. The machine can potentially produce di erent results in separate executions starting with the same input. For non-deterministic machines that accept or reject the input strings, a string is said to be accepted by the machine if there is at least one execution that accepts the string. Therefore, if a given execution of a non-deterministic machine results in the rejection of the input, it is not necessarily true that the string is rejected by the machine. 1.3.1 Finite Automata The simplest machines are those with no memory (actually, they have a xed amount of memory which cannot be increased, i.e., the memory is independent of the input size). They are called nite-state machines. A special case is a nite automaton (FA). FA's are pattern recognizers. Figure 1.7 shows the model. An FA can be formally described as M =< Q q0 F >, where Q is the nite set of states, is the input alphabet, is the transition function which de nes the actions of the FA, q0 is the initial state and F is the set of accepting or nal states. The input head can be either 1-way or 2-way, and the transition function may (in the case of non-deterministic) or may not (in the case of deterministic) allow multiple moves from a state on the same input. These two options result in four varieties of FA depending on whether they are 1-way or 2-way and deterministic or non-deterministic. (The 2-way varieties are provided input delimiters.) The four varieties are called 1DFA, 1NFA, 2DFA, 2NFA. All four varieties of FA are equivalent in their recognizing power. In other words, any language that can be accepted by an FA of any variety, can also be accepted by some FA of any of the other varieties. Therefore we see that for memoryless machines, having a 2-way head or non-determinism does not result in any gains. The languages accepted by FA are known as Regular Sets and can be described using Regular Expressions. It is important to note that even though all four varieties of FA are equivalent c 12 O.H.Ibarra a1 a2 a3 a4 ... 1-way read-only head Finite State Control {q1 , q 2, ..., q k} Figure 1.7: A Model of a Finite Automaton in terms of the language that they can recognize, some are more e cient (\succinct") than others. In particular, it can be shown that there are certain languages that can be accepted by an O(n) state 1NFA but requires at least O(2n ) states by any 1DFA accepting the same language. Consider the following set of languages. For every positive integer n, let Ln = fx1y j x in f0 1g jyj = n ; 1g. Ln can be accepted by a 1NFA with n + 1 states as follows 0,1 1 0,1 0,1 .... 0,1 n-1 states Start For example, L2 can be accepted by the following 1NFA. 0,1 1 0,1 Start Ln can be accepted by a 2DFA with O(n) states. c 13 O.H.Ibarra Claim: Any 1DFA accepting Ln needs at least 2n states. Proof: Assume there is a 1DFA A accepting Ln. Let w be any binary string of length n, i.e., jwj = n. 6 w - s(w) = state A enters after processing w q0 Subclaim: For any w and w0 s.t. jwj = jw0 j = n and w 6= w0 s(w) 6= s(w0). Proof: Suppose not, i.e., s(w) = s(w0 ) = p 6 w q0 6 q0 6 s(w) = p w0 6 s(w0 ) = p Since w and w0 are di erent, they disagree in at least one bit position. Let the point of 00 discrepancy be at position k + 1 from the right, k 0. Thus, w = w1 1w2 and w0 = w1 0w2 0 where w2 and w2 are of length k. Now consider 2 new strings. w = w0n;k;1 = w1 1w2 0n;k;1 00 w0 = w0 0n;k;1 = w1 0w2 0n;k;1 Since the machine will be in the same state (s(w) = s(w0 )) after reading the rst n symbols of both w and w0 , the remaining symbols for both are the same (0n;k;1 ), and we have a deterministic machine, A will accept both w and w0 . But this is impossible since w0 is not in Ln . It follows that s(w) 6= s(w0 ). Since there are 2n distinct strings of length n, there are 2n distinct s(w)'s. Hence A must have at least 2n states. c 14 O.H.Ibarra Note that Ln can be accepted by a two-way deterministic nite automaton, (2DFA) with O(n) states. Thus 2DFA's are more succint than 1DFA's. Here is another example. Consider the language: L = fx#xjx is a binary string and jxj = ng. It is easy to construct a 2DFA with O(n) states accepting L. Claim: Any 1NFA accepting L needs at least 2n states. Proof: Assume there is a 1NFA A accepting L with fewer than 2n states. Let w1 be any binary string of length n, i.e., jw1 j = n. Suppose the word w1 #w1 is given as input to A. Let 1 be the state after A has read the symbol #. Let w2 be any binary string of length n di erent from w1 , i.e., w1 = w2 . Suppose the word 6 w2 #w2 is given as input to A. Let 2 be the state after A has read the symbol # with this input. Subclaim: 1 6= 2 . Proof: Suppose that = 2 . Then consider the behavior of A when given the string w1 #w2 as input. Clearly, after reading the symbol #, A will be in state 1 . Since 1 = 2 , we see that the state of the machine is identical to the state when w2 #w2 is processed, after reading #. Hence A would also accept the string w1 #w2 , which is clearly not in the language. This is a contradiction. Hence our subclaim is true. 1 Therefore for each distinct wi of length n, we must have a distinct state i that is reached after reading the symbol #. Since there are 2n distinct binary strings of length n, we must have at least 2n di erent states in A. The above two proofs make use of what is called the Cut-and-Paste Argument. 1.3.2 Pushdown Automata These machines are similar to the Finite Automata, except that they have an additional \pushdown stack" on which an unlimited amount of information can be stored but which operates on a last-inrst-out manner. The stack can only be accessed using the PUSH and POP operations. As with FA, there are four varieties of PDAs: 1DPDA, 2DPDA, 1NPDA and 2NPDA. Unlike the FA, these four varieties of PDA do not have the same recognizing power. In particular, 1NPDAs are more powerful than 1DPDAs because they can recognize the language L = fxxR jx is a binary string g. c 15 O.H.Ibarra i {0 i 1i 0 |i > 0} 2NPDA 1NPDA {xxR} 2DPDA 1DPDA i {0 i 1i 0 |i > 0} FA {0 i 1i |i > 0} Figure 1.8: Relationships between the four PDA varieties Also, the 2-way varieties can recognize the language L = f0i 1i 0i ji > 0g which cannot be recognized by the 1-way varieties. The relationship between the four varieties is summarized in Figure 1.8. A general result regarding PDAs is: Any language that can be recognized by a 2DPDA can be recognized by a RAM (Random Access Machine) in linear time. 1.3.3 Linear Bounded Automaton These machines are similar to 2-way FA, except that they can overwrite the input. The ability to overwrite is useless if the overwritten cell is never visited again, hence, the 1-way varieties of LBA are uninteresting. Thus there are two interesting varieties of LBA - 2-way Deterministic LBA or 2DLBA and 2-way Non deterministic LBA or 2NLBA. The term Bounded is important and it captures the fact that the machine is not allowed to write beyond the given input. The relationship between 2DLBAs and 2NLBAs is an open question. For a DLBA, there is an upper bound on the number of steps that it can take if it halts (i.e. it is not looping): s tn (n + 2), where s is the number of states of the nite control, t is the size of the alphabet and n is the length of the input string (the `2' is because of input delimiters). The proof is left to the reader. 1.3.4 Expanding LBA or Turing Machines If we allow an LBA to write beyond the input string, then it becomes equivalent to a Turing Machine. A (Basic) Turing Machine consists of a nite control, a 2-way read/write input head where the input tape is in nite in one direction. Figure 1.9 shows the model for the basic Turing Machine. The basic Turing Machine can also be viewed as a generalization of a 2-way deterministic nite automaton. We need to make two extensions to the 2DFA: c 16 O.H.Ibarra a1 a2 . . . ai . . . an λ λ λ . . . 2-way R/W head Finite control λ=Blank Figure 1.9: Model of the Basic Turing Machine 1. The input head allowed to modify the input tape. 2. The input tape is made in nite to the right. The Turing Machine (TM) can therefore overwrite the input and can also use an arbitrarily large number of cells on the right hand side of the input tape. Initially, the input tape contains the input beginning at the leftmost cell and the rest of the tape contains blanks, denoted by ,t or B . The head can move in both directions on the input tape. A TM can be described as a 8-tuple: M =< Q P ; q0 qaccept qreject >, where, Q is the set of states in the nite control, P is the input alphabet, P ; is the tape alphabet (it includes , the blank symbol, and possibly other symbols.), is the transition function that governs the working of the TM it maps Q ; ! Q ; f+1 ;1g . Let us now see an example of Turing machine construction: TM (basic model) to recognize (accept) the language L = f 0i 1i j i 1 g Note that in the following, the subscripts of states are formed by the letters o and e, which stand for odd and even, and denotes the blank symbol. = f 0, 1 g, Q = f s, qo , qe , qoo , qoe, qeo , qee, r, f , qaccept g, ; = f 0, 1, X, X, g, F = f qaccept g (s, 0) = (qo , X, +1) (qo , 0) = (qe , 0, +1) (qo , X) = (qo , X, +1) (qe , 0) = (qo , X, +1) (qe , X) = (qe , X, +1) (qo , 1) = (qoo , X, +1) (qoo , 1) = (qoe , 1, +1) (qoo , X)= (qoo , X, +1) (qoe , 1) = (qoo , X, +1) c O.H.Ibarra 17 (qoe , X)= (qoe , X, +1) (qe , 1) = (qeo , X, +1) (qeo , 1) = (qee , 1, +1) (qeo , X)= (qeo , X, +1) (qee , 1) = (qeo , X, +1) (qee , X) = (qee , X, +1) (qoo , ) = (r, , -1) (qee , ) = (r, , -1) (r, a) = (r, a, -1) for a = 0, 1, X (r, X) = (f , X, +1) (f , X) = (f , X, +1) (f , 0) = (qo , X, +1) (f , ) = (qaccept , , -1) The TM works as follows: It makes repeated scans of the input. In each scan it marks o alternate 0's and 1's keeping track of the parity of each. If the parity of 0 is di erent from the parity of 1 in any scan, it rejects. If the parities are equal in every scan then the lengths of the 0s and 1s must be equal, hence it accepts. Since in each scan the number of unmarked symbols is halved, the TM makes O(log n) scans. In each scan the complete input is read. Clearly, on an input x of length n, the TM takes at most O(n log n) steps to verify that x is in the language L. Turing Machines can be viewed as 1. Pattern Recognizers or Language Acceptors 2. Function Computers Note that in the de nition of the basic TM, we have not allowed the input head to remain stationary - the only allowed directions for the head are +1 or -1 i.e. right or left only. This is however only a super cial limitation since we can simulate the stationary behaviour as a combination of a left and right move. Speci cally, (q a) = (p b 0) (q a) = (p0 b + 1) (p0 x) = (p x ;1) 8x 2 ; 1.4 A Programming Language for TMs A TM can be represented as a program using the following constructs. Note that the registers referred to in the constructs can only store 1 symbol a is any symbol in the worktape alphabet. c 18 O.H.Ibarra read R write a left right Ra R1 R2 if R1 = R2 goto l goto l accept reject halt read symbol under the read/write head and store in register R. write the symbol a under the read/write head. move read/write head 1 cell (square) to the left. move read/write head 1 cell to the right. store symbol a to register R. copy symbol in register R2 to register R1 . l is a label. Note that the following instructions can easily be simulated in the TM programming language: if R1 6= R2 goto l if R = a goto l if R = a goto l 6 while R1 = R2 do while R1 = R2 do 6 while R = a do while R = a do 6 Here an example: TM program to recognize the language L = f x j x in f0 1g , x = xR g 1 2 read A if A = goto 4 write right read B if B = goto 2 6 left read C if C = goto 4 if C = A goto 5 6 write c 19 O.H.Ibarra 3 4 5 left read B if B = goto 3 6 right goto 1 accept halt reject halt 1.5 Equivalence of Programs and TMs Claim: The program speci cation and the speci cation for Turing machines are the same. Proof: The proof is done in two parts. First we show that the program speci cation can be simulated by the speci cation then we show that the speci cation can be simulated by the program speci cation. Program speci cation can be simulated by speci cation: Suppose we are given a TM M speci ed by a program P using the constructs described above, and P has k instructions (lines) i1 , i2 , . . . , ik and m registers r1 , r2 , . . . , rm . We will construct an equivalent TM M 0 described by speci cation as follows. 1. The states of M 0 have the form hc i1 i2 : : : ik r1 r2 : : : rm i, where (a) c is the program counter, indicating the next instruction to be executed (b) i1 , i2 , . . . , ik are the instructions (these stay the same) (c) r1 , r2 , . . . , rm are the current values of the registers of P 2. The starting state of M 0 is h1 i1 i2 : : : ik : : : i. 3. The accepting states of M 0 have the form hc i1 i2 : : : ik r1 r2 : : : rm i, where ic is an accepting instruction. 4. The transition rules have the form: 0 0 (hc i1 : : : ik r1 : : : rm i s) = (hc0 i1 : : : ik r1 : : : rm i s0 d) where (a) c0 = l if ic is a branch statement to l and the condition is true c 20 O.H.Ibarra (b) c0 = c + 1 if ic is not a branch statement, or ic is a branch statement and the condition is false (c) d = ;1 (+1) if ic : left (right) 0 otherwise 0 (d) rj = s if ic : read rj 0 (e) rj = a if ic : rj a 0 (f) rj = ri if ic : rj ri (g) s0 = a if ic : write a It is easy to verify that M 0 is equivalent to M . speci cation can be simulated by program speci cation: Suppose we are given a TM M speci ed by a transition function , and M has s states q1 q2 : : : qs , and m tape symbols t1 t2 : : : tm . (Hence there are at most sm transition rules, since some transitions may not be de ned.) We will construct an equivalent program P using the constructs given above as follows. P has s + m + 2 registers: Q1 Q2 : : : Qs , T1 T2 : : : Tm , STATE and SYMBOL. Intuitively, at any given time STATE and SYMBOL hold the current state and symbol of M . P starts by initializing the registers: Q1 q1 Q2 q2 ::: Qs qs T1 t1 T2 t2 ::: Tm tm STATE q1 0: read SYMBOL Next P identi es the transition rule to execute next and then jumps to the appropriate code segment: c 21 O.H.Ibarra if STATE = Q1 goto 1 if STATE = Q2 goto 2 ::: if STATE = Qs goto s 1: if SY MBOL = T1 goto 1,1 if SY MBOL = T2 goto 1,2 ::: if SY MBOL = Tm goto 1,m ::: s: if SY MBOL = T1 goto s,1 if SY MBOL = T2 goto s,2 ::: if SY MBOL = Tm goto s,m i,j: <do the transition rule for (qi tj ) > < assume (qi tj ) =< q t d > if unde ned, then halt > write t SY MBOL STATE t Qq right <if d = +1 > left <if d = ;1 > accept <if q is an accepting state> reject <if q is a rejecting state> goto 0 <otherwise, do next move> It is easy to verify that P is equivalent to M . c 22 O.H.Ibarra 1.6 Some De nitions Turing machines can be viewed as Language Recognizers because when given input they either accept, reject or go into an in nite loop. Based on the type of the Turing machine that accepts a language, the following terms are de ned Languages that are accepted by Turing machines that always halt (i.e. never go into an in nite loop) are called Decidable or Recursive (R) languages. The TMs that accept these languages are called deciders. Languages that are accepted by Turing machines which may not halt for all inputs are called Recursively Enumerable (RE). The set of Recursive languages is properly included in the set of Recursively Enumerable languages, i.e., R RE . Turing machines can also be viewed as function computers. Consider for example a Turing machine that computes the function f (x y) = x + y when given the inputs x and y separated by a # sign on the input tape. If the inputs are in unary and the output is to be in unary, the algorithm is trivial - one needs to just replace the # symbol with a 1 and then replace the rightmost 1 with a blank. If on the other hand, the inputs are in binary and the output is to be in binary also, a more complex algorithm is required. As another example, consider the monus function: : f (x y ) = x ; y = ( x ; y if y > 0 0 otherwise If the input is in unary, with x followed by y separated by # symbol. The monus can be constructed by replacing one '1' of x for every '1' of y. If the symbols of y are exhausted before those of x, the answer is given by the remaining size of x. If the symbols of x are exhausted rst or at the same time, the answer is 0. A Turing machine could also be viewed as a transformer, such as one that takes unary input and produces binary output. 1.7 Variants of Turing Machines There are several varieties of Turing Machines, all of which have the same computing power. 1.7.1 2-track Tape The BTM has a single track tape - only one symbol can be stored in each cell. A 2-track tape is one which stores two symbols per cell. Would a TM with a 2-track tape be more powerful than the c 23 O.H.Ibarra a1 a2 . . . ai . . . an λ λ λ . . . ... λ λ ... λ λ ... 2-way R/W head Finite control Figure 1.10: 2-track Tape Turing Machine BTM? We can simulate a 2-track TM with a single track TM as follows. Given a machine A that has two tracks, we construct a machine B with one track that simulates the working of A. Let ; be the tape alphabet of machine A. Then the tape alphabet for B is ;0 = ; ;. In other words, we consider the each cell of the 1-track tape of B to hold a composite symbol which consists of two symbols of the tape of A, one symbol for each track. The transition rules of B are derived from those of A by appropriate replacement of the symbols. We therefore see that having 2 tracks does not increase the ability of the TM to recognize languages. However, the 2-track tape can make the programming of a TM easier, as in the following example. 1.7.2 2-way In nite Tape The tape of the BTM is in nite only in one direction (to the left). Would a TM with a tape that is in nite in both directions be more powerful? . . . λ a1 . . . ai . . . an λ λ λ . . . 2-way R/W head Finite control Figure 1.11: 2-way In nite Tape Turing Machine 1.7.3 Other Varieties There are other variations of TM's: multi-tape TM, multi-dimensional TM, TM with several heads per tape, etc. 1.8 Grammars A Grammar is formally de ned as G =< N T P S >, where, N is a set of symbols called nonterminals, T is a set of symbols called terminals, P is the set of rules or productions and S is a c O.H.Ibarra 24 symbol from the set N called the start symbol. The productions (or rules) given in the set P are of the form ! , where can be any string of terminals and non-terminals with containing at least one non-terminal. The rules allow the occurrence of the string to be replaced by the string . The language de ned by the grammar is the set of strings of terminals that can be generated by starting with the symbol S and applying the rules in P . By restricting the form that the rules can take, several types of grammars can be de ned. Some of these grammars correspond to the models of machines described above, i.e., any language generated by a grammar of a certain type can be recognized by some machine of the corresponding model and any language recognized by a machine of a given model can be generated by some grammar of the corresponding grammar. Let ! be any production in the grammar. The following types of grammars are de ned: Type 3 Grammars. There are two types of such grammars - the Left Linear Grammars (LLGs) and the Right Linear Grammars (RLGs). For both these grammars, must consist of exactly one nonterminal. For LLGs, can be either a terminal string or a single nonterminal followed by a terminal string, i.e., the productions all look like: A ! Bx or A ! x, where A B are nonterminals and x is a terminal string (possibly ). For RLGs, all productions look like: A ! xB or A ! x, where A B are nonterminals and x is a terminal string. The set of language generated by LLG and RLGs is exactly the set of language accepted by Finite Automata, i.e., the Regular languages. Thus Type 3 grammars correspond to FA. 1 (Type 2 2 ) A grammar which has productions of both types above is called a Linear Grammar and is not a Type 3 grammar. This grammar type can be informally dubbed as a Type 2 1 2 Grammar. An example of such a grammar is: S ! 0Aj A ! S1 which generates the language L = f0i 1i ji 0g which is not a regular language. It has been shown that Linear Grammars correspond to 1-reversal 1NPDAs. That is 1NPDAs which may push symbols onto the stack only as long as they have not popped any symbol from the stack. Once a symbol is popped, then no more symbols can be pushed onto the stack. Type 2 Grammars. These grammars are also called Context Free Grammars or CFGs. They are characterized by productions that have only one nonterminal symbol on the left hand side, i.e. j j = 1. These grammars correspond to 1NPDAs. Type 1 Grammars. These grammars are also called Context Sensitive Grammars or CSGs. The only restriction on the productions is that the number of symbols on the left hand side should be less than or equal to the number of symbols on the right hand side of the production, i.e. j j j j. These grammars correspond to the 2-way Nondeterministic Linear Bounded c 25 O.H.Ibarra Automata or NLBAs. It should be noted that a CSG cannot generate the null symbol . Therefore, a CSG can generate any language that is generated by a CFG if is not generated by the CFG. Although parsing of CFGs is not very di cult (it can be done in O(n3 )), parsing of CSGs is very di cult (all known algorithms require O(cn )) for some constant c. Type 0 Grammars. This is the nal class of grammars and it is unrestricted. This set corresponds to the Turing Machines or the Expandable LBAs. Table 1.1 summarizes the above information. Type Type 0 Type 1 Type 2 1 Type 2 2 Type 3 Name unrestricted CSG CFG LG LLG (RLG) Machine Restriction Turing Machines ! NLBA jj jj 1NPDA =A 1-reversal 1NPDA ! xByjx FA = A and ! Byjx ( ! yB jx) Table 1.1: Relationship between Grammars and Machines There is another class of grammars which corresponds to 1DPDAs, called the LR(k) grammars. There is no known grammatical characterization for the following models of machines: 2NPDA, 2DPDA and DLBA. 1.9 Exercises 1. Show that Hilbert's Tenth Problem (HTP) over Z , the set of all integers (positive, negative and 0) is decidable if and only if HTP over N , the set of non-negative integers, is decidable. Hint: You might need the fact that every non-negative integer n is the sum of three triangular numbers, and conversely. A non-negative integer k is triangular if k = i(i+1) for some integer 2 i (positive, negative or 0). 2. Show that the language L1 for the expression E1 can be accepted by a DFA. + 3. Suppose A is a DFA accepting a language L 4 , where 4 = f(s1 s2 s3 s4 )jsi 2 f0 1gg. + 0 0 consists of all strings in L with the rst component De ne the language L 3 , where L deleted. Show how to construct an NFA A0 from A accepting L0 . 4. Describe how to construct, given a polynomial p(x1 xn) with integer coe cients (i.e. an instance of HTP), a program P with no inputs such that P halts if and only if p(x1 xn ) has a solution. c O.H.Ibarra 26 5. From Problem 4 show that the following problems are undecidable: (a) Given: Program P and input x. Question: Will P halt on input x? (b) Given: Program P and input x and statement label l. Question: Will P on input x ever execute statement labeled l? 6. Show that HTP when restricted to one-variable polynomials is decidable. 7. Show that HTP when restricted to linear polynomials (over several variables) is decidable. Hint: Show that given a linear equation, E = a1 x1 + + an xn, the language L(E ) = f(x1 xn)jxi 2 N s.t. a1 x1 + + anxn = 0g is accepted by a DFA. Just describe infomally the DFA. Try an example, e.g., 3x1 ; 5x2 + x3 ; 7x4 + 6. ...
View Full Document

Ask a homework question - tutors are online