{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

chapter1 - Chapter 1 Introduction Theory of computation is...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Chapter 1 Introduction Theory of computation is an attempt to formalize the notion of computation, i.e., the notion of an algorithmic process. Computational Complexity aims to determine the solvability/unsolvability of computational problems, to develop techniques for discovering e cient algorithms, and to explain why some problems are inherently di cult in terms of the time and/or space they take to solve. For example, the problem of sorting numbers by comparison, can be solved in O(n log n) time by using merge sort. Since this bound has been achieved, it can be viewed as an upper bound the question that remains is - can we do better? It has been shown that if n numbers are sorted using comparisons, then we need at least O(n log n) steps. Hence, for this problem the lower bound has been established and realized, and merge sort is an asymptotically optimal comparison-based algorithm. Consider the problem of multiplying two n n matrices. A naive algorithm requires O(n3 ) operations. Is this the best we can do? Other more sophisticated algorithms are known which take less time: there is an algorithm to multiply two n n matrices using O(n2:376:::) operations. Can we do better? The answer to this question is not known since no non-trivial lower bound has been established for matrix multiplication. A trivial lower bound is O(n2 ). The partition problem is de ned as follows. Given a set of positive integers, c1 c2 cn , is it possible to partition them into two sets, P1 P2 , such that the sum of the integers in each set is the same i.e., X X ci = ci ? ci 2P1 ci 2P2 Although this looks like a simple problem, all known algorithms require exponential running time. This is an example of a what are called NP-complete problems. Finally let us consider Hilbert's Tenth Problem (HTP) and its application to the equivalence problem for programs. De nition 1 Hilbert's Tenth Problem (HTP): Given: A polynomial p(x1 x2 xn) of n variables and integer coe cients (e.g. p(x1 x2 x3 ) = 1 c 2 O.H.Ibarra 45x2 x2 ; 3x1 x3 + x2 ; 5). 1 2 Question: Does p(x1 x2 that p(a1 an ) = 0)? xn) have an integral root (i.e. are there integer values a1 an such Is HTM decidable? I.e. does there exist an algorithm which when given an arbitrary polynomial p will answer Yes if p has an integral root and No otherwise? Yes (p has an integral solution) p ALGO No (p does not have an integral solution) If p is over a single variable then it is decidable. Why? The problem was posed in 1900 and in 1970 it was shown to be undecidable, i.e., it was shown that no algorithm exists. It should also be noted that the result holds for solutions over the set of non-negative integers, f0 1 2 g. The proof of this is left to the reader. Hint: To show that an algorithm (i.e. decidability) for nonnegative integers case implies an algorithm (decidability) for the integers case, look at all possible combinations of positive and negative values for the variables. For the converse, you'll need the triangular integer hint: Any non-negative integer can be represented as the sum of 3 triangular numbers. n is a triangular number if n = i(i+1) , for some integer i (+,-, or 0). 2 Two applications of the undecidability of HTP are now presented. 1. Given: two non-looping programs (i.e., programs without loops in some programming language) P1 and P2 . Question: Are they equivalent, i.e., is P1 P2? This is the problem of Equivalence of Programs. We show that the equivalence of programs is undecidable (over the integral domain). Proof: Suppose the equivalence of programs is decidable, i.e., 9 an algorithm for it. Then we will show that there is an algorithm for HTP, which is a contradiction since HTP is known to be undecidable. Suppose p(x1 xn) is a polynomial. We de ne the following two programs: P1 : input(x1 xn) y p(x1 xn ) /* use +,-,* to compute the value of p and store in y */ if y 6= 0 then output(`0') end else output(`1') Hence P1 will always output `0' if p(x1 xn) has no solution. c 3 O.H.Ibarra P2 : input(x1 xn ) output(`0') end Thus we see that P1 P2 if and only if p(x1 xn ) has no solution. Corollary 1 6 9 an algorithm (i.e. it is unsolvable) for program optimization. Proof: Suppose 9 an algorithm for program optimization. Given a polynomial p(x1 xn), we construct the program P1 as described above. We give this program as input to the optimization algorithm which gives as output the program P1optimal , the optimized version of P1 . Now P1optimal will be the same as P2 if and only if p(x1 xn) has no solution. Hence using the optimization algorithm, we can solve HTP - which is a contradiction. Hence no algorithm for optimization exists. 2. The second application is in Number Theory. A logician named Alonzo Church showed that there is no algorithm (i.e., it is undecidable) that can decide, in general, whether statements in number theory are true or false. To be more precise: De nition 2 Let the universe (or domain) be the set of non-negative integers, N = f0 1 2 3 g. Statements (or formulas) are built using operations + , quanti ers 9 8, connectives _ ^ : over the universe N and variables x1 x2 . Examples: 8x9y x + x = y] - this is a True statement. 9y8x x + x = y] - this is a False statement. 8q9p8x y |p > q ^ (x y {z 1 ) xy = p}] - this is a True statement (it states that there > 6 can be rewritten using only_^ and : are an in nite number of primes). The formulas can be written such that all quanti ers appear in front of the formula such as: F = Q1 x1 Q2 x2 Qn xn ] where Qi is either 9 or 8, is a formula without quanti ers over variables x1 xn . Theory(N + ) is the set of all true formulas over N using only the operators + and . Theory(N +) is the set of all true formulas over N using only the operator +. Claim: Theory(N + ) is undecidable, i.e., 9 no algorithm to decide given a formula, F , in Theory(N + ) whether F is true or false. (Thus some theorems or true statements in Theory(N + ) are not provable.) In other words, theorem proving cannot be \mechanized". c 4 O.H.Ibarra Proof: Given a polynomial p(x1 xn), let p1 (x1 xn) be the positive terms and p2(x1 be the negative terms. Then de ne the formula F to be: F = 9x1 9x2 9xn p1(x1 xn ) = p2 (x1 xn)] E.g. For p = 45x2 x2 ; 3x1 x3 + x2 ; 5, p1 = 45x2 x2 + x2 and p2 = 3x1 x3 +5. The corresponding 1 2 1 2 formula would be: 9x19x29x3 45x2 x2 + x2 = 3x1 x3 + 5] 1 2 which can be simpli ed by introducing more variables. For example: 9x19x29x39z 45x2 x2 + x2 = z ^ z = 3x1x3 + 5] 1 2 9x19x29x39z9w 45x2 x2 + w = z ^ z = 3x1 x3 + 5 ^ w = x2 x2 ] 1 Now F is true if and only if p has a solution. Thus if there exists an algorithm to determine whether F is true or false, then we can use it to determine whether p has a solution, which is a contradiction. Hence there is no algorithm to decide the validity of formulas in Theory(N,+,*). Claim: Theory(N +) is decidable. Proof: (Sketch) Consider the formula F = 9x8y9w9zE , where E is the expression (x + x = y) ^ (x + y = z ) _ :(z = w)] Suppose we want to check whether F is true. Denote by: E1 the subexpression x + x = y E2 the subexpression x + y = z E3 the subexpression z = w De ne the following languages: L1 = f(x y z w)jx y z w in N x + x = yg L2 = f(x y z w)jx y z w in N x + y = z g L3 = f(x y z w)jx y z w in N z = wg x y z w will be represented in binary with leading ( rst) bit the LSB. So, for example, x = 5 is 1010 y = 7 is 1110 z = 3 is 1100 w = 12 is 0011 xn ) c 5 O.H.Ibarra Note that we can assume that the binary numbers have the same length. Thus, the tuple (5 7 3 12) in binary can be represented as (1 1 1 0)(0 1 1 0)(1 1 0 1)(0 0 0 1). Thus the binary representation of (5 7 3 12) is a string over the alphabet, 4 = f(s1 s2 s3 s4 )jsi 2 f0 1gg The \binary" representation (version) of L1 is the language over the alphabet 4: f(a0 b0 c0 d0 )(a1 b1 c1 d1 ) (an bn cn dn )jx = a0 an y = b0 bn z = c0 cn w = d0 dn are binary strings and x + x = yg. For notational convenience we will also use L1 to denote its binary representation. We can construct a DFA A1 to accept L1 . A1 simply reads the input string and performs the binary addition (in this case adding x to itself) and checking that the binary sum is equal to y. A1 only looks at the rst and second tracks and ignores the third and fourth tracks. Similarly, we can construct DFA's A2 and A3 to accept L2 and L3 , respectively. Now let E4 be the expression :E3 , then the language L4 corresponding to E4 is the complement of L3 . We can construct from A3 a DFA A4 accepting L4 . Let E5 be the expression E2 _ E4 . Clearly, L5 = L2 L4 . We can construct from A2 and A4 a DFA A5 to accept L5 . Finally, let E6 (this is E ) be the expression E1 ^ E5 . Then L6 = L1 \ L5 . Again, we can construct from A1 and A5 a DFA A6 to accept L6 . Now consider the expression 9zE6 . Call this expression E7 . Then L7 = f(x y w)j9z such that(x y w z) 2 L6 g + Note that now L7 3 , where alphabet 3 = f(s1 s2 s3 )jsi 2 f0 1gg. We can construct an NFA B7 to accept L7 . Note that the input to B7 has only three tracks (corresponding to x y w) - the track corresponding to z is not part of the input. B7 is constructed from A6 . B7 \simulates" the computation of A6 , but \guesses" (using non-determinism) the bits of z as it reads the input. B7 is then converted to a DFA A7 . We can do the same thing for the expression E8 = 9w9zE . We can construct from A7 a DFA A8 to accept L8 +, where now 2 = f(s1 s2)jsi 2 f0 1gg. For E9 = 8y9w9zE , we note 2 + that this expression is equivalent to E9 = :9y:(9w9zE ). The language L9 1 , where 1 = f(s1 )js1 2 f0 1gg. We can construct a DFA A9 to accept L9 as follows: Construct a DFA B8 from A8 that accepts the complement of L8 . Thus B8 accepts the language corresponding to :(9w9zE ). Next construct from B8 a DFA C8 to accept the language corresponding to 9y:(9w9zE ) (we already know how to do this!). c 6 O.H.Ibarra Then construct from C8 a DFA A9 to accept the complement of the language accepted by C8 . Thus A9 accepts the language corresponding to the expression :9y:(9w9zE ). Finally, we construct from A9 a DFA A10 to accept the language L10 corresponding to the + formula F = 9x8y9w9zE . Note that L10 0 , where 0 = f()g, where () is a symbol corresponding to an empty tuple. It is easily veri ed that A10 accepts any input if and only if F is true. So all we need to check is whether A10 accepts the null string . The constructions in the example above can be formalized to prove the claim. 1.1 Machine Models We can classify algorithms in terms of thier resource (memory) requirements: A. Fixed memory procedures. Some procedures use only a xed amount of memory, independent of the size of the input (i.e. the device carrying out such a procedure only needs a xed amount of memory). For example, the addition of two integers represented in binary. (a1 ,b1 ), (a2 ,b2 ),....,(an,bn ) - Device doing Addition - s1,s2 ,...,sn This device needs only 1 bit of memory for the carry bit. B. Unbounded memory procedures. Some procedures use memory that grows with the input size (i.e. the device carrying out such a procedure needs an unbounded memory). For example, the multiplication of two integers represented in binary. (a1 ,b1 ), (a2 ,b2 ),....,(an,bn ) - Device doing Multiplication - p1,p2,...,pn Figure 1.1: This device needs memory that is a function of n. Access to memory may be restricted or unrestricted. Some examples of restricted memory access machines are pushdown stacks, queues, and dequeues. Some examples of unrestricted memory access machines are disks, drums, tape units, and RAMs. c 7 O.H.Ibarra C. A general machine. We now introduce a general model of computation. By restricting this machine in various ways, we get machines such as the Turing machine, pushdown automaton, and nite automaton. For a general model, we need to have the following components: { { { { Input/Output Potentially unbounded memory CPU Program Input tape (2-way read only tape) c a1 a2 a3 ... an $ 2-way read only head Finite State Control {q1 , q 2, ..., q k} 1-way write only head Output tape( 1-way write only tape) Auxiliary Memory Figure 1.2: A General Model of a Computing Machine The above gure shows the components of such a model. The tapes are divided into cells which the control can read one at a time. The nite state control represents the CPU and program components. This control has access to the input tape which holds the input to the machine. This tape is a read only two way tape. In other words, the head can move in both directions - left and right or remain at the same cell, but it cannot modify the contents of any cell. The nite control also has access to the an auxillary memory which models the unbounded memory of the machine. This memory may be organized in various ways such as a pushdown stack or a queue or in nite length tapes depending upon the machine. The control also has access to an output tape onto which it writes the output. This output tape is one way, i.e. the control cannot move back on this tape once it has written a cell - it only moves forward or remains at the same cell. The machine shown above can be described as M =< Q ; / $ q0 >. Where c Q is the set of states of the nite control, c 8 O.H.Ibarra is the input alphabet, is the output alphabet, ; is the memory or worktape alphabet, c / is the left end marker for the input, $ is the right end marker for the input, q0 is the initial state in which the machine begins the computation, is the program of the machine consisting of a nite set of rules. Programming this machine is done by using rules of the following form. (q a ) ! (q0 d 0 b) q 2 fq1 q2 ::: qk g is the current state. a 2 is the current symbol under the ROH. is a \local portion" of the auxiliary memory. q0 2 fq1 q2 ::: qk g is the next state. d is the direction of the ROH movement (;1, 0, or +1). 0 is the change in the local portion of the auxiliary memory. b is the output symbol written on the WOH (could be ). For example, if the auxiliary memory is a pushdown stack then machine is a pushdown automaton, and a possible rule is (q5 0 a) ! (q3 +1 pop ). 1.2 Examples of Machines A simpli ed version of the general machine is shown in the gure below. We have combined the input tape and the memory to form an in nite tape in one direction which initially holds the input. The input tape is now a 2-way read/write tape. Let us now consider some speci c examples of machines. Example 1 If the machine has no memory and a 1-way input, then it is called a 1-way Finite Automaton or 1FA. Finite automata are simple machines that can recognize Regualar Expressions. The working of Finite Automata can also be described by state diagrams. Consider the following example of a binary adder which adds two binary numbers. The numbers are represented in binary and are given to machine as input. The machine works correctly only if the digits are in least signi cant c 9 O.H.Ibarra c / a1 a2 a3 ... $ 2-way read/write head Finite State Control {q1 , q 2, ..., q k} 1-way write only head Output tape( 1-way write only tape) Figure 1.3: A Simpler Model of a Computing Machine c / a1 a2 a3 ... $ 1-way read-only head Finite State Control {q1 , q 2, ..., q k} 1-way write only head Output tape( 1-way write only tape) Figure 1.4: A Model of a 1-way Finite Automaton. End markers are not needed for one-way FA's. c 10 O.H.Ibarra bit rst order. The input alphabet consists of pairs of bits, one for each number: P = f(0 0) (0 1) (1 0) (1 1)g The output alphabet consists of bits: = f0 1g. Finite automata can be described by state diagrams such as the one shown in Figure 1.5 for the binary adder. (1,1)/0 (0,0)/0 (0,1)/1 (1,0)/1 q0 (1,0)/0 (0,1)/0 (1,1)/1 q1 (0,0)/1 Start Figure 1.5: State Diagram for a Binary Adder Some FA have no output, and are called pattern recognizers. Given an input string the machine will enter an accept or a reject state after processing the input string, depending on whether the input string is part of the language that the machine recognizes. An example is a machine which is given a sequence of Heads (H ) and Tails, (T ) and it accepts the string only if every Head is followed by at least 2 Tails. Figure 1.6 shows the state diagram for this machine. T T q0 T q1 H q2 T q3 H H Start dead T,H Figure 1.6: State Diagram of a Pattern Recognizer P The machine can be described as A =< Q q0 F >, where the input alphabet of the P machine, = fH T g, Q = fq0 q1 q2 q3 deadg is the set of states, q0 is the initial state, F = fq1g is the set of nal or accepting states and the transition function, is de ned by the diagram. c O.H.Ibarra 11 1.3 Non-determinism Non-determinism is the mathematical model for parallel computation. In terms of machines, nondeterminism introduces choices. The translation rules, , or program of the machine describes the next state of the machine given the current state. In a deterministic machine there is no choice of the next state. For a non-deterministic machine, in contrast, the machine can have more than one choice of the next state. Due to this choice, it is not clear which of the multiple paths available will be followed in any given execution of the machine. The machine can potentially produce di erent results in separate executions starting with the same input. For non-deterministic machines that accept or reject the input strings, a string is said to be accepted by the machine if there is at least one execution that accepts the string. Therefore, if a given execution of a non-deterministic machine results in the rejection of the input, it is not necessarily true that the string is rejected by the machine. 1.3.1 Finite Automata The simplest machines are those with no memory (actually, they have a xed amount of memory which cannot be increased, i.e., the memory is independent of the input size). They are called nite-state machines. A special case is a nite automaton (FA). FA's are pattern recognizers. Figure 1.7 shows the model. An FA can be formally described as M =< Q q0 F >, where Q is the nite set of states, is the input alphabet, is the transition function which de nes the actions of the FA, q0 is the initial state and F is the set of accepting or nal states. The input head can be either 1-way or 2-way, and the transition function may (in the case of non-deterministic) or may not (in the case of deterministic) allow multiple moves from a state on the same input. These two options result in four varieties of FA depending on whether they are 1-way or 2-way and deterministic or non-deterministic. (The 2-way varieties are provided input delimiters.) The four varieties are called 1DFA, 1NFA, 2DFA, 2NFA. All four varieties of FA are equivalent in their recognizing power. In other words, any language that can be accepted by an FA of any variety, can also be accepted by some FA of any of the other varieties. Therefore we see that for memoryless machines, having a 2-way head or non-determinism does not result in any gains. The languages accepted by FA are known as Regular Sets and can be described using Regular Expressions. It is important to note that even though all four varieties of FA are equivalent c 12 O.H.Ibarra a1 a2 a3 a4 ... 1-way read-only head Finite State Control {q1 , q 2, ..., q k} Figure 1.7: A Model of a Finite Automaton in terms of the language that they can recognize, some are more e cient (\succinct") than others. In particular, it can be shown that there are certain languages that can be accepted by an O(n) state 1NFA but requires at least O(2n ) states by any 1DFA accepting the same language. Consider the following set of languages. For every positive integer n, let Ln = fx1y j x in f0 1g jyj = n ; 1g. Ln can be accepted by a 1NFA with n + 1 states as follows 0,1 1 0,1 0,1 .... 0,1 n-1 states Start For example, L2 can be accepted by the following 1NFA. 0,1 1 0,1 Start Ln can be accepted by a 2DFA with O(n) states. c 13 O.H.Ibarra Claim: Any 1DFA accepting Ln needs at least 2n states. Proof...
View Full Document

{[ snackBarMessage ]}