This preview shows page 1. Sign up to view the full content.
Unformatted text preview: Chapter 1
Introduction
Theory of computation is an attempt to formalize the notion of computation, i.e., the notion of an
algorithmic process. Computational Complexity aims to determine the solvability/unsolvability of
computational problems, to develop techniques for discovering e cient algorithms, and to explain
why some problems are inherently di cult in terms of the time and/or space they take to solve.
For example, the problem of sorting numbers by comparison, can be solved in O(n log n) time
by using merge sort. Since this bound has been achieved, it can be viewed as an upper bound the question that remains is  can we do better? It has been shown that if n numbers are sorted
using comparisons, then we need at least O(n log n) steps. Hence, for this problem the lower bound
has been established and realized, and merge sort is an asymptotically optimal comparisonbased
algorithm.
Consider the problem of multiplying two n n matrices. A naive algorithm requires O(n3 )
operations. Is this the best we can do? Other more sophisticated algorithms are known which take
less time: there is an algorithm to multiply two n n matrices using O(n2:376:::) operations. Can
we do better? The answer to this question is not known since no nontrivial lower bound has been
established for matrix multiplication. A trivial lower bound is O(n2 ).
The partition problem is de ned as follows. Given a set of positive integers, c1 c2
cn , is it
possible to partition them into two sets, P1 P2 , such that the sum of the integers in each set is the
same i.e.,
X
X
ci =
ci ?
ci 2P1 ci 2P2 Although this looks like a simple problem, all known algorithms require exponential running time.
This is an example of a what are called NPcomplete problems.
Finally let us consider Hilbert's Tenth Problem (HTP) and its application to the equivalence
problem for programs. De nition 1 Hilbert's Tenth Problem (HTP):
Given: A polynomial p(x1 x2 xn) of n variables and integer coe cients (e.g. p(x1 x2 x3 ) =
1 c 2 O.H.Ibarra 45x2 x2 ; 3x1 x3 + x2 ; 5).
1
2
Question: Does p(x1 x2
that p(a1
an ) = 0)? xn) have an integral root (i.e. are there integer values a1 an such Is HTM decidable? I.e. does there exist an algorithm which when given an arbitrary polynomial p
will answer Yes if p has an integral root and No otherwise?
Yes (p has an integral solution)
p
ALGO
No (p does not have an integral solution) If p is over a single variable then it is decidable. Why? The problem was posed in 1900 and
in 1970 it was shown to be undecidable, i.e., it was shown that no algorithm exists. It should also
be noted that the result holds for solutions over the set of nonnegative integers, f0 1 2 g. The
proof of this is left to the reader. Hint: To show that an algorithm (i.e. decidability) for nonnegative integers case implies an algorithm (decidability) for the integers case, look at all possible
combinations of positive and negative values for the variables. For the converse, you'll need the
triangular integer hint: Any nonnegative integer can be represented as the sum of 3 triangular
numbers. n is a triangular number if n = i(i+1) , for some integer i (+,, or 0).
2
Two applications of the undecidability of HTP are now presented.
1. Given: two nonlooping programs (i.e., programs without loops in some programming language) P1 and P2 .
Question: Are they equivalent, i.e., is P1 P2? This is the problem of Equivalence of
Programs.
We show that the equivalence of programs is undecidable (over the integral domain).
Proof: Suppose the equivalence of programs is decidable, i.e., 9 an algorithm for it. Then we
will show that there is an algorithm for HTP, which is a contradiction since HTP is known
to be undecidable.
Suppose p(x1
xn) is a polynomial. We de ne the following two programs: P1 : input(x1
xn)
y p(x1
xn ) /* use +,,* to compute the value of p and store in y */
if y 6= 0 then output(`0') end else output(`1') Hence P1 will always output `0' if p(x1 xn) has no solution. c 3 O.H.Ibarra P2 : input(x1 xn ) output(`0')
end Thus we see that P1 P2 if and only if p(x1 xn ) has no solution. Corollary 1 6 9 an algorithm (i.e. it is unsolvable) for program optimization.
Proof: Suppose 9 an algorithm for program optimization. Given a polynomial p(x1 xn), we construct the program P1 as described above. We give this program as input to the
optimization algorithm which gives as output the program P1optimal , the optimized version of
P1 . Now P1optimal will be the same as P2 if and only if p(x1
xn) has no solution. Hence
using the optimization algorithm, we can solve HTP  which is a contradiction. Hence no
algorithm for optimization exists.
2. The second application is in Number Theory. A logician named Alonzo Church showed that
there is no algorithm (i.e., it is undecidable) that can decide, in general, whether statements
in number theory are true or false. To be more precise: De nition 2 Let the universe (or domain) be the set of nonnegative integers, N = f0 1 2 3 g.
Statements (or formulas) are built using operations + , quanti ers 9 8, connectives _ ^ : over the universe N and variables x1 x2 . Examples:
8x9y x + x = y]  this is a True statement.
9y8x x + x = y]  this is a False statement.
8q9p8x y p > q ^ (x y {z 1 ) xy = p}]  this is a True statement (it states that there
>
6
can be rewritten using only_^ and : are an in nite number of primes). The formulas can be written such that all quanti ers appear in front of the formula such as: F = Q1 x1 Q2 x2 Qn xn ] where Qi is either 9 or 8, is a formula without quanti ers over variables x1 xn . Theory(N + )
is the set of all true formulas over N using only the operators + and . Theory(N +) is the
set of all true formulas over N using only the operator +.
Claim: Theory(N + ) is undecidable, i.e., 9 no algorithm to decide given a formula, F ,
in Theory(N + ) whether F is true or false. (Thus some theorems or true statements in
Theory(N + ) are not provable.) In other words, theorem proving cannot be \mechanized". c 4 O.H.Ibarra Proof: Given a polynomial p(x1 xn), let p1 (x1
xn) be the positive terms and p2(x1
be the negative terms. Then de ne the formula F to be:
F = 9x1 9x2 9xn p1(x1 xn ) = p2 (x1 xn)] E.g. For p = 45x2 x2 ; 3x1 x3 + x2 ; 5, p1 = 45x2 x2 + x2 and p2 = 3x1 x3 +5. The corresponding
1
2
1
2
formula would be:
9x19x29x3 45x2 x2 + x2 = 3x1 x3 + 5]
1
2
which can be simpli ed by introducing more variables. For example: 9x19x29x39z 45x2 x2 + x2 = z ^ z = 3x1x3 + 5]
1
2
9x19x29x39z9w 45x2 x2 + w = z ^ z = 3x1 x3 + 5 ^ w = x2 x2 ]
1
Now F is true if and only if p has a solution. Thus if there exists an algorithm to determine
whether F is true or false, then we can use it to determine whether p has a solution, which is a
contradiction. Hence there is no algorithm to decide the validity of formulas in Theory(N,+,*).
Claim: Theory(N +) is decidable.
Proof: (Sketch) Consider the formula F = 9x8y9w9zE , where E is the expression (x + x =
y) ^ (x + y = z ) _ :(z = w)] Suppose we want to check whether F is true. Denote by: E1 the subexpression x + x = y
E2 the subexpression x + y = z
E3 the subexpression z = w
De ne the following languages: L1 = f(x y z w)jx y z w in N x + x = yg
L2 = f(x y z w)jx y z w in N x + y = z g
L3 = f(x y z w)jx y z w in N z = wg
x y z w will be represented in binary with leading ( rst) bit the LSB. So, for example,
x = 5 is 1010
y = 7 is 1110
z = 3 is 1100
w = 12 is 0011 xn ) c 5 O.H.Ibarra Note that we can assume that the binary numbers have the same length. Thus, the tuple
(5 7 3 12) in binary can be represented as (1 1 1 0)(0 1 1 0)(1 1 0 1)(0 0 0 1). Thus the
binary representation of (5 7 3 12) is a string over the alphabet, 4 = f(s1 s2 s3 s4 )jsi 2
f0 1gg The \binary" representation (version) of L1 is the language over the alphabet 4:
f(a0 b0 c0 d0 )(a1 b1 c1 d1 ) (an bn cn dn )jx = a0 an y = b0 bn z = c0 cn w =
d0 dn are binary strings and x + x = yg.
For notational convenience we will also use L1 to denote its binary representation.
We can construct a DFA A1 to accept L1 . A1 simply reads the input string and performs the
binary addition (in this case adding x to itself) and checking that the binary sum is equal
to y. A1 only looks at the rst and second tracks and ignores the third and fourth tracks.
Similarly, we can construct DFA's A2 and A3 to accept L2 and L3 , respectively.
Now let E4 be the expression :E3 , then the language L4 corresponding to E4 is the complement of L3 . We can construct from A3 a DFA A4 accepting L4 .
Let E5 be the expression E2 _ E4 . Clearly, L5 = L2 L4 . We can construct from A2 and A4
a DFA A5 to accept L5 .
Finally, let E6 (this is E ) be the expression E1 ^ E5 . Then L6 = L1 \ L5 . Again, we can
construct from A1 and A5 a DFA A6 to accept L6 .
Now consider the expression 9zE6 . Call this expression E7 . Then L7 = f(x y w)j9z such that(x y w z) 2 L6 g
+
Note that now L7
3 , where alphabet 3 = f(s1 s2 s3 )jsi 2 f0 1gg. We can construct an
NFA B7 to accept L7 .
Note that the input to B7 has only three tracks (corresponding to x y w)  the track corresponding to z is not part of the input. B7 is constructed from A6 . B7 \simulates" the
computation of A6 , but \guesses" (using nondeterminism) the bits of z as it reads the input.
B7 is then converted to a DFA A7 .
We can do the same thing for the expression E8 = 9w9zE . We can construct from A7 a DFA
A8 to accept L8 +, where now 2 = f(s1 s2)jsi 2 f0 1gg. For E9 = 8y9w9zE , we note
2
+
that this expression is equivalent to E9 = :9y:(9w9zE ). The language L9
1 , where
1 = f(s1 )js1 2 f0 1gg. We can construct a DFA A9 to accept L9 as follows: Construct a DFA B8 from A8 that accepts the complement of L8 . Thus B8 accepts the
language corresponding to :(9w9zE ).
Next construct from B8 a DFA C8 to accept the language corresponding to 9y:(9w9zE )
(we already know how to do this!). c 6 O.H.Ibarra Then construct from C8 a DFA A9 to accept the complement of the language accepted
by C8 . Thus A9 accepts the language corresponding to the expression :9y:(9w9zE ).
Finally, we construct from A9 a DFA A10 to accept the language L10 corresponding to the
+
formula F = 9x8y9w9zE . Note that L10
0 , where 0 = f()g, where () is a symbol
corresponding to an empty tuple. It is easily veri ed that A10 accepts any input if and only
if F is true. So all we need to check is whether A10 accepts the null string .
The constructions in the example above can be formalized to prove the claim. 1.1 Machine Models
We can classify algorithms in terms of thier resource (memory) requirements:
A. Fixed memory procedures.
Some procedures use only a xed amount of memory, independent of the size of the input
(i.e. the device carrying out such a procedure only needs a xed amount of memory).
For example, the addition of two integers represented in binary.
(a1 ,b1 ), (a2 ,b2 ),....,(an,bn )  Device doing
Addition  s1,s2 ,...,sn This device needs only 1 bit of memory for the carry bit.
B. Unbounded memory procedures.
Some procedures use memory that grows with the input size (i.e. the device carrying out such
a procedure needs an unbounded memory).
For example, the multiplication of two integers represented in binary.
(a1 ,b1 ), (a2 ,b2 ),....,(an,bn )  Device doing Multiplication  p1,p2,...,pn Figure 1.1:
This device needs memory that is a function of n.
Access to memory may be restricted or unrestricted. Some examples of restricted memory
access machines are pushdown stacks, queues, and dequeues. Some examples of unrestricted
memory access machines are disks, drums, tape units, and RAMs. c 7 O.H.Ibarra C. A general machine.
We now introduce a general model of computation. By restricting this machine in various
ways, we get machines such as the Turing machine, pushdown automaton, and nite automaton. For a general model, we need to have the following components: {
{
{
{ Input/Output
Potentially unbounded memory
CPU
Program
Input tape (2way read only tape)
c a1 a2 a3 ... an $ 2way read only head
Finite State
Control
{q1 , q 2, ..., q k} 1way write only head Output tape( 1way write only tape)
Auxiliary
Memory Figure 1.2: A General Model of a Computing Machine
The above gure shows the components of such a model. The tapes are divided into cells
which the control can read one at a time. The nite state control represents the CPU and
program components. This control has access to the input tape which holds the input to
the machine. This tape is a read only two way tape. In other words, the head can move in
both directions  left and right or remain at the same cell, but it cannot modify the contents
of any cell. The nite control also has access to the an auxillary memory which models the
unbounded memory of the machine. This memory may be organized in various ways such
as a pushdown stack or a queue or in nite length tapes depending upon the machine. The
control also has access to an output tape onto which it writes the output. This output tape
is one way, i.e. the control cannot move back on this tape once it has written a cell  it only
moves forward or remains at the same cell.
The machine shown above can be described as M =< Q
; / $ q0 >. Where
c
Q is the set of states of the nite control, c 8 O.H.Ibarra is the input alphabet,
is the output alphabet,
; is the memory or worktape alphabet,
c
/ is the left end marker for the input,
$ is the right end marker for the input,
q0 is the initial state in which the machine begins the computation,
is the program of the machine consisting of a nite set of rules.
Programming this machine is done by using rules of the following form.
(q a ) ! (q0 d 0 b) q 2 fq1 q2 ::: qk g is the current state.
a 2 is the current symbol under the ROH.
is a \local portion" of the auxiliary memory.
q0 2 fq1 q2 ::: qk g is the next state.
d is the direction of the ROH movement (;1, 0, or +1).
0 is the change in the local portion of the auxiliary memory. b is the output symbol written on the WOH (could be ).
For example, if the auxiliary memory is a pushdown stack then machine is a pushdown
automaton, and a possible rule is (q5 0 a) ! (q3 +1 pop ). 1.2 Examples of Machines
A simpli ed version of the general machine is shown in the gure below. We have combined the
input tape and the memory to form an in nite tape in one direction which initially holds the input.
The input tape is now a 2way read/write tape.
Let us now consider some speci c examples of machines.
Example 1 If the machine has no memory and a 1way input, then it is called a 1way Finite
Automaton or 1FA.
Finite automata are simple machines that can recognize Regualar Expressions. The working
of Finite Automata can also be described by state diagrams. Consider the following example of
a binary adder which adds two binary numbers. The numbers are represented in binary and are
given to machine as input. The machine works correctly only if the digits are in least signi cant c 9 O.H.Ibarra c
/ a1 a2 a3 ... $ 2way read/write head
Finite State
Control
{q1 , q 2, ..., q k} 1way write only head Output tape( 1way write only tape) Figure 1.3: A Simpler Model of a Computing Machine c
/ a1 a2 a3 ... $ 1way readonly head
Finite State
Control
{q1 , q 2, ..., q k} 1way write only head Output tape( 1way write only tape) Figure 1.4: A Model of a 1way Finite Automaton. End markers are not needed for oneway FA's. c 10 O.H.Ibarra bit rst order. The input alphabet consists of pairs of bits, one for each number:
P
= f(0 0) (0 1) (1 0) (1 1)g
The output alphabet consists of bits:
= f0 1g. Finite automata can be described by state diagrams such as the one shown in Figure 1.5
for the binary adder.
(1,1)/0
(0,0)/0
(0,1)/1
(1,0)/1 q0 (1,0)/0
(0,1)/0
(1,1)/1 q1
(0,0)/1 Start Figure 1.5: State Diagram for a Binary Adder
Some FA have no output, and are called pattern recognizers. Given an input string the machine
will enter an accept or a reject state after processing the input string, depending on whether the
input string is part of the language that the machine recognizes. An example is a machine which
is given a sequence of Heads (H ) and Tails, (T ) and it accepts the string only if every Head is
followed by at least 2 Tails. Figure 1.6 shows the state diagram for this machine. T
T
q0 T q1 H q2 T q3 H H Start
dead
T,H
Figure 1.6: State Diagram of a Pattern Recognizer
P The machine can be described as A =< Q
q0 F >, where the input alphabet of the
P
machine, = fH T g,
Q = fq0 q1 q2 q3 deadg is the set of states, q0 is the initial state, F = fq1g is the set of nal or
accepting states and the transition function, is de ned by the diagram. c O.H.Ibarra 11 1.3 Nondeterminism
Nondeterminism is the mathematical model for parallel computation. In terms of machines, nondeterminism introduces choices. The translation rules, , or program of the machine describes the
next state of the machine given the current state. In a deterministic machine there is no choice of
the next state. For a nondeterministic machine, in contrast, the machine can have more than one
choice of the next state. Due to this choice, it is not clear which of the multiple paths available will
be followed in any given execution of the machine. The machine can potentially produce di erent
results in separate executions starting with the same input.
For nondeterministic machines that accept or reject the input strings, a string is said to be
accepted by the machine if there is at least one execution that accepts the string. Therefore, if
a given execution of a nondeterministic machine results in the rejection of the input, it is not
necessarily true that the string is rejected by the machine. 1.3.1 Finite Automata
The simplest machines are those with no memory (actually, they have a xed amount of memory
which cannot be increased, i.e., the memory is independent of the input size). They are called
nitestate machines. A special case is a nite automaton (FA). FA's are pattern recognizers.
Figure 1.7 shows the model. An FA can be formally described as M =< Q
q0 F >, where Q is the nite set of states,
is the input alphabet,
is the transition function which de nes the actions of the FA, q0 is the initial state and
F is the set of accepting or nal states.
The input head can be either 1way or 2way, and the transition function may (in the case of
nondeterministic) or may not (in the case of deterministic) allow multiple moves from a state on
the same input. These two options result in four varieties of FA depending on whether they are
1way or 2way and deterministic or nondeterministic. (The 2way varieties are provided input
delimiters.) The four varieties are called 1DFA, 1NFA, 2DFA, 2NFA. All four varieties of FA are
equivalent in their recognizing power. In other words, any language that can be accepted by an
FA of any variety, can also be accepted by some FA of any of the other varieties. Therefore we
see that for memoryless machines, having a 2way head or nondeterminism does not result in
any gains. The languages accepted by FA are known as Regular Sets and can be described using
Regular Expressions. It is important to note that even though all four varieties of FA are equivalent c 12 O.H.Ibarra a1 a2 a3 a4 ... 1way readonly head
Finite State
Control
{q1 , q 2, ..., q k} Figure 1.7: A Model of a Finite Automaton
in terms of the language that they can recognize, some are more e cient (\succinct") than others.
In particular, it can be shown that there are certain languages that can be accepted by an O(n)
state 1NFA but requires at least O(2n ) states by any 1DFA accepting the same language.
Consider the following set of languages. For every positive integer n, let Ln = fx1y j x in f0 1g jyj =
n ; 1g.
Ln can be accepted by a 1NFA with n + 1 states as follows 0,1
1 0,1 0,1 .... 0,1 n1 states Start
For example, L2 can be accepted by the following 1NFA. 0,1
1 0,1 Start
Ln can be accepted by a 2DFA with O(n) states. c 13 O.H.Ibarra Claim: Any 1DFA accepting Ln needs at least 2n states.
Proof: Assume there is a 1DFA A accepting Ln. Let w be any binary string of length n, i.e.,
jwj = n. 6 w  s(w) = state A enters after processing w q0 Subclaim: For any w and w0 s.t. jwj = jw0 j = n and w 6= w0 s(w) 6= s(w0).
Proof: Suppose not, i.e., s(w) = s(w0 ) = p 6 w q0 6
q0 6
s(w) = p w0 6
s(w0 ) = p Since w and w0 are di erent, they disagree in at least one bit position. Let the point of
00
discrepancy be at position k + 1 from the right, k 0. Thus, w = w1 1w2 and w0 = w1 0w2
0
where w2 and w2 are of length k.
Now consider 2 new strings. w = w0n;k;1 = w1 1w2 0n;k;1
00
w0 = w0 0n;k;1 = w1 0w2 0n;k;1 Since the machine will be in the same state (s(w) = s(w0 )) after reading the rst n symbols
of both w and w0 , the remaining symbols for both are the same (0n;k;1 ), and we have a
deterministic machine, A will accept both w and w0 . But this is impossible since w0 is not in
Ln . It follows that s(w) 6= s(w0 ). Since there are 2n distinct strings of length n, there are 2n
distinct s(w)'s. Hence A must have at least 2n states. c 14 O.H.Ibarra Note that Ln can be accepted by a twoway deterministic nite automaton, (2DFA) with O(n)
states. Thus 2DFA's are more succint than 1DFA's.
Here is another example.
Consider the language: L = fx#xjx is a binary string and jxj = ng.
It is easy to construct a 2DFA with O(n) states accepting L. Claim: Any 1NFA accepting L needs at least 2n states.
Proof: Assume there is a 1NFA A accepting L with fewer than 2n states. Let w1 be any
binary string of length n, i.e., jw1 j = n. Suppose the word w1 #w1 is given as input to A. Let
1 be the state after A has read the symbol #.
Let w2 be any binary string of length n di erent from w1 , i.e., w1 = w2 . Suppose the word
6
w2 #w2 is given as input to A. Let 2 be the state after A has read the symbol # with this
input. Subclaim: 1 6= 2 .
Proof: Suppose that = 2 . Then consider the behavior of A when given the string w1 #w2
as input. Clearly, after reading the symbol #, A will be in state 1 . Since 1 = 2 , we see
that the state of the machine is identical to the state when w2 #w2 is processed, after reading
#. Hence A would also accept the string w1 #w2 , which is clearly not in the language. This
is a contradiction. Hence our subclaim is true.
1 Therefore for each distinct wi of length n, we must have a distinct state i that is reached
after reading the symbol #. Since there are 2n distinct binary strings of length n, we must
have at least 2n di erent states in A.
The above two proofs make use of what is called the CutandPaste Argument. 1.3.2 Pushdown Automata
These machines are similar to the Finite Automata, except that they have an additional \pushdown
stack" on which an unlimited amount of information can be stored but which operates on a lastinrstout manner. The stack can only be accessed using the PUSH and POP operations. As with
FA, there are four varieties of PDAs: 1DPDA, 2DPDA, 1NPDA and 2NPDA. Unlike the FA, these
four varieties of PDA do not have the same recognizing power. In particular, 1NPDAs are more
powerful than 1DPDAs because they can recognize the language L = fxxR jx is a binary string g. c 15 O.H.Ibarra i
{0 i 1i 0 i > 0} 2NPDA 1NPDA
{xxR} 2DPDA 1DPDA i
{0 i 1i 0 i > 0} FA {0 i 1i i > 0} Figure 1.8: Relationships between the four PDA varieties
Also, the 2way varieties can recognize the language L = f0i 1i 0i ji > 0g which cannot be recognized
by the 1way varieties. The relationship between the four varieties is summarized in Figure 1.8.
A general result regarding PDAs is: Any language that can be recognized by a 2DPDA can be
recognized by a RAM (Random Access Machine) in linear time. 1.3.3 Linear Bounded Automaton
These machines are similar to 2way FA, except that they can overwrite the input. The ability
to overwrite is useless if the overwritten cell is never visited again, hence, the 1way varieties of
LBA are uninteresting. Thus there are two interesting varieties of LBA  2way Deterministic LBA
or 2DLBA and 2way Non deterministic LBA or 2NLBA. The term Bounded is important and it
captures the fact that the machine is not allowed to write beyond the given input. The relationship
between 2DLBAs and 2NLBAs is an open question.
For a DLBA, there is an upper bound on the number of steps that it can take if it halts (i.e.
it is not looping): s tn (n + 2), where s is the number of states of the nite control, t is the size
of the alphabet and n is the length of the input string (the `2' is because of input delimiters). The
proof is left to the reader. 1.3.4 Expanding LBA or Turing Machines
If we allow an LBA to write beyond the input string, then it becomes equivalent to a Turing
Machine. A (Basic) Turing Machine consists of a nite control, a 2way read/write input head
where the input tape is in nite in one direction. Figure 1.9 shows the model for the basic Turing
Machine.
The basic Turing Machine can also be viewed as a generalization of a 2way deterministic nite
automaton. We need to make two extensions to the 2DFA: c 16 O.H.Ibarra a1 a2 . . . ai . . . an λ λ λ . . .
2way R/W head
Finite
control λ=Blank Figure 1.9: Model of the Basic Turing Machine
1. The input head allowed to modify the input tape.
2. The input tape is made in nite to the right.
The Turing Machine (TM) can therefore overwrite the input and can also use an arbitrarily
large number of cells on the right hand side of the input tape. Initially, the input tape contains
the input beginning at the leftmost cell and the rest of the tape contains blanks, denoted by ,t or
B . The head can move in both directions on the input tape. A TM can be described as a 8tuple:
M =< Q P ; q0 qaccept qreject >, where,
Q is the set of states in the nite control,
P
is the input alphabet,
P
; is the tape alphabet (it includes , the blank symbol, and possibly other symbols.),
is the transition function that governs the working of the TM it maps Q ; ! Q ; f+1 ;1g
.
Let us now see an example of Turing machine construction:
TM (basic model) to recognize (accept) the language L = f 0i 1i j i 1 g
Note that in the following, the subscripts of states are formed by the letters o and e, which
stand for odd and even, and denotes the blank symbol.
= f 0, 1 g, Q = f s, qo , qe , qoo , qoe, qeo , qee, r, f , qaccept g, ; = f 0, 1, X, X, g,
F = f qaccept g
(s, 0) = (qo , X, +1)
(qo , 0) = (qe , 0, +1)
(qo , X) = (qo , X, +1)
(qe , 0) = (qo , X, +1)
(qe , X) = (qe , X, +1)
(qo , 1) = (qoo , X, +1)
(qoo , 1) = (qoe , 1, +1)
(qoo , X)= (qoo , X, +1)
(qoe , 1) = (qoo , X, +1) c O.H.Ibarra 17 (qoe , X)= (qoe , X, +1)
(qe , 1) = (qeo , X, +1)
(qeo , 1) = (qee , 1, +1)
(qeo , X)= (qeo , X, +1)
(qee , 1) = (qeo , X, +1)
(qee , X) = (qee , X, +1)
(qoo , ) = (r, , 1)
(qee , ) = (r, , 1)
(r, a) = (r, a, 1) for a = 0, 1, X
(r, X) = (f , X, +1)
(f , X) = (f , X, +1)
(f , 0) = (qo , X, +1)
(f , ) = (qaccept , , 1)
The TM works as follows: It makes repeated scans of the input. In each scan it marks o
alternate 0's and 1's keeping track of the parity of each. If the parity of 0 is di erent from
the parity of 1 in any scan, it rejects. If the parities are equal in every scan then the lengths
of the 0s and 1s must be equal, hence it accepts. Since in each scan the number of unmarked
symbols is halved, the TM makes O(log n) scans. In each scan the complete input is read.
Clearly, on an input x of length n, the TM takes at most O(n log n) steps to verify that x is
in the language L.
Turing Machines can be viewed as
1. Pattern Recognizers or Language Acceptors
2. Function Computers
Note that in the de nition of the basic TM, we have not allowed the input head to remain stationary  the only allowed directions for the head are +1 or 1 i.e. right or left only. This is however
only a super cial limitation since we can simulate the stationary behaviour as a combination of a
left and right move. Speci cally,
(q a) = (p b 0) (q a) = (p0 b + 1) (p0 x) = (p x ;1) 8x 2 ; 1.4 A Programming Language for TMs
A TM can be represented as a program using the following constructs. Note that the registers
referred to in the constructs can only store 1 symbol a is any symbol in the worktape alphabet. c 18 O.H.Ibarra read R
write a
left
right Ra
R1 R2
if R1 = R2 goto l goto l
accept
reject
halt read symbol under the read/write head and store in register R.
write the symbol a under the read/write head.
move read/write head 1 cell (square) to the left.
move read/write head 1 cell to the right.
store symbol a to register R.
copy symbol in register R2 to register R1 .
l is a label. Note that the following instructions can easily be simulated in the TM programming language: if R1 6= R2 goto l
if R = a goto l
if R = a goto l
6
while R1 = R2 do
while R1 = R2 do
6
while R = a do
while R = a do
6
Here an example:
TM program to recognize the language L = f x j x in f0 1g , x = xR g
1
2 read A
if A = goto 4
write
right
read B
if B = goto 2
6
left
read C
if C = goto 4
if C = A goto 5
6
write c 19 O.H.Ibarra 3 4
5 left
read B
if B = goto 3
6
right
goto 1
accept
halt
reject
halt 1.5 Equivalence of Programs and TMs
Claim: The program speci cation and the speci cation for Turing machines are the same.
Proof: The proof is done in two parts. First we show that the program speci cation can be simulated by the speci cation then we show that the speci cation can be simulated by the
program speci cation.
Program speci cation can be simulated by speci cation:
Suppose we are given a TM M speci ed by a program P using the constructs described above, and
P has k instructions (lines) i1 , i2 , . . . , ik and m registers r1 , r2 , . . . , rm .
We will construct an equivalent TM M 0 described by speci cation as follows.
1. The states of M 0 have the form hc i1 i2 : : : ik r1 r2 : : : rm i, where
(a) c is the program counter, indicating the next instruction to be executed
(b) i1 , i2 , . . . , ik are the instructions (these stay the same)
(c) r1 , r2 , . . . , rm are the current values of the registers of P
2. The starting state of M 0 is h1 i1 i2 : : : ik : : : i. 3. The accepting states of M 0 have the form hc i1 i2 : : : ik r1 r2 : : : rm i, where ic is an accepting instruction.
4. The transition rules have the form:
0
0
(hc i1 : : : ik r1 : : : rm i s) = (hc0 i1 : : : ik r1 : : : rm i s0 d) where
(a) c0 = l if ic is a branch statement to l and the condition is true c 20 O.H.Ibarra (b) c0 = c + 1 if ic is not a branch statement, or ic is a branch statement and the condition
is false
(c) d = ;1 (+1) if ic : left (right) 0 otherwise
0
(d) rj = s if ic : read rj
0
(e) rj = a if ic : rj a
0
(f) rj = ri if ic : rj ri
(g) s0 = a if ic : write a
It is easy to verify that M 0 is equivalent to M .
speci cation can be simulated by program speci cation: Suppose we are given a TM M speci ed
by a transition function , and M has s states q1 q2 : : : qs , and m tape symbols t1 t2 : : : tm .
(Hence there are at most sm transition rules, since some transitions may not be de ned.) We will
construct an equivalent program P using the constructs given above as follows.
P has s + m + 2 registers: Q1 Q2 : : : Qs , T1 T2 : : : Tm , STATE and SYMBOL. Intuitively, at
any given time STATE and SYMBOL hold the current state and symbol of M .
P starts by initializing the registers: Q1 q1 Q2 q2 :::
Qs qs T1 t1 T2 t2 :::
Tm tm STATE q1 0: read SYMBOL
Next P identi es the transition rule to execute next and then jumps to the appropriate code
segment: c 21 O.H.Ibarra if STATE = Q1 goto 1
if STATE = Q2 goto 2
::: if STATE = Qs goto s
1: if SY MBOL = T1 goto 1,1
if SY MBOL = T2 goto 1,2
::: if SY MBOL = Tm goto 1,m
::: s: if SY MBOL = T1 goto s,1
if SY MBOL = T2 goto s,2
::: if SY MBOL = Tm goto s,m
i,j: <do the transition rule for (qi tj ) >
< assume (qi tj ) =< q t d > if unde ned, then halt > write t
SY MBOL
STATE t
Qq right <if d = +1 >
left <if d = ;1 >
accept <if q is an accepting state>
reject <if q is a rejecting state>
goto 0 <otherwise, do next move>
It is easy to verify that P is equivalent to M . c 22 O.H.Ibarra 1.6 Some De nitions
Turing machines can be viewed as Language Recognizers because when given input they either
accept, reject or go into an in nite loop. Based on the type of the Turing machine that accepts a
language, the following terms are de ned
Languages that are accepted by Turing machines that always halt (i.e. never go into an
in nite loop) are called Decidable or Recursive (R) languages. The TMs that accept
these languages are called deciders.
Languages that are accepted by Turing machines which may not halt for all inputs are called
Recursively Enumerable (RE).
The set of Recursive languages is properly included in the set of Recursively Enumerable languages,
i.e., R RE .
Turing machines can also be viewed as function computers. Consider for example a Turing
machine that computes the function f (x y) = x + y when given the inputs x and y separated by a
# sign on the input tape. If the inputs are in unary and the output is to be in unary, the algorithm
is trivial  one needs to just replace the # symbol with a 1 and then replace the rightmost 1 with
a blank. If on the other hand, the inputs are in binary and the output is to be in binary also, a
more complex algorithm is required.
As another example, consider the monus function:
: f (x y ) = x ; y = ( x ; y if y > 0
0 otherwise
If the input is in unary, with x followed by y separated by # symbol. The monus can be
constructed by replacing one '1' of x for every '1' of y. If the symbols of y are exhausted before
those of x, the answer is given by the remaining size of x. If the symbols of x are exhausted rst
or at the same time, the answer is 0.
A Turing machine could also be viewed as a transformer, such as one that takes unary input
and produces binary output. 1.7 Variants of Turing Machines
There are several varieties of Turing Machines, all of which have the same computing power. 1.7.1 2track Tape
The BTM has a single track tape  only one symbol can be stored in each cell. A 2track tape is
one which stores two symbols per cell. Would a TM with a 2track tape be more powerful than the c 23 O.H.Ibarra a1 a2 . . . ai . . . an λ λ λ . . .
... λ λ ...
λ λ ...
2way R/W head
Finite
control Figure 1.10: 2track Tape Turing Machine
BTM? We can simulate a 2track TM with a single track TM as follows. Given a machine A that
has two tracks, we construct a machine B with one track that simulates the working of A. Let ;
be the tape alphabet of machine A. Then the tape alphabet for B is ;0 = ; ;. In other words,
we consider the each cell of the 1track tape of B to hold a composite symbol which consists of two
symbols of the tape of A, one symbol for each track. The transition rules of B are derived from
those of A by appropriate replacement of the symbols.
We therefore see that having 2 tracks does not increase the ability of the TM to recognize
languages. However, the 2track tape can make the programming of a TM easier, as in the following
example. 1.7.2 2way In nite Tape
The tape of the BTM is in nite only in one direction (to the left). Would a TM with a tape that
is in nite in both directions be more powerful?
. . . λ a1 . . . ai . . . an λ λ λ . . .
2way R/W head
Finite
control Figure 1.11: 2way In nite Tape Turing Machine 1.7.3 Other Varieties
There are other variations of TM's: multitape TM, multidimensional TM, TM with several heads
per tape, etc. 1.8 Grammars
A Grammar is formally de ned as G =< N T P S >, where, N is a set of symbols called nonterminals, T is a set of symbols called terminals, P is the set of rules or productions and S is a c O.H.Ibarra 24 symbol from the set N called the start symbol. The productions (or rules) given in the set P are of
the form ! , where
can be any string of terminals and nonterminals with containing at
least one nonterminal. The rules allow the occurrence of the string to be replaced by the string
. The language de ned by the grammar is the set of strings of terminals that can be generated
by starting with the symbol S and applying the rules in P . By restricting the form that the rules
can take, several types of grammars can be de ned. Some of these grammars correspond to the
models of machines described above, i.e., any language generated by a grammar of a certain type
can be recognized by some machine of the corresponding model and any language recognized by a
machine of a given model can be generated by some grammar of the corresponding grammar. Let
! be any production in the grammar. The following types of grammars are de ned: Type 3 Grammars. There are two types of such grammars  the Left Linear Grammars (LLGs) and the Right Linear Grammars (RLGs). For both these grammars, must consist
of exactly one nonterminal. For LLGs, can be either a terminal string or a single nonterminal
followed by a terminal string, i.e., the productions all look like: A ! Bx or A ! x, where
A B are nonterminals and x is a terminal string (possibly ).
For RLGs, all productions look like: A ! xB or A ! x, where A B are nonterminals and x
is a terminal string. The set of language generated by LLG and RLGs is exactly the set of
language accepted by Finite Automata, i.e., the Regular languages. Thus Type 3 grammars
correspond to FA.
1
(Type 2 2 ) A grammar which has productions of both types above is called a Linear Grammar
and is not a Type 3 grammar. This grammar type can be informally dubbed as a Type 2 1
2
Grammar. An example of such a grammar is:
S ! 0Aj
A ! S1
which generates the language L = f0i 1i ji 0g which is not a regular language. It has been
shown that Linear Grammars correspond to 1reversal 1NPDAs. That is 1NPDAs which may
push symbols onto the stack only as long as they have not popped any symbol from the stack.
Once a symbol is popped, then no more symbols can be pushed onto the stack. Type 2 Grammars. These grammars are also called Context Free Grammars or CFGs. They are characterized by productions that have only one nonterminal symbol on the left hand
side, i.e. j j = 1. These grammars correspond to 1NPDAs. Type 1 Grammars. These grammars are also called Context Sensitive Grammars or CSGs. The only restriction on the productions is that the number of symbols on the left hand side
should be less than or equal to the number of symbols on the right hand side of the production,
i.e. j j j j. These grammars correspond to the 2way Nondeterministic Linear Bounded c 25 O.H.Ibarra Automata or NLBAs. It should be noted that a CSG cannot generate the null symbol .
Therefore, a CSG can generate any language that is generated by a CFG if is not generated
by the CFG. Although parsing of CFGs is not very di cult (it can be done in O(n3 )), parsing
of CSGs is very di cult (all known algorithms require O(cn )) for some constant c. Type 0 Grammars. This is the nal class of grammars and it is unrestricted. This set
corresponds to the Turing Machines or the Expandable LBAs. Table 1.1 summarizes the above information.
Type
Type 0
Type 1
Type 2
1
Type 2 2
Type 3 Name
unrestricted
CSG
CFG
LG
LLG (RLG) Machine
Restriction
Turing Machines
!
NLBA
jj jj
1NPDA
=A
1reversal 1NPDA ! xByjx
FA
= A and ! Byjx ( ! yB jx) Table 1.1: Relationship between Grammars and Machines
There is another class of grammars which corresponds to 1DPDAs, called the LR(k) grammars.
There is no known grammatical characterization for the following models of machines: 2NPDA,
2DPDA and DLBA. 1.9 Exercises
1. Show that Hilbert's Tenth Problem (HTP) over Z , the set of all integers (positive, negative
and 0) is decidable if and only if HTP over N , the set of nonnegative integers, is decidable.
Hint: You might need the fact that every nonnegative integer n is the sum of three triangular
numbers, and conversely. A nonnegative integer k is triangular if k = i(i+1) for some integer
2
i (positive, negative or 0).
2. Show that the language L1 for the expression E1 can be accepted by a DFA.
+
3. Suppose A is a DFA accepting a language L
4 , where 4 = f(s1 s2 s3 s4 )jsi 2 f0 1gg.
+
0
0 consists of all strings in L with the rst component
De ne the language L
3 , where L
deleted. Show how to construct an NFA A0 from A accepting L0 . 4. Describe how to construct, given a polynomial p(x1
xn) with integer coe cients (i.e. an
instance of HTP), a program P with no inputs such that P halts if and only if p(x1
xn )
has a solution. c O.H.Ibarra 26 5. From Problem 4 show that the following problems are undecidable:
(a) Given: Program P and input x.
Question: Will P halt on input x?
(b) Given: Program P and input x and statement label l.
Question: Will P on input x ever execute statement labeled l?
6. Show that HTP when restricted to onevariable polynomials is decidable.
7. Show that HTP when restricted to linear polynomials (over several variables) is decidable.
Hint: Show that given a linear equation, E = a1 x1 + + an xn, the language L(E ) =
f(x1 xn)jxi 2 N s.t. a1 x1 + + anxn = 0g is accepted by a DFA. Just describe
infomally the DFA. Try an example, e.g., 3x1 ; 5x2 + x3 ; 7x4 + 6. ...
View Full
Document
 Winter '08
 Ibarra,O

Click to edit the document details