**Unformatted text preview: **Introduction to Computing CSCI 109 China – Tianhe-‐2 Gaurav S. Sukhatme Spring 2016 Lectures 15-‐16-‐17: Abstract Machines and Theory Feb 24-‐26-‐29, 2016 Abstract machines and theory What is theore9cal computer science? u Finite state machines Reading: u The Turing machine St. Amant Ch. 8 u The Church-‐Turing thesis Gleick Ch. 7 (opLonal) u The hal9ng problem u The analysis of algorithms u The analysis of problems u 1 What is theoretical computer science? The study of what can be computed and how eﬃciently it can be computed u Based on abstract mathema9cal models u Insight into the behavior of a real computer… …any real computer u 2 Finite state machines Up Fed Crying On Down Up Oﬀ Asleep Hungry More hungry Hungry Awake, not crying Down Light switch 1 day old infant 10 3 Finite state machines Add M Add M Add M M=F M+1 Add F Add F Add M F+1 M+2 Add M M+3 M+4 Add F Add F Add F Add F Add F Add F Add M St. Amant pp. 152 Add M F+2 F+3 Add M F+4 Add M Add F 4 Finite state machine limitations If a ﬁnite state machine with n states is given n+1 symbols, it will revisit one state at least once u No way to tell if revisited state is being visited for the 1st 9me or 5th 9me u The pigeonhole principle u Problems an FSM cannot solve: u Given a string of As and Bs, tell if the number of As and Bs are the same v Tell if a string is a palindrome v 5 The Turing machine An inﬁnite tape, with squares marked on it u Each square can be blank or hold a symbol u A machine moves over the tape u When posi9oned over a square, the machine can u v
v
v v u Read a symbol from the tape Erase a symbol from the tape Write a symbol onto the tape (erasing what was on the square) Do nothing The machine can move either one square to right or lec 6 Alan Turing 7 Turing Machine Animation (Vimeo) 8 What can a Turing machine do? Given a string of As and Bs, tell if the number of As and Bs are the same u Move back and forth, erasing AB pairs v #As = #Bs if no extra As (or Bs) lec over v Wri9ng programs for a Turing machine is very tedious u However… u 9 The Church-‐Turing thesis Turing machines are capable of solving any eﬀecLvely solvable algorithmic problem Turing machines are thus excellent models for what (all) real computers are capable of doing 10 Problems not ‘eﬀectively computable’ Are there problems that are not eﬀec9vely computable ? u Yes, inﬁnite such problems u Could these be computable someday ? u Perhaps, but not on a Turing-‐equivalent computer u 11 A halting question Deciding whether a program will halt (terminate) u Does the following program halt ? !
u Input number!
While number is not 0!
Print number!
number = number – 1 !
! 12 The halting problem Deciding whether a given program will halt (terminate) on an arbitrary input u In 1936, Turing proved that a general algorithm to the hal9ng problem for all possible programs over all possible inputs cannot exist u The hal2ng problem is undecidable ! 13 Proof sketch for halting problem u Want to decide if program A halts on input a u Need to write a program, lets call it halter!
Inputs: A, a v Output: Yes (if program A halts on input a) or No (if program A does not halt on input a) v halter(A,a)!
if(some complex computation)is true!
Return Yes!
else!
Return No! 14 The halting problem is undecidable In 1936, Turing proved that a general algorithm to the hal9ng problem for all possible programs over all possible inputs cannot exist u In other words, a general version of halter (one that works for all programs and their inputs) does not exist u 15 Proof by contradiction Assume that a general version of halter exists u Show that this assump9on leads to a contradic9on u Invent a new program called clever!
u clever(program,input)!
!result = halter(program,input)!
!if result is No!
!
!return Yes!
!else!
!
!loop forever!
16 Proof by contradiction What does clever do when given itself as input? u clever(program,input)!
!result=halter(program,input)!
!if result is No!
! !return Yes!
!else!
! !loop forever!
17 Proof by contradiction u u If halter says clever(program,input)!
that clever halts, result=halter(program,input)!
then clever !if result is No!
loops forever !
!return Yes!
(which means it doesn’t halt) !else!
If halter says that clever does not halt then clever halts and returns Yes ! !loop forever! Conclusion: The program halter cannot exist 18 Russell’s Paradox u In a hypothe9cal small town: u More generally: Let R be the set of all sets that are not members of v There is one male barber themselves v The barber shaves all those v If R is not a member of and only those men in town itself, then its deﬁni9on who do not shave themselves dictates that it must contain v All men stay clean shaven itself, and if it contains itself, either by shaving themselves then it contradicts its own or going to the barber deﬁni9on as the set of all u Who shaves the barber? sets that are not members of themselves v 19 Analysis of algorithms How long does an algorithm take to run? 9me complexity u How much memory does it need? space complexity u 20 Estimating running time How to es9mate algorithm running 9me? u Write a program that implements the algorithm, run it, and measure the 9me it takes v Analyze the algorithm (independent of programming language and type of computer) and calculate in a general way how much work it does to solve a problem of a given size v Which is beker? u 21 Analysis of an algorithm: Turing machine example u Some number of As followed by some number of Bs on a tape u More As than Bs? u Turing machine program v v v
v ﬁnd extremal AB pairs and erase them one at a 9me if no As lec and 1 or more Bs lec then output N if tape is blank output N if 1 or more As lec and no Bs lec then output Y A A A A B B B B A A A B B B B A A A B B B A A B B B A A B B A B B A B B N A A A A B B B B A A A B B B B A A A B B B A A B B B A A B B A B B A B B N 22 Analysis of an algorithm: Turing machine example u How long does the program take for an input string of length 8? 8 steps + 7 steps + 6 steps + … 1 step = 36 steps u How long does the program take for an input that is ‘n’ symbols long? n steps + (n-‐1) steps + (n-‐2) steps + … 1 step = n(n+1)/2 = (n2/2)+(n/2) (n2/2)+(n/2) is approximately equal to (n2/2) as n becomes large n A n A A A B B B B A A A B B B B A A A B B B A A B B B A A B B A B B A B n2/2 B N 23 Analysis of binary search n = 8, the algorithm takes 3 steps u n = 32, the algorithm takes 5 steps u For a general n, the algorithm takes log2n steps u 24 Growth rates of functions Linear u Quadra9c u Exponen9al u 25 Analysis of problems u u u u Study of algorithms illuminates the study of classes of problems If a polynomial 9me algorithm exists to solve a problem then the problem is called tractable If a problem cannot be solved by a polynomial 9me algorithm then it is called intractable This divides problems into three groups: v
v
v Problems with known polynomial 9me algorithms Problems that are proven to have no polynomial-‐9me algorithm Problems with no known polynomial 9me algorithm but not yet proven to be intractable 26 Tractable and Intractable u Tractable problems (P) v
v
v u Sor9ng a list Searching an unordered list Finding a minimum spanning tree in a graph u Intractable v Lis9ng all permuta9ons (all possible orderings) of n numbers Might be (in)tractable v v Subset sum: given a set of numbers, is there a subset that adds up to a given number? Travelling salesperson: n ci9es, n! routes, ﬁnd the shortest route These problems have no known polynomial 9me solu9on However no one has been able to prove that such a solu9on does not exist 27 Subset sum problem Given a set of integers and an integer s, does any non-‐empty subset sum to s? u {1, 4, 67, -‐1, 42, 5, 17} and s = 24 No u {4, 3, 17, 12, 10, 20} and s = 19 Yes {4, 3, 12} u If a set has N elements, it has 2N subsets. u Checking the sum of each subset takes a maximum of N opera9ons N
u To check all the subsets takes 2 N opera9ons N N/2, but all u Some cleverness can reduce this by a bit (2 becomes 2
known algorithms are exponen9al) u 28 Travelling salesperson problem u u u Given a list of ci9es and the distances between each pair of ci9es, what is the shortest possible route that visits each city exactly once and returns to the origin city? Given a graph where edges are labeled with distances between ver9ces. Start at a speciﬁed vertex, visit all other ver9ces exactly once and return to the start vertex in such a way that sum of the edge weights is minimized There are n! routes (a number on the order of nn -‐ much bigger than 2n) 29 Tractability, Intractability, Undecidability u
u u u u ‘Proper9es of problems’ (NOT ‘proper9es of algorithms’) Tractable: problem can be solved by a polynomial 9me algorithm (or something more eﬃcient) Intractable: problem cannot be solved by a polynomial 9me algorithm (all solu9ons are proven to be more ineﬃcient than polynomial 9me) Unknown: not known if the problem is tractable or intractable (no known polynomial 9me solu9on, no proof that a polynomial 9me solu9on does not exist) Undecidable: decision problem for which there is (provably) no algorithmic solu9on on a Turing machine v Non-‐existence v Not a ma?er of eﬃciency 30 Tractability, Intractability, Undecidability u
u u u u ‘Proper9es of problems’ (NOT ‘proper9es of algorithms’) Tractable: problem can be solved by a polynomial 9me algorithm (or something more eﬃcient) Intractable: problem cannot be solved by a polynomial 9me algorithm (all solu9ons are proven to be more ineﬃcient than polynomial 9me) Unknown: not known if the problem is tractable or intractable (no known polynomial 9me solu9on, no proof that a polynomial 9me solu9on does not exist) Undecidable: decision problem for which there is (provably) no algorithmic solu9on on a Turing machine v Non-‐existence v Not a ma?er of eﬃciency 31 Subset sum problem Given a set of integers and an integer s, does any non-‐empty subset sum to s? u {1, 4, 67, -‐1, 42, 5, 17} and s = 24 No u {4, 3, 17, 12, 10, 20} and s = 19 Yes {4, 3, 12} u If a set has N elements, it has 2N subsets. u Checking the sum of each subset takes a maximum of N opera9ons N
u To check all the subsets takes 2 N opera9ons N N/2, but all u Some cleverness can reduce this by a bit (2 becomes 2
known algorithms are exponen9al) u 32 P and NP u u P: set of problems that can be solved in polynomial 9me Easy to solve Consider subset sum No known polynomial 9me algorithm v However, if you give me a solu9on to the problem, it is easy for me to check if the solu9on is correct – i.e. I can write a polynomial 9me algorithm to check if a given solu9on is correct v u NP: set of problems for which a solu9on Easy to check can be checked in polynomial 9me 33 P=NP? All problems in P are also in NP u Are there any problems in NP that are not also in P? u In other words, is u P = NP ? Central open ques9on in Computer Science u 34 Abstract machines and theory What is theore9cal computer science? u Finite state machines Reading: u The Turing machine St. Amant Ch. 8 u The Church-‐Turing thesis Gleick Ch. 7 (opLonal) u The hal9ng problem u The analysis of algorithms u The analysis of problems u 35 Overview Problems Solu9on: Algorithms + Data Structures Low-‐level Low-‐level Low-‐level instruc9ons instruc9ons
instruc9ons Execu9ons managed by Opera9ng System Pseudocode Compile Program Program
to Program Program CPU, Memory, Disk, I/O 36 ...

View
Full Document

- Fall '13
- Turing, u