{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

CSCI109-Lecture#15-16-17-Abstract-Machines-Theory-Feb24-Feb26-Feb29-2016-2

CSCI109-Lecture#15-16-17-Abstract-Machines-Theory-Feb24-Feb26-Feb29-2016-2

Info icon This preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Introduction to Computing CSCI 109 China – Tianhe-­‐2 Gaurav S. Sukhatme Spring 2016 Lectures 15-­‐16-­‐17: Abstract Machines and Theory Feb 24-­‐26-­‐29, 2016 Abstract machines and theory What is theore9cal computer science? u Finite state machines Reading: u The Turing machine St. Amant Ch. 8 u  The Church-­‐Turing thesis Gleick Ch. 7 (opLonal) u  The hal9ng problem u  The analysis of algorithms u The analysis of problems u  1 What is theoretical computer science? The study of what can be computed and how efficiently it can be computed u Based on abstract mathema9cal models u Insight into the behavior of a real computer… …any real computer u  2 Finite state machines Up Fed Crying On Down Up Off Asleep Hungry More hungry Hungry Awake, not crying Down Light switch 1 day old infant 10 3 Finite state machines Add M Add M Add M M=F M+1 Add F Add F Add M F+1 M+2 Add M M+3 M+4 Add F Add F Add F Add F Add F Add F Add M St. Amant pp. 152 Add M F+2 F+3 Add M F+4 Add M Add F 4 Finite state machine limitations If a finite state machine with n states is given n+1 symbols, it will revisit one state at least once u No way to tell if revisited state is being visited for the 1st 9me or 5th 9me u The pigeonhole principle u  Problems an FSM cannot solve: u  Given a string of As and Bs, tell if the number of As and Bs are the same v  Tell if a string is a palindrome v  5 The Turing machine An infinite tape, with squares marked on it u  Each square can be blank or hold a symbol u  A machine moves over the tape u  When posi9oned over a square, the machine can u  v  v  v  v  u  Read a symbol from the tape Erase a symbol from the tape Write a symbol onto the tape (erasing what was on the square) Do nothing The machine can move either one square to right or lec 6 Alan Turing 7 Turing Machine Animation (Vimeo) 8 What can a Turing machine do? Given a string of As and Bs, tell if the number of As and Bs are the same u  Move back and forth, erasing AB pairs v  #As = #Bs if no extra As (or Bs) lec over v  Wri9ng programs for a Turing machine is very tedious u However… u  9 The Church-­‐Turing thesis Turing machines are capable of solving any effecLvely solvable algorithmic problem Turing machines are thus excellent models for what (all) real computers are capable of doing 10 Problems not ‘effectively computable’ Are there problems that are not effec9vely computable ? u Yes, infinite such problems u  Could these be computable someday ? u  Perhaps, but not on a Turing-­‐equivalent computer u  11 A halting question Deciding whether a program will halt (terminate) u Does the following program halt ? ! u  Input number! While number is not 0! Print number! number = number – 1 ! ! 12 The halting problem Deciding whether a given program will halt (terminate) on an arbitrary input u In 1936, Turing proved that a general algorithm to the hal9ng problem for all possible programs over all possible inputs cannot exist u  The hal2ng problem is undecidable ! 13 Proof sketch for halting problem u  Want to decide if program A halts on input a u  Need to write a program, lets call it halter! Inputs: A, a v  Output: Yes (if program A halts on input a) or No (if program A does not halt on input a) v  halter(A,a)! if(some complex computation)is true! Return Yes! else! Return No! 14 The halting problem is undecidable In 1936, Turing proved that a general algorithm to the hal9ng problem for all possible programs over all possible inputs cannot exist u  In other words, a general version of halter (one that works for all programs and their inputs) does not exist u  15 Proof by contradiction Assume that a general version of halter exists u  Show that this assump9on leads to a contradic9on u  Invent a new program called clever! u  clever(program,input)! !result = halter(program,input)! !if result is No! ! !return Yes! !else! ! !loop forever! 16 Proof by contradiction What does clever do when given itself as input? u  clever(program,input)! !result=halter(program,input)! !if result is No! ! !return Yes! !else! ! !loop forever! 17 Proof by contradiction u  u  If halter says clever(program,input)! that clever halts, result=halter(program,input)! then clever !if result is No! loops forever ! !return Yes! (which means it doesn’t halt) !else! If halter says that clever does not halt then clever halts and returns Yes ! !loop forever! Conclusion: The program halter cannot exist 18 Russell’s Paradox u  In a hypothe9cal small town: u  More generally: Let R be the set of all sets that are not members of v  There is one male barber themselves v  The barber shaves all those v  If R is not a member of and only those men in town itself, then its defini9on who do not shave themselves dictates that it must contain v  All men stay clean shaven itself, and if it contains itself, either by shaving themselves then it contradicts its own or going to the barber defini9on as the set of all u  Who shaves the barber? sets that are not members of themselves v  19 Analysis of algorithms How long does an algorithm take to run? 9me complexity u How much memory does it need? space complexity u  20 Estimating running time How to es9mate algorithm running 9me? u  Write a program that implements the algorithm, run it, and measure the 9me it takes v  Analyze the algorithm (independent of programming language and type of computer) and calculate in a general way how much work it does to solve a problem of a given size v  Which is beker? u  21 Analysis of an algorithm: Turing machine example u  Some number of As followed by some number of Bs on a tape u  More As than Bs? u  Turing machine program v  v  v  v  find extremal AB pairs and erase them one at a 9me if no As lec and 1 or more Bs lec then output N if tape is blank output N if 1 or more As lec and no Bs lec then output Y A A A A B B B B A A A B B B B A A A B B B A A B B B A A B B A B B A B B N A A A A B B B B A A A B B B B A A A B B B A A B B B A A B B A B B A B B N 22 Analysis of an algorithm: Turing machine example u  How long does the program take for an input string of length 8? 8 steps + 7 steps + 6 steps + … 1 step = 36 steps u  How long does the program take for an input that is ‘n’ symbols long? n steps + (n-­‐1) steps + (n-­‐2) steps + … 1 step = n(n+1)/2 = (n2/2)+(n/2) (n2/2)+(n/2) is approximately equal to (n2/2) as n becomes large n A n A A A B B B B A A A B B B B A A A B B B A A B B B A A B B A B B A B n2/2 B N 23 Analysis of binary search n = 8, the algorithm takes 3 steps u  n = 32, the algorithm takes 5 steps u  For a general n, the algorithm takes log2n steps u  24 Growth rates of functions Linear u Quadra9c u Exponen9al u  25 Analysis of problems u  u  u  u  Study of algorithms illuminates the study of classes of problems If a polynomial 9me algorithm exists to solve a problem then the problem is called tractable If a problem cannot be solved by a polynomial 9me algorithm then it is called intractable This divides problems into three groups: v  v  v  Problems with known polynomial 9me algorithms Problems that are proven to have no polynomial-­‐9me algorithm Problems with no known polynomial 9me algorithm but not yet proven to be intractable 26 Tractable and Intractable u  Tractable problems (P) v  v  v  u  Sor9ng a list Searching an unordered list Finding a minimum spanning tree in a graph u  Intractable v  Lis9ng all permuta9ons (all possible orderings) of n numbers Might be (in)tractable v  v  Subset sum: given a set of numbers, is there a subset that adds up to a given number? Travelling salesperson: n ci9es, n! routes, find the shortest route These problems have no known polynomial 9me solu9on However no one has been able to prove that such a solu9on does not exist 27 Subset sum problem Given a set of integers and an integer s, does any non-­‐empty subset sum to s? u  {1, 4, 67, -­‐1, 42, 5, 17} and s = 24 No u  {4, 3, 17, 12, 10, 20} and s = 19 Yes {4, 3, 12} u  If a set has N elements, it has 2N subsets. u  Checking the sum of each subset takes a maximum of N opera9ons N u  To check all the subsets takes 2 N opera9ons N N/2, but all u  Some cleverness can reduce this by a bit (2 becomes 2 known algorithms are exponen9al) u  28 Travelling salesperson problem u  u  u  Given a list of ci9es and the distances between each pair of ci9es, what is the shortest possible route that visits each city exactly once and returns to the origin city? Given a graph where edges are labeled with distances between ver9ces. Start at a specified vertex, visit all other ver9ces exactly once and return to the start vertex in such a way that sum of the edge weights is minimized There are n! routes (a number on the order of nn -­‐ much bigger than 2n) 29 Tractability, Intractability, Undecidability u  u  u  u  u  ‘Proper9es of problems’ (NOT ‘proper9es of algorithms’) Tractable: problem can be solved by a polynomial 9me algorithm (or something more efficient) Intractable: problem cannot be solved by a polynomial 9me algorithm (all solu9ons are proven to be more inefficient than polynomial 9me) Unknown: not known if the problem is tractable or intractable (no known polynomial 9me solu9on, no proof that a polynomial 9me solu9on does not exist) Undecidable: decision problem for which there is (provably) no algorithmic solu9on on a Turing machine v  Non-­‐existence v  Not a ma?er of efficiency 30 Tractability, Intractability, Undecidability u  u  u  u  u  ‘Proper9es of problems’ (NOT ‘proper9es of algorithms’) Tractable: problem can be solved by a polynomial 9me algorithm (or something more efficient) Intractable: problem cannot be solved by a polynomial 9me algorithm (all solu9ons are proven to be more inefficient than polynomial 9me) Unknown: not known if the problem is tractable or intractable (no known polynomial 9me solu9on, no proof that a polynomial 9me solu9on does not exist) Undecidable: decision problem for which there is (provably) no algorithmic solu9on on a Turing machine v  Non-­‐existence v  Not a ma?er of efficiency 31 Subset sum problem Given a set of integers and an integer s, does any non-­‐empty subset sum to s? u  {1, 4, 67, -­‐1, 42, 5, 17} and s = 24 No u  {4, 3, 17, 12, 10, 20} and s = 19 Yes {4, 3, 12} u  If a set has N elements, it has 2N subsets. u  Checking the sum of each subset takes a maximum of N opera9ons N u  To check all the subsets takes 2 N opera9ons N N/2, but all u  Some cleverness can reduce this by a bit (2 becomes 2 known algorithms are exponen9al) u  32 P and NP u  u  P: set of problems that can be solved in polynomial 9me Easy to solve Consider subset sum No known polynomial 9me algorithm v  However, if you give me a solu9on to the problem, it is easy for me to check if the solu9on is correct – i.e. I can write a polynomial 9me algorithm to check if a given solu9on is correct v  u  NP: set of problems for which a solu9on Easy to check can be checked in polynomial 9me 33 P=NP? All problems in P are also in NP u Are there any problems in NP that are not also in P? u In other words, is u  P = NP ? Central open ques9on in Computer Science u  34 Abstract machines and theory What is theore9cal computer science? u Finite state machines Reading: u The Turing machine St. Amant Ch. 8 u  The Church-­‐Turing thesis Gleick Ch. 7 (opLonal) u  The hal9ng problem u  The analysis of algorithms u The analysis of problems u  35 Overview Problems Solu9on: Algorithms + Data Structures Low-­‐level Low-­‐level Low-­‐level instruc9ons instruc9ons instruc9ons Execu9ons managed by Opera9ng System Pseudocode Compile Program Program to Program Program CPU, Memory, Disk, I/O 36 ...
View Full Document

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern