to Introduction Articial Intelligence
Lecture 1
Dr. Tom Nurkkala January 18, 2007
1
What is AI?
Basic Landscape Key Point 1. Fundamentally, AI seeks to understand and build intelligent entities. Two dimensions 1. Thinking (behaving) vs. Acting (doing) 2. Human (emulating human performance) vs. Ideal (rational) All four possibilities have been explored. Humanly Rationally Acting Turing Test Rational Agent Thinking Cognitive Modeling Laws of Thought Table 1: Taxonomy of Denitions of AI
1.1
Acting Humanly: Turing Test
Turing Test Alan Turing, 1950. An operational denition of intelligence, based on behavior indistinguishable from human behavior. Test passes if human interrogator cannot distinguish AI from human. Requires Natural language processingcommunicate in English (or other human language) Knowledge representationencode, store, and retrieve knowledge 1
Automated Reasoninguse knowledge to answer questions Machine Learningadapt, learn, recognize patterns Original Turing Test abstracts out physical interaction Total Turing Test adds Computer Visionperceive objects visually Roboticsmanipulate objects, move These disciplines comprise most of AI Some disagree: not necessary to duplicate an exemplar; better to study underlying principles. Some researchers (e.g., Cynthia Breazeal at MIT with the Kismet robot) disagree with this view, insisting that AI must be embedded in the world.
1.2
Thinking Humanly: Cognitive Modeling
For machines to think like humans, we have to understand how humans think. Need a theory of mind We dont. Analogy of brain science to stethoscope against the side of an all-relay computer. Cognitive Modeling Two approaches: 1. Introspection 2. Psychological experiments General Problem Solver (Newell and Simon, 1961) wanted problems solved: Correctly Like a human would solve them Cognitive Science AI computer models plus experimental psychology yield testable theories of human mind. Early on, in conict (e.g., algorithm working well must mean thats the model) Now, cross fertilization.
2
1.3
Thinking Rationally: Laws of Thought
Logic: Laws of Thought Irrefutable reasoning processes (e.g., syllogisms), date back to Aristotle (384322 bc) Correct premises yield correct conclusions Build on logic and programs that perform logical deduction to create intelligent systems. Obstacles 1. Hard to state informal knowledge in logical terms 2. Hard to capture less than 100% certainty 3. Hard to solve combinatorially complex problems in practice
1.4
Acting Rationally: Rational Agent
Types of Agents Basic Agent Something that acts Not a mere program 1. Autonomous control 2. Perception of environment 3. Persistence in time 4. Adapt to change 5. Adopting goals Rational Agent An agent that acts to achieve a best expected outcome Relationship to other approaches Requires same skills as for Turing Test Broader scope than Laws of Thought rationality; act even when no provably correct way to act
3
Rational Agents Advantages to Rational Agent Approach More general than Laws of Thought More amenable to scientic development than human thought or human behavior Standard of rationality clearly dened and general Scope of Rationality Perfect rationality Always doing the right thing Not feasible in complex environments (too expensive computationally) Still a good starting point for analysis. Limited rationality Acting appropriately under limited time bounds Focus of book : General principles of rational agents and on components for constructing them.
2
Foundations of AI
History of disciplines that contributed to AI
2.1
Philosophy
Basic Philosophical Issue Key Question 1. How can formal rules be used to draw valid conclusions (automatically)? Aristotle (4th c. bc)Syllogisms and Laws of Thought Thomas Hobbes (17th c.)Reasoning equivalent to numerical computation Leonardo da Vinci (ca. 1500)designed (didnt build) a mechanical calculator Blaise Pascal (17th c.)Pascaline (arithmetical machine) Gottfried Wilhelm Leibniz (d. 18th c.)mechanical device to operate on concepts instead of numbers
4
Minds and Brains Key Question 2. How does mind arise from brain? Dualism Human being comprised of physical and non-physical Materialism Only physical exists; minds are what brains do. Ren Descartes (17th c.)rst discussed distinction between mind and matter e Materialism runs afoul of free will A mind governed by physical law is no more free than a rock falling under the force of gravity free will the way that perception of choices appears to the choice process. Epistemology Key Question 3. Assuming materialism, whats the source of knowledge? Empiricism Francis Bacon (17th c.), John Locke (17th c.)nothing is known other than what was rst in the senses Induction David Hume (18th c.)general rules are acquired by repeated associations Logical Positivism Ludwig Wittgenstein (20th c.), Bertrand Russell (20th c.), Rudolph Carnap (20th c.)all knowledge either analytical (true by nature of form) or empirical (true by merit of observation by sense perception) Knowledge and Action Key Question 4. What is the connection between knowledge and action? Aristotle Actions justied by logical connection between goals and outcome of actions Start with the end in mind. Yields a regression planning system. Implemented by General Problem Solver Problem: decide what to do when several actions will achieve the goal.
5
2.2
Logic
Mathematics
Key Question 5. What are the formal rules that yield valid conclusions? Ancient Greeks George Boole (19th c.)propositional (a.k.a. Boolean) logic Gottlob Frege (20th c.)rst-order logic Computation Key Question 6. What is computable? What are the limits of logic and computation? Uncomputable Cant be computed at all Intractable Cant be computed practically (e.g., before the heat death of the universe). Usually problems with exponential growth in complexity O(k n ) for input size n, k a constant. NP-Complete Describes a large class of combinatorial search and reasoning problems are thought to be intractable. No known algorithms with better than exponential time, O(k n ). Can be veried in polynomial time, O(nk ). No proof that exponential time is required. Computation First algorithm: Euclids GCD algorithm al-Khowarazmi (9th. c.)Persia; rst studied algorithms Boolealgorithms for deduction David Hilbert (20th c.)decidability problem (Entscheidungsproblem): is there an algorithm for deciding any logical proposition involving natural numbers? Kurt Gdel (20th c.)No! Incompleteness Theorem (in a system complex enough o to describe natural numbers, there exist true statements that cannot be proven true algorithmically) Alan Turing (20th c.)Which functions are computable? Turing Machinecomputes any computable function. There are functions no Turing Machine can compute (Halting Problem: whether an arbitrary program will terminate)
6
Probability Key Question 7. How do we reason in the face of uncertainty? Thomas Bayes (18th c.)Bayesian analysis underlies most modern approaches to uncertain reasoning.
2.3
Economics
Optimization Key Question 8. How to make decisions to maximize payo ? Economicshow people make choices that lead to preferred outcomes (utility). Adam Smith (18th c.)Wealth of Nations; intelligent agents maximizing their own well-being Decision Theorycombines probability and utility theories; formal framework for decisions under uncertainty Adversaries and Planning Key Question 9. How do we makes such decisions when facing an adversary? Game Theory Adversarial search Alpha-beta pruning Key Question 10. How do we decide if the payo is far in the future (i.e., payo results from a series of actions in sequence)? Operations Research Markov processes Satiscing (Herbert Simon, 20th c.)making decisions that are good enough rather than (laboriously calculated) optimality better describe human behavior.
7
2.4
Neuroscience
Wetware Key Question 11. How do brains work? Neurosciencestudy of the nervous system, particularly the brain Brain consists of neurons Connection with 10 to 105 other neurons Connectivity changes over time (memory) Dierent areas of the brain responsible for dierent cognitive function. Can observe brain activity (EEG, PET) Chips vs Brains Computer 108 gates 1010 bits RAM 1011 bits disk Cycle Time 109 sec Bandwidth 1010 bits/sec Memory (updates/sec) 109 Computation Storage Brain 1011 neurons 1011 neurons 1014 synapses 103 sec 1014 bits/sec 1014
Table 2: Processing capacity in the computer and the human brain Key Point 2. Although a computer is one million times faster in raw switching speed, the brain is 100,000 times faster at what it does.
2.5
Psychology
Psychology Behaviorism John Watson (20th c.)reject any theory involving metal processes, dismissed as unscientic; only objective measures of precepts (stimulus) and resulting action (response); failed to understand humans very well Cognitive Psychology William James (20th c.)brain as information processing device, perception as logical inference; reestablished mental terms like beliefs, goals Cognitive Science (MIT, 1956)intersection of logic, linguistics, computing, psychology, Neuroscience, etc.; addresses memory, language, logical thought; strongly inuenced by Computer Science 8
2.6
Computer Engineering
Engineering Hardware Lisp Machines Moores LawPerformance doubles every 18 months (to 2020) Software Operating Systems Languages (Lisp, Prolog, special purpose) Tools Ideas that come from AI Time sharing Interpreters PCs, windows, mice Rapid development environments Linked-lists Automatic storage management (garbage collection) Concepts in symbolic, functional, dynamic, and object-oriented Theory/Cybernetics
Control programming
2.7
Control Theory Key Question 12. How can artifacts operate under their own control? Self-regulating feedback systemswater clock, steam governor (Watt), thermostat Control Theory Purposive behavior as a regulatory mechanism trying to minimize error (dierence between current state and goal state) Norbert Weiner (20th c.)coined term Cybernetics in book by that name (1948) Goal: systems that maximize an objective function over time (i.e., designing of systems that behave optimally) Sounds like AIWhy two disciplines? Control theorycalculus and matrix algebra; xed sets of continuous variables; feasible only for linear systems. AIlogical inference, computation; problems outside of control theorys focus. 9
2.8
Linguistics
Human Language Skinner (1957)Verbal Behavior, a behaviorist approach to language learning Noam Chomsky (20th c.)Syntactic Structures are present in the brain itself Contra Skinner, nearly killed behaviorism Behaviorism doesnt address novelty in language Very complexphonology, syntax, semantics, discourse, intention, subject matter, etc. Early Knowledge Representation work arose from linguistics research Whats it called? Computational Linguistics (if youre a Linguist) Natural Language Processing (if youre a Computer Scientist)
3
3.1
History of AI
Gestation (19431955)
Imitating Neurons Warren McColloch and Walter Pitts (1943) draw on Basic knowledge of neurons Propositional logic Idea of Turning Machine Formulate idea of articial neuron. Can compute any computable function (e.g., all logical connectives). Donald Hebb (1949) how to set connection strengths between neuronsHebbian Learning Marvin Minsky and Dean Edmonds (1950) build rst neural net computer (Snarc). Forty neurons from 3,000 vacuum tubes.
10
Imitation Game Alan Turing (1950) writes Computing Machinery and Intelligence describing Turing Test (Turing called it The Imitation Game) Machine learning Genetic algorithms Original paper at http://www.loebner.net/Prizef/TuringArticle.html.
3.2
Birth (1956)
Dartmouth Workshop Dartmouth workshop, Summer 1956 Introduced the major gures of early AI to one another Coined the term Articial Intelligence Established AI as a stand-alone discipline, distinct from its intellectual forebears. Attendees: John McCarthy, Marvin Minsky, Claude Shannon, Allen Newell, Herb Simon. Newell and Simons Logic Theorist Proved most theorems from a chapter of Russel and Whiteheads Principia Mathematica In one case found a shorter proof than Principia
3.3
Enthusiasm (19521969)
General Solvers Newell and Simon GPSGeneral Problem Solver Designed to imitate human problem solving Considered goals and sub-goals (Thinking Humanly) Physical Symbol System HypothesisA physical symbol system has the necessary and sucient means for general intelligent action A systemhuman or machineexhibiting intelligence must operate by manipulate symbols 11
Blue
Red Red Green
Green
Green Blue
Red
Challenged by connectionist approaches Hebert Gelernter (1959)Geometry Theorem Prover Arthur Samuel (staring 1952)checkers programs; learned how to improve, quickly eclipsing Samuel himself James Slagle (1963)Saint solved basic integration problems. Former faculty at Minnesota Daniel Bobrow (1967)Student solved algebra story problems M&M John McCarthy moved to MIT; in 1958 he: Dened Lisp, the dominant AI language (one year older than Fortran, the oldest high-level language still in use) Invented time sharing in response to limited availability of computer time. Published Programs with Common Sense, describing Advice Taker. Used knowledge and search to solve problems, embodied common-sense world knowledge, could integrate new knowledge. Marvin Minsky moved to MIT in 1958 McCarthy: representation and reasoning in formal logic Minsky: getting programs to work; anti-logical McCarthy founds AI lab at Stanford in 1963 Minsky: Systems focused on limited domains (micro-worlds). Blocks World Blocks World Most famous micro-world Rearrange blocks using a robot hand
12
Fundamental work in Vision (David Human, 1971; David Waltz, 1975) Constraint propagation (David Waltz) Learning Theory (Patrick Winston, 1980) Natural Language Understanding (Terry Winograd, 1972) Planning (Scott Fahlman, 1974)
3.4
Dose of Reality (19661973)
Reality Bites AI has always made vast predictions of future success Systems appeared promising on limited problems, but failed to scale to larger problems or more general domains. Why? No knowledge of subject matter. The spirit is willing but the esh is weak (Matthew 26.41) from English to Russian and back again: The vodka is good but the meat is rotten. Intractable problems. Combinatorial explosion. Early problems small, combinatorial solutions reasonable. Didnt scale up. For example, theorem proving with more than a few dozen facts. Limitations on basic structures used to generate intelligent behavior. Minsky and Papert, Perceptrons shows that this simple form of neural network could not represent much (e.g., logical or) A program being able to nd a solution in principle does not mean it an nd a solution in practice.
3.5
Knowledge Systems (19691979)
Expert Systems Early AI embraced general-purpose problem solving using elementary reasoning steps (Weak Methods) Didnt scale up to large or dicult problems. Instead, use domain-specic knowledge Larger reasoning steps for typical cases Dendral 13
Ed Feigenbaum, Bruce Buchanan, Joshua Lederberg, Stanford, 1969 Infer molecular structure from information provided by a mass spectrometer Cant try all possible structures because too expensive computationally Consult chemists; looked for well-known patterns of spectral peaks that suggest common substructure First successful knowledge-intensive system Large number of special-purpose rules Expert Systems Mycin Feigenbaum, Buchanan, Edward Shortlie Diagnose blood infections Rivaled performance of expert physicians All knowledge acquired from experts (no existing framework (Chemistry) as for Dendral) Rules reected uncertainty using certainty factors Shrdlu Terry Winograd as part of Blocks World Natural language understanding Too dependent on syntax Apparent power (e.g., resolving pronoun referents) due to limited domain Language and Knowledge Roger Shank (early 1980s) Language understanding Actually more about representing and reasoning with knowledge required for language understanding Needed improved schemes for representing domain knowledge. Marvin Mikskys (1975) frames oered structured approach Large taxonomic hierarchy, attribute-based 14
3.6
Industrial AI (1980)
Big Business R1 First successful commercial expert system Congured computers for Digital Equipment Corporation Saved $40M/year Expert systems became all the rage Fifth Generation Project (Japan, 1980s) Build intelligent computers running Prolog Microelectronics and Computer Technology Corporation (MCC) the US response. Focus on AI Never met ambitions goals AI Winter Industry boomed until about 1988 ($ billions) Companies failed to deliver Investment dried up
3.7
Neural Networks Redux (1986)
Neural Nets Ride Again Minsky and Papert had killed o neural net work in the 1960s Renewed interest with New algorithms for learning (esp. back-propagation) Became known as connectionist models of intelligent systems Competitor to the physical symbol systems Which to prefer is an open question Perhaps best seen as complimentary views Important collection: Parallel Distributed Processing (Rumelhart and McClelland, 1986)
15
3.8
AI as Science (1987)
AI Grows Up Neats Think AI theories should be grounded in mathematical rigor Scrues Try out lots of AI ideas, write programs, and see what works AI has shifted from scruness to neatness More common to build on existing theories than propose new ones New theories based on experimental evidence rather than intuition Embracing elds previously rejected (e.g., control theory, statistics, formal methods) Re-entering broader scientic community instead of being isolationist Scientic method applied Hypotheses subjected to empirical experiments Results analyzed and statistically valid Code and data shared Experiments validated independently
3.9
Intelligent Agents (1995)
Agents: All the Rage Much Progress solving sub-problems Vision, sonar, speech recognition, knowledge representation, search, logic, probability, planning, etc. Focus turning to whole agent problem The Situated movement Consider agents embedded in real environment Continuous sensory inputs One important environment: the Internet (ubiquity of -bot) Insights from work on agents Must be able to cope with unreliable information and uncertainty Important to work with other elds that use agents (e.g., economics) 16
4
AI State of the Art
Up to the Minute Key Question 13. What can AI do today? Autonomous Planning and Scheduling Spacecraft control Goal-directed planning, detection, diagnosis, problem recovery Game Planing IBM Deep Blue World chess champion Autonomous Control CMU NavLab Computer-controlled mini-van Crossed the US without human control over 98% of the time Up to the Minute Diagnosis Medical diagnosis in several areas of medicine (e.g., pathology) Explanation, justication for decisions Logistics Planning DODs Dynamic Analysis and Replanning Tool Logistics planning of 50,000 vehicles, cargo, people Embarkation, destination, route, conict resolution Paid back all of DARPAs 30-year investment in AI Robotics Robotic surgical assistants Cooperating autonomous robots in reconnaissance Exploration of the Solar System
17