BOOK.pdf - AUTONOMOUS LEARNING FROM THE ENVIRONMENT WEI MIN...

This preview shows page 1 out of 415 pages.

Unformatted text preview: AUTONOMOUS LEARNING FROM THE ENVIRONMENT WEI - MIN SHEN Microelectronics and Computer Technology Corporation Computer Science Press W. H. Freeman and Company To the people, especially those in China. Foreword by Herbert A. Simon Contemporary artificial intelligence systems depend heavily upon their programmers for their knowledge about the world in which they must act. Robotics is only beginning to provide the sensory and motor capabilities that they require when they must interact with the physical environment. And even when such capabilities are in place, or when the interaction with the task environment is symbolic instead of physical, AI systems still need intelligent strategies for exploring their environments to acquire information about them, and to build their internal representations of them. Dr. Shen’s admirable book addresses these fundamental problems of how learning from and about the environment can be automated. It provides both a basic framework, within which Dr. Shen examines our present understanding of these matters, and an important example of a program, LIVE, that possesses novel and important capabilities for learning about its environment autonomously. Tasks calling for intelligence fall into two broad categories. In the one case, the intelligent system knows the task situation exactly, hence need not distinguish between the real world that its actions will affect and the mental world in which it plans its actions. In the other case, the intelligent system knows the actual situation only in part, hence must be concerned with incompleteness and inaccuracies of its picture of reality; for its plans will frequently fail to reach the intended goals or have undesired side effects, and it must have means for recognizing these failures, remedying them as far as possible, and re-establishing its contact with the external reality. Proving mathematical theorems belongs to the first category of AI tasks: the theorem prover is provided with the axioms and inference rules of its mathematical ”world,” never has to stray beyond that world and can always be sure that its iii iv inferences will be valid in it. The domain of robotics belongs to the second category. There are always important differences between the world as perceived by the robot planner, and the world upon which the robot acts. In this situation, an intelligent system must have means for gathering information about the world and correcting its picture of it. Of course, the category of systems that must handle uncertainty extends far beyond robotics, encompassing all intelligent systems that deal, physically or mentally, with the conditions of the world we live in. This book is concerned with this second, very broad and fundamental category of artificial intelligence tasks, and with systems capable of coping with the uncertainties and incomplete information that are intrinsic to such tasks. There is another way of dividing up AI tasks that is based upon a distinction between information the intelligent system is given and information it must acquire by some kind of learning process. A performance system is designed to work in a defined task domain, accepting particular goals and seeking to reach them by some kind of highly selective search. The system must be told what goal is to be reached and must be given a description of the structure and characteristics of the task domain in which it is to operate: its problem space. In real life, the problem space can only be a highly simplified and approximate description of the actual outside world. In contrast, a learning system, is capable of acquiring a problem space, in whole or part, by interacting with the external environment and without being instructed about it directly. Learning, in turn, can be passive or active (or both). Passive learning uses sensory organs to gather information about the environment, but without acting on the environment. Active learning moves about in the environment in order to gather information, or actually operates on the environment to change it (performs experiments). Since no intelligent system can grasp the whole of reality in its problem representation, it must build highly simplified and special problem spaces to deal with the special classes of situations it faces at given times. It must be able to alter its problem spaces frequently, and sometimes radically. Clearly, systems that must distinguish between the real world and the world of thought, taking account of the differences between the actual and the expected, have to detect these differences and respond to them in adaptive ways. This book is concerned specifically with intelligent systems that have learning capabilities enabling them to correct their pictures of their environments and consequently to behave instrumentally in the world – the real world rather than the imagined one. To build theory about complex systems, whether these systems be natural or artificial, we need to pursue both empirical and formal approaches – neither by it- v self is sufficient. Up to the present time, most of what we have learned about artificial intelligence (and about human intelligence, for that matter) has been learned by building systems capable of exhibiting intelligent behavior. ”The moment of truth,” it has been said, ”is a running system.” At the same time, merely exhibiting such a system is not enough: we must discover how, why, and how well it works. That calls for a conceptual structure within which intelligent systems can be analysed and compared. This book indeed provides such a conceptual framework for addressing the general problem of learning from the environment. It emphasizes that if a system is to learn from its environment it must have capabilities for induction – it must be able to assemble its observations into a coherent problem space that can serve as its model of the environment. As part of its modeling activity, it must be able to induce concepts and regularities and laws to describe the world it is living in. It must be able to make predictions and to modify its model when the predictions are falsified. It must be able to detect when the variables it can observe provide only an incomplete description of the external reality, and must be able to respond to this incompleteness by postulating new hidden variables (theoretical terms). A framework that encompasses all of these requirements also encompasses many central topics in artificial intelligence: concept learning, heuristic search, law discovery, prediction and others. The first half of the book draws upon the research on all of these topics, bringing this work into focus and exploring its implications for the design of systems that learn. It shows how a wide range of techniques – some drawn from AI, some from operations research, others from statistical decision theory and elsewhere – can be used for model construction, for making predictions, for exploring the environment actively, for correcting and elaborating the model. One major value of the book for me has been to put together in a principled way and as a single continent a large body of literature that had hitherto formed rather isolated islands. Dr. Shen’s book never loses sight of the fact that theory in AI is centrally concerned with what is computable and how it can be computed. ”Computability,” from an AI standpoint, has almost nothing to do with Godel theorems that show that some valid propositions must always lie beyond the capabilities of any single formal system, or with the equivalence or inequivalence of certain languages or architectures with Turing Machines. It has only a little more to do with those theorems on computational complexity that deal with worst cases as problem size increases without limit, or with theorems that show that an algorithm will ultimately converge (after a computation of unspecified duration) to a desired result. Computability in AI has to do with what can be computed over reasonable vi time intervals using the kinds of computing devices that are actually available, or are likely to be available within a visible time. This is an imprecise criterion, but the only one of real interest and the only one that distinguishes AI from pure mathematics. Dr. Shen, while he does not ignore formal results, keeps this criterion clearly in mind in both his empirical and his theoretical work. One way to study the computational capabilities of systems that learn from the environment is to construct robots to operate in a physical world. Another way is to construct two ”worlds” within the computer – in one of them actions are taken and consequences follow; the other represents the problem solver’s representation of that real world, its problem space. A theory of autonomous learning from the environment can be developed and tested initially in either of these contexts, and the theory developed in this book applies to both. Ultimately, of course, the autonomous system must be tested in the real world, but simulation can provide a very effective and economical initial test platform. The particular AI system, LIVE, that Dr. Shen has built and observed, and that occupies much of the second half of the book, represents the second strategy. It is a conceptual rather than a physical robot; the world that LIVE learns about is stored in computer memory alongside (but separated from) its own problem space. The two worlds communicate only through the sensory capabilities with which LIVE is endowed and its capabilities for action. This research strategy for studying learning from the environment avoids the arduous tasks of building physical visual and auditory sensors and mechanical arms and legs, but does not lose sight of the fundamental issue: the actual and possible discrepancies between the mental picture and the scene of action. Research on human discovery processes had shown that surprise – the accident that, as Pasteur put it, happens to the prepared mind – plays a frequent and important role in scientific discovery. Fleming’s discovery of penicillin, Hans Kreb’s discovery of the role of ornithine in the synthesis of urea, the Curies’ discovery of radium are well-known examples. Autonomous Learning from the Environment shows clearly why surprise (the departure of expectations from observations) is useful for guiding the modification of the learner’s problem space, and in particular how the formation of expectations enhances the knowledge obtainable from new observations. Most readers, I think, will experience more than one surprise as they explore the pages of this book, and if they do not already know how to exploit surprise for purposes of discovery, they will find help on that topic too. On this and many other topics, Weimin Shen has provided us with an indispensable vade mecum for our explorations of systems that learn from their environments. Preface I decided to write a book on how machines learn from their environments after I finished my Ph.D. in 1989 while training under Herbert A. Simon. There were already many books on machine learning in general, but I felt that none of them addressed the questions that have motivated many researchers in the field of autonomous learning from the environment: What does it mean that X learns from environment Y ? Can we understand and duplicate such intelligent processes on man-made systems? These questions, although interest us all, have only been addressed by separated research areas with specific focuses. Yet the important inquires, such as what are the necessary tasks of autonomous learning and what technologies we have today to accomplish these tasks (regardless of the field— mathematics, psychology, or computer science), can be addressed in a global view. This is what I have attempted to do in this book. The readers must judge whether I have succeeded. The thesis of this book is that intelligent behavior of any creature, animals or machines alike, is ultimately rooted in its physical abilities to perceive and act in its environment. As Charles S. Peirce said more than a hundred years ago: “our idea of anything is our idea of its sensible effects.” We say ice is cold, because we touch it and sense the effect. The secret of autonomous learning, I think, lies inside the process of how such sensible effects are learned by each individual from its environment. I hope the book can reach two groups of readers. For those who want to learn the subject of learning from the environment, I hope the book provides enough material to help them to become active researchers in the field. For those who already stand at the frontier of the research, I hope this book, with lots of naive and sketchy ideas, can trigger some interesting thoughts of their own. In attempting to address such an ambitious and broad subject, I am bound to miss some significant ideas, issues and approaches, for which I ask the reader’s forbearance. This book is more about the “mind” and less about the “body” of vii viii autonomous systems. Most discussions in the book are on computational methods and nothing is said about their hardware, such as cameras or manipulators. The research in autonomous learning from the environment is still in its infancy today. This book is only a snapshot of a sprout, whose seed can be found in my Ph.D. dissertation completed in Carnegie Mellon University, and whose roots now stretch out into many scientific fields in seeking better nutrition. Although there is a long way to go before such a sprout can grow into a standing tree, I believe the future of this research is bright. Truly autonomous learning systems, once they are built, can bring humanity endless benefit and prosperity. I want to thank many of my great teachers and friends in Carnegie Mellon University. I owe a great deal of gratitude to my advisor Herbert A. Simon, who opened for me the doors to western science and guided me in the maze of scientific research. To me, he is both my academic advisor and a noble man who has dedicated himself to sciences. This book is only a by-product of what I have learned from him. Professor Jaime G. Carbonell, Tom Mitchell and Chuck Thorp, who were my co-advisors, have shown great support. In our many pleasant meetings, they always understood the ideas that I was trying to explain better than I understood them myself. When I was writing my dissertation, professor Allen Newell gave me some very valuable advice and I can never forget his unique and warmest handshakes. It is sad that we have lost such a great pioneer in our field. In those CMU years, I also overcame my Confucian diffidence and learned how to discuss ideas with people (even if I do not know what I am talking about). Among the valued participants in these discussions were Jerry Burch, Murray Campbell, Jill Fain, Allan Fisher, Yolanda Gil, Klaus Gross, Angela Hickman, Peter Highnam, Peter Jansen, David Long, Ken McMillan, PengSi Ow, Jeffrey Schlimmer, Peter Shell, Han Tallis and many others. Working at Microelectronics and Computer Technology Corporation has made me much more mature as a researcher and as a citizen. While writing this book, I had numerous discussions with Mark Derthick, Michael Huhns, and Avi Saha, who broadened my view of machine learning and reinforced my confidences when I needed it most. Special thanks to Phil Cannata, who knows how to bring out the best in people. Working with him is a great pleasure. The editors at Computer Science Press, W. H. Freeman and Company, Gray Carlson, Burt Gabriel, William Gruener, and Penny Hull provided much help. William Gruener gave me the courage to start the project. I want to thank Jeffrey Ullman, Paul Rosenbloom, David Chapman, Richard Sutton, and several anonymous reviewers for their valuable criticisms and suggestions on the earlier drafts of this book. I am still, however, responsible for the errors and mistakes in this ix book. Finally, I think no man can stand alone. My earlier teachers, Wang Jian, You Juin-Hong, and Lou Chi-Ming taught me that life can be meaningful. My beautiful wife, Sai Ying, and lovely sons, Ted and Lucas, have made me complete. My parents and sister in China have supported me unconditionally and sacrificed a great deal to make me who I am now. Wei-Min Shen February 1994 x Contents 1 2 3 Introduction 1.1 Why Autonomous Learning Systems? 1.2 What Is Autonomous Learning? . . . 1.3 Approaches to Autonomous Learning 1.4 What Is in This Book? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 2 3 4 The Basic Definitions 2.1 Learning from the Environment . . . . . . . . . . . . . 2.2 The Learner and Its Actions, Percepts, Goals and Models 2.3 The Environment and Its Types . . . . . . . . . . . . . . 2.3.1 Transparent Environment . . . . . . . . . . . . . 2.3.2 Translucent Environment . . . . . . . . . . . . . 2.3.3 Uncertain Environment . . . . . . . . . . . . . . 2.3.4 Semicontrollable Environment . . . . . . . . . . 2.4 Examples of Learning from the Environment . . . . . . 2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 7 8 13 16 16 17 17 18 24 The Tasks of Autonomous Learning 3.1 Model Abstraction . . . . . . . . . . . . . . . 3.1.1 The Choice of Model Forms . . . . . . 3.1.2 The Evaluation of Models . . . . . . . 3.1.3 The Revision of Models . . . . . . . . 3.1.4 Active and Incremental Approximation 3.2 Model Application . . . . . . . . . . . . . . . 3.3 Integration: The Coherent Control . . . . . . . 3.4 Views from Other Scientific Disciplines . . . . 3.4.1 Function Approximation . . . . . . . . 3.4.2 Function Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 25 25 27 29 30 31 32 34 34 37 xi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii CONTENTS 3.5 4 3.4.3 Classification and Clustering . . . . . . . . . . . . . . . . 3.4.4 Inductive Inference and System Identification . . . . . . . 3.4.5 Learning Finite State Machines and Hidden Markov Models 3.4.6 Dynamic Systems and Chaos . . . . . . . . . . . . . . . . 3.4.7 Problem Solving and Decision Making . . . . . . . . . . 3.4.8 Reinforcement Learning . . . . . . . . . . . . . . . . . . 3.4.9 Adaptive Control . . . . . . . . . . . . . . . . . . . . . . 3.4.10 Developmental Psychology . . . . . . . . . . . . . . . . . Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Model Abstraction in Transparent Environments 4.1 Experience Spaces and Model Spaces . . . . . . . . . . . . . . . 4.2 Model Construction via Direct Recording . . . . . . . . . . . . . 4.3 Model Abstraction via Concept Learning . . . . . . . . . . . . . . 4.4 Aspects of Concept Learning . . . . . . . . . . . . . . . . . . . . 4.4.1 The Partial Order of Models . . . . . . . . . . . . . . . . 4.4.2 Inductive Biases: Attribute Based and Structured . . . . . 4.4.3 Correctness Criteria: PAC and Others . . . . . . . . . . . 4.5 Learning from Attribute-Based Instances: Version Space and Related Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6 Learning from Structured Instances: The FOIL Algorithm . . . . 4.7 Complementary Discrimination Learning (CDL) . . . . . . . . . 4.7.1 The Framework of Predict-Surprise-Identify-Revise . . . . 4.8 Using CDL to Learn Boolean Concepts . . . . . . . . . . . . . . 4.8.1 The CDL1 Algorithm . . . . . . . . . . . . . . . . . . . . 4.8.2 Experiments and Analysis . . . . . . . . . . . . . . . . . 4.9 Using CDL to Learn Decision Lists . . . . . . . . . . . . . . . . 4.9.1 The CDL2 Algorithm . . . . . . . . . . . . . . . . . . . . 4.9.2 Experiments and Analysis . . . . . . . . . . . . . . . . . 4.10 Using CDL to Learn Concepts from Structured Instances . . . . . 4.11 Model Abstraction by Neural Networks ....
View Full Document

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture