Download Document
Showing page : 1 of 312
This preview has blurred sections. Sign up to view the full version! View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: LICENSE INFORMATION: This is a single-user copy of this eBook. It may not be copied or distributed. Unauthorized reproduction or distribution of this eBook may result in severe criminal penalties. FUZZY LOGIC for Embedded Systems Applications FUZZY LOGIC for Embedded Systems Applications by Ahmad M. Ibrahim, Ph.D. Senior Member, IEEE AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS SAN DIEGO SAN FRANCISCO SINGAPORE SYDNEY TOKYO Newnes is an imprint of Elsevier Science. Copyright 2004, Elsevier Science (USA). All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher. Recognizing the importance of preserving what has been written, Elsevier Science prints its books on acid-free paper whenever possible. Library of Congress Cataloging-in-Publication Data ISBN: 0-7506-7699-X British Library Cataloging-in-Publication Data A catalogue record for this book is available from the British Library. The publisher offers special discounts on bulk orders of this book. For information, please contact: Manager of Special Sales Elsevier Science 200 Wheeler Road Burlington, MA 01803 Tel: 781-313-4700 Fax: 781-313-4880 For information on all Newnes publications available, contact our World Wide Web home page at: http://www.newnespress.com 10 9 8 7 6 5 4 3 2 1 Printed in the United States of America Contents Preface ................................................................................................ xi About the Author ............................................................................. xv Whats on the CD-ROM? ................................................................. xvi 1. Embedded Systems: An Overview ............................................... 1 Definition and Examples ................................................................ 1 Further Features of Embedded Systems .......................................... 2 Design Metrics ............................................................................... 4 Implementation ............................................................................. 6 1.4.1 Processor Technologies ....................................................... 6 1.4.2 Integrated Circuit Technologies .......................................... 7 1.4.3 Design Technologies ......................................................... 11 1.5 Fuzzy Logic and Embedded Systems ............................................. 12 Concluding Remarks ............................................................................. 13 Bibliography ......................................................................................... 14 Web Resources ..................................................................................... 16 1.1 1.2 1.3 1.4 2. Fuzzy Sets .................................................................................... 23 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 Introduction ................................................................................. 23 Classical Sets ................................................................................ 23 Set Operations ............................................................................. 25 Boolean Logic .............................................................................. 31 Basic Concepts of Fuzzy Sets ........................................................ 31 Other Representations of Fuzzy Sets ............................................. 32 Determination of Membership Functions ..................................... 35 Fuzzy Sets Properties .................................................................... 35 Operations on Fuzzy Sets ............................................................. 37 v Contents 2.9.1 2.9.2 Logic Operations .............................................................. 37 Algebraic Operations on Fuzzy Sets .................................. 42 3. Fuzzy Relations ............................................................................ 53 3.1 Introduction ................................................................................. 53 3.2 Classical Relations ........................................................................ 53 3.3 Classical Reasoning ...................................................................... 58 3.4 Fundamentals of Fuzzy Relations .................................................. 59 3.5 Operations on Binary Fuzzy Relations ........................................... 59 3.6 Types of Fuzzy Relations ............................................................... 62 3.7 Fuzzy Reasoning .......................................................................... 63 Concluding Remarks ............................................................................. 65 Bibliography ......................................................................................... 66 Web Resources ..................................................................................... 67 4. Embedded Fuzzy Applications ................................................... 69 Introduction ................................................................................. 69 Conventional Control Systems ..................................................... 69 4.2.1 Description ....................................................................... 69 4.2.2 Analysis ............................................................................ 71 4.2.3 Design ............................................................................. 72 4.2.4 PID Control ...................................................................... 72 4.3 Fuzzy Logic Controller (FLC) ......................................................... 73 4.3.1 Description ....................................................................... 73 4.3.2 Design ............................................................................. 74 4.3.3 Defuzzification ................................................................. 75 4.3.4 Analysis ............................................................................ 78 4.4 Simplified Examples of Applications ............................................. 83 4.4.1 Washing machine ............................................................ 83 4.4.2 Vacuum cleaner ............................................................... 89 Concluding Remarks ............................................................................. 89 Bibliography ......................................................................................... 90 Web Resources ..................................................................................... 93 4.1 4.2 vi Contents 5. Critique of Fuzzy Logic ............................................................... 99 5.1 Introduction ................................................................................. 99 5.2 Objectives of Fuzzy Logic.............................................................. 99 5.3 Whats in a Name? ..................................................................... 100 5.4 Fuzziness vs. Probability ............................................................. 101 5.5 Fuzzy Logic vs. Multiple-Valued Logic ......................................... 102 5.6 Philosophical Issues .................................................................... 103 5.7 Engineering Applications Issues ................................................. 105 Concluding Remarks ........................................................................... 107 Bibliography ....................................................................................... 108 6. Neural Networks ....................................................................... 111 Introduction ............................................................................... 111 Background ............................................................................... 111 Learning .................................................................................... 112 The Basic Neuron Model ............................................................ 112 The Perceptron........................................................................... 116 The Multilayer Perceptron .......................................................... 118 Recurrent Network ..................................................................... 122 6.7.1 Hopfield Networks ......................................................... 122 6.7.2 Boltzmann Machine Networks ....................................... 124 6.8 Kohonen Self-Organizing Networks ........................................... 125 6.8.1 Background ................................................................... 125 6.8.2 Description ..................................................................... 126 6.8.3 Determining the winning neuron ................................... 127 6.8.4 Learning algorithm ......................................................... 127 6.9 Adaptive Resonance Theory, ART ............................................... 128 6.9.1 Background ................................................................... 128 6.9.2 ART ................................................................................ 128 Concluding Remarks ........................................................................... 131 Bibliography ....................................................................................... 132 Web Resources ................................................................................... 134 6.1 6.2 6.3 6.4 6.5 6.6 6.7 vii Contents 7. Hybrid Systems .......................................................................... 141 7.1 Introduction ............................................................................... 141 7.2 Fuzzy Neuron ............................................................................. 141 7.3 Multilayer FNN Architectures ...................................................... 143 7.4 Fuzzy ART .................................................................................. 145 7.6 Fuzzy ARTMAP ........................................................................... 147 7.7 Neural Fuzzy Systems ................................................................. 148 Concluding Remarks ........................................................................... 151 Bibliography ....................................................................................... 152 Web Resources ................................................................................... 156 8. Hardware Implementations ...................................................... 161 Introduction ............................................................................... 161 Digital Techniques ...................................................................... 162 Analog Techniques ..................................................................... 164 8.3.1 Voltage mode ................................................................ 164 8.3.2 Current mode ................................................................ 167 8.3.3 Mixed Mode .................................................................. 178 8.4 Fuzzy Analog Memory ............................................................... 181 8.5 Neurons ..................................................................................... 182 Concluding Remarks ........................................................................... 185 Bibliography ....................................................................................... 186 Web Resources ................................................................................... 191 8.1 8.2 8.3 9. Software Tools ........................................................................... 205 9.1 9.2 Introduction ............................................................................... 205 Software Overview ..................................................................... 206 Quizzes ............................................................................................ 241 Embedded Systems ............................................................................. 241 Fuzzy Sets & Relations ......................................................................... 242 Embedded Fuzzy Applications ............................................................. 244 Hybrid Systems ................................................................................... 248 Hardware Realization .......................................................................... 249 viii Contents Genetic Algorithms ......................................................................... 251 1. 2. Introduction ............................................................................... 251 Classified Annotated Web Bibliography ..................................... 255 2.1 Glossaries ....................................................................... 255 2.2 Introductions and Tutorials ............................................. 256 2.3 Research Papers ............................................................. 259 2.4 Interactive Demos .......................................................... 259 2.5 Java Related Resources ................................................... 260 2.6 Matlab Related Resources .............................................. 261 2.7 Other Languages Resources ........................................... 261 2.8 Fuzzy Logic and Genetic Algorithms ............................... 262 Symbols and Acronyms .................................................................. 265 Circuit Symbols and MOSFETs ........................................................ 269 Glossary ........................................................................................... 275 Index ................................................................................................ 289 ix [This is a blank page.] Preface Fuzzy logic and its applications are now well-established and arguments for and against it have reached a steady state. There is an overwhelming volume of literature on the topic making it a difficult task for a practicing engineer, beginner researcher, or an advanced student to grasp the topic and then apply the acquired knowledge with only a small investment of time and money. This book is intended to present fuzzy logic and its applications for embedded systems succinctly yet comprehensively, with a self-contained, simple, readable approach. Simplicity here means the omission of extraneous sentences and phrases, and the exclusion of non-applied mathematical and research-oriented details. It is intended for the intelligent reader with an alert mind. The approach, the organization, and the presentation of this book are also hoped to enhance the accessibility to existing knowledge beyond its contents. An extensive bibliography not only of printed material but also of annotated Web links is provided at the end of each chapter. The book is divided into nine chapters in addition to a set of quizzes, an appendix, a list of symbols and acronyms, and a glossary. Chapter 1 gives an overview of embedded systems and their implementation techniques. The chapter introduces the wide scope of embedded systems. The relationship between fuzzy logic and embedded systems is also outlined. Chapters 2 and 3 introduce succinctly the concepts of fuzzy sets, fuzzy operations, and fuzzy relations with illustrative examples. The discussion is geared toward what would be needed in order to design fuzzy embedded systems. In Chapter 4, embedded fuzzy logic applications are introduced with simplified case studies. Contrasting fuzzy logic control with conventional control is emphasized. It is hoped that by the end of this chapter that the reader would be able to apply fuzzy logic to the design of an embedded system of interest. The reader may wish to consult Chapter 8, Fuzzy Software Tools, select a few tools and start experimenting with fuzzy systems design and simulation. A critique of fuzzy logic is presented in Chapter 5, a topic that is not commonly discussed in the engineering literature on fuzzy logic. It is, however, important for engineers to understand the limitations and implications of any methodology they intend to use. No tool is suitable for everything; it must be used skillfully within its xi Preface bounds of applicability. Moreover, most engineers are interested in intellectual discussions relevant to the topic at hand as long as the ideas are communicated properly and to the point. The discussion in this chapter is brief, but with an extensive bibliography provided for those who may like to further research any of the points discussed. Chapter 6 presents the fundamentals of artificial neural networks, which are closely related to fuzzy logic but with fundamentally different concepts. The chapter introduces basic structures and learning algorithms of neural networks. Hardware realizations are outlined in Chapter 8, which discusses both analog and digital implementations. Chapter 9 provides the reader with an opportunity to gain hands-on experience with a minimum investment of time and money. The chapter gives an overview of numerous software tools for fuzzy logic systems, neural networks, and neuro-fuzzy systems. The software reviewed includes C, C++, and Java source codes, in addition to M-files for MATLAB. Design and analysis software tools with graphical user interfaces are also discussed, as well as software that is meant to demonstrate fundamental concepts. All software tools, or working demos of them, are available for downloading through the Web along with documentation and examples in most cases. Since most of the ideas presented are now well established there are no references given within the chapter text. However, further reading and expansion on the information presented is facilitated by the selected bibliography provided at the end of each chapter. The Web resources selected are those available at no charge, relevant to the topics discussed, and expected to be reliable. The idea of citing references from the Web does not yet hold wide acceptance in some academic circles. The major argument against the idea is the questionable reliability of the content and its sustained availability. It should be remembered that content does not become reliable just because it is printed on paper, and unreliable when it is posted freely on the Web. Nevertheless, many of the resources cited as accessible through the Web are documents that were published on paper as well, and the Web simply enhances their availability. The use of the Web is essential for some of the resources such as interactive demos. It should also be remembered that books are known to go out of print and many may not be physically accessible with ease. Of course, an available, well written book constitutes a better and more convenient source of knowledge. Accessibility through the Web would be the next best thing, but it requires guidance, as provided here, to make the most out of it. The set of quizzes with answers provided is meant to help the reader ponder about the subjects introduced without being side-tracked from the final goal that could be better reached by using the resources of chapter 9; thus learning through practice. xii Preface The appendix introduces the fundamental concepts of Genetic Algorithms (GAs). It is an optimization technique sometimes used in conjunction with the design of fuzzy and neural systems. A classified, annotated Web bibliography is provided for the reader who may wish to study the topic further. A list of symbols and acronyms is provided along with the circuit symbols of MOSFETs that appear in literature on fuzzy logic embedded implementations to help the reader avoid possible confusion when consulting the resources. A glossary of terms related to embedded systems, fuzzy logic and neural networks is provided. Although it refers to terms used in the various chapters of the book, it also introduces some terms to expand the coverage and provoke further interest in the general topic. If an instructor is to use this book as a teaching resource, numerous applicationoriented exercises and mini-projects can be assigned to the students if the instructor wishes to do so. For example: The quizzes provided could be the basis for class discussion on the fundamentals of fuzzy logic and neural networks. An item from the numerous Web resources provided could be selected and students asked to research it further and expand on the short review given. Students could be asked to research entries from the glossary further. Students could be asked to run a selected demo simulation or Java applet from the resources provided in Chapter 9, observe its action, document their observations, and relate it to the theory. Students could be assigned a system of interest such as a refrigerator, vacuum cleaner, auto-focus, level control, speed control, etc. They could identify the inputs and outputs, design a linguistic model, then simulate it using one of the tools discussed. Students could be also asked to produce their own source code and executable file to solve one of the above problems. A topic from Chapter 5, Fuzzy Logic Critique, could be selected for further research and discussion. xiii Preface It is hoped that through this book the reader will: Gain an understanding of the wide range of embedded systems and their future trend. Be able to use fuzzy sets and fuzzy logic algebra. Recognize when and why it would be advantageous to use fuzzy logic. Understand the fundamentals of neural networks and recognize when it is advantageous to use them. Gain familiarity with the hardware implementation approaches of fuzzy logic [This is a blank page.] for embedded systems applications. Be able to experiment with the design of fuzzy systems for embedded systems applications. Be able to pursue further details of a topic within fuzzy logic embedded systems applications with relative ease. I would like to express my gratitude to Dr. Alexander Berezin, McMaster University, Hamilton, Ontario, Canada for his encouragement and useful discussions. I would also like to thank Dr. Lawrence Hulsman, University of Arizona, Tucson, USA for his continuous encouragement and inspiration. The comments provided by Mahmoud Gadala, Senior Engineer, Pratt & Whitney Canada Inc, Montreal, Quebec, Canada are highly appreciated. I would also like to thank Dr. Ahmed Hussein, University of Northern British Columbia, Prince George, British Columbia, Canada, Dr. Erol Inelmen, Bogazici University, Istanbul, Turkey, Prof. Predrag Pesikan, DeVry, Mississauga, Ontario, Canada, and Dr. Aleksander Malinowski, Bradley University, Peoria, Illinois, USA; their interest and input are particularly valued. xiv About the Author Dr. Ibrahim is a senior member of the Institute of Electrical and Electronics Engineers (IEEE), a member of the Association of Professional Engineers of Ontario (APEO), the Material Research Society (MRS), the American Association of Engineering Education (ASEE), and the International Banknote Society (IBS). He lectured in the area of electronics on three continents to a diverse population of students and presented seminars and workshops to practicing engineers. He has a wide range of publications including papers in refereed journals and conferences. Dr. Ibrahim has organized and chaired sessions on Current Trends in Electronics Education during the IEEE Conferences IECON01, Denver, Colorado, USA, and IECON02, Sevilla, Andalucia, Spain. He was a guest editor for the 2003 Special Issue of the International Journal of Engineering Education on Current Trends in Electronics Education. He was a member of the International Advisory Committee of the First and Second International Conferences on Distance Learning held at the Belarussian State University of Informatics and Radioelectroics, Minsk, Belarus in the years 2001 and 2002. Dr. Ibrahim has a B.Sc. (EE) degree from Ain Shams University, Cairo, Egypt, and a Masters (Eng. Phys.) and Ph.D. (EE) degrees from McMaster University, Hamilton, Ontario, Canada. At present he is a senior professor at DeVry, Mississauga, Ontario, Canada. xv Whats on the CD-ROM? Included on the accompanying CD-ROM: A fully searchable eBook version of the text in Adobe pdf format Links to numerous useful fuzzy-related web sites Additional informative resources and documents Access the Read_Me file for more details on the CD-ROM contents. xvi CHAPTER 1 Embedded Systems: An Overview 1.1 Definition and Examples A system whose principal function is not computational but is controlled by a computational system embedded within it is referred to as a computational embedded system, which is usually shortened to embedded system. The word embedded implies that the computational system lies within an overall system. The user of the overall system may not even be aware of the computational systems existence. The phrase computational system refers to computers, microprocessors, microcontrollers, DSP chips, or custom-made hardware along with the software that may be associated with them. Desktop or laptop computers, although they contain microprocessors, are not considered embedded systems since their principal function is computational. If such a computer is customized and built permanently into an identifiable system with the sole purpose of controlling that particular system, then one can refer to it as part of an embedded system. Embedded systems are used in a wide range of applications. They can be loosely categorized as: Consumer Electronics including digital cameras, televisions, cell phones, and camcorders. Home Appliances including microwave ovens, rice cookers, thermostats, washing machines, and drying machines. Business and Office Equipment including alarm systems, card readers, product scanners, cash registers, fax machines, and copiers. Transportation Systems including automobiles (transmission control, fuel injection, antilock brakes, and cruise control), train systems, and avionic systems. Factory Control including machine control, instrumentation, and robotics. Medical Systems including life-support systems, testing systems, and diagnostic systems. The list of embedded system applications is growing continuously and is almost endless. 1 Chapter 1 1.2 Further Features of Embedded Systems Embedded systems share common features that include the following: Hardware and software An embedded computing system typically employs both software and hardware components. The emphasis in the design varies depending on the constraints of the design specifications. The capabilities of integrated circuit (IC) chips have been following the so-called Moores law, which predicts that IC technology will double its performance every 18 months. Figure 1.1 gives an illustration of Moores Law, showing the increase in the number of transistors in processors over time. 1,000,000 100,000 K Transistors Pentium 4 Processor Pentium ll Processor Pentium Processor 1 Billion Transistors Pentium lll Processor Pentium Pro Processor 10,000 1,000 100 10 1 '75 i486 Processor i386 Processor 80286 8086 '80 '85 '90 '95 '00 '05 '10 '15 Source: Intel Figure 1.1: The increase in the number of transistors in processors over the years. There is, of course, a physical limitation on the dimensions of a transistor which, when reached, means different physical laws become applicable. This may lead to practical application of quantum computing and a new meaning to embedded systems. It appears that the amount of software used per consumer product is doubling roughly every two years. The trend appears to be continuing and the number of familiar and new consumer products using all the capabilities of IC technology is increasing as prices become suitable for consumers. One has to be careful, however, when talking about the amount of software. The number of lines of code and their rate of execution would not be enough to evaluate the performance of a system independent of what a code instruction can do. Manufacturers appear to depend on software to improve the user interface of products. They also depend more and more on software in introducing the product diversity needed in the market while maintaining the minimum hardware variety to ensure economical mass production. Some of the features of embedded computing systems include: 2 Embedded Systems: An Overview Tightly-Constrained Specifications Embedded computing systems typically have particularly tight design specifications. The physical dimensions of the system usually need to be small, requiring the system to fit on a single chip. The cost is required to be low as well. In most cases, the system must continually react to changes in its environment and to process the data fast enough as the changes occur (real-time operation). Reliability Bugs in embedded computing systems cannot be tolerated, as opposed to personal computing where software bugs are mere inconveniences. Rigorous systematic testing and commissioning are required. Implementing an embedded system with hardware or software problems could lead to serious consequences. For example, one significant software error in a life-support system or a transportation system could be fatal. The need for assured reliability becomes increasingly demanding considering the fact that numerous products with embedded systems have a relatively short life cycle. They tend to become obsolete due to technological changes. Networked Processors Some embedded computing systems may require the use of more than one processor, each to control one subsystem. These processors need to be networked to share data and to meet the needs of the overall system. For example, an automobile has subsystems that require electronic control. These include fuel injection and ignition, automatic transmissions, air-conditioning systems, shock absorbers, anti-lock braking systems, and air bags. A network of inexpensive processors could lead to a cheaper and simpler system implementation than a single processor performing all the tasks. Internet-enabled The increased popularity of the internet and multimedia has created a demand for embedded consumer product networking. It is predicted that the traditionally separated worlds of entertainment (audio/video equipment), communication (telephone and e-mail), and computing will become inseparable. All, in addition to home appliances (washing machines and refrigerators), would need to be linked to the internet or its successor. Table 1.1 gives a summary of major applications and initiatives of embedded internet application areas. 3 Chapter 1 Table 1.1: Summary of major applications and initiatives of embedded inherent application areas. Category Examples of Commercial Applications Home Appliances and Consumer Electronics Office Systems Factory Control Medical Systems Communications Equipment diagnostics and service, Web-based content distribution Copier, printer, and fax machine configuration and system management Machine controller configuration and diagnostics Diagnosis, treatment, and data communications Switch and network router configuration and management PDAs, cellphones, and pagers for Web-based content distribution GPS and traffic reports Maintenance and diagnostics Entertainment Automotive Systems 1.3 Design Metrics Design metrics are measurable features of the implemented systems. They are useful in characterizing and comparing various implementations of embedded systems. They include: Safety This feature overrides any other. A system that is not safe is not useful. The system needs to be guaranteed that it causes no harm to the individuals using it, the environment, or other systems it may come in contact with. Cost The overall cost includes the cost of designing, testing the prototype, and the cost of manufacturing each unit. Time-to-market This is the time it takes to develop a system and make it available on the market. It includes the time needed to design and test a prototype and the manufacturing time. Time-to-market needs to be much shorter than the lifetime of the product in the market before consumers consider it to be obsolete. This time will have a direct impact on the expected revenues from marketing the product. 4 Embedded Systems: An Overview Size In numerous embedded systems the physical size and weight of the system are important. They are related in many cases to the hardware and software used. Software size is measured in bytes and number of instructions of the code used. The number of gates or transistors are used to measure the size of the hardware. Performance Performance of an embedded system typically refers to the time the system takes to perform key tasks. This would include the response time, i.e. the time between the start of execution of a task and its finish, and the number of tasks performed per unit time. Sometimes the performance of a processor is cited in instructions per unit time; one has to be careful when comparing processors on such a basis to consider what an instruction can do. Power Consumption Power consumption is an important feature. The power consumed by the embedded system determines the lifetime of the systems battery, the cooling requirements of the hardware, and reliability. Figure 1.2 shows that dissipated power has increased over the years as the number of integrated transistors has increased. 100 P6 Watts/cm2 10 8086 286 1 8085 8080 8008 4004 1971 1974 1978 1985 1992 2000 486 386 0.1 Figure 1.2: Trend of power dissipation over the years. (Source: Intel) Design Modification Flexibility This feature refers to the level of ease with which the functionality of the system can be modified with minimum cost to produce a new prototype. Well-documented systems enable designers, even those who were not involved in the initial design, to maintain and modify a given system economically. 5 Chapter 1 Bench Marking The EDN Embedded Microprocessor Benchmark Consortium (EEMBC, pronounced embassy) was formed in 1997 to develop performance benchmarks for processors for embedded applications. EEMBCs benchmarks are meant to address real-world applications in areas such as automotive/industrial/consumer, networking, telecommunications, and office automation. This would provide designers with a certified method of comparing and judging performance. It is important to observe that the metrics are typically correlated, either positively or negatively. Challenges occur and hence there is a need to compromise when the correlation is negativei.e., improving one feature often leads to degrading another. 1.4 Implementation Embedded systems implementation relates to three general sets of technologies; processor, hardware and software technologies; IC chip technologies; and design and test technologies. The following sections give a short overview of these categories. 1.4.1 Processor Technologies General-Purpose Processors A general-purpose processor, or a microprocessor, is a predesigned programmable digital system that can be used for numerous diverse applications. It is used in embedded systems by programming its memory to execute the required operations. Such an approach to embedded system design has the advantage of design modification flexibility through program modification. It also has the advantage of a lower prototype development cost and hence a lower cost for the production of a small number of units. In addition, the development time is relatively short. However, the cost is higher than custom-designed processors if the number of units required to be manufactured is large. The size and power consumption could be larger than necessary because of the existence of unneeded processor hardware. Several software tools are typically available for programming a chosen general-purpose processor. The existence of an Integrated Development Environment (IDE) simplifies the design process further since it enables writing of the source code, its compilation, and linking it into an executable file on the intended microprocessor, all from within a single application A microcontroller is a special type of microprocessor, designed for embedded applications. It usually features on-chip program and data memory to enable single-chip design solutions. 6 Embedded Systems: An Overview Limited-Purpose Processors Limited purpose processors include single-purpose and special-purpose processors. A single-purpose processor is a digital circuit designed to execute one particular program. Other terms used include coprocessor, accelerator, and peripheral. A special-purpose processor is a digital circuit that can execute numerous programs, but its design is optimized for a particular application area, for example digital signal processing (DSP). The performance of such processors is typically faster than that of general-purpose processors. Also, they are smaller and their power consumption is lower. However, they are less flexible. The unit cost for small quantities and the nonrecurring engineering costs are also higher. 1.4.2 Integrated Circuit Technologies Processors are implemented using integrated circuit technologies. Integrated circuits (ICs) are sometimes referred to as chips. They are typically classified based on the number of transistors with which the chip is built: SSI (Small-Scale Integration): up to 100 per chip; for example, logic gate chips. MSI (Medium-Scale Integration): from 100 to 3000; for example, flip-flops and counters chips. LSI (Large-Scale Integration): from 3000 to 100,000; for example, peripheral interface chips. VLSI (Very Large-Scale Integration): from 100,000 to 1,000,000; for example, microprocessors, and memory chips. ULSI (Ultra Large-Scale Integration): more than 1 million. The line between VLSI and ULSI is vague, and the term ULSI is most commonly used in Japanese literature. Silicon (Si), and gallium arsenide (GaAs), are semiconductor materials used for IC fabrication. The physical properties of GaAs lead to inherently faster devices, but it is more expensive than Si. Two technologies are used for silicon-based ICs: Bipolar Junction Transistors (BJT), and Metal-Oxide-Semiconductor Field Effect Transistors (MOSFET). Bipolar technology leads to device families such as TTL (transistor-transistor logic) and I2L (integrated injection logic). The bipolar technology offers high speed and high current drive, but leads to higher power dissipation and lower circuit complexity. The MOSFET technology offers low power dissipation, a very high level of integration, and better electrical characteristics. Circuit realization techniques may utilize: 7 Chapter 1 Enhancement type n-channel (EnMOS) Both enhancement and depletion type n-channel (EDnMOS) Both p-channel and n-channel enhancement MOSFET, which is known as CMOS (complementary metal-oxide semiconductor). CMOS-based implementation is becoming the standard process for all but the highest speed devices. It offers lower power dissipation but the circuit complexity for realization is lower compared to those which may be realized with EDnMOS. BiCMOS devices are based on both bipolar and CMOS technologies. They give the best of the two technologies, but lead to increased cost and fabrication complexity. An IC is fabricated by creating several layers of different characteristics in a Si wafer. Such layers can be created by depositing photosensitive chemicals on the surface of a silicon wafer and shining light through masks to change the properties of the chemicals. An etching process follows to remove parts of the chemicals as required to expose the surface according to a given pattern and make the required changes in the properties as needed. The layers are thus built using an appropriately designed set of masks (referred to as a layout). Figure 1.3 shows current and future CMOS structures. Gate Source Silicon Raised source and drain Source Oxide Silicon Insulator New gate dielectric Drain Gate Drain Figure 1.3: Intel CMOS current and future structure. (Source: Intel) 8 Embedded Systems: An Overview The top illustration of Figure 1.3 shows the basic structure of a CMOS transistor currently in use (year 2003). All CMOS circuits are built with transistors such as this, and connected to each other according to the circuit design. The Pentium 4 processor has 42 million such transistors integrated on a piece of silicon as small as a fingernail. The lower part of Figure 1.3 illustrates the structure of the new TeraHertz transistor that is expected to be in use within a few years. Its goal is to maintain the level of power consumption of present day microprocessors, although the future ones will be built with many more transistors. Technology generation has been defined by the feature size, , which is the smallest feature on an ICfor example, the length of the transistor channel. Current feature size is in the submicron range. Figure 1.4 shows the trend in feature size over the years. Gate Length (microns) 1.0 0.1 75 80 85 90 95 00 05 Figure 1.4: Feature size of Intel microprocessors over the years. (Source: Intel) There are numerous design styles for IC implementation of given logic functions or algorithms, each with its merits and demerits. These include: Semicustom ASIC In an application-specific IC, (ASIC), the chip is partially finished in a standard form, and then it is completed as the need arises based on the design to be implemented. ASICs include FPGAs, PLDs, and CPLDs. 9 Chapter 1 Field Programmable Device, FPD FPD is a general term that refers to any type of integrated circuit used for implementing hardware by user programming. These devices are also referred to as Programmable Logic Devices, PLDs. They are similar in principle to the EPROM (Electrically Programmable Read Only Memory), but they have many more potential applications. SPLD refers to simple PLD, and CPLD refers to complex PLD. A CPLD consists of several interconnected programmable SPLD-like blocks on a single chip. CPLDs feature logic resources with a large number of inputs. A Field Programmable Gate Array (FPGA), is an FPD with a general structure that allows very high logic capacity. It offers fewer logic resources and a higher ratio of flip-flops to logic resources than a CPLD. Gate Array, GA Design implementation with a GA is done with mask design and processing, not by user programming. The manufacturing process has two phases. The first is based on generic masks, or standard masks, and results in an array of uncommitted transistors on the GA IC chip. These chips are later customized as the need arises by defining the metal interconnects between the transistors of the array. Standard-Cells Based Integrated Circuits, CBIC In standard-cells based design, all of the commonly used logic cells are designed, characterized, and stored in a standard cell library in a computer. A typical library may consist of a few hundred cells for inverters, NAND gates, NOR gates, latches, flip-flops, etc. Multiple implementation technologies may be used for each cell to provide an adequate choice of electrical characteristics as they may be needed in the design. The characterization of each cell is done for several different categories, including delay-time vs. load capacitance, timing simulation model, circuit simulation model, fault simulation model, cell data for place-and-route, and mask data. CBIC based design is highly flexible in allowing cells to be placed, modified and connected with a chip area typically 1015% smaller compared to gate arrays. Each design requires the production of a unique full mask set leading to longer time-to-market and higher non-recurring engineering costs making it a more suitable deign style at high volumes. This design style is sometimes referred to as full-custom design, but in a strict sense it is less than full custom because the cells used are pre-designed and may be utilized in numerous varied chip designs. Full-Custom Design In a full-custom design, all the layers of a particular embedded system design are optimized; the optimization includes interconnect lengths and transistor 10 Embedded Systems: An Overview sizes. The design productivity is relatively low, the cost is high, and the timeto-market is long, but excellent performance with optimum size and power consumption can be achieved. A comparison between the characteristics of various implementation styles is given in Table 1.2. Table 1.2: Summary of selection criteria for various technology implementation styles. Criterion PLDs Semicustom GAs CBICs Full Custom Time to market Performance Architectural flexibility Volume dependence Solution efficiency Application support Design change cost Development cost Short High Medium to High Medium High High Low Medium High Higher High High Some High Medium to high Long Very high Highest High High Very high None Very high Very high Low Much Medium Low High Some High Medium to high 1.4.3 Design Technologies A top-down design process comprises progressively refined abstraction levels that include: System Specification At this level, the designer describes the functionality of the system. Behavioral Specifications The designer refines the system specifications by assigning them to processors; this leads to behavioral specifications for each processor. 11 Chapter 1 Register Transfer Specifications Behavior specification is further refined by converting the processor behavioral specifications to a set of register transfer (RT), and state machines. Logic Specifications The RT level is further refined into logic specifications expressed through Boolean equations. A machine code for general-purpose processors and gate-level netlists for single-purpose processors is then reached. The design process can be optimized by using appropriate software tools to progress through the levels described. Using libraries of existing implementations could enhance productivity. Such libraries exist at all of the levels, including systemlevel libraries that might consist of complete systems solving particular problems. Testing correct functionality to prevent time-consuming debugging is most commonly achieved through simulations. At the logic level, gate-level simulators provide output timing waveforms for given input waveforms. General-purpose processor simulators execute machine codes, and hardware description language (HDL) simulators execute RT-level description and provide output timing waveforms for given input waveforms. At the behavioral level, HDL simulators enable hardware/software verification. Model checkers verify the completeness and correctness of the specifications. 1.5 Fuzzy Logic and Embedded Systems An objective of fuzzy logic has been to make computers think like people. Fuzzy logic can deal with the vagueness intrinsic to human thinking and natural language and recognizes that its nature is different from randomness. Using fuzzy logic algorithms could enable machines to understand and respond to vague human concepts such as hot, cold, large, small, etc. It also could provide a relatively simple approach to reach definite conclusions from imprecise information. Almost every application, including embedded control applications, could reap some benefits from fuzzy logic. Its incorporation in embedded systems could lead to enhanced performance, increased simplicity and productivity, reduced cost and timeto-market, along with other benefits. Fuzzy logic has the advantage of modeling complex, nonlinear problems linguistically rather than mathematically and using natural language processing (computing with words). The use of fuzzy logic requires, however, the knowledge of a human expert to create an algorithm that mimics his/her expertise and thinking. Also, studying the stability of a fuzzy system is a demanding task. Numerous applications, including embedded ones, combine the use of fuzzy logic and neural networks. Neurofuzzy techniques take advantage of both fuzzy logic and neural networks, leading to systems that can: 12 Embedded Systems: An Overview Mimic the human decision-making process Handle imprecise or vague information Learn by example and hence do not require the knowledge of a human expert Self-learn and self-organize. Process numeric, linguistic, or logical information Concluding Remarks Embedded systems have become a ubiquitous part of our daily life at work and play. They have gained importance for productivity and convenience. Industrial control systems, medical instruments, transportation vehicles, bank machines, washing machines, vacuum cleaners, and many other devices now depend on embedded systems. It appears as if our current civilization is built around embedded systems. The demand for higher performance and sophistication of embedded systems is increasing. Increased market demands require embedded systems to be developed even further at a rapid pace. Fuzzy logic and neural approaches can provide a mechanism for getting the most out of embedded system capabilities and making them more intelligent. They can also accelerate the development cycle and reduce the time-tomarket of new products to meet ever-increasing demands. It is important for engineers to know about the capabilities of fuzzy logic and neural networks as possible design approaches from which one may select the most suitable for the problem at hand. 13 Chapter 1 Bibliography 1. I. D. Agranat, Engineering Web Technologies for Embedded Applications, IEEE Internet Computing, 4045, May/June, 1998. 2. W. Banks and G. Hayward, Fuzzy Logic in Embedded Microcomputers and Control Systems, Byte Craft Limited, Waterloo, Ontario, 2001. 3. S. Brown and J. Rose, FPGA and CPLD Architectures: A Tutorial, IEEE Design & Test of Computers, 4256, Summer 1996. 4. J. P. Calvez, Performance Assessment of Embedded Hw/Sw Systems, Proceedings of the International Conference on Computer Design, ICD95, 1995, pp. 5257. 5. M. Chiodo, P. Giusto, A. Jurecska, M. Marelli, H. C. Hsieh, A. S. Vincentelli, and L. Lavagno, Hardware-Software Codesign of Embedded Systems, IEEE Micro, August 1994, pp. 2636. 6. J. Debardelaben, V. K. Madisetti, and A. J. Gadient, Incorporating Cost Modeling into Embedded System Design, IEEE Design & Test of Computers, July 1997, pp. 2435. 7. R. E. Filman, Embedded Internet Systems Come Home, IEEE Internet Computing, 52-53, January/February, 2001. 8. D. D. Gajski and F. Vahid, Specification and Design of Embedded HardwareSoftware Systems, IEEE Design & Test of Computers, Spring 1995, pp. 5367. 9. B. H. Lee, Embedded Internet Systems, IEEE Internet Computing, 24-29, May/June, 1998. 10. D. Mlyneck and Y. Leblebici, Design of VLSI Systems, vlsi.wpi.edu/webcourse, 1998. 11. D. Mulchandani, Java for Embedded Systems, IEEE Internet Computing, 3039, May/June, 1998. 12. G. Rozenberg and F. Vandrager, Lectures on Embedded Systems, Springer, New York, 1998. 13. M. Schaaf and F. Maurer, Integrating Java and COBRA, IEEE Internet Computing, 7278, January/February, 2001. 14. M. Schlett, Trends in Embedded Microprocessor Design, IEEE Computer, 4449, August 1998. 15. D. C. Schmidt, Middleware Techniques and Optimization for Real-time Embedded Systems, The 12th International Symposium on System Synthesis, Boca Raton, Florida, 1-4 November 1999, pp. 1217. 14 Embedded Systems: An Overview 16. D. Sciuto, Design Tools for Embedded Systems, IEEE Design & Test of Computers, 1113, April/June 2000. 17. M. J. Sebastian Smith, Application-Specific Integrated Circuits, Reading, MA: Addison-Wesley, 1997. 18. Texas Instruments, Implementation of Fuzzy Logic: Selected Applications, Technical Report SPRA028, January 1993. 19. F. Vahid and T. Givargis, Embedded System Design, John Wiley & Sons Inc., 2002. 20. A. R. Weiss, The Standardization of Embedded Benchmarking: Pitfalls and Opportunities, Proceedings of the IEEE International Conference on Computer Design, 10-13 October, Austin, Texas, 1999, pp. 492498. 21. T. Wilmshurst, An Introduction to the Design of Small-scale Embedded Systems, Palgrove, New York, 2001. 22. W. Wolf, Computers as Components: Principles of Embedded Computing System Design, Morgan Kaufmann Publishers, San Francisco, 2001. 23. W. Wolf, CAD Techniques for Embedded Systems-on-Silicon, Proceedings of the IEEE International Conference on Computer Design, 1013 October, Austin, Texas, 1999, pp. 291308. 24. L. A. Zadeh, Making Computers Think Like People, IEEE Spectrum, August 1994, pp. 2632. 25. L. A. Zadeh, Fuzzy Logic, IEEE Computer Magazine, April 1988, pp. 8393. 15 Chapter 1 Web Resources 1. EETimes www.eet.com/ This is the Web site of EETimes, a leading weekly newspaper for the Electronics Industry. It provides a section for embedded systems (www.embedded.com/) with news, editorials, articles and downloadable items. 2. Interactive Week www.zdnet.com/intweek/ This the Web site of ZDNet News, a site devoted to the electronics industry. It provides news, technology updates, reviews, downloads and prices. The site is searchable. 3. Embedded Technology www.embeddedtechnology.com/ A site devoted to embedded technology. It provides links to reviews, services and downloads. It enables buying and selling online and gives access to free newsletters and trade publications. The site is searchable. 4. Embedded Systems References www.microcontroller.com/ A Web site intended to provide resources to the professional embedded systems developer. The main categories include: Microcontrollers, Semiconductors, News, Embedded Marketplace, and more. The site is searchable. 5. Programmable Logic Software www.optimagic.com/software.html This site provides links to sites related to design software for programmable Logic (FPGA, CPLD, and PLD), both free and commercial. They include: Schematic-Based Tools such as: VIEWlogic, Mentor Graphics, Cadence Design Systems, OrCAD, ALDEC Active-HDL, Capilano Computing Systems Design Works, VeriBest, Protel, Tanner Research, Morphologic rapid-development system, MyCAD Logic Synthesis and Optimization including: FPGA Express, FPGA Compiler, Exemplar Logic, Synplicity, VIEWlogic, Cadence Design Systems, Accolade Design Automation, Compass Design Automation Design Verification, Programmable Logic Vendor Software including: Xilinx, Altera, Vantis, Lattice, Lucent, Actel, Cypress Warp2, Atmel FPGA, and University or Research Languages. 16 Embedded Systems: An Overview 6. Compact HTML for Small Appliances www.w3.org/TR/1998/NOTE-compactHTML-19980209/ An article in HTML format about compact HTML for small appliances by Tomihisa Kamada, ACCESS Co.,Ltd, Japan. The Compact HTML proposed in the document defines a subset of HTML for small information appliances such as smart phones, smart communicators, mobile PDAs, etc. 7. Real Time, Embedded and Specialized Systems www.omg.org/realtime/ This is the Web site of the Real-time, Embedded, and Specialized Systems Platform Task Force (RTESS PTF). It relates to systems that have one or more of the following characteristics: real-time, embedded, fault tolerant, highly available, high performance, and safety critical. It provides industry links and presentations, and plans to provide white papers on relevant issues. 8. Embedded Microprocessor Benchmark Consortium (EEMBC) www.eembc.org The Embedded Microprocessor Benchmark Consortium develops and certifies benchmarks and benchmark scores to help designers select embedded processors. EEMBC was formed in 1997, its membership includes more than 45 leading companies. According to the statement made on the Web site, every processor submitted for bench marking is tested for up to 46 different parameters, each representing a different workload and capability in telecom, networking, consumer, office automation, and automotive industrial applications. Some sections of the Web site require membership. 9. Bluetooth SIG www.bluetooth.com/ A searchable Web site dedicated to Bluetooth technology, which is a worldwide specification for low-power, low-cost, short-range wireless networking that provides links between mobile computers, mobile phones, other portable handheld devices, and connectivity to the internet. 10. Windows NT Embedded Web Site www.microsoft.com/windows/embedded/nt Microsoft Windows embedded systems Web site. It provides information, news, products, etc. It is searchable. 17 Chapter 1 11. Analog Devices www.analog.com/ This is the Web site of Analog Devices. It provides specifications and application notes related to embedded systems and devices, among other information. The site is searchable. 12. Intel www.intel.com/research/silicon This Web page provides links to Intels research activities in areas such as nano technology, microprocessors, human computer interface and more. The figures of Chapter 1 were obtained from this Web site. 13. Sun Microsystems www.javasoft.com This site describes itself as the source for Java Technology. It provides links to information about Java technologies, downloading software tools, solutions, tutorials, and more. 14. Texas Instruments http://focus.ti.com/docs/apps/appshomepage.jhtml This is the Texas Instruments web page for semiconductor applications including: Audio systems, Broadband Solutions, Digital Control, Video and Imaging, Optical Networking, and Wireless Communications. 15. Free On-line Dictionary of Computing www.instantweb.com/~foldoc/ The On-line Dictionary of Computing, FOLDOC, is a searchable dictionary of acronyms, jargon, programming languages, tools, architecture, operating systems, networking, theory, conventions, standards, mathematics, telecoms, electronics, institutions, companies, projects, products, history; in fact, anything to do with computing. The editor is Denis Howe. 16. Microcontrollers craft a networked future www.eetimes.com/story/OEG20010521S0061 An article by Bernard Cole that appeared in EE Times May 21, 2001. The article indicated that activity is increasing in 8- and 16-bit, and 16- and 32bit deeply embedded microcontrollers. Even as market attention fixates on 32-bit embedded RISC and DSP in everything from set-top boxes and cell phones to internet-enabled personal digital assistants, the deeply embedded microcontroller space is gaining acreage as opportunities open up. In communications and networking, these parts are playing roles as ancillary processors as they expand the capabilities of existing embedded systems by adding internet ability. 18 Embedded Systems: An Overview 17. Intel MCS 96 Microcontrollers www.intel.com/design/mcs96/saback.htm Information on the Intel MCS 96 16-bit microcontroller family. It discusses Intel expanding its popular MCS 96 microcontroller family with the next generation of 16-bit microcontroller architecture. 18. Fujitsu Microcontrollers www.fme.fujitsu.com/products/micro/16bit/ Information from Fujitsu about its spectrum of 16 bit microcontrollers, covering general-purpose and application-specific types with a rich variety of features, including the latest technologies such as on-chip Flash ROM. 19. National Semiconductor Microcontrollers News www.national.com/news/0,1737,2000+39,00.html This provides news from National Semiconductor about microcontrollers. The site has an archive that is searchable by date and by category. 20. The McClean Report www.icinsights.com/prodsrvs/mcclean/mcclean_section10.html This is an abstract of Section 10 (Microcontrollers and Digital Signal Processors) of the McClean report, which provided analysis and forecasts for the Integrated Circuit industry. 21. Trends in Embedded Microprocessors Design www.realtime-info.be/magazine/98q4/1998q4_p014.pdf An article by M. Schlett, Hitachi Europe. It appeared in IEEE Computer, 31, 8, 4449, 1998. 22. Serial Communications Systems in the Automobile www.mcjournal.com/articles/arc100/article4.htm An article by Ross Bannatyne, Transportation Systems Group, Motorola Inc. It relates to vehicles, electronic systems architecture, which has become a network of real-time distributed control systems. 23. Aptronix www.aptronix.com/tech/ The Web site of Aptronix, a company specializing in fuzzy logic, expert systems, data mining and neural networks as applied to automation and controls. The page provides links to numerous fuzzy embedded applications. 19 Chapter 1 24. InterScience www.intersci.com/projects/fuzzy.html According to the the information provided, the compnay has been processing real-time video with the use of Field Programmable Gate Arrays (FPGAs) since 1992. Applications implemented include frame subtraction, convolutions, imagebased triggering and compression engines. The page has information about fuzzy logic and links to information on embedded signal processing, automated systems for testing EEPROMs, automated testers, and more. 25. Omron http://oeiweb.omron.com/oei/index.htm A company specializing in industrial automation and electronic control components. The site is searchable and provides links to information about numerous applications of fuzzy logic in embedded systems applications. 26. Byte Craft Limited Publishing www.bytecraft.com/publishing.html Two books are available for free in PDF format: First Steps with Embedded Systems, and Fuzzy Logic in Embedded Microcomputers and Control Systems. 27. Getting Started with Programmable Logic http://tutor.al-williams.com/pld-1.htm A tutorial on programmable logic from AWS Electronics, League City, Texas, USA. It is presented in HTML format. 28. Links to VLSI Information www.mrc.uidaho.edu/vlsi/ Web links related to VLSI provided by the Microelectronics and Communications Institute, University of Idaho, USA. 29. CPU Info Center http://bwrc.eecs.berkeley.edu/CIC/ A site maintained by Berkeley Wireless Research Center. It provides links to information related to microprocessors, for example: Online Technical Documents, CPU & System, Performance Information, Embedded Microprocessor Information, History of CPUs, and Processor Road-maps. 30. Microelectronics A Tool for Innovation www.madess.cnr.it/pubblicazioni/microelectronics_2001.pdf A report on the National Italian Project MADESS II. It has extensive reviews and statistics on microelectronics and its applications globally. 20 Embedded Systems: An Overview 31. Conference Proceedings http://jamaica.ee.pitt.edu/Archives/ProceedingArchives/ This site provides links to archives of numerous conferences related to the design of embedded systems, including: Hardware/software Codesign Conference, Design Automation Conference, Design, Automation & Test in Europe Conference, International Conference on Computer Aided Design. The proceedings are searchable by author and topic and complete papers are available in PDF format. 21 [This is a blank page.] CHAPTER 2 Fuzzy Sets 2.1 Introduction One can view fuzzy sets as a generalization of classical sets, or crisp sets as they are sometimes called. Classical sets and their operations are particularly useful in expressing classical logic and they lead to Boolean logic and its applications in digital systems. Fuzzy sets and fuzzy operations, on the other hand, are useful in expressing the ideas of fuzzy logic leading to applications such as fuzzy controllers. The following sections start with a short review of classical sets: their notations and operations. This is followed by fuzzy sets and their operations. 2.2 Classical Sets A set is defined as a collection of objects that may share certain characteristics. For example, one may define a set of positive integers, a set of students with passing grades, and a set of honest politicians. Each individual object is referred to as an element or member of the set. In a classical set an object x is either a member of a given set A (expressed as x A) or not a member (expressed as x A); partial membership is not allowed. There are numerous ways to define a set: One may specify the properties of its elements. For example, A = {x|x is an odd number <10} One may list all the members of the set. For example, A = {1, 3, 5, 7, 9} One may use a formula to define the set. For example, A = {xi = xi + 1, i = 1,...,5, where xi = 1} The set could also be defined as the result of a logical operation. For example, A = {x|x is an element that belongs to B OR C} 23 Chapter 2 A membership function, , can be used to define a set. A(x) = 1 A(x) = 0 if x A, and if x A for all values of x. Example Let all the numbers under consideration, i.e. the universe of discourse, be defined as {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}. Then, the set of odd numbers can be expressed as {(1,1), (2,0), (3,1), (4,0), (5,1), (6,0), (7,1), (8,0), (9,1), (10,0)}. Where each member of the universe of discourse is associated with a membership value in the form (#, ). The numbers 1, 3, 5, 7, and 9 are associated with = 1 because they form the set of odd numbers extracted from the universe of discourse. This method of defining a set can be easily extended to define a fuzzy set by allowing partial membership. Universal Set The set that consists of all the elements of interest for a particular application (the universe of discourse) is referred to as the universal set. It is the mother of all sets; any set that is not a universal set is a subset. One may write A I to mean that a set A (any set) is actually a subset of the universal set I. Empty Set A set that has no elements is referred to as an empty set and is denoted by the symbol . It is a shell that represents a description that does not happen to fit anything. (Contrary to what some may think, the set of Honest Politicians is not an example of an empty set!) An empty set is a subset of any other set, i.e. A I. Power Set The set which consists of all possible subsets of a given set A is called a power set and is denoted as P(A) = {x|x A} Example If A = {1, 2, 3}, then P(A) = {, {1}, {2}, {3}, {1,2}, {1,3}, {2,3}, {1, 2, 3}} 24 Fuzzy Sets Cardinality The number of elements of a given set is called the cardinality of the set. The cardinality of a set A is denoted by # A or |A|. The cardinality is a finite number for finite sets. Example If A = {1, 2, 3, 5, 7}, then # A = 5. 2.3 Set Operations Sets can be manipulated through numerous operations such as the complement, union, intersection, subtraction, and cartesian product. These will be defined and explained in the following sections. COMPLEMENT The COMPLEMENT, or ABSOLUTE COMPLEMENT of a given set A is denoted by A . It is defined by A = { x x I and x A }. It is demonstrated graphically in Figure 2.1. I A A Figure 2.1: The complement of set A. If the membership function of set A is A ( x ) and that of A is A ( x ) , then one can write A ( x) = 1 A ( x) . This is illustrated graphically in Figure 2.2. 25 Chapter 2 a) (x) 1 0 x A(x) b) 1 0 x A(x) c) 1 0 x Figure 2.2: Complement in terms of membership function. a) the universe of discourse b) set A c) the complement of A Example Let I = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10} If A = {(1,1), (2,1), (3,1), (4,1), (5,1), (6,0), (7,0), (8,0), (9,0), 10,0)} = {1, 2, 3, 4, 5} Then, A = {(1,0), (2,0), (3,0), (4,0), (5,0), (6,1), (7,1), (8,1), (9,1), (10,1)} = {6,7,8,9,10}. Example If I = {0, 1} and A = {1}, then A = {0} The complement of the complement of A is A itself. This property is known as involution, which is similar to the logical double negation. 26 Fuzzy Sets UNION The UNION of sets A and B is defined by A B = { x x A or x B }. It is illustrated graphically in Figure 2.3. A B I Figure 2.3: The union of two sets. If the membership function of set A is A ( x ) and that of set B is B ( x ) , then the membership of the set resulting from A B can be represented graphically as shown in Figure 2.4. A(x) 1 0 x B(x) 1 0 x AUB(x) 1 0 x Figure 2.4: Union in terms of membership function. 27 Chapter 2 Example Let A = {1, 2, 3, 4, 5, 6} and B = {3, 5, 7, 8}, then A B = {1, 2, 3, 4, 5, 6, 7, 8}. Example If A = {1}, B = {0}, and I = {0,1}, then A B = I INTERSECTION The INTERSECTION of sets A and B is defined by A B = { x x A and x B }. It is illustrated graphically in Figure 2.5, and in terms of the membership function Figure 2.6. A B I Figure 2.5: Intersection of two sets. A(x) 1 0 x B(x) 1 0 x A B(x) 1 U 0 x Figure 2.6: Intersection in terms of membership function. 28 Fuzzy Sets Example Let A = {1, 2, 3, 4, 5, 6} and B = {2, 4, 6, 8,10}, then A B = {2, 4, 6}. Example Let A = {1} and B = {0}, then A B = . SUBTRACTION The difference of sets A and B is a set that consists of all the elements which belong to A, but do not belong to B. It can be expressed as A B = A A B = { x x A and x B }. The operation is illustrated in Figure 2.7. a) A B b) A B Figure 2.7: Subtraction of two sets. a) A B b) B A 29 Chapter 2 Cartesian Product The Cartesian product of sets A and B is defined as A B {(a,b)| a A, b B}. Example Let A = {1,2} and B = {3, 4, 5}, then A B = {(1,3), (1,4), (1,5), (2,3), (2,4), (2,5)}, and B A = {(3,1), (3,2), (4,1), (4,2), (5,1), (5,2)}. Table 2.1 summarizes some basic properties of classical set operations. It useful to observe the duality some properties exhibit by replacing , , with I , , , respectively. It is also useful to remember that the law of contradiction and the law of excluded middle are fundamental to classical logic (and our way of thinking) to contrast with fuzzy logic later. Table 2.1: Some basic properties of classical set operations. (note that the negation operation can be expresses as A or A ) Law of contradiction Law of excluded middle De Morgans laws A A = A A = I ( A B) = A B ( A B) = A B Involution (Double negation) Commutative A=A A B = B A A B = B A Associative A ( A B) = ( A B) C A ( B C ) = ( A B) C Distributive A ( B C ) = ( A B) ( A C ) A ( B C ) = ( A B) ( A C ) 30 Fuzzy Sets 2.4 Boolean Logic The basic building blocks of digital systems, including computers and embedded systems, are electronic switching devices, such as transistors, or algorithms. Since such devices assume two conditions: ON or OFF, the universe of discourse can be represented by {0, 1}. A set with one element {0}, referred to as a singleton, can represent the OFF condition, another singleton{1}can represent the ON condition. Operations on such sets are special cases of the general set operations, they are referred to as binary or Boolean operations. The correspondence between basic set operations and binary logic operations are outlined in Table 2.2. Table 2.2: Basic binary operations. Operation NEGATION OR AND Symbol Corresponding Set Operation COMPLEMENT INTERSECTION UNION +, , The circuits that perform the AND, OR, and NEGATION operations are referred to as logic gates. They are available as IC chips. Numerous circuits built using such gates are also available as IC chips, the number of gates per chip defines the level of integration as discussed in Chapter 1. 2.5 Basic Concepts of Fuzzy Sets A fuzzy set is a set where degrees of membership between 1 and 0 are allowed; it allows partial membership. Fuzzy sets can thus better reflect the way intelligent people think. For example, an intelligent person will not classify people as either friends or enemies; there is a range between these two extremes. Not recognizing that there are degrees in every trait can lead to erroneous decisions. Vague human expressions such as tall, hot, cold, etc. can be expressed by fuzzy sets of the form A = { ( x, A ( x )) x X } where X represents the universe of discourse and A(x) assumes values in the range from 1 to 0. 31 Chapter 2 Example Let the values of temperature in C under consideration be T = {0, 5, 10, 15, 20, 25, 30, 35, 40}. Then, the term hot can be defined by a fuzzy set as follows HOT = {(0,0), (5,0.1), (10,0.3), (15,0.5), (20,0.6), (25,0.7), (30,0.8), (35,0.9), (40,1.0)}. This fuzzy set reflects the point of view that 0 C is not hot at all, 5, 10, and 15 C are somewhat hot, and 40 C is indeed hot. Another person could have defined the set differently. 2.6 Other Representations of Fuzzy Sets Fuzzy sets can be described in numerous ways; all of them allow partial membership to be expressed. The ordered pair method introduced in the previous section appears in a different format as follows A = 1 x1 + 2 x2 + 3 x3 + ... The symbol / here does not denote division, nor does the symbol + denote summation. The summation symbol is used to connect the terms and thus it means a union of single-term subsets. Example Let X = { x1 , x2 , x3 , x4 }. One can define a fuzzy set as: A = 0.8 x1 + 0.4 x2 + 0.1 x3 + 0.9 x4 . Fuzzy sets can also be defined by assigning a continuous function to describe the membership either analytically or graphically. Some commonly used membership functions are illustrated in Figure 2.8. The triangular membership function in Figure 2.8-a can also be expressed as ( x ) = a (b-x)/(b-c) ; = a (d-x)/(d-c) ; = 0; bxc cxd otherwise bxc cxd dxe otherwise The trapezoidal function shown in Figure 2.8-b can be expressed as: ( x ) = a (b-x)/(b-c) ; =a; = a (e-x)/(e-d) ; =0; 32 Fuzzy Sets The Gaussian function of Figure 2.8-c can be expressed as: (x) = a exp(- x-b)2/22) a) (x) a b c d x b) (x) a b c d e x c) (x) a b x Figure 2.8: Common membership functions. a) Triangular b) Trapezoidal c) Gaussian 33 Chapter 2 Example Figure 2.9 shows three sets defined graphically to represent the fuzzy sets SLOW, MEDIUM, and FAST to reflect a way of thinking about values of speed in the range of 0 to 40 km/hr. (x) slow 1 medium fast 15 20 25 30 35 40 Speed Figure 2.9: Possible graphical representation of fuzzy sets. Example The vague expression around ten can be expressed as a fuzzy set graphically as shown in Figure 2.10. (x) 1 8 10 12 x Figure 2.10: Around ten expressed as a fuzzy set graphically. 34 Fuzzy Sets 2.7 Determination of Membership Functions Discrete and continuous membership functions of a fuzzy set are intended to capture a persons thinking. The fuzzy set in the example of Section 2.5 was defined based on ones thinking of what coldness is. Fuzzy membership functions can still be determined subjectively in practical problems based on an experts opinion. In such a situation one can think of membership functions as a technique to formalize empirical problem solving that is based on experience rather than the knowledge of theory. The experts way of thinking can be captured either directly or through a special algorithm. Such determination could become more focused by physical measurements if the need arises. Available frequency histograms and other probability data can also help in constructing the membership function. It is important, however, to note that membership function values, or grades of membership, are not probabilities and they do not have to add to 1. Membership construction can be further simplified by selecting their form from the smaller family of the commonly used ones, such as those shown in Figure 2.8. 2.8 Fuzzy Sets Properties Empty fuzzy set A fuzzy set is referred to as empty if and only if the value of the membership function is zero for all possible members under consideration. In a short hand form this statement would read A = iff A ( x ) = 0 x X . (iff and are short hand forms for if and only if and for all values of, respectively). Universal fuzzy Set A fuzzy set is universal if and only if the value of the membership function is one for all members under consideration. Equal fuzzy sets Two fuzzy sets A and B are said to be equal iff A ( x ) = B ( x ) for all x X . -cuts A fuzzy set may be completely characterized by its -cuts, defined as follows strong -cuts: weak -cuts: A = { x A ( x ) > }; [0,1] A = { x A ( x ) }; [0,1] Thus, an -cut is a crisp set that consists of all the elements of a fuzzy set whose membership functions have values greater than a specified value , or greater than or equal to a specified value; the first condition leads to strong -cuts and the second to weak -cuts. All the cuts of a fuzzy set form a family of crisp sets. 35 Chapter 2 Example Let A = 0.2 1 + 0.5 2 + 0.6 3 + 1 4 + 0.7 5 + 0.3 6 + 0.1 7 . Then, the weak -cuts for [0,1] from 0.1 to 1 with step width of 0.1 are as follows A0. 1 = {1, 2, 3, 4, 5, 6, 7} A0.2 = A0.3 = {1, 2, 3, 4, 5, 6} A0.4 = A0.5 = {2, 3, 4, 5} A0.6 = {3, 4, 5} A0.7 = {4, 5} A0.8 = A0.9 = A1.0 = {4} Support The support of a fuzzy set A is a crisp set supp(A) of all x X such that A ( x ) > 0 . It is a strong -cut for = 0. The element x X at which A ( x ) = 0.5 is referred to as the cross-over point. A fuzzy set whose support is a single element in X with A ( x ) = 1 is referred to as a fuzzy singleton. Core The core of a fuzzy set A is a crisp set core(A) of all x X such that A ( x ) = 1 . The core of a fuzzy set may be an empty set. Height The height, h(A) of a fuzzy set A is the largest value of A for which the -cut is not empty. In other words, it is the largest value of the membership function attained by an element in the set. A fuzzy set with h(A) = 1 is referred to as normal, otherwise it will be referred to as sub-normal. The concepts of -cuts, support, core, and height are illustrated in Figure 2.11. 36 Fuzzy Sets A(x) 1.0 am h(A) =1 A is normal core Am (x) support x A(x) 1.0 0.8 h(A)<1 A is sub-normal core (A) = x Figure 2.11: A graphical illustration of -cuts, support, core, and height. 2.9 Operations on Fuzzy Sets 2.9.1 Logic Operations The three basic logic operations on classical (crisp) sets were introduced in Section 2.2. The definitions of sets were generalized in Section 2.5 leading to fuzzy sets upon which similar operations can be performed. The generalizations of operations on sets to operations on fuzzy sets are not unique. The ones introduced here are referred to as the standard fuzzy set operations. They are the operations most commonly used in engineering applications. 37 Chapter 2 COMPLEMENT The absolute complement of a fuzzy set A is denoted by A and its membership function is defined by A ( x) = 1 A ( x) for all x X Example Let A = 0.7 1 + 1 1 + 0.6 3 + 0.2 4 + 0 5 . Then, A = 0.3 1 + 0 2 + 0.4 3 + 0.8 4 + 1 5 . UNION The union of two fuzzy sets A and B is a fuzzy set whose membership function is defined by A B ( x ) = max [ A ( x ), B ( x ) ] Example Let A = 0.3 1 + 0 2 + 0.4 3 + 0.8 4 + 1 5 , and B = 0.2 1 + 0.3 2 + 0.1 3 + 0.2 4 + 0.4 5 . Then, A B = 0.3 1 + 0.3 2 + 0.4 3 + 0.8 4 + 1 5 INTERSECTION The intersection of two fuzzy sets A and B is a fuzzy set whose membership function is defined by A B ( x) = min[ A ( x), B ( x)]. Example Let A = 0.3 1 + 0 2 + 0.4 3 + 0.8 4 + 1 5 , and B = 0.2 1 + 0.3 2 + 0.1 3 + 0.2 4 + 0.4 5 Then, A B = 0.2 1 + 0 2 + 0.1 3 + 0.2 4 + 0.4 5 Some properties of fuzzy set operations are given in Table 2.3. It is particularly important to observe that the laws of contradiction and excluded middle are not applicable. How far a set deviates from such laws can be a measure of fuzziness. A graphical illustration of basic fuzzy sets operations is presented in Figure 2.12. 38 Fuzzy Sets Table 2.3 Some properties of fuzzy sets operations Law of contradiction Law of excluded middle A A A A I ( A B ) = A B De Morgans laws Involution (Double negation) ( A B ) = A B A=A A B = B A A B = B A Commutative Associative A ( A B) = ( A B) C A ( B C ) = ( A B) C A ( B C ) = ( A B) ( A C ) A ( B C ) = ( A B) ( A C ) Distributive 39 Chapter 2 A a) 1 0 2 3 6 10 x B b) 1 0 1 2 3 10 x Figure 2.12: Graphical illustration of basic fuzzy sets operations a) Set A b) Set B 40 Fuzzy Sets U A B c) 1 0 2 3 6 10 x A U B d) 1 0 1 2 3 6 10 x A e) 1 0 3 6 10 x Figure 2.12: Graphical illustration of basic fuzzy sets operations (cont.) c) A B e) A d) A B 41 Chapter 2 2.9.2 Algebraic Operations on Fuzzy Sets Cartesian multiplication The Cartesian multiplication of two sets A and B is a fuzzy set C such that C=AB = C ( x ) / ( a, b ) a A, b B, C (c ) = min A ( a ) , B (b ) Example Let A = 0.2/3 + 1/5 + 0.5/7, and B = 0.8/2 + 0.3/6. Then, A B = min[0.2,0.8]/(3,2) + min[0.2,0.3]/(3,6) + min[1,0.8]/(5,2) + min[1,0.3]/(5,6) + min[0.5,0.8]/(7,2) + min[0.5,0.3]/(7,6) = 0.2/(3,2) + 0.2/(3,6) + 0.8/(5,2) + 0.3/(5,6) + 0.5/(7,2) + 0.3/(7,6). { } Algebraic multiplication The algebraic product of two fuzzy sets A and B leads to a fuzzy set C such that AB = { A ( a ) B (b ) / x x A, x B} . Example Let A = 0.2/3 + 1/5 + 0.5/7, and B = 0.1/3 + 0.3/7 + 0.2/8. Then, AB = (0.2)(0.1)/3 + (1)(0)/5 + (0.5)(0.3)/7 + (0)(0.2)/8 = 0.02/3 + 0.15/7 Exponent Raising a set A to the power of is a special case of algebraic multiplication. It is defined by A = ( A ( x )) / x x A . { } 42 Fuzzy Sets Example Let A = 0.5/4 + 1/5 + 0.4/6. Then, A2 = (0.25)2/4 + (1)2/5 + (0.4)2/6 = 0.25/4 + 1/5 + 0.16/6, and A0.5 = (0.5)0.5/4 + (1)0.5/5 + (0.4)0.5/6 = 0.7/4 + 1/5 + 0.63/6. As pointed out earlier, a fuzzy set A can be expressed as a linguistic concept such as hot, cold, young, old, etc. The result of using nested linguistic modifiers such as very, very very, etc. can be expressed using A. Example Let the range of temperature under consideration be 5, 10, 20, 30, and 40C. Let also the set A represent the concept hot from a given point of view: A = 0.1/5 + 0.2/10 + 0.4/20 + 0.6/30 + 0.7/40. Then, very hot could be expressed by A2 = 0.01/5 + 0.04/10 + 0.16/20 + 0.36/30 + 0.49/40. It follows that very very hot can be expressed by (A2)2 = A4. Concentration and Dilation These two operations are unique to fuzzy sets; they do not have counterparts in classical sets. The concentration operation is defined as: CON (A) = A2 The dilation process is defined as: DIL (A) = A0.5 Example Let A = 0.5/4 + 1/5 + 0.4/6. Then, CON (A) = 0.25/4 + 1/5 + 0.16/6, and DIL (A) = 0.7/4 + 1/5 + 0.63/6. CON2 (A) The operations concentration and dilation could be composed with themselves, eg, = A4, and DIL2 (A) = A 4 . In general one can write CON (A) DIL 1 = A2, and (A) = A0.5. with being an integer 2. 43 Chapter 2 The CON operation reduces the value of A(x) except for A(x) = 1. As , the membership function becomes two-valued: 1 and 0. On the other hand, the DIL operation increases the value of A(x) except for A(x) = 1. As , A(x) 1. The effect of the CON operation on the membership function is illustrated graphically in Figure 2.13. A(x) A CON 1(A) CON 2(A) 2>1 x Figure 2.13: Graphical illustration of the effect of the on the membership function. CON operation Algebraic sum The algebraic sum of two sets A and B is a fuzzy set with a membership function given by: A (x ) + B (x ) A (x )B (x ) where the symbol + here denotes algebraic summation. Example Let C be the algebraic summation of two fuzzy sets A and B expressed by: C = A + B, A = 0.2/1 + 1/2 + 0.4/3, and B = 0.3/1 + 0.4/2 + 0.2/7. Then, the set C would consist of four members: 1, 2, 3, and 7 with membership functions C ( x1 ), C ( x2 ), C ( x3 ), C ( x4 ) respectively. They can be determined as follows: 44 Fuzzy Sets C (x1) = 0.2 + 0.3 (0.2)(0.3) = 0.44 C (x2) = 1 + 0.4 (1)(0.4) =1 C (x3) = 0.4 + 0 + (0)(0.4) = 0.4 C (x4) = 0.2 + 0 + (0)(0.2) = 0.2 Accordingly, C = 0.44/1 + 1/2 + 0.4/3 + 0.2/7. (One should remember that the symbol + here is not algebraic summation.) Bounded sum The symbol denotes the bounded sum of two fuzzy sets. The operation leads to a fuzzy set with a membership function defined by: A B ( x ) = min 1, ( A ( x ) + B ( x )) Example Let A = 0.5/3 + 1/4 + 0.8/5, and B = 0.2/3 + 0.4/5. Then, A B = 0.7/3 + 1/4 + 1/5. Bounded difference The symbol denotes the bounded difference of two fuzzy sets. The operation leads to a fuzzy set with a membership defined by: A B ( x ) = min 1, ( A ( x ) + B ( x )) Example Let A = 0.5/3 + 0.3/5 + 0.7/8, and B = 0.4/3 + 0.1/5. Then, A where B = (x1)/3 + (x2)/5 + (x3)/8 (x1) = min[1,(0.5 0.4)] = 0.1, (x2) = min[1,(0.3 0.1)] = 0.2, and (x3) = min[1,(0.7 0.0)] = 0.7. hence, A B = 0.1/3 + 0.2/5 + 0.6/8. 45 Chapter 2 Some identities that relate sets and algebraic operations are given in Table 2.4. They will be of particular importance when discussing some of the embedded applications of fuzzy sets. Table 2.4: Identities relating sets and algebraic operations. A ( x ) B ( x ) = B ( x ) ( ( x ) B A ( x )) A ( x ) B ( x ) = A ( x ) + ( B ( x ) A ( x ) B ( x ) = ( A ( x ) A ( x )) A ( x )) B ( x )) + ( B ( x ) Fuzzification Fuzzification is the operation of transforming a crisp set to a fuzzy set, or a fuzzy set to a fuzzier set. The operation translates crisp input or measured values into linguistic concepts. This, in a way, is similar to what people may do in numerous situations to reach a decision. For example, if one is told that the temperature is going to be 10 C, one translates this crisp input value into a linguistic concept such as mild, cold, or warm according to ones inclination, then reaches a decision about the need to wear a jacket or not. If one fails to fuzzify (for example, due to lack of familiarity with the Celsius temperature scale) then the decision process cannot continue or a possibly erroneous decision would be reached. So, you have been fuzzifying all along (without knowing it) whenever you made correct decisions. A common fuzzification algorithm of a set A = keeping i constant and transforming xi to a fuzzy set that depicts the expression about xi, K(xi). The fuzzy set K(xi) is referred to as the kernel of fuzzification. Fuzzified set A is expressed as ~ A = 1K(x1) + 2K(x2) + ... + nK(xn) where the symbol ~ means fuzzified. { x x X} is performed by i i i The method is referred to sometimes as support fuzzification (s-fuzzification) in order to distinguish it from another possible fuzzification method referred to as grade fuzzification (g-fuzzification) in which xi is kept constant and i is expressed as a fuzzy set. It is interesting to observe that both the dilation and fuzzification operations lead to a fuzzier set. However, dilation of a crisp set yields the same crisp set, which is not typically the case with the fuzzification operation. 46 Fuzzy Sets It is useful to devise a measurement to determine how fuzzy a set is. All sets that allow partial membership are fuzzy, but some are fuzzier or can be made fuzzier than others. One can intuitively suggest that the sharper the boundaries, i.e. the closer the values of the membership to 1 or 0, the less fuzzy a set is. A more formal measure of fuzziness could be desirable to define. A possible measure of fuzziness can be put forward by evaluating the deviation of a set from its complement, i.e. A ( x ) A ( x ) . Such a difference can be expressed as A ( x ) (1 A ( x ) = 2 A ( x ) 1 The largest difference occurs when (x) is at its largest value, which is 1, implying that the largest difference would be 1. The degree of the lack of distinction between a fuzzy set and its complement can then be expressed by 1 2 A ( x ) 1 . A fuzzy measure f(A) for a finite set can then be defined by f ( A ) = 1 2 A ( x ) 1 xX ( ) Note that f(A) 0 for all fuzzy sets and f(A) = 0 for all crisp sets. Also f(A) is at its maximum when (x) = 0.5 since this is where the membership function is farthest away from both 1 and 0. Concluding Remarks Fuzzy sets are the tools which convert the concepts of fuzzy logic into algorithms leading to applications. They are used to express precisely what one means by vague expressions such as hot, cold, tall, and short. By allowing partial membership, fuzzy sets can provide computers with algorithms that extend their binary logic and enable them to make human-like decisions. The term fuzzy in this context does not mean imprecise, but exactly the opposite. It may appear difficult to reconcile the claim that the objective of fuzzy sets is to enable computing with words with the fact that fuzzy sets are of a mathematical nature. Do humans think in terms of triangular membership functions, Cartesian multiplications, etc.? Certainly not, but one may think of fuzzy sets and the associated mathematics as the media through which the way we think is transferred to a computer, rather than trying to accommodate our thinking to the computer needs. Of course, it would have been more efficient to have a means for direct transfer of thoughts without the intermediate stage of mathematics. The avoidance of mathematics occurs in the way to describe a system. Attempting to transfer our way of thinking into fuzzy set formulation could have an interesting side effect. It gives us an opportunity to ponder about our own thoughts and actions and reflect on the wisdom of our choices and judgements. 47 Chapter 2 Bibliography 1. G. J. Klir and T. A. Floger, Fuzzy sets, Uncertainty, and Information, Prentice Hall International, London, UK, 1988. 2. G. J. Klir, U. H. St. Clair, and B. Yuan, Fuzzy Set Theory, Prentice Hall, Upper Saddle River, NJ, 1997. 3. S. Lipschuhutz, Discrete Mathematics, McGraw-Hill, Toronto, 1976. 4. K. H. Rosen, Discrete Mathematics and its Applications, McGraw-Hill, Toronto, 1999. 5. Y. Shirai, T. Yamakawa, and F. Ueno, A CAD-oriented Synthesis of Fuzzy Logic Circuits, Systems Computers Controls, 15, 7683, 1984. 6. L. A. Zadeh, Fuzzy Sets, Information Control 8, 338353, 1965. 7. L. A. Zadeh, A Fuzzy-Set Theoretic Interpretation of Linguistic Hedges, J. Cybern, 2, 3, 434, 1972. 8. L. A. Zadeh, Outline of a New Approach to the Analysis of Complex Systems and Decision Processes, IEEE Trans. Sys. Man Cybern, SMC-3, 2844, 1973. 48 Fuzzy Sets Web Resources 1. Notes on Set Theory www.math.csusb.edu/notes/sets/sets.html A set of notes presented in HTML format by Nikos Drakos, Computer Based Learning Unit, University of Leeds. It discusses basic topics related to Set Theory including: notations and terminology, set operations, and Venn diagrams. 2. Set Theory www.math.niu.edu/~rusin/known-math/index/03EXX.html An article from the Mathematical Atlas which evolved from a much more informal collection of mathematical pointers collected by Prof. Dave Rusin of Northern Illinois University. It provides extensive Web links to topics such as the history of set theory, applications and related fields, an annotated list of textbooks, and a list of software and tables. It also provides links to Web sites with similar interests. 3. Sets, Logic and Foundations and Philosophy of Mathematics www.mathlinks.info/em035_set_log_foun_philo.htm A collection of Web links from Math.Links maintained by Ronald N. Gibson. The links are classified into categories that include: Academic Programs in Logic and Mathematical Foundations, Articles, Courses, Lectures, Texts, Tutorials, Journals, Organizations, and more. 4. Set Theory Page www.cis.syr.edu/~sanchis/setory.html A collection of links to Web sites by Prof. Luis Sanchis, Syracuse University. Web sites listed include: Situated Set Theory, Set Theory, Bounded Set Theory, Holmes: New Foundations Home Page, Constructive Set Theory, Programming with Sets, Forster Bibliography, E-mail Addresses of Set Theorists, Homepages of Set Theorists, Beginnings of Set Theory, Logic Eprints, Fixed Point Theory, and much more. 5. Java Set Theory Machine www.jaytomlin.com/music/settheory/ A page maintained by Jay Tomlin. It provides an interactive Java applet related to the Musical Set Theory. It also provides a short tutorial about the topic. The source code is available for downloading. 49 Chapter 2 6. Logic and Set Theory www.humboldt.edu/~mef2/logicsites.html An extensive collection of web links maintained by Professor Martin Flashman, Department of Mathematics, Humboldt State University, Arcata, California, USA. The links are organized in categories including: Logic, Set Theory, Turing Machine, History, and Major References. 7. The Mathematics of Set Theory www.jboden.demon.co.uk/SetTheory/ This Web site declares that its aim is to cover a wide range of topics from ZFC Set Theory. Topics include: A brief History of Axiomatic Set Theory, Some Logical Paradoxes, The axioms of ZFC ,Order Relations, Equivalence Relations & Equipollence, Number Systems, Ordinals & Cardinals , and Glossary. Extensive hyperlinks are provided. 8. A History of Set Theory www-history.mcs.st-and.ac.uk/history/HistTopics/Beginnings_of_set_theory.html An article by J. J. OConnor and E. F. Robertson. It discusses the history of Set Theory. It provides extensive hyperlinks. It has 25 references. 9. Boolean Logic www.howstuffworks.com/boolean.htm A tutorial in HTML format by Marshall Brain. Topics covered include: Introductory Information, Simple Gates, Flip-flops, and more. 10. Logic Gates and Boolean Logic educ.queensu.ca/~compsci/units/BoolLogic/titlepage.html A collection of annotated resources by Mark Mamo and Shane Bauman. Topics include: Introduction to Boolean Logic Black Box Circuits Summary of Logic Gates Sample Questions on Logic Gates, Circuits and Truth Tables Discovering the Rules of Boolean Algebra Simplifying Boolean Expressions 50 Fuzzy Sets 11. George Boole www.digitalcentury.com/encyclo/update/boole.html An article in HTML format by Natalie Voss. It provides information about George Boole and his work in addition to numerous related Web Links. 12. Max Black www-gap.dcs.st-and.ac.uk/~history/Mathematicians/Black.html An article in HTML format by J. J. OConnor and E. F. Robertson with hyperlinks and seven references. 13. Lotfi Zadeh www.cs.berkeley.edu/~zadeh This site provides a bibliography of Lotfi Zadeh, who is considered the creator of fuzzy logic with his seminal paper published in 1965. 14. Fuzzy Logic Tutorial www.seattlerobotics.org/encoder/mar98/fuz/flindex.html A tutorial by Steven Kaehler that covers topics including: Introduction to Fuzzy Logic Why Use Fuzzy Logic The Rule Matrix Membership Functions Putting It All Together A Set of Sample Cases 15. Introduction to Fuzzy Logic www.dementia.org/~julied/logic/index.html A page maintained by Julie Grosman. It provides an introduction to fuzzy logic, its history, an overview of sets & operations, and its applications. It also has an interactive ten-question quiz. 16. Fuzzy Calculator http://home.planet.nl/~n.dubois/FzCalc/Fcalc.htm An interactive calculator for fuzzy operations by Nico du Bois. The calculator has a graphical interface through which one can define two three-element fuzzy sets by entering a number as an element and the corresponding membership function. The desired operation is performed by clicking on the related button of the calculator. The result is a three element set defined by the value of its elements and their corresponding membership values. 51 Chapter 2 17. Fuzzy Sets and Systems Links www.abo.fi/~rfuller/fuzs.html A collection of Web links by Robert Fullr. It provides links to Personal Home Pages of Fuzzy Researchers, Who is Who in Fuzzy Database, Fuzzy-Mail Archives, Fuzzy Logic Sources of Information, Fuzzy logic at Webopedia, Computational, Conferences and Workshops on Fuzzy Systems, Professional Organizations, Journals & Books, Research groups, and more. 52 CHAPTER 3 Fuzzy Relations 3.1 Introduction Relationships among objects are fundamental to decision making and dynamic systems applications. A classical (crisp) relation represents the presence or absence of a connection or association among elements of two or more sets. Fuzzy relations, on the other hand, allow degrees of strength to such connections and associations. This chapter gives a brief overview of classical relations: basic concepts, presentation, and operations to pave the way to the introduction of fuzzy relations along the same lines. The chapter ends with a discussion of fuzzy reasoning. 3.2 Classical Relations A binary relation associates elements of only two sets to each other. A relation R from set A to set B is a subset of the Cartesian product of the two: R A B. The relation can be represented by the notation aRb. Relations may also be defined among three sets (ternary relations), four sets (quaternary relations), five sets (quinary relations), and so on. Presentation of relations A relation can be defined by listing all the member pairs (a,b) R. It can also be expressed using a coordinate diagram, a matrix, a mapping diagram, or an arrow diagram. Example Let A = {desk, bag, book}, and B = {wood, plastic, paper}. The Cartesian product of these two sets leads to A B = {(wood,desk), (wood,bag), (wood,book), (plastic,desk), (plastic,bag), (plastic,book), (paper,desk), (paper,bag), (paper,book)}. From this set one may select a subset such that R = {(wood,desk), (plastic,bag), ((paper,bag), (paper,book)}. The subset R can be represented using a coordinate diagram as shown in Figure 3.1. 53 Chapter 3 desk bag book wood plastic paper Figure 3.1: The coordinate diagram of a relation. The relation could equivalently be represented using a matrix as shown below. R desk bag book wood plastic paper 1 0 0 0 1 0 0 1 1 The mapping representation is yet another method of defining the relation between two sets. It is illustrated in Figure 3.2. desk bag book wood plastic paper Figure 3.2: Mapping representation of a relation. 54 Fuzzy Relations A binary relation in which each element from the first set is not mapped to more than one element in the second is referred to as a function and is expressed by R:AB Example Figure 3.3 gives illustrations of R : A B. A pine gold eagle tree bird metal B a) b) A eagle pine cedar gold cuckoo silver metal tree bird B Figure 3.3: Illustrations of R : A B The mapping can be arranged differently, leading to an arrow diagram. Example Let A = {1, 2, 3, 4, 5, 6}, and let R be a relation on A defined as R = {(1,1), (1,2), (1,6), (3,2), (3,4), (4,5), (5,5), (5,6)}. The relation can be represented by the arrow diagram shown in Figure 3.4. 55 Chapter 3 Figure 3.4: An arrow diagram representation of R. 1 6 2 5 4 3 Operations on relations a) Inverse of a relation Let R = {(a,b)|(a,b) A B}, then the inverse, R1, is defined as R1 = {(b,a)|(a,b) R}. It follows that bR1a iff aRb, and (R1)1. If A = B and R = R1, the relation is referred to as involuntary, self-inverse, or symmetric. If R is represented by a matrix MR, then the matrix M R1 representing R1 is T the transpose of M R1 = M R . The transpose of a matrix is obtained by writing the rows, in order, as columns. Example 1 1 0 Let MR = 0 0 0 , 1 0 0 b) Composition then MR1 1 0 1 = 1 0 0 . 0 0 0 Composition is an operation on two compatible binary relations and results in a single binary relation. Two binary relations P and Q are compatible for composition iff P A B, and Q B C. In other words, the second set in P must be the same as the first set in Q. The composition of the two sets is denoted by P Q. It can be determined graphically or using matrix representation where 56 Fuzzy Relations MRQ = MRMQ Some properties of the composition operation are summarized in Table 3.1. Table 3.1: Some properties of the composition operation. Associative Not commutative Inverse (P Q) R = P (Q R) PQQP (P Q)1 = Q1 P1 Example Let A = {a1,a2,a3}, B = {b1,b2,b3}, and C = {c1,c2,c3}. Let P = {(a1,b1), (a1,b2), (a2,b3), (a3,b3)}, and Q = {(b1,c1), (b2,c3), (b3,c1)}. The relations P and Q are illustrated in Figure 3.5. From the illustration one may infer that: P Q = {(a1,c1), (a1,c3), (a2,c1), (a3,c1)}. P a1 a2 a3 b1 b2 b3 Q c1 c2 c3 Figure 3.5: Illustration of the relation P and Q. One can reach similar conclusions using the matrix representation as follows: b1 b2 b3 1 1 0 a1 MP = 0 0 1 a2 , and 0 0 1 a3 57 Chapter 3 c1 c2 c3 1 0 0 b1 MQ = 0 0 1 b2 1 0 0 b3 c1 c2 c3 Then, MP Q 1 0 1 a1 = 1 0 0 a2 1 0 0 a3 This matrix leads to P Q = {(a1,c1), (a1,c3), (a2,c1), (a3,c1)}. as expected. 3.3 Classical Reasoning In classical binary logic, reasoning is based on two complementary mechanisms: deduction (modus ponens) and induction (modus tollens). Deduction is used to obtain conclusions by means of forward inference and induction is used to deduce causes by means of backward inference. The two mechanisms are contrasted in Table 3.2. In that table A and B are crisp sets and the symbol means implies. Table 3.2: Deduction and Induction. Deduction Rule Premise Conclusion IF Induction IF x is A y is B x is A y is B x is A y is B y is not B x is not A In other words, given the rule: :IF x is A, THEN y is B and the observation that x is A, one concludes by deduction that : y is B . In a mathematical shorthand: ( p ( p q )) q Given the same rule but the observation that y is not B, one concludes by induction that: x is not A. In a mathematical shorthand: ( q ( p q )) p 58 Fuzzy Relations 3.4 Fundamentals of Fuzzy Relations Fuzzy relations are fuzzy sets defined on universal sets which are Cartesian products. They capture the strength of association among elements of two or more sets, not just whether such an association exists or not. Example Let A = {a1,a2,a2} and b = {b1,b2,b3}. Let R be a relation from A to B given by R = 0.1/(a1,b3) + 0.5/(a1,b2) + 0.3/(a2,b1) + 0.4/(A2,B3) + 1.0/(A3,B3) + 0.1/(A3,B1) The corresponding fuzzy matrix would be a1 a2 a3 0 0.3 0.1 b1 MR = 0.5 0 0 b2 0.1 0.4 1 b3 The corresponding graph is shown in Figure 3.6. a1 b1 0.1 0.5 0.3 Figure 3.6: Illustration of a fuzzy relation. a2 0.1 b2 0.4 a3 1.0 b3 3.5 Operations on Binary Fuzzy Relations Inverse of a fuzzy relation The inverse of a fuzzy relation R on A B is denoted by R1. It is a relation on B A defined by R1 (b,a) = R(a,b) for all pairs (b,a) B A. 59 Chapter 3 Example 0.3 1.0 0 If MR = 1.0 0.5 0.3 , then 0.0 0.0 0.8 0.3 1.0 0.0 = 1.0 0.5 0.0 . 0.0 0.3 0.8 M R 1 Composition of fuzzy relations As explained in Section 3.2, the composition of two crisp binary relations P and Q requires that the relations be compatiblei.e., A B and B C share the set B. The composition R = P Q consists of pairs (a,b) from A C that are connected through at least one element in B. In fuzzy relations, the connections have degrees of strength stemming from the fact P and Q are fuzzy sets. There are several types of fuzzy relations compositions. The most common in engineering applications is the max-min composition. In this composition scheme the strength of the connection is determined by the smaller strength connection of the two in a chain that connects a to c. Among the chains that connect the two elements, the one with the largest strength is the one that is selected to characterize the relation. In mathematical shorthand, we can write P Q = {max min[P (a,b), Q (b,c)]/(a,c)| a A,b B,c C} The properties of crisp relations given in Table 3.1 are valid for fuzzy relations composition as well. Example Let A = {a1,a2}, B = {b1,b2}, and C = {c1,c2}. Let P be a relation from A to B defined by: b1 b2 0.4 0.2 a1 MP = 0.8 0.3 a2 60 Fuzzy Relations Let Q be a relation from B to C defined by: c1 c2 0.3 0.1 b1 MQ = 0.4 0.9 b2 Then, 0.4 P Q= 0.8 r =1 r3 0.2 0.3 0.1 0.3 0.4 0.9 r2 r4 where r1,r2,r3, and r3 are calculated as follows: r1 = = = r2 = = = r3 = = = r4 = = = max[min(0.4,0.3), max[0.3,0.2] 0.3 max[min(0.4,0.1), max[0.1,0.2] 0.2 max[min(0.8,0.3), max[0.3,0.3] 0.3 max[min(0.8,0.1), max[0.1,0.3] 0.3 MP min(0.2,0.4)] min(0.2,0.9)] min(0.3,0.4)] min(0.3,0.9)] 0.3 0.2 = 0.3 0.3 The same result could be obtained graphically by using an arrow diagram and applying the max-min rule of composition outlined earlier. This is illustrated in Figure 3.7. Hence, Q 61 Chapter 3 a) a1 a2 A 0.4 0.8 0.2 0.3 B 0.3 C c1 b1 b2 0.4 0.1 0.9 c2 P Q A b) C 0.3 0.2 0.3 0.3 a1 a2 c1 c2 PoQ Figure 3.7: Illustration of the max-min composition. a) The relations P and Q b) The relation P Q As mentioned previously, other types of composition of fuzzy relations could be defined. For example: min-max composition (sometimes denoted by P Q ) which is the dual of the max-min composition. max-star composition where multiplication, summation, or some other binary operation is denoted by in place of the min operation in the maxmin composition. Two special cases of max-star composition are the max-product (max-prod) and the max-average (max-av) compositions. 3.6 Types of Fuzzy Relations If R is a fuzzy relation on A A, and x A, y A, and z A, then the relation R can be classified as summarized in Table 3.3. 62 Fuzzy Relations Table 3.3: Summary of fuzzy relations types. Reflexive Anti-reflexive Symmetric Anti-symmetric max-min transitive Identity relation Zero relation Universal R (x,x) = 1 and R (x,y) < 1, x y R (x,x) = 0 and R (x,y) < 1, x y R (x,y) = R (y,x) R (x,y) R (y,x) R (x,z) max min[ (x,y), (x,z)] Diagonal elements of MR are all ones All elements of MR are zeros All elements of MR are ones Examples The following are examples of MR matrices for reflexive, anti-reflexive, symmetric and antisymmetric relations, respectively: 1.0 0.3 0.4 0.3 0.2 0.4 0.2 0.7 0.0 1.0 0.5 , 0.5 0.6 1.0 0.2 0.5 0.8 1.0 0.5 . 0.6 0.2 0.2 0.8 0.3 0.7 0.5 0.0 0.7 , 0.7 0.8 0.2 , and 0.3 0.0 0.5 0.2 0.4 3.7 Fuzzy Reasoning Fuzzy reasoning is based on inference rules of the form IF <premise>, THEN <consequence> as is the case in classical logic, but fuzzy sets, rather than crisp sets, are used. Fuzzy sets define linguistic variables and hence fuzzy inference rules can model a system linguistically. Fuzzy algorithms are mathematically equivalent to fuzzy relations and fuzzy inference is equivalent to fuzzy composition. There are numerous ways that have been put forward to express an inference rule. A direct, simple inference rule takes the form: IF x is A, THEN y is B where A and B are fuzzy sets. 63 Chapter 3 If the number of rules is large it becomes more convenient to employ a fuzzy relations approach. The IF/THEN rules are converted to fuzzy relations, then fuzzy composition is used to infer conclusions. The conversion from IF/THEN rules to fuzzy relations could be defined in more than one way. A simple method is given by: R=AB=AB An inference rule could have more than one proposition. For example, a rule of inference with two propositions would take the form: if x is A, and y is B, then z is C where A, B, and C are fuzzy subsets of X, Y, and Z, respectively. The rule may be written as: A and B C Since A and B are subsets of different sets X and Y, respectively, it would be inaccurate to write A B C . The situation is illustrated in Figure 3.8, from which one concludes that x is A and y is B is ( x, y ) is ( A Y ) ( X B ) ( x, y ) is A B Hence, the proposition A and B can be expressed as A and B = A B Then the relation R can be expressed as R = A and B C =ABC Y AY (AY) (BX) B BX A X Figure 3.8: A B with A and B being subsets of different sets X and Y. 64 Fuzzy Relations A fuzzy algorithm has several rules, such as Rule 1: Rule 2: . . . Rule n: IF IF IF x is A1, THEN y is B1 x is A2, THEN y is B2 x is An, THEN y is Bn The rules can be also written as A1 B1 A2 B2 . . . An Bn This n-rule system can be converted to n relations: R1,R2,...,Rn. These relations can be combined into one relation, R, using fuzzy intersection operations or fuzzy union operations depending on how the rules are perceived to be connected. R = R1 R2 Rn , or R = R1 R2 Rn The difference in the way the rules are perceived to be related would obviously lead to different results. Concluding Remarks A crisp relation represents the presence or absence of association or interaction between the elements of two or more sets. As the concept of crisp sets was generalized to allow degrees of membership, so was the concept of relations. Fuzzy relations allow degrees of association or interaction between elements to better reflect ones way of thinking and decision making. They may be thought of as fuzzy sets defined over multidimensional universes of discourse. Fuzzy relations play a central role in fuzzy algorithms, which are collections of fuzzy rules of the form of IF/THEN. Such rules are nothing other than fuzzy relations in disguise. Embedded fuzzy system applications are based on hardware and software implementations of fuzzy algorithms using embedded technology approaches. 65 Chapter 3 Bibliography 1. M. Benedicty and R. F. Sledge, Discrete Mathematical Structures, HBJ, Toronto, 1987. 2. E. Cox, The Fuzzy Systems Handbook, AP Professional, Boston, USA, 1994. 3. A. Kaufman and M. M. Gupta, Introduction to Fuzzy Arithmetic, Van Nostrand Reinhald, New York, USA, 1991. 4. G. J. Klir and T. A. Folger, Fuzzy Sets, Uncertainty, and Information, Prentice Hall, London, UK, 1998. 5. G. J. Klir, U. H. St. Clair, and B. Yuan, Fuzzy Set Theory, Prentice Hall, Upper Saddle River, NJ, USA, 1997. 6. E. H. Mamdani, Application of Fuzzy Algorithms for Control of a Simple Dynamic Plant, Proceedings of IEEE, 121, 12, 15851588, 1974. 7. M. Sugeno,An Introductory Survey on Fuzzy Control, Information Sciences, 36, 5983, 1985. 8. T. Tanaka, An Introduction to Fuzzy Logic for Practical Applications, Springer, New York, USA, 1997. 9. T. Terano, K. Asai, and M. Sugeno, Fuzzy Systems Theory and Applications, Academic Press, Inc., Boston, USA, 1992. 66 Fuzzy Relations Web Resources 1. On Fuzzy Relations, Metrics and Cluster Analysis http://dmi.uib.es/people/valverde/gran1/GRAN1.html An article in HTML format by J. Jacas and L. Valverde. It gives an introduction to fuzzy logic and discusses: Building fuzzy transitive relations, Generators, m-cluster coverages. It provides 20 references. 2. Modelling Linguistic Expressions Using Fuzzy Relations www.scch.at/servlet/resource.ResourceLoader?id=5 A 15-page technical report with 13 references available in PDF format. It is authored by M. De Cock, U. Bodenhofer, and E. Kerre, and appeared in The Proceedings of the 6th International Conference on Soft Computing, Iizuka, Japan, October 1-4, 2000, pp. 353360. 3. Numerical Representation of Transitive Fuzzy Relations http://online.sfsu.edu/~sergei/pr-05.pdf A 12-page paper with 12 references by S. Ovchinnikov, Mathematics Department, San Francisco Stat University. It investigates numerical representations of fuzzy transitive relations. 4. Fuzzy Researchers www.abo.fi/~rfuller/persons.html This site provides a list of home pages of fuzzy logic researchers. It is compiled by Robert Fuller, bi Akademi, Finland. 67 [This is a blank page.] CHAPTER 4 Embedded Fuzzy Applications 4.1 Introduction As outlined in Chapter 1, there are various definitions of an embedded system. An embedded system involves computing, but not as its main purpose. Fuzzy logic has a wide range of applications in such systems. The common thread that links all these applications is that an expert (a human operator) can, at least in principle, perform the task required. However, the mathematical model is too complicated to be of practical use, and therefore a linguistic model becomes advantageous. In some cases where mathematical models do exist and lead to satisfactory results, fuzzy logic may still be used either to improve or simplify the process or to compare with a benchmark. There are numerous embedded applications of fuzzy logic such as fuzzy control, fuzzy signal processing, and fuzzy image processing. The discussion here will concentrate on fuzzy control. Analysis and design of conventional control systems are overviewed first, to be contrasted afterwards with fuzzy logic control analysis and design. 4.2 Conventional Control Systems 4.2.1 Description A typical block diagram of a feedback control system is shown in Figure 4.1. r(t) + e(t) Controller m(t) Process c(t) Measurement Figure 4.1: A typical block diagram of a feedback control system. 69 Chapter 4 The system consists of three main blocks: the process to be controlled, measurement, and the controller. The output of the process is measured and converted to an equivalent electrical signal using suitable sensor circuitry. The signal that represents the output, c(t), is then compared with an input reference signal, r(t), resulting in an error signal, e(t). This error signal actuates the controller to generate the control action signal, m(t). The signals mentioned in the above descriptions are expressed as functions of timei.e., the description is in the time domain. The system is described using a differential equation with the time-derivative operator d/dt being the basic operator used. The system could have been equivalently described in the frequency domain, or the s-domain. The Laplace operator, s, is the basic operator used in the s-domain description of the system. It is equivalent to the time-derivative operator in the time domain (with the initial conditions being zero). The derivative operator is also equivalent to the j operator for sinusoidal signals, which is used in the frequency domain description of a system. The ratio of the output to the input signals expressed in the s-domain is the transfer function of the system, G(s), with s being, in general, a complex number expressed by s = + j . The basic forms of the transfer function include: Gain, G(s) = K Differentiator, first-order lead, G ( s ) = s Integrator, first-order lag, G ( s ) = ( s ) 1 Second-order lag, G ( s ) = s2 + 2n s + 2 n ( ) 1 where K is a constant (gain factor), is the time-constant, is the damping factor, and n is the natural frequency. The transfer function of a system is, in general, composed of such basic transfer functions. It can then be expressed in one of two equivalent forms: Pole-zero form: G (s) = K ( s + z1 )( s + z2 )( s + z3 )... ( s + p1 )( s + p2 )( s + p3 )... where z1, z2, z3, and p1, p2, p3, are the zeros and poles of the system, respectively. Time-Constant form: G (s) = K ( 1s + 1)( 2 s + 1)( 3 s + 1)... ( a s + 1)( b s + 1)( c s + 1)... 70 Embedded Fuzzy Applications 4.2.2 Analysis The objectives of analyzing a given feedback system are mainly to determine: the stability of the system and its extent the transient response the steady-state response A system is said to be stable when its output does not increase without bound or oscillate with time. The steady-state and transient response parameters are defined in Figure 4.2. To analyze a feedback control system one needs to: Determine a mathematical model for each of the system building units. Transfer functions are the most convenient to use for that purpose. Represent the system using a block diagram and determine its overall transfer function. Determine the characteristics by means of a step-response, pole-zero map, root-locus, Bode plot, Nyquist diagram, or Nicols Chart. Simply from the transfer function point of view, the stability of a single-input, single-output system can be determined based on the location of the poles in the complex plane (splane): the left half of the complex plane is the stable region. Output Overshoot 1 0.9 Steady-state error D = Dead-time R = Rise-time 0.1 0.0 D R Settling time Time Dead time: the time elapsed between the application of an input signal and the start of an output signal. Rise time: the time the output takes to increase from 10% to 90% of the final value. Overshoot: the maximum difference between the transient and steady-state values. Settling time: the time for the response to become within 5% of the final value. Steady-state error: the value of e(t) as t . Figure 4.2: A possible step response of a stable system with the steady-state and transient response parameters defined. 71 Chapter 4 4.2.3 Design The objective of the design of a feedback control system is to meet a set of given performance specifications. They are usually given in the time-domain or the frequency domain. The time-domain specifications have two components: Transient performance, which could be given in terms of overshoot percentage, rise time, settling time, and the predominant time constant. Steady-state performance, which is given in terms of steady-state error. The frequency-domain specifications are usually given in terms of gain and phase margins, cut-off rate, and resonance peak and frequency. 4.2.4 PID Control A commonly used control scheme is known as the three-term PID controller, where PID stands for Proportional Integral Derivative. It is also used as a benchmark against which any other control scheme is compared. The concept of a PID controller is illustrated in Figure 4.3. KP + + e(t) KI dt + m(t) KD d dt Figure 4.3: PID controller concept. 72 Embedded Fuzzy Applications The control action signal of a PID controller is composed of three components: Proportional action, p(t), where the signal is proportional to the error signal at the present moment. p (t ) = K p e (t ) Integral action, i(t), where the signal is proportional to the commutative values of the error signal up to the present moment. i (t ) = Ki e (t ) dt 0 t Derivative action, d(t), which is proportional to the rate of change of the error signal at the present moment. d d (t ) = K d e (t ) dt where Kp, Ki, Kd and are constants. The overall control action, m(t), can be expressed as: t m (t ) = K p e (t ) + Ki e (t ) dt + K d 0 d e (t ) dt In the s-domain, this can be expressed as: K M (s ) = K p + 1 + KD s E (s ) s As Kp is increased, the system speed increases (with a tendency to overshoot), and the steady-state error decreases, but is not eliminated. As Kd is increased, the damping factor increases, thereby reducing the overshoot. The derivative control is susceptible to noise and it is never used alone. As Ki is increased, the steady-state error goes to zero and the system tends towards instability. Integral control is also never used alone. 4.3 Fuzzy Logic Controller (FLC) 4.3.1 Description The block diagram shown in Figure 4.1 is still applicable in fuzzy applications, but the controller is a fuzzy controller rather than a PID controller. Figure 4.4 shows the basic structure of a fuzzy logic controller. The main building units of an FLC are a fuzzification unit, a fuzzy logic reasoning unit, a knowledge base, and a defuzzification unit. Defuzzification is the process of converting inferred fuzzy control actions into a crisp control action. 73 Chapter 4 Fuzzy Reasoning Fuzzy Knowledge base Defuzzification E Fuzzification U Figure 4.4: Basic structure of a fuzzy logic controller. The fuzzy knowledge-base has a rule-base that maps a fuzzy input variable, E, into a fuzzy output, U. This can be expressed by a linguistic statement such as: E U (condition E implies condition U) which may be written as: IF E THEN U. There is an equivalency between the above expression and the relation obtained by Cartesian multiplication, as explained in Chapter 3i.e., R = E U IF E THEN U. The fuzzy knowledge-base also has a database defining the variables. A fuzzy variable is defined by a fuzzy set, which in turn is defined by a membership function. Fuzzy reasoning is used to infer the output contributed from each rule. The fuzzy outputs reached from each rule are aggregated and defuzzified to generate a crisp output. The essence of a fuzzy logic controller is thus based on a linguistic model (rulebase and the defined membership functions) as opposed to a mathematical model, as is the case with a PID controller. Fuzzy logic controllers are used to reduce the development time or to improve the performance of an existing PID controller. In the case of highly complex systems, fuzzy logic could be the only solution. 4.3.2 Design In the design of an FLC system it is assumed that: A solution exists. The input and output variables can be observed and measured. An adequate solution (not necessarily an optimum one) is acceptable. A linguistic model can be created based on the knowledge of a human expert. 74 Embedded Fuzzy Applications In order to model a system linguistically, one needs to: Identify the input and output variables of the process to be controlled (the plant). For example: speed, temperature, humidity, etc. Define subsets that cover the universe of discourse of each variable and assign a linguistic label to each one. For example, the linguistic variable speed may be defined as three fuzzy subsets: slow, medium, and fast as shown in Figure 2.9. Form a rule-base by assigning relationships between inputs and outputs. Determine a fuzzification method. Determine a defuzzification method to be used to generate a crisp output from the fuzzy outputs generated from the rule-base. The section to follow elaborates on the process of defuzzification and its numerous methods. 4.3.3 Defuzzification For a given input, several IF/THEN rules could be launched at the same time. Each rule would have a different strength, because a given input may belong to more than one fuzzy set, but with different membership values. For example, an input temperature of 80 C may belong to the fuzzy subset very_high with = 0.8 and to the fuzzy subset medium with = 0.3 . Thus, when this temperature occurs two, rules will fire: IF IF very_high THEN action-1 medium THEN action-2 If action-1 is defined by fuzzy set F1 and action-2 is defined by fuzzy set F2, then the two sets are aggregated (commonly using the UNION operation) leading to the fuzzy set F, as illustrated in Figure 4.5. In general, the output of the fuzzy reasoning would involve more than two fuzzy sets; therefore, one can write: F = Fi i =1 k Assuming the support of F is X = {x1, x2, x3,} then for xi X , F(xi) = wi indicates the degree to which each is suggested by the rule-base as a good output for the given input. The defuzzification operation is applied on F to determine the best crisp output. 75 Chapter 4 F 1 0.5 y F 2 1 y F 1 0.5 y Figure 4.5: Two rules firing at the same time leads to fuzzy sets that could be aggregated and result in a fuzzy set F. Numerous defuzzification methods have been suggested in the literature; however, sometimes different authors name the same method differently. No method has proved to be always more advantageous than the others. The selection of which method to use depends primarily on the experience of the designer. One may take into consideration the computational complexity involved, applicability to all situations considered, and plausibility of the results from an engineering point of view. Defuzzification methods include the following: Centroid method This method is also known as the center of mass, or center of gravity method. It is probably the most commonly used defuzzification method. The defuzzified output, x* is defined by x* = ( x ) xdx ( x ) dx F F where the symbol denotes algebraic integration. 76 Embedded Fuzzy Applications Center of largest area This method is applicable when the output consists of at least two convex fuzzy subsets which do not overlap. The result is biased towards a side of one membership function. First maxima This method is applicable when the output is peaked; the smallest value of the domain with maximum membership is selected. Max-membership This method is also known as the height method. The output is defined by: F x * F ( x ) , for all x X Weighted average method This method is valid for symmetrical output membership functions. Each membership function is weighted by its maximum membership value. The output is defined by: ( ) x* = where denotes algebraic summation, and xi is the maximum of the ith membership function. The method is illustrated in Figure 4.6 where two fuzzy subsets are considered. In that figure, the defuzzified output would be (x ) x (x ) F i F i i x* = ( 0.4 ) a + ( 0.8) b 0.4 + 0.8 1 0.8 0.6 0.4 0.2 a b Figure 4.6: Two symmetrical membership functions. 77 Chapter 4 Mean-max membership The method is also known as the middle of the maxima. The output is defined by: x = * x i =1 n i n a+b 2 In Figure 4.6, the defuzzified output would be: x* = Center of sums This method uses the algebraic sum of the individual fuzzy subsets instead of their union. Although the calculations become faster, this method leads to adding the intersecting areas twice. 4.3.4 Analysis As discussed earlier, fuzzy logic is particularly useful in the control of highly nonlinear processes difficult to control by conventional methods. The central idea is to use the knowledge of an expert (human operator) to create a linguistic model. However, conventional analysis techniques of control systems are based on mathematical models and hence cannot be used for the analysis of a fuzzy logic control system. One may argue that if the model is based on the knowledge of an expert then the analysis is not needed. However, one must realize that human knowledge could be incomplete. In addition, fidelity of the mapping using rule-base, inference rules, and defuzzification methods needs to be assured. Extensive simulations, in general, would be required to establish confidence in the reliability of the system. One should examine the static properties of a designed FLC system and observe aspects such as: The consistency of the IF/THEN rules. Inconsistencies could occur due to an attempt to accommodate contradictory design criteria or simply due to faulty formulation of the rules. The existence of enough rules to assure smooth control action, but not too many to slow the operation. A proper control action should be inferable for every state of the process. Fuzzy sets are properly assigned in the defined universe of discourse. Overlapping of neighboring membership functions ensures completeness and robustness, but may lead to distortions in the output. The defuzzification method used is suitable to ensure correct and smooth control action. 78 Embedded Fuzzy Applications As for the dynamic properties, numerous techniques have been put forward for the analysis of the stability of an FLC system. They can be categorized into two groups: The first group consists of methods that require a mathematical model of the process to be controlled (the plant). This category includes the cases of linear and nonlinear plants. The describing function technique and the circle criterion are example of the first. The invariance principle and Lyapunov function are examples of the second. The second group consists of methods that do not require a mathematical model. They include phase plane trajectory, energetistic stability, and linguistic stability methods. Energetistic Stability Criterion The energetistic stability criterion involves time-consuming calculations; however, it provides an intuitive method to analyze the stability of a fuzzy dynamic system. It is based on the general physical law that any dynamic system is stable if its total energy decreases with time until it reaches a steady state. Various authors have suggested algorithms based on this idea. The approach of Kiszka et al. is summarized here. It starts with the basic equation of a simple dynamic system, X k +1 = X k U k R, k = 1,2,3, where Xk and Xk+1 are fuzzy sets at the kth and (k+1)th time instants, respectively, and R is a fuzzy relation describing the fuzzy control system. If there is no input, the system is said to be free or unforced. This occurs when Uk = zero for all values of k, where zero is a fuzzy singleton as illustrated in Figure 4.7 and is described by 1 u = u1 ( k ) Uk (u ) = zero = 0 otherwise U (u) = zero for all values of k k 1 0 u1 u2 u3 uj u Figure 4.7: Illustration of the fuzzy singleton, Uk = zero. 79 Chapter 4 Let , P = Uk R then Xk +1 = Xk P , k = 0,1,2,3, A state is an equilibrium state if Xk +1 = Xk = XS for all k. This leads to: XS = XS P The energy of a fuzzy relation P is defined by Kiszca et al. by: E (P) = where w ( xi , y j ) is the Cartesian product of the ith element in the support of the fuzzy set X in the universe of discourse WX, the jth element in the support of fuzzy set Y in the universe of discourse WY , f P ( xi , y j ) , is the corresponding value of the mem- 1 n m w ( xi , y j ) f P ( xi , y j ) n m i =1 j =1 ( ) ( ) bership in the normalized fuzzy relation matrix MP, and n and m are the cardinalities of the support sets for fuzzy sets X and Y, respectively. Example Let WX = [2,4,6], WY = [1,3,5], and P is a relation from X to Y defined by 0.2 0.3 0.1 MP = 0.1 0.4 0.3 0.5 0.1 0.2 Then, the energy would be: E (P) = 1 (3)(3) {2 1 0.2 + 2 3 0.1 + 2 5 0.5 + 4 1 0.3 + 4 3 0.4 + 4 5 0.1 + 6 1 0.1 + 6 3 0.3 + 6 5 0.2} = 2.89 The energy expression for a free fuzzy dynamic system would be: E ( Xk +1 ) = E ( Xk P ) But, X1 = X0 P , X2 = X1 P = X0 P P = X0 P2 , . . . Xk = X0 P k where X0 is the initial fuzzy system state and P k = P P P P (k times). 80 Embedded Fuzzy Applications Then, the energy expression can be written as: E ( Xk +1 ) = E X0 P k +1 ( ) ) The change in energy, E , between two consecutive states would be: E = E X0 P k E X0 P k 1 ( ) ( Since E(X0) is a constant, E does not depend on the initial conditions. Therefore, the rational characteristic energy function, EC ( P, k ) , can be expressed as EC ( P, k ) = E P k E P k 1 ( ) ( ) This expression could be used to determine the stability as follows: a) The system is stable if EC ( P, k ) 0 for k . c) The system is oscillatory with a period T, if b) The system is unstable if EC ( P, k ) 0 for k . EC ( P, k ) = EC ( P, k + T ) for k . The following is a simple illustrative example of the energetistic stability criterion as put forward by Kiszka et al. Example Let the universe of discourse be X = Y =[1,2]. Determine the stability of the system if the fuzzy relation P is given by: 0.1 0.5 0.1 0.3 a) P = b) P= 0.3 0.2 0.7 1.0 To determine the stability of a system, one starts by calculating several consecutive values of E(Pk) using the max-min composition rule discussed in Chapter 3. This gives a11 a 21 with a12 b11 b12 c11 c12 = a22 b21 b22 c21 c22 c11 = max min ( a11 , b11 ),min ( a12 , b21 ) c12 = max min ( a11 , b12 ),min ( a12 , b22 ) c21 = max min ( a21 , b11 ),min ( a22 , b21 ) c22 = max min ( a21 , b12 ),min ( a22 , b22 ) 81 Chapter 4 Then for the first system: 0.1 0.5 0.1 0.5 P2 = P P = 0.3 0.2 0.3 0.2 0.3 0.2 = 0.2 0.3 0.3 P3 = P2 P = 0.2 0.2 = 0.3 0.2 P 4 = P3 P = 0.3 0.3 = 0.2 0.2 0.1 0.5 0.3 0.3 0.2 0.3 0.2 0.3 0.1 0.5 0.2 0.3 0.2 0.2 0.3 One can then use the definition of energy E (P) = 1 n m w ( xi , y j ) f P ( xi , y j ) to calculate n m i =1 j =1 ( ) E ( P ), E P2 , E P3 , E P 4 . E (P) = 1 {1 1 0.1 + 1 2 0.5 + 2 1 0.3 + 2 2 0.2} 2 2 1 = (2.5) 4 ( ) ( ) ( ) Similar calculations yield: 1 1 1 (2.3) , E P3 = (2.2 ) , and E P4 = (2.2 ). 4 4 4 One may conclude that the system is oscillatory. E P2 = ( ) ( ) ( ) 82 Embedded Fuzzy Applications For the second system: 0.3 0.1 0.3 0.3 0.3 = 1.0 0.7 1.0 0.7 1.0 0.3 0.1 0.3 0.3 0.3 = 1.0 0.7 1.0 0.7 1.0 0.3 0.3 P4 = P5 = 0.7 1.0 One may then conclude that this system is stable. 0.1 P2 = 0.7 0.3 P3 = 0.7 4.4 Simplified Examples of Applications Two examples are given here of fuzzy logic control in embedded systems: a washing machine and a vacuum cleaner. The objective is to illustrate how to start thinking about designing a practical fuzzy logic system, the core of which is the definition of the inputs, outputs and fuzzy control rules. The two examples presented do not require extensive knowledge. Hence, the rules, the fuzzy subsets, and the corresponding results of simulation are easily understood. By relating them to each other and reflecting on how to enhance their operation, they should provide an insight into the operation of more complex systems that may require specialized knowledge. No calculation details are given here. Usually such design and analysis is carried out with the help of a software tool. Some of the software tools available will be discussed in Chapter 9. Web links to some fuzzy embedded systems, including washing machines, vacuum cleaners, and car anti-lock brake systems are provided at the end of the chapter. 4.4.1 Washing machine There are several washing machines on the market that use fuzzy logic. The discussion here, however, does not refer to any particular washing machine; it is rather concerned with a hypothetical washing machine. The objective is to provide a simple example of the initial stages of designing a fuzzy control for a washing machine. Ease of use is a desirable feature, along with ability to set the washing parameters based on the laundry load characteristics. The characteristics of the laundry load (inputs) include: the actual weight, fabric types, and amount of dirt. The washing parameters (outputs) include: amount of detergent, washing time, agitation, water level, and temperature. Controlling these parameters could lead to a cleaner laundry, conserve water, save detergent, electricity, time, and money. 83 Chapter 4 The design of a machine to meet such specifications could be a demanding task. It is obvious that there is no simple mathematical model that could be of practical use to relate the inputs to the outputs, but a knowledgeable operator could do the task manually. This is where fuzzy logic comes in. A rule-base could be created based on the knowledge of the operator to control the process. Analysis would still be necessary to assure that plausible results are reached. Consider, for simplicity, a machine with two inputs and one output, the inputs being: The dirtiness of the load as measured by the opacity of the washing water using an optical sensor system. The weight of the laundry load as measured by a pressure sensor system. The output is the amount of detergent dispensed. The dirtiness is defined in the range from 0 to 100, by defined fuzzy subsets: Almost_Clean, Dirty, Soiled, and Filthy as shown in Figure 4.8. The weight of the laundry is defined in the range from 0 to 100 by fuzzy sets: Very_Light, Light, Heavy, and Very_Heavy as shown in Figure 4.9. The output, for simplicity, is defined by the singleton subsets shown in Figure 4.10. 1.00 Degree of Membership Almost_Clean 0.75 Dirty Soiled Filthy 0.50 0.25 0.00 0 10 20 30 40 50 60 70 80 90 100 DIRTINESS Figure 4.8: Subsets defining Dirtiness. 84 Embedded Fuzzy Applications 1.00 Degree of Membership Very_Light 0.75 Light Heavy Very _Heavy 0.50 0.25 0.00 0 10 20 30 40 50 60 70 80 90 100 WEIGHT Figure 4.9: Subsets defining Weight. 1.00 Very_Light Little Much Very _Much Maximum Degree of Membership 0.75 0.50 0.25 0.00 0 10 20 30 40 50 60 70 80 90 100 DETERGENT Figure 4.10: The Detergent subsets. Suggested control rules are given below, and summarized in Table 4.1. if DIRTINESS is ALMOST_CLEAN and WEIGHT is VERY_LIGHT then DETERGENT is VERY_LITTLE if DIRTINESS is DIRTY and WEIGHT is VERY_LIGHT then DETERGENT is LITTLE if DIRTINESS is SOILED and WEIGHT is VERY_LIGHT then DETERGENT is MUCH if DIRTINESS is FILTHY and WEIGHT is VERY_LIGHT then DETERGENT is VERY MUCH if DIRTINESS is ALMOST_CLEAN and WEIGHT is LIGHT then DETERGENT is VERY_LITTLE if DIRTINESS is DIRTY and WEIGHT is LIGHT then DETERGENT is LITTLE if DIRTINESS is SOILED and WEIGHT is LIGHT then DETERGENT is MUCH if DIRTINESS is FILTHY and WEIGHT is LIGHT then DETERGENT is MUCH if DIRTINESS is ALMOST_CLEAN and WEIGHT is HEAVY then DETERGENT is MUCH if DIRTINESS is DIRTY and WEIGHT is HEAVY then DETERGENT is MUCH if DIRTINESS is SOILED and WEIGHT is VERY_HEAVY then DETERGENT is VERY_MUCH if DIRTINESS is FILTHY and WEIGHT is HEAVY then DETERGENT is MAXIMUM if DIRTINESS is ALMOST_CLEAN and WEIGHT is VERY_HEAVY then DETERGENT is MUCH if DIRTINESS is DIRTY and WEIGHT is VERY_HEAVY then DETERGENT is VERY_MUCH if DIRTINESS is SOILED and WEIGHT is VERY_HEAVY then DETERGENT is MAXIMUM if DIRTINESS is FILTHY and WEIGHT is VERY_HEAVY then DETERGENT is MAXIMUM 85 Chapter 4 Table 4.1: Fuzzy control rules for a washing machine. Weight Dirtiness Almost_Clean Dirty Soiled Filthy Very_Light Very_Little Little Much Very_Much Light Little Little Much Much Heavy Much Much Very_Much Very_much Very_Heavy Much Very_Much Maximum Maximum The definition of the input sets, output sets, and the control rules reflect a particular point of view. If the analysis results shown in Figures 4.10 and 4.11 are not acceptable then the rules, or even the definition of the fuzzy subsets could be modified. More rules could be used for smoother operation, as well as more complex rules that include operations other than AND. The design could be expanded to include more inputs. The diagrams for this example were generated using Fuzzy Logic Designer software developed by Byte Dynamics, Inc. Detailed discussion of software tools is presented in Chapter 9. DETERGENT vs DIRTINESS 100 90 80 DETERGENT 70 60 50 40 30 20 10 0 0 10 20 30 40 50 60 70 80 90 100 DETERGENT Figure 4.11: Detergent vs. Dirtiness. 86 Embedded Fuzzy Applications 100 90 80 DETERGENT vs WEIGHT DETERGENT 70 60 50 40 30 20 10 0 0 10 20 30 40 50 60 70 80 90 100 WEIGHT Figure 4.12: Detergent vs. Weight. If different rules were used, different results would be obtained. For example, if the rules used are: if DIRTINESS is ALMOST_CLEAN or WEIGHT is VERY_LIGHT then DETERGENT is VERY_LITTLE if DIRTINESS is FILTHY or WEIGHT is VERY_HEAVY then DETERGENT is VERY_MUCH if DIRTINESS is DIRTY or WEIGHT is HEAVY then DETERGENT is MUCH if DIRTINESS is FILTHY and WEIGHT is VERY_HEAVY then DETERGENT is MAXIMUM The results of the simulations would be as shown in Figures 4.13 and 4.14. One may find them objectionable (do you agree?). 87 Chapter 4 100 90 80 DETERGENT vs WEIGHT DETERGENT 70 60 50 40 30 20 10 0 0 10 20 30 40 50 60 70 80 90 100 DIRTINESS Figure 4.13: Detergent vs. Dirtiness, objectionable results. 100 90 80 DETERGENT vs DIRTINESS DETERGENT 70 60 50 40 30 20 10 0 0 10 20 30 40 50 60 70 80 90 100 DIRTINESS Figure 4.14: Detergent vs. Weight, objectionable results. 88 Embedded Fuzzy Applications 4.4.2 Vacuum cleaner Once again, there are several vacuum cleaners on the market that use fuzzy logic. Links to a few of them are provided in the Web resources section at the end of this chapter. The objective of the discussion here is to highlight that the initial design steps follow the same line of reasoning as that of the washing machine. All fuzzy control applications follow a similar approach. First you need to identify the inputs and the outputs, then relate them through IF/THEN rules. The inputs here could be dirtiness, type of surface, and type of dirt, and the output could be the force of vacuuming. One can define fuzzy subsets for the Surface such as : Wood, Curtain, and Carpet, and for Dirtiness as: Almost_Clean, Dirty, Soiled, and Filthy. The strength of vacuuming could be defined by the fuzzy subsets: Very_Weak, Weak, Normal, Strong, and Very_Strong. Rules relating the inputs and the output could be as summarized in Table 4.2. Simulation analysis could be carried out on the rules, and set definitions could be modified to make it satisfactory if it is not. Table 4.2: Fuzzy control rules for a vacuum cleaner. Dirtiness Surface Wood Curtain Carpet Almost_Clean Very_Week Very_Week Week Dirty Week Normal Normal Soiled Normal Strong Strong Filthy Strong Very_Strong Very_Strong Concluding Remarks The application of fuzzy logic principles enables solving problems that would have been difficult or even impossible to solve if mathematical models were used. Fuzzy logic, however, would not create a solution if one did not exist in the first place through the knowledge of an expert; such knowledge is the basis of a fuzzy algorithm. To learn how to solve a problem is the domain of neural networks, discussed in Chapter 6. It is also important to realize that what is reached through fuzzy algorithms is a solution, but not necessarily the optimum one. An optimum solution could be reached from a pool of solutions using genetic algorithms. This topic is introduced in the Appendix. 89 Chapter 4 Bibliography 1. C. von Altrock, Fuzzy Logic in Automotive Engineering, Circuit Cellar, 88, 19, 1997. 2. J. C. Bezdek and S. K. Pal, Fuzzy Models for Pattern Recognition, IEEE Press, New York, 1992. 3. O. Bishop, Understand Electronic Control Systems, Newnes, Oxford, 2000. 4. M. M. Bourke and D. G. Fisher, Convergence, Eigen Fuzzy Sets and Stability Analysis of Relational Matrices, Fuzzy Sets and Systems, 81, 227234, 1996. 5. F. Bouslama and A. Ichikawa, Application of Limit Fuzzy Controllers to Stability Analysis, Fuzzy Sets and Systems, 49, 103120, 1992. 6. Y. Y. Chen, Stability Analysis for Fuzzy Control - A Lyapunov Approach, Proc. of IEEE Int. Conf. on Systems, Man and Cybernetics, VA, USA, 1987, pp. 10871031. 7. D. P. Filev and R. R. Yager, On the Analysis of Fuzzy Logic Controllers, Fuzzy Sets and Systems, 68, 3966, 1994. 8. J. Gang and C. Laijiu, Linguistic Stability Analysis of Fuzzy Closed Loop Control Systems, Fuzzy Sets and Systems, 82, 2734, 1996. 9. O. Georgieva, Stability of Quasilinear Fuzzy System, Fuzzy Sets and Systems, 73, 249258, 1995. 10. M. M. Gupta, G. M. Trojan, and J. B. Kiszaka, Controllability of Fuzzy Control Systems, IEEE Transactions on Systems, Man, and Cybernetics, SMC-16, 576582, 1986. 11. L. P. Holmbald and J. J. stergraad, Control of a Cement Kiln by Fuzzy Logic, in Fuzzy Information and Decision Processes, M .M. Gupta and R. Sanchez (eds.), North Holland, 1982. 12. G. Hwang and S. Lin, A Stability Approach to Fuzzy Control Design for Nonlinear Systems, Fuzzy Sets and systems, 48, 279287, 1992. 13. C. Jianqin and C. Laijiu, Study on Stability of Fuzzy Closed-loop Control Systems, Fuzzy Sets and Systems, 57, 159168, 1993. 14. T. A. Johansen, Fuzzy Model Based Control: Stability, Robustness, and Performance Issues, IEEE Transactions on Fuzzy Systems, 2, 3, 221234, 1994. 15. A. A. Kania, On Stability of Formal Fuzziness Systems, Information Sciences, 22, 5168, 1980. 90 Embedded Fuzzy Applications 16. R. Katoh, T. Yamashita, and S. Singh, Stability Analysis of Control Systems Having PD Type of Fuzzy Controller, Fuzzy Sets and Systems, 74, 321334, 1995. 17. W. J. M. Kickert and E. H. Mamdani, Analysis of Fuzzy Logic Controller, Fuzzy Sets and Systems, 1, 2944, 1978. 18. P. J. King and E. H. Mamdani, The Application of Fuzzy Control Systems to Industrial Processes, Automatica, 13, 232242, 1977. 19. J. B. Kiszaka, M. M. Gupta, and P. N. Nikiforuk, Energetistic Stability of Fuzzy Dynamic Systems, IEEE Trans. on Systems, Man, and Cybernetics, SMC-14, 6, 783791, 1985. 20. S. Kitamura and T. Kurozumi, Extended Circle Criterion and Stability Analysis of Fuzzy Control Systems, Fuzzy Engineering toward Human Friendly Systems, T. Terano, M. Sugeno, M. Mukaidono (eds.), IOS Press, 1992, pp. 634643. 21. B. Kosko, Fuzzy Engineering, Prentice Hall, Upper Saddle River, New Jersey, 1997. 22. H. A. Malki, H. Li, and G. Chen, New Design and Stability Analysis of Fuzzy Proportional-Derivative Control Systems, IEEE Transactions on Fuzzy Systems, 2, 4, 245354, 1994. 23. E. H. Mamdani, Application of Fuzzy Algorithms for the Control of a Dynamic Plant, Proc. IEEE, 12, 15851588, 1974. 24. R. J. Marks II, editor, Fuzzy Logic Technology and Applications, IEEE Press, New York, 1994. 25. M. Mizumoto, Realization of PID Controls by Fuzzy Control Methods, Proc. of the IEEE Int. Conf. on Fuzzy Systems, San Diego, March 812, 1992, pp. 709715. 26. K. S. Ray and D. D. Majumder, Application of Circle Criteria for Stability Analysis of Linear SISO and MIMO Systems Associated with Fuzzy Logic Controllers, IEEE Trans. on Systems, Man, and Cybernetics, SMC-14, 2, 345349, 1984. 27. K. S. Ray, A. H. Ghosh, and D. D. Majumder, L2-Stability and Related Design Concepts for SISO Linear System Associated with Fuzzy Systems, IEEE Transactions on Systems, Man, and Cybernetics, SMC-14, 6, 932939, 1984. 28. L. Rezink, Fuzzy Controllers, Newnes, Oxford, 1997. 91 Chapter 4 29. T. J. Ross, Fuzzy Logic with Engineering Applications, McGraw-Hill, Inc., New York, 1995. 30. K. Tanaka and M. Sugeno, Stability Analysis and Design of Fuzzy Control Systems, Fuzzy Sets and Systems, 45, 135156, 1992. 31. G. Vachtesvanos and S. Farinwata, Fuzzy Logic Control: A Systematic Design and Performance Assessment Methodology, in Fuzzy Logic: Implementation and Applications, M. J. Patyra and D. M. Mlynek (eds.), John Wiley & Sons, 1996, pp. 3962. 32. J. Wilkie, M. Johnson, and R. Katebi, Control Engineering, Palgrave, New York, 2002. 33. R. R. Yager and D. F. Filv, Essentials of Fuzzy Modeling and Control, John Wiley & Sons, Inc., New York, 1994. 34. Y. Yan, M. Tyan, and J. Power, Using Fuzzy Logic, Prentice Hall, New York, USA, 1994. 35. S. Y. Yi and M. J. Chung, Systematic Design and Stability Analysis of a Fuzzy Logic Controller, Fuzzy Sets and Systems, 72, 271298, 1995. 92 Embedded Fuzzy Applications Web Resources 1. A Demonstration of Real-time Control of an Inverted Pendulum www.aptronix.com/fuzzynet/java/pend/pendjava.htm This is a demonstration of a one-stage inverted pendulum with a fuzzy Java controller provided by FuzzyNet OnLine. 2. Simulation of the Problem Fuzzy Control of a Pendulum www.erudit.de/erudit/demos/cartball/ This Web site is maintained by the European Network for Fuzzy Logic and Uncertainty Modeling in Information Technology, ERUDIT. It provides a demo to illustrate the fuzzy control of an inverted pendulum. The demo can run through the Web or can be downloaded as a standalone version. 3. Fuzzy Logic On-Line Demos www.emsl.pnl.gov:2080/proj/neuron/fuzzy/demos.html This Web site provides links to numerous fuzzy demos including: The Fuzzy Truck Simulator (Auburn University) Fuzzy Logic Overview (Battelle Memorial Institute) Amplitude Control Broom Balancing (Cambridge University) Demo of Fuzzy Furnace Controller (Rick Hillegas), and more. The site is provided by Pacific Northwest National Laboratory, USA. 4. Fuzzy Truck Parking www.iit.nrc.ca/IR_public/fuzzy/FuzzyTruck.html The demo shows an application of fuzzy logic to park a truck by backing it up to a loading platform. It was created by the Integrated Reasoning Group of the Institute for Information Technology of the National Research Council of Canada. 5. Fuzzy Truck Simulator www.vgt.bme.hu/okt/tananyag/fuzzy/fuzzy12/home.html The simulation goal is to design a fuzzy system to back up to a truck so that it arrives at a loading dock at a right angle (relative to the horizontal axis). The applet shows that through direct manipulation of the fuzzy set rules, the system is dynamic enough to compensate to a certain degree. The source code is also provided. The site is maintained by Christopher Britton, Budapest University of Technology and Economics. 93 Chapter 4 6. Tuning of Fuzzy PID Controllers www.iau.dtu.dk/~jj/pubs/fpid.pdf This is a PDF file of a 22-page paper with 10 references with the above title by Jan Jantzen, Technical University of Denmark. 7. About robustness of Fuzzy PD and PID www.erudit.de/erudit/events/esit2000/proceedings/BB-02-2-P.pdf A paper by B. Butkiewicz, Warsaw University of Technology. The main idea of the paper is to show that the behavior of a fuzzy controller depends mainly on the rules and plant parameters, not on the reasoning and defuzzification methods. The paper appeared in the Proceedings of the European Symposium on Intelligent Techniques, 2000, Aachen, Germany, September 1415, 2000, pp. 350356. 8. Online Papers and Reports www.angelfire.com/me/pcominos/paperlink.htm About a hundred papers and reports related to control systems available in zipped PS format or PDF format. Among them some are related to fuzzy PID control including: Fuzzy-PID Controllers vs. Fuzzy-PI Controllers by Matilde Santos, J. M. de la Cruz, S. Dormido, and Between Fuzzy-PID and PIDConventional Controllers: a Good Choice by Matilde Santos, J. M. de la Cruz, S. Dormido, A. P. de Madrid 9. Fuzzy Systems for Control Applications www.site.uottawa.ca/~petriu/Fuzzy-tutor.PDF A 30-page tutorial available in PDF format. It is authored by E. M. Petriu, University of Ottawa. The tutorial is easy to follow, very well illustrated, and provides bibliographical references. 10. Fuzzy Control Methods and their Real System Applications www.ic-tech.com/Fuzzy%20Logic/ A report consisting of 67 slides presented in HTML format by IC Tech, Inc., April 2000. Topics include: Fuzzy Control Methods and their Real System Applications Data Acquisition and Knowledge Acquisition Learning in Neural Networks Boolean Logic and Fuzzy Logic Fuzzy Systems Design of a Fuzzy Controller Automotive Applications of Fuzzy Control 94 Embedded Fuzzy Applications 11. Stability Analysis and Controller Design for Dynamic Fuzzy Systems Based on New Fuzzy Inference Approach. www.ubka.uni-karlsruhe.de/vvv/1997/elektrotechnik/3/3.pdf A paper from the Elektronische Volltextarchiv, EVA (the archives of scientific publications written by members of the University of Karlsruhe). It is authored by E. Schfers and V. Krebs. It presents a new inference method with interpolating rules as a basis for the analysis of dynamic fuzzy systems. 12. Fuzzy Logic in Embedded Microcomputers and Control Systems www.bytecraft.com/fuzlogic.pdf A 48-page report by W. Banks and G. Hayward, published by Byte Craft Limited, Waterloo, Ontario, Canada and reprinted in April, 2001. The report gives a short tutorial on fuzzy logic, then discusses fuzzy logic implementation on embedded microcomputers. It also provides bibliographical references. 13. Application of Fuzzy Logic to Multimedia Technology www.sys.uea.ac.uk/~king/restricted/ A report in HTML format authored by M. Razaz, J. R. King, A. Duke, and A. Rogers in collaboration with BT Laboratories. The paper describes the application of fuzzy logic to determine the availability of a person within a multi-user virtual environment system. The collected information is then used to determine whether a user is there and the level of busyness of such a user. Availability is defined using fuzzy logic concepts. The paper then reports on a new fuzzy software system that has been designed and implemented to be used to determine a users availability within the described environment. 14. A Laboratory Course on Fuzzy Control http://eewww.eng.ohio-state.edu/~passino/PapersToPost/FC-lab-course.pdf A paper available in PDF format authored by S. Yurkovich and K. Passino appeared in the IEEE Transactions on Education, 42,1, 1521, 1999. The authors describe a control laboratory course offered at the Ohio State University for senior undergraduates and graduate students. 15. Fuzzy Control of a DC Motor http://alds.stts.edu/APPNOTE/Fuzzy/2359.PDF A 14-page application note in PDF format provided by STMicroelectronics. It details all aspects of a fuzzy control system for a DC motor. 95 Chapter 4 16. An Approach to Motor Control with Fuzzy Logic http://alds.stts.edu/APPNOTE/Fuzzy/2077.PDF Another 14-page application note in PDF format provided by STMicroelectronics, discussing motor control using fuzzy logic. 17. Fuzzy Systems for Control Applications: the fuzzy backer-upper www.mathematica-journal.com/issue/v4i1/columns/freeman/ 64-69_Jim41.mj.pdf A report in PDF format by J. A. Freeman, Loral Space Information Systems. The report appeared in The Mathematica Journal, 4, 1, 6469, 1994. The report introduces the fuzzy logic tool from the automation toolbox. A fuzzy control system is developed that automatically backs up a truck to a specified point on a loading dock. 18. Fuzzy Tracking Control Design for Nonlinear Dynamic Systems www.ee.nthu.edu.tw/bschen/papers/c080-fuzzy.pdf A paper by C.-S. Tseng, B.-S. Chen, and H.-J. Uang appeared in the IEEE Transactions on Fuzzy Systems, 9, 3, 381392, 2001. 19. Texas Instruments Fuzzy Logic Control www-s.ti.com/sc/psheets/spra028/spra028.pdf A 48-page report in pdf format provided by Texas Instruments, 1993. 20. Enhanced Control of an Alternating Current Motor using Fuzzy Logic and a TMS320 Digital Signal Processor http://alds.stts.edu/APPNOTE/Fuzzy/SPRA057.PDF A 66-page application report in PDF format, provided by Texas Instruments, 1995. 21. Vacuum Cleaner Refernce Platform http://e-www.motorola.com/brdata/PDFDB/docs/AN1843.pdf A 40-page Motorola Semiconductor Application Note authored by K. Berringer, published in the year 2000. 22. LG Electronics Fuzzy Washing Machine www.dreamlg.com/en/lgewasher/index.jsp Announcements and technical data from LG Electronics. 23. Samsung Fuzzy Washing Machine www.samsungelectronics.com.my/washing_machine/tech_info/index.html Announcements and technical data from Samsung. 96 Embedded Fuzzy Applications 24. Hitachi Microwave Oven Control www.has.hitachi.com.sg/grp3/sicd/application/appl-pdf/28-h012.pdf Technical information about using fuzzy logic for microwave oven control. 25. Intel Fuzzy Control Anti-lock Brakes www.intel.com/design/mcs96/designex/2351.htm A report in HTML format authored by D. Elting, M. Fennich, R. Kowalczyk and B. Hellenthal. Topics covered include: Fuzzy Logic Overview, Fuzzy ABS, Model BUILDER, fuzzyTECH 3.0, and Intel Fuzzy ABS Features. 26. Aptronix Fuzzy Design Notes www.aptronix.com/fuzzynet/index.htm These design notes include the design of a washing machine, camera auto focus, air conditioner control, and car automatic transmission. 27. fuzzyTECH Application Library/Technical www.fuzzytech.com/ This is the Web site of fuzzyTech. It has links to: News, Products, Fuzzy Application Library, Demo, and more. The Application Library includes topics such as: What is Fuzzy Logic, Technical Applications, Business and Finance Applications, and more. The Technical Applications include: Industrial Automation, Fuzzy in Appliances, Automotive Engineering, Antilock Braking System, and more. 28. A Practical Study on the Implementation of Fuzzy Logic Controllers http://citeseer.nj.nec.com/cordon98practical.html A 26-page technical report with 33 references authored by O. Cordn, F. Herrera and A. Peregrn, Universidad de Granada, Spain, July 1998. Topics covered include: Fuzzy Logic Controllers, Design of Fuzzy Logic Controllers, Framework of Implementing Fuzzy Logic Controllers, Software Implementing Fuzzy Logic Controllers, and Comparative Study of Approximate and Exact Methods. 29. HP Technical Reports A Fuzzy Inference Design on Hewlett-Packard Logic Synthesis System by R. L. Chen www.hpl.hp.com/techreports/93/HPL-93-70.html Cochannel Interference in Wireless LANS and Fuzzy Clustering by P. Golding and A. E.Jones www.hpl.hp.com/techreports/97/HPL-97-95.html 97 Chapter 4 30. Fuzzy Filtering www.innovatia.com/software/papers/fuzzy.htm An article by Dan Simon, Innovatia Software. The paper discusses fuzzy logic, a more heuristic approach to filtering. The application is fuzzy filtering for phase-locked loops; the approach can be applied to most estimation problems. 31. International Journal of Fuzzy Systems, IJFS www.fuzzy.org.tw/English/default.htm This is a publication of the Chinese Fuzzy Systems Association. All articles are in PDF format and available for free downloading. The first issue appeared in September 1999. 98 CHAPTER 5 Critique of Fuzzy Logic 5.1 Introduction Most introductory writings about fuzzy logic mention that it was surrounded with controversy since its inception, then typically give no details. Critical discussions of fuzzy logic are scattered over literature that may not be accessible to practicing engineers. The limitations of fuzzy logic applications are not typically discussed either. It is, however, important for engineers to recognize and understand, to some degree, the issues that surround fuzzy logic. Here, some critical views and limitations of fuzzy logic in engineering applications are outlined in a relatively easy-to-follow approach. An extensive selected bibliography that relates to the issues discussed is provided for readers who may wish to pursue any of the issues in more depth. 5.2 Objectives of Fuzzy Logic Numerous individuals have taken extreme positions supporting or opposing the concepts put forward by Lotfi Zadeh regarding fuzzy sets and fuzzy logic. Some maintain that probability theory can handle any kind of uncertainty; others think that fuzziness is probability in disguise, or that probability is the only sensible way to deal with uncertainty. Some found that the term fuzzy suits the innumerate rather than the educated. Even stronger oppositions were expressed in statements such as: Fuzzy logic is the cocaine of science (attributed to William Kahan), and Fuzzification is a kind of scientific permissiveness ... I cannot conceive of fuzzification as a viable alternative to scientific method (attributed to Rudolff Kalman). On the other extreme, some became highly enthusiastic and put forward claims that surround fuzzy logic with an aura of mystique. Some even claim that the concepts of fuzzy set theory and fuzzy logic are changing the world. And so these men of Hindostan Disputed loud and long, Each of his own opinion Exceeding stiff and strong, Though each was partly in the right, And all were in the wrong! John Godfrey Saxe (18161887) 99 Chapter 5 Perhaps the most reasonable critique of fuzzy logic should focus on the objectives of fuzzy logic as stated by its founder, Lotfi Zadeh, and ask: have these objectives been achieved? The objectives were to make computers think like people and to enable computing with words. When asked about the future of fuzzy logic, Professor Zadeh supposedly answered: my crystal ball is fuzzy. No claims about mystical nature or changing the world were made, but a distinction between vagueness and probability was emphasized. It is obvious that computers are not yet thinking like people. Nor are we computing with words, but we are closer to achieving these objectives than ever before. Numerous embedded systems use fuzzy logic, alone or with other techniques, and as a result, new devices and systems have thus been enabled. One can state that, from an engineering point of view, fuzzy logic is quite successful, but like any other methodology it has its limitations. It is important for engineers to understand fuzzy logic and its limitations to get the maximum out of this important design methodology and apply it when it is advantageous, or seek an alternative otherwise. It is good to remember that: Engineering is the art of doing that well with one dollar, which any bungler can do with two dollars after a fashion. Arthur M. Wellington [1847-1895], the Economic Theory of the Locomotion of Railways, Wiley, New York, 1900. 5.3 Whats in a Name? We had a funny discussion over the term fuzzy. The company is highly respected and asked us to find another name. Jens-Jorgen stergaard It is amazing to observe that because the term fuzzy was coined for the new concepts that were introduced, it caused strong reaction, both favorable and unfavorable. It is actually fortunate that Lotfi Zadeh coined the term fuzzy rather than using a less exotic term such as Empirical Computer Aided Design Methodology. There would have been less opposition, but also greatly lessened enthusiasm and interest in investigating potential applications. One should realize that the term fuzzy is used as a designation for a different class of sets and a different approach to reasoning. There is nothing fuzzy about fuzzy logic. It is the logic of fuzziness, not being fuzzy itself. On the contrary, a detailed expert knowledge of the system is usually needed. It could be helpful to draw a parallel with the term imaginary. Rene Descartes introduced the term imaginary to designate a new class of numbers. He argued that although one can imagine that every nth degree equation had n roots, there were no real numbers for some of these imagined roots. Electronics engineers realize that imaginary numbers have nothing to do with mysticism or mythology. They have real applications; imaginary is just a term that designates a different class of numbers. 100 Critique of Fuzzy Logic 5.4 Fuzziness vs. Probability Debate began with the argument that probability theory cannot capture all aspects of uncertaintyfor example, the uncertainty stemming from the intrinsic vagueness associated with human natural language. Probabilitists disagree; for example, Lavriolette and Seaman explain: Probability should be conceived as a measure of an individuals uncertainty about an event or object. Probability does not exist, just as pounds or kilograms do not exist. Probability is a measure of personal uncertainty, which does exist. We contend that the adherents of FST [Fuzzy Set Theory] have failed to demonstrate that their approach solves problems which are insoluble probabilistically. FST has certainly been used to solve problems, but the fuzzy set theorists have failed to prove the existence of uncertain events which cannot be represented probabilistically. This failure coupled with their disdain for operational measures of efficacy, has led them to employ if it works, it is good as a measure of success. It has also led them to claim that since probabilitists cannot solve this problem, it cannot be solved with probability. ... the need for expediency does not preclude the development of other, superior solutions. Lindley supported that view and added: Probability is the only satisfactory measure of ones personal uncertainty about the world and I put forward the challenge that anything that can be done by alternatives to probability can be better done by probability. One should observe, however, that even if it is true that probability could be used to formulate human knowledge and experiences, the details have never been developed to the extent reached by fuzzy logic theory. One should also observe that fuzzy logic has a relatively short history compared to probability theory. Fuzziness supporters point out that probability and fuzziness are related and may be complementary, but are two different concepts. Fuzziness measures the degree to which an event occurs, not whether it occurs or not as is the case with probability. Probability assumes that the event class is crisply defined and that the laws of noncontradiction ( A A = ) and the law of excluded middle ( A A = I ) hold. Fuzziness occurs when these laws are violated. Bertrand Russell pointed out much earlier that these two laws can be violated by introducing what became known as Russells Paradox. An army barber was ordered to shave a man if and only if he does not shave himself. Who shaves the barber? The most misleading similarity between fuzziness and probability is that both systems quantify uncertainty with numerical values in the interval [1,0]. However, it is important to note that membership values of various elements in a given fuzzy set do not have to add up to one. Another misleading similarity stems from the fact that 101 Chapter 5 the two systems algebraically manipulate sets and prepositions associatively, commutatively, and distributively. These similarities are misleading because a key distinction comes from the systems they are attempting to describe. For example, the uncertainty of tomorrows weather could be stated by saying that the probability that it will rain tomorrow is 70%; by tomorrow this uncertainty changes to certainty. On the other hand, the membership value one may assign to a chair in the set COMFORTABLE would remain the same tomorrow. The way one thinks about the degree of comfort experienced by a given chair will not change. The term fuzzy probability has been introduced in the fuzzy logic literature to describe two conceptually distinct hybrid fuzzy-probabilistic theories. The two theories have common features such as: they are based on and extend the classical probability theory. they use fuzzy numbers to quantify the probability of occurrence of an event. they follow almost the same probability calculus. The two differ, however, in the assumptions they make about the origin of the uncertainty regarding the probability of a given event. The first assumes fuzziness to occur in probabilities due to fuzziness in the definition of the event in question. The second assumes fuzziness in probabilities as a way of modeling vagueness in subjective linguistic probability assignments; it could rather be referred to as linguistic probability. 5.5 Fuzzy Logic vs. Multiple-Valued Logic Multiple-valued logics are non-classical logics; they do not restrict the number of truth values to only two. They can be categorized into families that comprise uniformly defined finite-valued and infinite-valued systems. They include: Lukasiewicz logics Gdel logics t-Norm related systems Dunn/Belnaps 4-valued system Product systems Multiple-valued logic, MVL, was created as a separate subject by Jan Lukasiewicz in the 1920s. One of the main motivations of developing Lukasiewicz MVL was to create a logical system that is free of some of the paradoxes in the twovalued Boolean logic. The Lukasiewicz logic with [0,1] as the truth value set could be viewed as a starting point for fuzzy logic. Fuzzy logic can be viewed as a generalization of MVL. In fuzzy logic, one infers new facts along with their truth values. In MVL, one infers only those facts that have truth values of 1 (absolutely true). Fuzzy 102 Critique of Fuzzy Logic set theory is application oriented. It went beyond Lukasiewicz MLV by developing linguistic semantics for logic using fuzzy hedges. This led to a multitude of practical applications. 5.6 Philosophical Issues No theory can ever be proven by experiment to be the correct theory. However, a theory is falsified, when it is shown that its predictions do not agree with some experiment. K. R. Poper The above statement seems to be widely accepted in engineering and philosophy. Theories, however, have their limitations and domain of applicability. The fact that a theory does not apply under certain conditions does not imply that it is not valid, but rather it is not the correct theory. It is conceivable that there exists for any theory an experiment that can falsify it. But, if a theory can lead to a valid solution, it can be used for engineering purposes. As far as the laws of mathematics refer to reality, they are not certain; and as far as they are certain, they do not refer to reality. Albert Einstein Quoted in J R Newman, The World of Mathematics, 1956. It is a fundamental engineering concept to recognize the domain of applicability of any given theory. Logic will not be deemed invalid just because the following experimental observations have been confirmed over and over again: I returned, and saw under the sun, that the race is not to the swift, nor the battle to the strong, neither yet bread to the wise, nor yet riches to men of understanding, nor yet favour to men of skill; but time and chance happeneth to them all. Ecclesiastes, Chapter 9 Ellen Hisdal wrote a series of wonderful papers detailing various difficulties with fuzzy set theory. Some of them are mentioned briefly here. The details of the arguments are provided in the papers cited in the bibliography. The discussion included: The possibility-certainty difficulty. According to the theory of fuzzy sets, the numerical values of membership functions are equal to those of the possibilities for a given label. The possibility x is defined to be numerically equal to the membership function of F when we are given the proposition x is F, e.g. John is young. But when we know that the possibility of an event is equal to 1, we do not know whether this event is certain or only a possibility. 103 Chapter 5 The depression of the inclusive OR curve. The max operator is supposed to represent the inclusive OR, not the exclusive one. The depression in the resultant curve of Figure 5.1 should not be there. (x) A=Tall B=Medium x (x)A B Tall OR Medium x Figure 5.1: The depression in the curve for the inclusive OR a) Fuzzy sets defining the labels tall and medium b) The resultant of tall OR medium using the max operation The VERY operator The membership value of VERY is defined to be equal to the square of for a given value of . It would have been more intuitive if the definition led to an increase in the membership values resulting in an upward shift of the membership graph. 104 Critique of Fuzzy Logic There is a lack of distinction between the result of AND and OR at a common crossover point. There are too many postulates and undefined concepts. The shapes of the membership functions have no theoretical basis. The union of a fuzzy set and its complement is, in general, not equal to the attributes of the universe. Susan Haack addressed other philosophically-oriented aspects of fuzzy logic. She criticized it directly by asking: do we need fuzzy logic? and reaching the conclusion that we do not. She also criticized fuzzy logic indirectly by asking: is truth flat or bumpy? and supported the argument that describing p as one-third true is sheer nonsense. She concluded that truth does not come in degrees and that fuzzy logic is not a viable competitor of classical logic. She explained that the term fuzzy logic may refer to two related, but distinct, enterprises: The interpretation of many-valued logics in terms of fuzzy set theory. She refers to it as base logics of fuzzy logic and accepts it. New logical systems in which the truth values are themselves fuzzy sets. She pointed out that this does not only challenge classical logic, but also the classical conception of what logic is; she refuses that. She suggested that it is base logics that have been given practical applications. Based on her views and arguments, we may summarize that what is useful in fuzzy logic theory is not particularly new and what is new is not particularly useful. We might then ask, why did the creation of so many useful applications have to wait until fuzzy logic appeared? What a coincidence! How do classical logic and reasoning explain it? 5.7 Engineering Applications Issues Fuzzy logic, like any other methodology or theory, has its limitations and domains of applicability. The solution it provides may not be always the best solution for any problem. It is important for individuals who are interested in using fuzzy logic for practical applications to recognize that and use fuzzy logic methodology only when it is advantageous. Limitations Limitations of fuzzy logic from an engineering point of view include: Fuzzy logic cannot solve problems that have no known solution; it requires the knowledge of an expert. It is particularly useful when there is an expert who can solve the problem, but there is no mathematical model to follow. If 105 Chapter 5 no one knows how to solve the problem, then it follows that no rules can be devised and fuzzy logic principles cannot be applied. Fuzzy logic algorithms do not have the ability to learn membership functions or rules during or after problem solving. Extensive verification and validation are required, especially where safety is a key factor. Assessing the stability of a fuzzy system before testing it is complicated and there is no standard, straightforward technique for stability determination. Fuzzy control vs. PID control Now it is the turn of engineers to make claims similar to those put forward by some mathematicians, scientists, and philosophers: some other technique could have been used for a better result. This time it is op-amps, resistors, and capacitors. National Semiconductor engineer and author Bob Pease explained that Mamdanis PID controller was said to give poor performance because it was misapplied. He also explained that, in general, literature that reports superior results of FLC over PID could be due to poor design of the PID. It could have been interesting in the early days of fuzzy logic applications to compare FLC performance results with those of PID to gain confidence that FLC can work. It would still have been a useful finding if FLC was not reported to be superior to PID. But the main strength after the the technique gained some acceptance, is not in competing with PID when PID could be used satisfactorily. The aim is to design a controller where PID cannot be used because there is no simple, accurate mathematical model of the system. Mathematical complexity An appealing aspect of fuzzy logic from an engineering applications point of view is its stated objective of making computers think like people (not necessarily intelligent people, but those whose expertise needs to be modeled for computational purposes regardless of their intelligence level). Its approach is to model with words systems that cannot be modeled mathematically, or mathematical models that are not of practical use. It appears that fuzzy logic applications depend on common sense and modernize the old engineering methodology of design based on experience rather than theoretical analysis only: the empirical design. Fuzzy logic, as it appears now in some literature, is quite complicated. The mathematics involved could be even more challenging than the typical mathematics needed for other engineering problem-solving methodologies. Fuzzy logic could become more complicated than it should be and more difficult to understand. 106 Critique of Fuzzy Logic Since the mathematicians have invaded the theory of relativity, I do not understand it myself anymore. Albert Einstein Quoted in P A Schilpp, Albert Einstein, Philosopher-Scientist , Evanston ,1949. Concluding Remarks Fuzzy logic theory was criticized since its inception. However, its applications, particularly in embedded systems, have achieved success and increased in number over the years. This observation prompted some critiques to quip: Fuzzy logic works in practice but not in theory! Some over-enthusiastic supporters of the theory promote it as the answer to all problems. For engineers, it is fundamental to understand all approaches available to solve a problem, to know their merits and recognize their limitations. Only then can they make educated and practical choices regarding the optimum solution of the problem at hand. In engineering, the concern is not with a particular philosophical dogma, but with a working model. The model would be satisfactory if all observations indicate that it achieves its objectives within set limits. Practice, guided by theory, is the domain of engineering. 107 Chapter 5 Bibliography 1. J. Bezdek, Fuzziness vs. Probability Again, IEEE Transactions on Fuzzy Systems, 2, 1, 13, 1994. 2. D. Dubois and H. Prade, Fuzzy Sets and Probability: Misunderstandings, Bridges and Gaps, Proc. of the 2nd IEEE Int. Conf. on Fuzzy Systems, 2, 10591068, 1993. 3. C. Elkan, The Paradoxical Success of Fuzzy Logic, IEEE Expert, 9, 38, 1994. 4. G. Epstein, Multiple-Valued Logic Design: An Introduction, Institute of Physics Publishing, Bristol, UK, 1993. 5. B. R. Gaines, Fuzzy and Probability Uncertainty Logics, Inform. & Contr., 38, 154169, 1978. 6. S. Gottwald, Many-Valued Logic, Stanford Encyclopedia of Philosophy, 2000 (plato.stanford.edu). 7. S. Haack, Deviant Logic, Fuzzy Logic: Beyond the Formalism, The University of Chicago Press, Chicago, 1996. 8. J. Halliwell and Q. Shen, Towards a Linguistic Probability Theory, Proc. of the 11th Int. Conf. on Fuzzy Sets and Systems, FUZZ-IEEE02, 2002. 9. E. Hisdal, Infinite-valued Logic Based on Two-valued Logic and Probability. Part 1.1 Difficulties with Present-Day Fuzzy-Set Theory and Their Resolution in the TEE Model, Int. J. man-Machine Studies, 25, 89111, 1986. 10. E. Hisdal, Infinite-valued Logic Based on Two-valued Logic and Probability. Part 1.2 Different Sources of Fuzziness, Int. J. man-Machine Studies, 25, 89111, 1986. 11. E. Hisdal, The Philosophical Issues Raised by Fuzzy Set Theory, Fuzzy Sets & Systems, 25, 349356, 1988. 12. E. Hisdal, Are Grades of Membership Probabilities? Fuzzy Sets & Systems, 25, 325348, 1988. 13. G. J. Klir, Is There More to Uncertainty than Some Probability Theorists Might Have Us Believe? Int. J. General Systems, 14, 347378, 1989. 14. G. J. Klir, Principles of Uncertainty: What are they? Why do we need them? Fuzzy Sets & Systems, 74, 1531, 1995. 15. B. Kosko, Fuzzy Thinking: The New Science of Fuzzy Logic, Hyperion, New York, 1993. 108 Critique of Fuzzy Logic 16. B. Kosko, Fuzziness versus Probability, Int. J. General Syst., 17, 211140, 1990. 17. M. Laviolette and J. W. Seaman, Jr., The Efficacy of Fuzzy Representations of Uncertainty, IEEE Transactions on Fuzzy Systems, 2, 415, 1994. 18. D. V. Lindley, The Probability Approach to Treatment of Uncertainty in Artificial Intelligence and Expert Systems, Stat Sci, 2, 1724, 1987. 19. D. McNeill and P. Freiberger, Fuzzy Logic: The Discovery of a Revolutionary Computer Technology - And How it is Changing Our World, Simon & Schuster, New York, 1993. 20. B. Pease, Whats All This Fuzzy Logic Stuff, Anyhow? Electronic Design, Part I, May 13, 1993, p.77, Part II, Nov. 1, 1993, p. 95, Part III, Nov. 11, 1993, p.105, Part IV, Nov. 6, 2000, Part V, Nov. 20, 2000. 21. W. Pedrycz, Why Triangular Membership Functions? Fuzzy Sets & Systems, 64, 2130, 1994. 22. K. C. Smith, Multiple-Valued Logic: A Tutorial and appreciation, IEEE Computer Magazine, 4, 1727, 1988. 23. W. Stallings, Fuzzy Set Theory versus Bayesian Statistics, IEEE Trans. Syst. Man Cybern,, 7, 216219, 1977. 24. O. Wolkenhauer and J. M. Edmunds, A Critique of Fuzzy Logic in Control, Int. J. Elect. Enging. Educ., 34, 235242, 1997. 25. L. A. Zadeh, Making Computers Think Like People, IEEE Spectrum, 9, 2632, 1984. 26. L. A. Zadeh, Fuzzy Logic = Computing with Words, IEEE Transactions on Fuzzy Systems, 4, 2, 103111, 1996. 27. L. A. Zadeh, What is Fuzzy Logic and What are its Applications? Scientific Computing Seminar, NERSC, May 17, 2002. (www.nersc.gov/research/ SCSeminars/200220517.html). 28. L. A. Zadeh, Some Reflections on the Anniversary of Fuzzy Sets and Systems, Fuzzy Sets & Systems, 100, 57, 1998. 109 [This is a blank page.] CHAPTER 6 Neural Networks 6.1 Introduction Both fuzzy logic and electronic (or artificial) neural networks are inspired by aspects of the computational power of humans. Fuzzy logic, as explained earlier, provides mechanisms for inference under cognitive uncertainty. It attempts to make computers think like people. Artificial neural networks, on the other hand, provide the means for pattern classification. They attempt to mimic the biological sensory system. The two fields developed independently, but they can complement each other in numerous applications. This chapter introduces fundamental ideas of neural networks. The following chapter complements this introduction by presenting neurofuzzy systems and their applications. 6.2 Background Since artificial neural networks (neural networks for short) are inspired by biological systems, it is useful to give a short account of these biological systems and provide an explanation of some of the commonly used terminology. It is useful to note that the term connectionism appears in some relevant literature. It has its roots in an early movement in cognitive science to model the brain based on interconnection of many simple units forming a networki.e., a neural network. The biological neuron is the basic building unit of the nervous system. The human brain is estimated to have billions of neurons, each with thousands of connections to other neurons. It is a living cell that does not duplicate itself, yet it can develop more connections with other neurons as the individual learns more. The main features of neurons are: The dendrites, which are branch-like structures that receive input signals to be processed by the cell body. The cell body (soma), which performs the central functions of the cell. The axon, which carries the output signals. It is terminated with the so-called terminal buttons. The point at which two or more neurons interact is known as the synapse. The synaptic cleft is the area between the terminal buttons of one neuron and the dendritic ends of another. The signal is transmitted across by chemicals known as neurotransmitters. 111 Chapter 6 Models of biological neural networks are built around nodes (or processing units) that simulate the action of a neuron. Input and output links to these nodes simulate synapses. Weights are assigned to the input links to simulate the action of the neurotransmitters. An algorithm is then used to adjust the weights of the input links so that the neurons produce the desired output, thus simulating the process of learning. Such algorithms are therefore referred to as learning or training algorithms. 6.3 Learning Learning, rather than being programmed, is an important feature of human computational abilities. Learning may be viewed as the change in behavior acquired due to practice or experience, and it lasts for a relatively long time. As it occurs, the effective coupling between the neurons is modified. Assessment of learning should not be limited to measuring the relatively permanent change in behavior that occurs or what the learner can do. This would be a limited view of learning. Learning may occur without any immediate measurable change in behavior. Such learning acts as a catalyst for measurable learning later. The first type of learning could be better described as training, which is usually what one hope to achieve by training an artificial neural network. The second type is the one that provides humans with the efficiency for further learning, the ability to reason, and leads to transferable skills. Both types complement each other and require time to achieve. Learning can occur in two modes: supervised and unsupervised. In supervised learning, examples (training sets of data) are required. Unsupervised learning does not require examples; it is adaptive or self-organizing. Unsupervised learning is very common in biological systems. It is also important for artificial neural networks; training data are not always available for the intended application of the neural network. 6.4 The Basic Neuron Model As outlined in the previous section, the biological neuron is the basic building unit of the nervous system. Similarly, the artificial neuron is the basic building unit of the artificial neural network. It is thus important to model mathematically, and hence computationally, or using VLSI, the features of the neuron described in the previous sections. The basic features may be outlined as: the output from a neuron is either ON or OFF. the output depends on the weighted sum of the inputs. A certain threshold must be reached to make the neuron fire. The weights associated with the inputs model the efficiency of the synapses, coupling. A more efficient synapse will have a larger weight. The neuron is trained by adjusting these weights. 112 Neural Networks These features are summarized in Figure 6.1 depicting the basic neuron model. x1 Soma w1 w2 x2 Inputs Axon y Output wn xn Dendrites Sumation & Thresholding Figure 6.1: Basic neuron model. The inputs are x1, x2,...xn and the output is y. The body adds the weighted inputs then compares this sum against the threshold. From Figure 6.1 one can write: Total weighted input = w1 x1 + w2 x2 + w3 x3 + + wn xn n = wi xi . i =1 If this sum exceeds the neurons threshold value then the neuron fires (the output is ON; otherwise it stays OFF). This is illustrated in Figure 6.2 Output Output 1 1 Inputs Inputs Figure 6.2: The thresholding function. a) Thresholding at . b) Thresholding at 0. 113 Chapter 6 The output of the neuron can be expressed by: n y = fh ( wi xi ) i =1 where fh is a step function (commonly known as the Heaviside function) defined by: fh ( x ) = 1, x > 0 = 0, x 0 The bias, or offset , could be introduced as one of the inputs with a permanent weight of 1, leading to a simpler expression: n y = fh ( wi xi ) i=0 The lower limit in the above equation is from 0 rather than 1. The value of input x0 is always set to 1. Using the Heaviside function leads to hard threshold. This neuron model was introduced by Warren McCulloch and Walter Pitts in the 1940s. The model is sometimes referred to as the McCulloch-Pitts neuron or MP neuron. Other functions with softer transitions are commonly used. These include the ramp and sigmoid functions. The sigmoid function can be expressed as: 1 y= 1 + e kx where y is the output, x is the total input and k is a gain factor that controls the sharpness of the transition from 1 to 0, as illustrated in Figure 6.3. The sigmoid function is particularly useful because its derivative is easy to compute: dy 1 = ky (1 y ) . , then If y = kx dx 1+ e This feature could save computational time executing training algorithms. Example The AND Problem Suppose the neuron is to be trained to respond to the patterns 00, 01 and 10 by an output of 0, and to the pattern 11 by an output of 1. This is the operation of a Boolean AND. The problem is illustrated in Figure 6.4. The solution is presented by the line L separating the two classes of patterns. Other lines could have been drawn to separate the patterns, i.e. there are numerous possible solutions to the problem. 114 Neural Networks y 1 k=10 k=1.0 k=0.5 k=0.3 0.8 0.6 0.4 0.2 0 -20 -15 -10 -5 0 x 5 10 15 20 Figure 6.3: The sigmoid function. x2 AND Inputs x1 x2 0 1 0 1 0 0 1 1 Output y 0 0 0 1 y=0 0 y=0 1 AND 1 y=0 y=1 L x1 Figure 6.4: A graphical illustration of the problem. A possible neuron specification to solve the problem is given in Figure 6.5. In that figure, when the input is 11 the weighted sum exceeds the threshold, leading to an output of one. It is obvious again that there are several other specifications that could have led to the same result. 115 Chapter 6 x1 0 0 1 1 x2 0 1 0 1 y 0 0 0 1 0.5 0.5 0.9 y y = (i xi ) x1 x2 Figure 6.5: Possible specifications of a neuron to solve the AND problem. The numbers on the input arrows are weights associated with the inputs and the number inside the circle is the threshold value. More complicated pattern recognition problems require more than one neuron. These interconnected neurons form a neural network. 6.5 The Perceptron The perceptron is the simplest neural network. It has the structure shown in Figure 6.6. Several neurons are arranged in one layer with the inputs connected to every neuron. Learning occurs by adjusting the weights associated with the inputs so that the network can classify the input patterns. In this case a training algorithm would be necessary. x1 w11 w12 y2 y1 Inputs x2 Outputs wnn yn xn Figure 6.6: The perceptron. 116 Neural Networks The highlights of such an algorithm are: Initialize weights and thresholds by assigning random values. Present a training patterni.e., an input and the desired output. Calculate the actual output: n y = f wi ( t ) xi ( t ) i=0 Adapt weights: If correct, then wi (t + 1) = wi (t ). If output 0 should be 1 (class A), then wi (t + 1) = wi (t ) + xi (t ). If output 1 should be 0 (class B), then wi (t + 1) = wi (t ) xi (t ). where 0 1 is a constant that controls the adaptation rate. Other methods to adapt weights have been used. The Widrow-Hoff delta rule calculates the difference between the weighted sum and the desired output. This difference is called the error and given the designation, . It is defined by: = d (t ) y (t ) where d(t) is the desired response and y(t) is the actual response. Then wi (t + 1) = wi (t ) + xi (t ) d (t ) = 0 if input is from class B. d (t ) = +1 if input is from class A, and Widrow called networks using this algorithm Adaptive Linear Neurons, ADALINES. Connecting many of them together leads to a structure referred to as many-ADALINES, MADALINES. The training algorithm of the perceptron attempts to determine the straight line that separates classes, but there are numerous cases where the classes are not linearly separable. The simplest example is the case of the exclusive-OR (XOR) problem. For input patterns 00 and 11, the output is to be 0 and for input patterns 01 and 10, the output is to be 1. The problem is illustrated in Figure 6.7. From that figure one can see that there is no one single line that separates the patterns. Such patterns are referred to as linearly inseparable. A single-layer perceptron cannot solve any problem where the patterns are not linearly separable. 117 Chapter 6 x2 XOR Inputs x1 x2 0 1 0 1 0 0 1 1 Output y 0 1 1 0 y=0 0 1 XOR 1 y=1 y=0 y=1 x1 Figure 6.7: Illustration of the problem. 6.6 The Multilayer Perceptron To overcome the difficulty of classifying linearly inseparable patterns, a multilayer perceptron is used. A nonlinear thresholding function is also usede.g., a sigmoid, not a step function. Layers of perceptrons with linear functions would not be more capable than a suitably chosen single layer. A typical architecture of a multilayer perceptron is illustrated in Figure 6.8. Inputs Outputs Input Layer Hidden Layer Output Layer Figure 6.8: A typical architecture of a multilayer perceptron. 118 Neural Networks It is a feedforward network that consists of three layers. An input layer is directly connected to the input and an output layer is directly connected to the output. In addition, there is a layer that is not directly connected to either the input or the output and hence is referred to as the hidden layer. The hidden and output layers have a perceptron-like structure, but with the threshold function of the neurons being sigmoid rather than a step function. From this point of view, the network shown in Figure 6.8 could be referred to as a two-layer network. The multilayer perceptron was introduced to solve the problem of linearly nonseparable patterns. Considering the XOR problem as a benchmark, it is worth investigating if a multilayer perceptron could indeed solve that problem. Example A possible multilayer perceptron solution to the XOR problem is shown in Figure 6.9. For input 00, the weighted sum for neuron A or neuron B is 0, the output is thus 0 and hence the weighted sum for neuron C is 0 leading to y = 0. For input 01, the weighted sum for neuron A is 1, and its output is 1. The weighted sum for neuron B is 1 which is less than the threshold of 1.5, thus the output is 0. The weighted sum for neuron C would be 1, leading to an output of 1. Similar reasoning applies to the input 10. For input 11, the weighted sum for neurons A and B would be 2, and hence the output of both would be 1. The weighted sum for neuron C, however, would be 0, leading to y = 0. A x1 Inputs x2 1 1 1 0.5 1 C 0.5 y Outputs -1 1.5 1 B Figure 6.9: A possible solution to the XOR problem using a multilayer perceptron. In summary, for patterns 00 and 11 the networks respond with an output of 0, and for patterns 01 and 10 the networks respond with an output of 1, thus solving the XOR problem. 119 Chapter 6 One would expect that the learning rule for multilayer perceptrons would be different from that of the perceptron. The new rule is called the generalized delta rule, or the backpropagation rule. It was put forward by Rumelhart, McClelland and Williams in the 1980s. An input pattern is provided and the output is calculated and then compared with the desired output. The weights are altered accordingly so that the network produces fewer errors the next time. The use of a sigmoid function is essential so that enough information concerning the output is available to neurons in the earlier layers. This enables these neurons to get their corresponding weights adapted in order to reduce the error next time. This type of learning is called supervised learning since a training set (input patterns and the corresponding desired output) is required. The weights of each neuron are adjusted in direct proportion to the error in the neuron to which it is connected. Backpropagating these errors through the network enables all of the weights to be correctly adapted, and thus the network learns the patterns of the training set. The multilayer perceptron learning algorithm can be presented as follows: Weights and threshold are initialized by assigning to them small random values. A training pattern is presented consisting of an input: x0 , x1 , x2 ,, xn 1 and a target (desired) output: t0 , t1 , t2 ,, tm 1 . The actual output of each layer is then calculated by: n 1 y pj = f ( wi xi ) i=0 where ypj is the output from layer j and f(.) is the sigmoid function. The output is passed to the next layer until the final output Opj is reached. Weights are adapted starting from the output layer and moving backwards: wij (t + 1) = wij (t ) + pj Opj where wij are the weights from neuron i to neuron j at time t, is a gain factor, and pj is the error for pattern p on neuron j. For the output neurons: pj = kOpj (1 Opj )(t pj Opj ) For the hidden neurons: pj = kOpj (1 Opj ) ( pk w jk ) where k k represents summation over the k neurons in the layer above neuron j. 120 Neural Networks The steps are repeated until a pattern is learned. Then a new pattern is presented and so on, until all the patterns of the training set are learned. It is useful to note that numerous variations of this algorithm have been put forward by various authors. The training time of the network depends on the convergence rate, i.e. the speed with which the network error approaches zero. To visualize the network behavior and the possible learning difficulty one may consider the energy function defined by: 2 1 E = (t pj Opj ) 2 This represents the amount by which the output deviates from the target (desired) output. Now consider a network in which only one weight is being adjusted. A possible energy function vs. weight graph is illustrated in Figure 6.10. In general, n adjustable weights lead to an n-dimensional surface. The objective of the training algorithm is to adjust the weights so that E is minimized. Each minimum (called a basin of attraction in an n-dimensional surface) represents a solution. A difficulty occurs if the network settles into a local minimum rather than a global one (B, for example, rather than A in Figure 6.10). If this occurs, the network stops learning. E B A w Figure 6.10: One-dimensional energy function. A local minimum occurs when two or more disjoint classes are categorized as being the same. This difficulty could be avoided by using a training set with clearly distinguishable training examples. Adding more hidden layers reduces the occurrence of local minima. Random values may be added to the weights to make the network escape settling in a local minimum. Before using a training algorithm for the multilayer perceptron, one has to decide the network configuration: the number of layers and the number of neurons per layer. In general, not more than two hidden layers should be needed, given enough neurons per layer. A network with the fewest possible number of neurons is an optimum network. It leads to better performance, e.g. reduction in training time, computational cost, and improved generalization capability. To reach an optimum network, there are two approaches: 121 Chapter 6 Trimming Concept Start with a network that has an excess of neurons and then trim it, or Construction Concept Start with a network that has a few neurons and then add more as needed. There are several approaches to implement the trimming and construction concepts. One can examine a trained network and remove neurons with minimal contributions. One can also modify the training algorithm so that each weight is allowed to decay to zero, hence the network optimizes itself during training by removing unnecessary connections and unneeded neurons can also be removed. As well, there are algorithms that build the networks by starting with several neurons at the bottom of the architecture and then adding layers. Each added layer has fewer neurons than the previous one until the process comes to an end with a single output neuron. Other algorithms that add layers as needed have been put forward in the literature. 6.7 Recurrent Network Learning with multilayer perceptrons discussed in the previous section is an example of supervised learning in feedforward networksi.e., networks that allow bilateral connections between neurons. Examples of such networks are Hopfield networks and Boltzmann networks. 6.7.1 Hopfield Networks A Hopfield network consists of a number of neurons that are fully connected. That is, every neuron is connected to every other neuron. A possible arrangement is shown in Figure 6.11. The weights are symmetrical; the weight wij connecting neuron i to neuron j is equal to wji, the weight connecting neuron j to unit i. In this model, neurons use the hard threshold function of the McCulloch-Pitts neuron. All neurons are equal in this network, but some become more equal after training! The inputs to the network are applied to all neurons at the same time. The network is then given time to cycle through a succession of states until it converges to a stable solution. The stable, steady-state values of the neurons would be the output of the network. The operation can thus be summarized as: All the nodes are assigned a starting value. An unknown input pattern is applied to all neurons. The network iterates to convergence. 122 Neural Networks This architecture is most suitable when exact binary (1 and 1 or 1 and 0) representations are possible such as in the case of black and white images (where input elements are pixel values), or with coded text (where input values are sets of bits). A network trained to recognize a set of patterns will be able to do so when presented with them again. Recognition will occur with the patterns even if corrupted, i.e. with the appearance not identical to that of the training set. The number of patterns such a network can memorize and remember accurately is about 15% of the number of the neurons in the network. wij = wji w1n w12 w11 w2n w22 w21 wnn wn2 wn1 wij = wji wii = 0 x1 x2 xn 6.11: The architecture of a Hopfield network in two different arrangements. 123 Chapter 6 A possible learning algorithm for the Hopfield network can be described as follows: Weights are assigned to all connections such that: wij = xis x sj for i j , and s=0 M 1 wij = 0 for i = j 0 i, j M 1 where xis = 1 is element i in the training set for class s, and the training sets have M patterns. An unknown pattern is introduced, then where i (t ) is the output of neuron i at time t. i ( 0 ) = xi , 0 i N 1 Iteration continues until steady-state is reached. The iteration is governed by, N 1 i (t + 1) = f ( wij i (t )) , 1 j N 1 i=0 The process is repeated with other patterns in the training set. 6.7.2 Boltzmann Machine Networks A Boltzmann machine network could be viewed as a modification of a Hopfield network. It provides a mechanism for the network to escape from local minima and move to a global minimum in the energy landscape. The idea parallels that of metallurgical annealing. In that process, a metal is heated close to its melting point then allowed to cool slowly down to room temperature. The process leads to the removal of crystal defects such as dislocations and hence allows the metal to reach a stable low-energy configuration. In thermodynamics, the probability of a system being in state A or B follows the Boltzmann probability distribution: Prob ( A ) E = exp Prob ( B ) T where E = E ( B ) E ( B ) , E ( A ) and E ( B ) are energies corresponding to the two states, and T is the temperature. The Boltzmann machine network is fully connected like the Hopfield network, but unlike the Hopfield network, neurons are randomly divided into three groups. Some are chosen to be input neurons, some to be output neurons, and the rest are not 124 Neural Networks connected directly to either the input or the output, i.e. they are hidden. Learning occurs in two phases: incremental and decremental. Incremental (reinforcement) The input and output neurons are clamped to their correct values. The network is then allowed to cycle through its states. Suppose it is the turn of the ith neuron, selected randomly, to evaluate its input and determine its next state. For that purpose a probabilistic rule is used. Such a method would allow transition from a higher to a lower energy state as well as a transition from a lower to a higher energy state, to allow the network to escape local minima. The probabilistic rule determines, regardless of the current state of the neuron, the next state to be 1 (or ON) with a probability, p, given by: 1 p= E 1 + exp T The probability of the next state being 0 (OFF) would be, of course, 1 p. T is a positive number that plays the role of temperature. It can be adjusted to have different values at various stages of converging to a steady state. E is the energy of the system if the neuron is OFF and the negative value of the energy if the neuron is ON. It plays the role of activation energy. If E is a positive number, then the energy of the system is higher if the neuron is OFF (at 0). Thus to minimize the energy, having the neuron ON (at 1) is preferable. Similarly, E if is a negative number, then having the neuron OFF (at 0 or 1) would be the preferable state. The temperature parameter, T, is decremented until the output reaches a steady state. The weight between two neurons is incremented if they are both ON. Decremental (forget bad associations) In this phase, only the input neurons are clamped. The network is then given time to reach thermal equilibrium again. The weights between two neurons that are both ON are decremented. The process is repeated until the weights reach a steady-state. 6.8 Kohonen Self-Organizing Networks 6.8.1 Background The neural networks discussed in the previous section require training patternsthat is, they have to be told the desired response for a given input and feedback obtained on their performance until they learn the patterns. Difficulty occurs if there is no training pattern or historical data for the problem the network is required to solve. A different class of networks is needed: networks that do not need examples to learn. 125 Chapter 6 Such networks undergo both self-organizing and unsupervised learning. They modify the weights associated with the neural connections based on the characteristics of the input patterns. A Kohonen network is an example of such a network. 6.8.2 Description A typical Kohonen network consists of a two-dimensional layer (or grid) of interconnected neurons (plus an input layer). All inputs are connected to every neuron and all neurons are connected to each other. The network starts with small random values assigned to the weights. All neurons receive an input pattern. Then, the winner (capitalistic neuron) takes all. Thus, neuron with the largest output and its neighbors are allowed to learn (to adapt their weights). The output of each neuron acts as an inhibitory input to other neurons but acts as an excitory in its neighborhood. This process is known as lateral inhibition. The inhibitory effect of a neuron decreases with distance from the winning neuron. This process assumes the appearance of a Mexican hat function as shown in Figure 6.12. Excitory Effect Distance from winning neuron Distance from winning neuron Inhibitory Effect Inhibitory Effect Figure 6.12: The Mexican hat function describes the relationship between the distance from the winning neuron and the connection strength. Repetition of the process reduces the neighborhood of the winning neuron to a predetermined size. Thus, a cluster of neurons that respond to a particular pattern is formed. Repeating the process leads to converting the random set of neurons into clusters or subsets, each recognizing an input pattern. 126 Neural Networks 6.8.3 Determining the winning neuron In general, an input can be expressed as a vector with N components. It can then be normalized by dividing each of its components by the length of the vector, leading to xi ( 2 2 2 x1 + x2 + x N ) 1 2 All normalized vectors have a distance of one unit from the origin, but in different directions. Vectors with two components will be on a unit circle. Those with three components will be on a unit sphere. Vectors with N components will be on a hypersphere of N dimensions. Example Suppose an input pattern A has normalized components x1 and x2. The input vector is compared with each of the weight vectors to find the nearest distance. The square of the distance between points (x1,x2) and (w11,w21) is d = ( x1 w11 ) + ( x2 w21 ) 2 2 2 2 2 2 = w11 + w21 + x1 + x2 2 ( w11 x1 + w21 x2 ) = 2 2 ( net weighted input ) = 2 (1 net weighted input ) The input vector and weight vector are the same when d = 0, i.e. when the net weighted input = 1. Thus, finding the nearest vector is the same as finding the neuron with the largest net input and hence the largest output. 6.8.4 Learning algorithm The previous discussion of a Kohonen network can be summarized as follows, leading to an algorithm for self-organizing: ( ) ( ) weights are initialized by setting them to small random values a new input is presented the input vector xi(t) is compared with each of the weight vectors to determine which is nearest the distance dj between the input and each output neuron j where the distance is d j = ( xi (t ) wij (t )) i=0 n 1 2 where xi(t) is the input neuron i at time t and wij(t) is the weight from input neuron to output neuron j at time t. 127 Chapter 6 The output neuron, j*, with minimum distance is selected. The output neuron j* and neighbors are updated for j NE j * ( t ) wij (t + 1) = wij (t ) + (t ) ( xi (t ) wij (t )) where (t ) is a gain term that decreases with time and NE j * ( t ) is the neighborhood of the winning neuron at time t. The steps are repeated by presenting a new pattern. 6.9 Adaptive Resonance Theory, ART 6.9.1 Background Adaptive resonance theory, ART, refers to a class of self-organizing neural architectures. They include ART1, ART2, ART3, and ARTMAP. ART1 is concerned with classification of Boolean pattern sets. ART2 extends the classification to analog patterns. ART3 allows any number of ART2 modules to be concentrated into a processing hierarchy. ART networks were introduced to solve the so-called plasticity-stability problem. It is commonly expected that people should be able to add more to what they have already learned. However, in artificial neural networks a catastrophic failure of memory may occur if we try to add more patterns to a stable, trained network. The ART networks product could be stable, yet have the ability to learn more, which is referred to as plasticity. ARTMAP is a supervised version that can rapidly self-organize stable categorical mappings between n-dimensional inputs of m-dimensional output vectors. The key idea is to spare some output neurons for new patterns to be learned. If an input pattern and a stored pattern are sufficiently similar, they are said to be resonating and hence recognized. If they are too dissimilar, the unused output neurons are utilized to form a cluster for a new class of patterns. The network gives no response if all neurons are used. The fundamental features of the ART network are presented in the following sections through ART1, which is historically the first (and simplest) member of the ART family. 6.9.2 ART As opposed to the networks discussed in the previous sections, the ART network layers are not homogenous. An ART network has two layers that perform different functions. In addition, there are external parts to the layers that control the flow of the data through the network. The structure of the network is shown in Figure 6.13. 128 Neural Networks The first layer is the input/comparison layer and the second is the output/recognition layer. The neurons in the output layer are connected so that lateral inhibition can occur; that is, it uses the winner-take-all principle. Feedback and feedforward connections exist between every neuron in the input and output layers. Each layer has logic controls designated CONTROL-1 and CONTROL-2 connected to each neuron in layers F1 and F2, respectively. Attenuational gain control channel + + CONTROL-2 Category Representation Field Layer F2 Output/Recognition + Feedback weights Feedforward weights T W REST + CONTROL-1 Feature Representation Field Layer F1 Input/Comparison Vigillance Test + + + Input Pattern Figure 6.13: ART1 architecture. A vigilance test evaluates the similarity of an input pattern to a stored pattern (exemplar) or template. It uses Boolean templates; each 1 in a template is referred to as a feature. Similarities are measured by the ratio S, of the number of features common between the pattern and the template to the number of features in the input pattern, where 0 S 1 . For similarity to be declared: S > , where is a prescribed value referred to as the vigilance threshold. It is used to inform the network how to determine whether or not an input pattern is similar to an existing stored pattern. The value ascribed to it determines the resolution of the classification. The operation of the network can be summarized as follows: 129 Chapter 6 1. Initialization The feedback weights are set to 1, tij ( 0 ) = 1 . 0 i N , and 0 j M 1 The feedforward weights are set such that wij ( 0 ) = 1 , 1+ N where N is the number of input neurons and M is the number of output neurons. The vigilance threshold, , is assigned a value such that 0 < < 1 . 2. The Recognition Phase An input pattern is applied. The similarity is computed: i = ( wij (t ) xi ) i=0 N 1 where j is the output of neuron j in the output layer and xi is the input of neuron i in the input layer, which can be either 1 or 0. The best matching exemplar is selected: j* = max j 3. The Comparison Phase The winning neurons feedback their pattern by adjusting the weights Tij to the input. The ratio S is calculated: tij x S= xi If S , then the classification is complete. If S < , then a search phase starts. 4. The Search Phase All outputs are reset (set to ZERO). The input is reapplied. The recognition and comparison phases are repeated. The process is repeated until S . If no classification is possible, the input is declared unknown and is allocated to a previously unassigned set of neurons in the output layer. 130 Neural Networks Concluding Remarks In this chapter neural networks were introduced, and the concept of learning was discussed. Although there are numerous architectures and training algorithms for neural networks, it is important to observe that they all share some general features, including: Neural networks are trained, not programmed. Training is the process of adjusting the weights throughout the network so that it responds correctly to input patterns. Training may be supervised or unsupervised. Supervised training requires a training set or historical data. The computation depends on parallel processing. This leads to fault tolerance and graceful degradation, as opposed to catastrophic failure. The memory is not localized, but rather distributed. A trained network can generalize. It can classify patterns that it has not seen before; for example, a distorted input from the training set. The network may not always learni.e., the error may not approach zero. Time is needed for the network to learn; a network with complex functions would take longer to learn. The learning time also depends on the architecture and learning algorithm used. The network may over-learn, or learn the noise associated with input patterns. The capacity of the network is, in general, dependent on the number of neurons used. 131 Chapter 6 Bibliography 1. A. Aarts and J. Korst, Simulated Annealing and Boltzmann Machines, John Wiley, New York, 1989. 2. I. Aleksander and H. Morton, An Introduction to Neural Computing, ITP Press, London, 1991. 3. D. J. Amit, Modeling Brain Function, Cambridge University Press, 1989. 4. R. Beale and J. Jackson, Neural Computing, Adam Higler, Bristol, 1991. 5. R. Bharath and J. Drosen, Neural Network Computing, Windcrest/ McGraw-Hill, New York, 1994. 6. G. A. Carpenter and S. Grossberg, ART-2: Self-organization of Stable Category Recognition Codes for Analog Input Patterns, Applied Optics, 26, 491930, 1987. 7. G. A. Carpenter and S. Grossberg, The ART Adaptive Pattern Recognition by Self-organizing Neural Networks, IEEE Computer, 21, 7790, 1988. 8. G. A. Carpenter and S. Grossberg, ART3: Hierarchical Search using Chemical Transmitters in Self-organizing Pattern Recognition Architectures, Neural Networks, 3, 12952, 1990. 9. G. A. Carpenter, S. Grossberg, and J. H. Reynolds, ARTMAP: Supervised Realtime Learning and Classification of Nonstationary Data by Self-organizing Neural Network, Neural Networks, 4, 56588, 1991. 10. M. Caudill and C. Butler, Naturally Intelligent Systems, The MIT Press, Cambridge, 1990. 11. M. Chester, Neural Networks: A Tutorial, PTR Prentice Hall, Englewood Cliffs, NJ, 1993. 12. S. Deutsch and A. Deutsch, Understanding the Nervous System: An Engineering Perspective, IEEE Press, New York, 1993. 13. K. Gurney, An Introduction to Neural Networks, Routledge, Taylor & Francis Group, London, 1999. 14. M. T. Hagan, H. B. Demuth, and M. Beale, Neural Network Design, PWS Publishing Company, Boston, 1996. 15. S. Haykin, Neural Networks: A Comprehensive Foundation, Macmillan College Publishing Company, New York, 1994. 16. J. Hertz, A. Krogh, and R. G. Palmer, Introduction to the Theory of Neural Computing, Addison-Wesley Publishing Company, New York, 1991. 132 Neural Networks 17. J. J. Hopfield, Neural Networks and Physical Systems with Emergent Collective Computational Properties, Proceedings of the National Academy of Sciences of the USA, 79, 255488, 1982. 18. T. Kohonen, The Self-organizing Map, Proceedings of the IEEE, 78, 146480, 1990. 19. R. P. Lippmann, An Introduction to Computing with Neural Networks, IEEE ASSP Magazine, 4, 22, 1987. 20. W. T. Miller III, R. S. Sutton, and P. J. Werbos (eds.), Neural Networks for Control, The MIT Press, Cambridge, 1995. 21. P. K. Simpson (ed.), Neural Networks Theory, Technology, and Applications, The IEEE Press, New York, 1996. 22. D. M. Skapura, Building Neural Networks, Addison-Wesley Publishing Company, New York, 1995. 23. Y. X. Zhong, How is the Consciousness Related to Neural Networks?, Proceedings of the 8th International Conference on Neural Information Processing, Shanghai, China, Nov. 1418, 2000, pp.14. 133 Chapter 6 Web Resources 1. Artificial Neural Networks Tutorial www.gc.ssr.upm.es/inves/neural/ann1/anntutorial.html A tutorial provided in three languages by Dr. Fuente and presented in html format. It covers basic concepts of neural networks, supervised and unsupervised models. It also provides examples and a bibliography. 2. Neural Networks Tutorial with Java Applets diwww.epfl.ch/mantra/tutorial/english/ An excellent tutorial of neural networks through Java applets. It is provided by Laboratoire de Microelectronique, cole Polytechnique Fdrale de Lausanne. The topics include: Single Neurons Artificial Neuron McCulloch-Pitts Neuron Spiking Neuron. (Requires Swing) Hodgkin-Huxley Model Axons and Action Potential Propagation Supervised Learning Single-layer networks Multi-layer networks Density Estimation and Interpolation Unsupervised Learning Reinforcement Learning Network Dynamics Hopfield Network. Pseudoinverse Network. Network of spiking neurons. (Requires Swing). Retina Simulation. (Runs very slow with some Netscape versions). 3. The Basics of Neural Networks Demystified www.contingencies.org/novdec01/workshop.pdf An essay with eight references by Louise Francis. The essay appeared in Contingencies, Nov/Dec, pp. 5661, 2001. 134 Neural Networks 4. Neural Networks Report www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html An introductory report on Neural Networks by Christo Stergiou and Dimitrios Siganos, Department of Computing, Imperial College, London. It has seventeen references, five of which are Web accessible. The report topics include: neuron models, the architecture of neural networks, the learning process, and applications of neural networks. 5. Neural Nets www.shef.ac.uk/psychology/gurney/notes/index.html A book-size tutorial by Kevin Gurney, Department of Psychology, University of Sheffield, UK. It is available in HTML and PS formats. Topics covered include: Computers and Symbols versus Nets and Neurons, Learning Rules, The Delta Rule, Multilayer Nets and Backpropagation, Hopfield Network, Competition and Self-organization, and more. 6. An Introduction to Neural Networks www.cs.stir.ac.uk/~lss/NNIntro/InvSlides.html This is an introduction to neural networks in HTML format based on a talk given by Prof. Leslie Smith, Centre for Cognitive and Computational Neuroscience, Department of Computing and Mathematics, University of Stirling, UK. It provides a very useful overview of neural networks, reviews some algorithms and architectures, and discusses their past applications and likely new applications. 7. Connectionism plato.stanford.edu/entries/connectionism/ An essay from the Stanford Encyclopedia of Philosophy. It defines the term and discusses related topics. They include: A Description of Neural Networks, Neural Network Learning and Backpropagation, Samples of What Neural Networks Can Do, Strengths and Weaknesses of Neural Network Models, Connectionist Representation, The Shape of the Controversy between Connectionists and Classicists, The Systematicity Debate, Connectionism and Semantic Similarity, and Connectionism and the Elimination of Folk Psychology. A bibliography and Internet resources are available. 135 Chapter 6 8. Lecture on Connectionism yoda.cis.temple.edu:8080/UGAIWWW/lectures95/meeden/nnet/nnet.html Lecture note on Connectionism by Nikos Drakos of the Computer Based Learning Unit, University of Leeds. Topics covered include: an introduction to connectionism, learning to read with NETtalk, and suggestions for laboratory experiments. It also provides ten references. 9. Connectionism, Confusion, and Cognitive Science www.bcp.psych.ualberta.ca/~mike/Pearl_Street/ Papers/Confuse/ confuse.html An HTML version of a paper by Michael R.W. Dawson (Biological Computation Project, Department of Psychology, University of Alberta, Edmonton, Canada) and Kevin S. Shamanski that appeared in The Journal Of Intelligent Systems, 1994. It discusses the topic from a non-engineering point of view. It concludes that connectionism produces systems that generate interesting and sophisticated behaviors. However, connectionism has little to offer in terms of theoretical accounts of the internal structures, procedures, and representations that produce this behavior. It leads to the possibility of constructing intelligence without first understanding it. 10. Neural Networks Papers www.dsp.pub.ro/articles/articles.htm A collection of Web links to presentations on topics that include: History of neural networks The connectionist paradigm Logic function modelling using neural networks Continuous function approximation Neural networks learning methods and algorithms Neural networks back propagation algorithm Neural networks competitive learning Self organizing feature maps Neural networks learning transformations. Reinforcement learning Neural networks knowledge extraction Networks of spiking neurons Boltzmann machines Belief networks Radial basis functions The site is maintained by the DSP Laboratory, Department of Electrical Engineering, University of Politechnica, of Bucharest, Romania. 136 Neural Networks 11. Artificial Neural Networks Technology www.dacs.dtic.mil/techs/neural/neural_ToC.html A report on neural networks by Dave Anderson and George McNeil, Data & Analysis Center for Software, New York, USA. It is available in HTML, PDF, PS, and text formats. Topics presented include: Operations Training an Artificial Neural Network How Neural Networks Differ from Traditional Computing and Expert Systems History of Neural Networks Detailed Description of Neural Network Components and How They Work Network Selection: Networks for Prediction Networks for Classification Networks for Data Association Networks for Data Conceptualization Networks for Data Filtering How Artificial Neural Networks Are Being Used Emerging Technologies 12. Neural Networks Glossary rfhs8012.fh-regensburg.de/~saj39122/jfroehl/diplom/e-glossary.html A small glossary on neural network terms. Entries are accessible alphabetically, and some have hyperlinks. It is maintained by Joehen Frhlich, Farchhochschule Regensburg, Germany. 13. Glossary of Neural Computing www.brainstorm.co.uk/NCTT/portfolo/part06/welcome.htm A list of about 70 terms related to neural computing, maintained by DTI NeuroComputing Web. 14. Pictures of the Biological Neurons glinda.lrsm.upenn.edu/~weeks/neurons.html A collection of pictures of biological neurons provided by Eric Weeks, Emory University, Atlanta, GA, USA. 137 Chapter 6 15. Neurons Gallery faculty.washington.edu/chudler/gall1.html A collection of neurons in colour with information provided about each. The page is maintained by Eric Chudler, Department of Anesthesiology, University of Washington, USA. 16. Freeware Tools for Neural Networks vortex.cs.wayne.edu/NeuralNets/freelist.htm This site provides a list of software related to neural networks. Some are free, others are not. The list includes: @BRAIN, 1st Class, Adaptive Logic Network Educational Kit, AIM Estimator , AIM Problem Solver, The ART Gallery, Aspirin/Migraines, Baysian learning for neural networks, BrainMaker, CAD/Chem-Customer Formulation System, Cascade Neural Network Simulator, Cascor, Fuzzy ART, Neural Networks at your Fingertips, NeuralShell, and much more. 17. Neural Networks Software www.ncrg.aston.ac.uk/NN/software.html A big list of software related to neural networks with short descriptions. The list is provided by the Neural Computing Reseach Group, NCRG, Aston University, Birmingham, UK. 18. Neural Networks Warehouse neuralnetworks.ai-depot.com/Applications.html This site provides links under numerous categories including: books, tutorials, libraries, software, programs, research, and applications. The site is owned and maintained by Alex J. Champandard, Artificial Intelligence Depot. 19. Bibliographies on Neural Networks liinwww.ira.uka.de/bibliography/Neural/index.html A huge collection of papers related to neural networks. The collection is searchable by title. It is also divided into sub-groups including: Bibliography on the Self-Organizing Map (SOM) and Learning Vector Quantization (LVQ) Bibliography for the IEEE Transactions on Neural Networks Bibliography for Advances in Neural Information Processing Systems Bibliography for the journal: Neural Computation Bibliography for the journal: Neural Networks Bibliography on Neurofuzzy Systems 138 Neural Networks Bibliography on constructive algorithms for neural networks Bibliography on Recurrent Neural Networks Bibliography on Invariant Pattern Recognition with Neural Networks Bibliography of Fault Tolerance related Neural Network literature Bibliography on Adaptive Resonance Theory (ART) and much more. 139 [This is a blank page.] CHAPTER 7 Hybrid Systems 7.1 Introduction Fuzzy logic systems and neural networks were inspired by human computational abilities. Thus, they share some common ground. For example, they do not require mathematical models and they work under imprecise, uncertain, and noisy environments. It is also interesting to observe that both are numerical in nature, although this is not a human trait. They also have some complementary characteristics. Fuzzy logic provides inference mechanisms under cognitive uncertainty; it enables computing with words. Fuzzy logic systems require rules set by a human expert; they cannot learn by example or on their own. In contrast, neural networks can learn, adapt, and generalize. In addition, neural networks exhibit fault tolerance due to their distributed structures. Neural methods can be incorporated into fuzzy systems and fuzzy logic methods can be introduced into neural networks to take advantage of their complementary characteristics. Such combinations lead to what is referred to here as hybrid systems, which are also called fuzzy-neural, neural-fuzzy, or fuzzy-neuro systems. Such systems combine the merits of both fuzzy logic systems and neural networks; namely, the explicit knowledge representation from fuzzy logic methods and the learning ability of neural networks. Using this combination of methods, a more versatile system with a human-like approach can be designed. 7.2 Fuzzy Neuron There are numerous possible ways to introduce fuzzy logic methods into a neural network. For example, the model of the neuron discussed in Section 6.1 could be generalized to incorporate fuzziness leading to a fuzzy neuron. It is also possible to introduce fuzziness in a network with non-fuzzy neurons by modifying the learning algorithms to allow for membership functions. In this section the concept of fuzzy neurons is illustrated. 141 Chapter 7 A possible model for a fuzzy neuron, FN, that can express and process fuzzy information was put forward by Kwan and Cai. It is illustrated in Figure 7.1. The FN has inputs xi associated with weights wi (i = 1 to n). It also has outputs yj (j = 1 to m); all of the outputs have values in the interval [0, 1]. These could represent membership values of a given pattern to a particular fuzzy set. The inputs may be thought of as the representation of a linguistic variable and the output expresses the membership values of assigned linguistic descriptions such as TALL, MEDIUM, SMALL, etc. These values could then be propagated to other neurons. x1 x2 w2 w1 g1 g2 y1 y2 h f gm ym Membership Value Determination wn xn Agregation Input pattern Thresholding membership values in fuzzy set j (j = 1 to m) Figure 7.1: A possible model for a fuzzy neuron. In mathematical terms, the FN operation could be described by y j = gi f hin=1 ( wi xi ) where h is an aggregation function (such as MIN, MAX, etc.) that replaces the summation operation in a non-fuzzy neuron model, f is an activation function, is the activation threshold, and {gi , j = 1,2,, m} are m output functions that represent the membership functions of the input pattern in all the m fuzzy sets. ( ) 142 Hybrid Systems If z is the aggregated weighted input hin=1 ( wi xi ) , then one could define: INPUT-FN MAX-FN with which an input layer is composed such that z = x. in which the aggregation function is a MAX operation, leading to n i =1 z = max ( wi xi ) Such a neuron could be referred to also as an OR-FN. MIN-FN in which the aggregation function is a MIN function, leading to n i =1 z = min ( wi xi ) Such a neuron could be referred to as an AND-FN. COMP-FN (competitive FN) in which the activation threshold, , is a variable and there is only one output such that y = g ( s ) = 0, s < = 1, s = t (c1 , c2 ,, cK ) , t being the threshold function, and ck ( k = 1 to K ) are competitive variables of the FN. All these neurons could appear in one fuzzy neural network, FNN, leading to a heterogeneously structured network as opposed to the homogeneous structures, where all the neurons have similar definitions, as discussed in Chapter 6. s = f ( z ) defines the state of the FN, and 7.3 Multilayer FNN Architectures A multilayer computational structure consisting of fuzzy neurons can be described as a multilayer fuzzy neural network. Numerous architectures have been suggested in the literature. The following two examples are provided to illustrate the concept. Example Pedrycz Structures A simplified three-layer fuzzy neural network structure is shown in Figure 7.2. Each layer consists of the same type of FNs, but different types are used for each layer. The input layer consists of INPUT-FNs, the hidden layer of MAX-FNs, and the output layer of a single MIN-FN. Figure 7.3 illustrates an alternative structure where the hidden layer is composed of MIN-FNs, and the output layer is a single MAX-FN. 143 Chapter 7 x1 x2 y xi INPUT-FNs MIN-FNs MAX-FNs Figure 7.2: A simplified three-layer fuzzy neural network with a hidden layer composed of MIN-FNs. x1 x2 y xi INPUT-FNs MAX-FNs MIN-FNs Figure 7.3: A simplified three-layer fuzzy neural network with a hidden layer composed of MAX-FNs. 144 Hybrid Systems Example Kwan-Cai Structure A four-layer feedforward fuzzy neural network, FNN, for 2-D pattern processing is shown in Figure 7.4. The first layer of the network accepts pixel values of an input pattern and transfers them into normalized values in the interval [0, 1]. The second layer fuzzifies the input pattern to an adjustable predetermined degree such that all distinct training patterns can be separated by the network and it produces an acceptable recognition rates. The mth output of this layer can be thought of as expressing a fuzzy concept. The third layer gives similarity of the input pattern to all learned patterns. The fourth layer is a defuzzier stage. It chooses the maximum similarity as the activation threshold of all the COMP-FNs. max x1 x2 xi INPUT-FNs MIN-FNs MAX-FNs COMP-FN Figure 7.4: Kwan-Cai FNN structure. 7.4 Fuzzy ART Adaptive Resonance Theory neural networks, ART, can be built on prior learning as outlined in Chapter 6. They can create new categories to accommodate inputs that do not belong to any of the categories learned before. Fuzzy ART generalizes ART1 to incorporate fuzzy computations. The generalization to learning both analog and binary input patterns is achieved by replacing the classical intersection operator in ART1 calculations with the fuzzy AND operationthat is, the MIN operation. Fuzzy ART still clusters patterns (vectors) by comparing an input pattern to existing ones, then creating a new category if the need arises. The highlights of the model put forward by Carpenter et al. are as follows: 145 Chapter 7 For each M-dimensional input I = ( I1 , I 2 ,, I M ) and cluster category j, a choice function is defined by Three parameters that control the dynamics of the fuzzy ART are defined: the choice parameter, > 0 , the learning rate parameter, [0,1] , and the vigilance parameter, [0,1] . Tj ( I ) = I wj + wj where w j = ( w j1 , w j 2 ,, w jM ) is the weight vector associated with category j. The number of potential categories N is arbitrary. The fuzzy ART weight vector wj represents the bottom-up and top-down weight vectors of ART1, the operator denotes a fuzzy AND (i.e., MIN) operation, and the norm is defined by: x = xi . i =1 M Tj ( I ) is simply written as Tj when I is fixed. The category choice is indexed by J, where: TJ = max Tj , j = 1,2,, N If more than one Tj is maximal, the category j with the smallest index J is chosen so that neurons become committed in order j = 1,2,3,. The matching function is defined by: Sj = I wj I If S j , resonance occurs and the learning process starts. If S j < , mismatch reset occurs and Tj is reset to 1 for the duration of the input presentation. A new index J is chosen, and the search continues until resonance occurs. Learning is achieved by updating the weight vector, wJ. w new = I wold + (1 ) wold . J J J ( ) It was reported to be advantageous to normalize the inputs. This could be achieved by processing each input vector a by setting I = a . An alternative a normalization rule known as complement coding could also be used. If a and ac represent the ON the OFF responses, respectively, then aic 1 ai . The complement coded input I becomes a 2M-dimensional vector I = (a,ac). Since 146 Hybrid Systems I = a, ac ( ) M M = ai + M ai i =1 i =1 =M then inputs that are complement coded are normalized. 7.6 Fuzzy ARTMAP A fuzzy ARTMAP, as described by Carpenter et al., is a system that can learn to classify inputs by a fuzzy set of features, or a pattern of fuzzy membership values expressing the extent to which each feature is present. An illustration of the system is shown in Figure 7.5. The system consists of two fuzzy ART modules linked by an inter-ART module, called a map field, Fab. It is used to form predictive associations between categories and carry out the match tracking rule: the vigilance parameter of ART-a increases in response to predictive mismatch at ART-b. Match tracking recognizes category structure so that predictive errors are not repeated when the input is presented again. Wjab Map Field Fab ART-a F2 a ART-b F2b Match Tracking F1b F0b Desired output, Ib (during training) ARTMAP F1a F0a Input, Ia Figure 7.5: Fuzzy architecture. 147 Chapter 7 The operational highlights of the network are as follows: Fields F0a and F0b complement-code the Ma-dimensional vector a and the Mb-dimensional vector b. For ART-a: Ia = A = (a,ac), and for ART-b: Ib = B = (b,bc). a a For ART-a, the output of the field F1a is denoted by x a x1a , x2 ,, x2 Ma , the output of field F is denoted by y y , y ,, y a 2 a a ) denote the output vector of field F , and y ( y , y ,, y ) , and w ( w , w ,, w ) . For the map field, x ( x , x ,, x ) denote the F output vector, w ( w , w ,, w ) denotes the weight vector from the j F neuron to For ART-b, x x , x ,, x b b 1 b 2 b b 1 b 2 b Nb ab b 2 Mb b a weight vectors are denoted by w j wa1 , wa2 ,, w2 Ma j j ( ( ( a 1 a 2 a 2 Ma ) , and the j ). ( ) th ART-a b 1 k b k1 b k2 b 2 Mb ab j F ab . ab j1 ab j2 ab jNb ab 1 ab 2 ab Nb ab th a 2 The map field is activated whenever one of the ART-a or ART-b categories is active. If both ART-a and ART-b are active, then the field map becomes active only if ART-a predicts the same category as ART-b. At the start of each input presentation, the ART-a vigilance parameter a is set to a baseline value. If xab < ab y b , where ab is the map field vigilance parameter, then a is increased until a A > A wa , where A is the input j in complement form to F2a and J is the index of the active F2a neuron. When this occurs, the ART-a search leads to either the activation of another F2a node or the shutdown of F2a for the rest of the input presentation. Learning rules determine how the map field weights w ab change. jk 7.7 Neural Fuzzy Systems In fuzzy set theory, the universe of discourse needs to be divided into fuzzy subsets. Each subset is defined by a membership function. This process requires the knowledge of a human expert. On the other hand, neural networks perform conceptually similar tasks. They can divide the data space into clusters based on certain features. In a fuzzy system context, clustering multidimensional data could lead to: One-dimensional membership functions based on a given metric, or Multidimensional membership functions that model fuzzy relationsin other words, fuzzy IF/THEN rules. Thus, a neural network could identify and extract membership functions and fuzzy rules without the knowledge of a human expert. One may ask: how this could be useful? 148 Hybrid Systems Recall that a basic idea of fuzzy logic is to map human thinking into a computer algorithm. It is possible that the physical environment under consideration may change over a period of time. Adapting of the fuzzy system through adjusting membership functions and fuzzy rules would be needed to better reflect the changes occurring in the actual system. Neural networks could also perform computationally intensive tasks, such as fuzzy logic inference. Knowledge of the rules (the antecedent and consequent clauses) is encoded in the network. Keller et al. suggested a four-layer network to do that task. The first is an input layer, the second is for checking the antecedent clause, the third is for clause combining, and the fourth layer is the output layer. Various methods and algorithms have been suggested in the literature. The following example illustrates an algorithm put forward by Adeli and Hung for determining the membership function. Example Membership determination using the Adeli-Hung algorithm (AHA). The neural network for this algorithm has a flat topologyi.e., two layers including the input layer. The number of input neurons equals the number of patterns, M, in each training instance. There are N training instances: X1,X2,,XN. The number of output neurons equals the number of clusters, N. The algorithm starts with M inputs and one output. The network can be described in shorthand by NN(M,1); this reads as a neural network with m inputs and one output. The final number of output neurons is yet to be determined. The first training instance gets assigned to the first cluster. Then, for the second instance: If it is classified to the first cluster, the output neuron representing the first cluster becomes active. If it is classified as a new cluster, an additional output neuron is added to the network. The process continues until all training instances are classified. The details of the AHA can be highlighted as follows: The mean vector of the training instance is defined by: XN = 1 N xi . It follows that for the (N + 1) training instance: N i =1 N N XN + X N +1 . N +1 N +1 X N +1 = 149 Chapter 7 A function giving the degree of difference is defined using Euclidean distance as: diff ( X,Ci ) = (x j =1 M j cij ) . 2 It represents the difference between a training instance, X, and a cluster C (mean of the patterns of the instances in the cluster) in an NN(M,P) network (a neural network with M input neurons and P output neurons). The cluster with the smallest degree of difference diffmin ( X , Ci ) is made active. Cactive = C min diff ( X,Ci ), i = 1,2,, p . { } If the clusters are completely disjoint, a binary matrix Z is used to record the cluster of each instance: zij = 1 , if instance i belongs to cluster j, and zij = 0 , if instance i does not belong to cluster j. If the clusters are partly overlapping, the boundaries of the clusters become fuzzy. A binary matrix is still used. The degree of membership of each instance in the cluster is based on the distance between the instance and the prototype of the cluster. The prototype for each cluster is defined as the mean of all instances in that cluster. Assuming np instances in a cluster p, the pattern vector of the instance number i in that cluster is p X p = xip , xip2 ,, xiM , and i 1 Cp ( = (c = p1 , c p2 ,c pM n ) ) 1 p p Xi . n p i =1 1 np where c pi = x i =1 np p ij , and j = 1,2,,M. Assuming triangular membership functions, the membership value of the instance number i in the p cluster is defined by p X p = f D w X ip ,Cp i If ( ) ( ) D ( X ,C ) > , then ( X ) = 1 w p i p p p i w where is a predetermined overlapping threshold value. p p If D (X ,C ) , then (X ) = 1 p i p i Dw Xp ,Cp i ( ), 150 Hybrid Systems Concluding Remarks Fuzzy logic and neural networks were inspired by human computational powers. Both provide solutions that could be difficult to reach otherwise. However, the solutions reached may not be the optimum solutions to the problems. Combining the two techniques in a hybrid fuzzy and neural system enables enhanced performance. The hybrid system exploits the merits of each technique. Still, the solution reached may not be the optimum one. Genetic algorithms, GA (introduced in Appendix A), can be employed for optimization purposes in fuzzy logic systems, neural networks, and hybrid fuzzy-neural systems. 151 Chapter 7 Bibliography 1. S. Abe and M.-S. Lan, Efficient Methods for Fuzzy Rule Extraction from Numerical Data, in Fuzzy Logic and Neural Network Handbook, C. H. Chen Ed., McGraw Hill, New York, 1996, pp.7.17.33. 2. H. Adali and S.-L. Hung, Machine Learning Neural Networks, Genetic Algorithms and Fuzzy Systems, John Wiley & Sons, New York, 1995. 3. C. von Altrock, Fuzzy Logic and Neural Fuzzy Applications Explained, Prentice hall PTR, Englewood Cliffs, NJ, 1995. 4. A. Blanco, M. Delgado and M.C. Pegalajar, Identification of Fuzzy Dynamic Systems Using Max-Min Recurrent Neural Networks, Fuzzy Sets and Systems, 122, 3, 451 467, 2001. 5. P. P. Bonissone, Y.-T. Chen, K. Goebel, and P. S. Khedkar, Hybrid Soft Computing Systems: Industrial and Commercial Applications, Proc. IEEE, 87, 9, 16411667, 1999. 6. F. Bouslama and K. Ichikawa, Application of Neural Networks to Fuzzy Control, Neural Networks, 6, 791799, 1993. 7. J. J. Buckley and T. Feuring, Fuzzy and Neural: Interactions and Applications, Physica-Verlag, Heidelberg, 1999. 8. J. J. Buckley and Y. Hayashi, Fuzzy Neural Networks: A Survey, Fuzzy Sets and Systems, 66, 113, 1994. 9. G. Carpenter and S. Grossberg, Fuzzy ARTMAP: A Synthesis of Neural Networks and Fuzzy Logic for Supervised Categorization and Nonstationary, Prediction, Fuzzy Sets, Neural Networks, and Soft Computing, (R. R. Yager and L. A. Zadeh, eds.) Van Nostrand Reinhold, New York, 1994. 10. G. A. Carpenter, S. Grossberg, and D. B. Rosen, Fuzzy ART: Fast Stable Learning and Categorization of Analog Patterns by Adaptive Resonance System, Neural Networks, 4, 759771, 1991. 11. G. Castellano, A.M. Fanelli, and C. Mencar. A Neuro-Fuzzy Network to Generate Human-understandable Knowledge from Data, Cognitive Systems Research Journal, 3, 2, 125144, 2002. 12. G. Castellano, A.M. Fanelli and T. Roselli. Fuzzy Rule Extraction by a KBN Approach, First International ICSC Conference on Neuro-Fuzzy Technologies, NF 2002, Havana, Cuba, January 16 19, 2002. 13. L. Caponetti, G. Castellano, and A.M. Fanelli, A Neuro-Fuzzy System for Document Image Segmentation and Region Classification, The 2nd IEEE International Workshop on Intelligent Signal Processing, WISP2001, Budapest, Hungary, May 2001. 152 Hybrid Systems 14. C. H. Chen, Fuzzy Logic and Neural Network Handbook, IEEE Press, New York, 1996. 15. J.-L. Chen and J.-Y. Chang, Fuzzy Perceptron Neural Network for Classifiers with Numerical Data and Linguistic Rules as Inputs, IEEE Trans. Fuzzy Systems, 8, 6, 730745, 2000. 16. C.-L. Chen and W.-C. Chen, Fuzzy Controller Design by Using Neural Network Techniques, IEEE Trans. Fuzzy Systems, 2, 235244, 1994. 17. M. Chiaberge and L. M. Reyneri, Cintia: A Neuro-Fuzzy Real-Time Controller for Low-Power Embedded Systems, IEEE Micro, 6, 4047, 1995. 18. F.-L. Chung and J.-C. Duan, On Multistage Fuzzy Neural Network Modeling, IEEE Trans. Fuzzy Systems, 8, 2, 125142, 2000. 19. F. L. Chung and T. Lee, Fuzzy Competitive Learning, Neural Networks, 3, 539551, 1994. 20. R. K. De, J. Basak, and S. K. Pal, Unsupervised Feature Extraction Using Neuro-Fuzzy Approach, Fuzzy Sets and Systems, 126, 3, 277291, 2002. 21. Y. Diano and K. M. Passino, Adaptive Neural/Fuzzy Control for Interpolated Nonlinear Systems, IEEE Trans. Fuzzy Systems, 10, 5, 583595, 2002. 22. J.-C. Duan and F.-L. Chung, Cascaded Fuzzy Neural Network Model Based on Syllogistic Fuzzy Reasoning, IEEE Trans. Fuzzy Systems, 9, 2, 293306, 2001. 23. M. J. Er and S. Wu, A Fast Learning Algorithm for Parsimonious Fuzzy Neural Systems, Fuzzy Sets and Systems, 126, 3, 337351, 2002. 24. M. French and E. Rogers, Input/Output Stability Theory for Direct NeuroFuzzy Controllers, IEEE Trans. Fuzzy Systems, 6, 3, 331345, 1998. 25. C. W. Frey and H.-B. Kuntze, A Neuro-Fuzzy Supervisory Control System for Industrial Batch Processes, IEEE Trans. Fuzzy Systems, 9, 4, 570577, 2001. 26. H. P. Graf and L. M. Reyneri, The Expanding World of Neural and Fuzzy Systems, IEEE Micro, 6, 1011, 1995. 27. M. M. Gupta and D. H. Rao, On the Principles of Fuzzy Neural Networks, Fuzzy Sets and Systems, 61, 118, 1994. 28. K. Hirota and W. Pedrycz, Neurocomputing with Fuzzy Flip-Flops, Proceedings of the International Joint Conference on Neural Network, 2, 18671870, 1993. 29. Q. Hu and D. B. Hertz, Fuzzy Logic Controlled Neural Learning, Information Sciences Applications, 2, 1, 1533, 1994. 153 Chapter 7 30. H. Isshibuchi, R. Fujioka, and H. Tanaka, Neural Networks that Learn from Fuzzy IF-Then Rules, IEEE Transactions on Fuzzy Systems, 1, 2, 8597, 1993. 31. J.-S. R. Jang, C.-T. Sun, and E. Mizutani, Neuro-Fuzzy and Soft Computing, Prentice Hall, Uppwer Saddle River, NJ, 1997. 32. C.-F. Juang and C.-T. Lin, An On-Line Self-Constructing Neural Fuzzy Inference Network and Its Applications, IEEE Trans. Fuzzy Systems, 6,1, 1232, 1998. 33. S. V. Kartalopoulos, Understanding Neural Networks and Fuzzy Logic, IEEE Press, New York, 1996. 34. N. K. Kasabov, Foundation of Neural Networks, Fuzzy Systems, and Knowledge Engineering, The MIT Press, Cambridge, 1996. 35. T. Kasuba, Simplified Fuzzy ARTMAP, AI Expert, 11, 1855, 1993. 36. J. M. Keller and H. Tahani, Implementation of Conjunctive and Disjunctive Fuzzy Logic Rules with Neural Networks, International Journal of Approximate Reasoning, 6, 221240, 1992. 37. B. Kosko, Neural Networks and Fuzzy Systems, Prentice Hall, Englewood Cliffs, NJ, 1992. 38. S.-Y. Kung, J. Yaur, and S.-H. Lin, Synergistic Modeling and Applications of Hierarchical Fuzzy Neural Networks, Proc. IEEE, 87, 9, 15501550 1574, 1999. 39. H. K. Kwan and Y. Cai, Fuzzy Neural Network and Its Application to Pattern Recognition, IEEE Trans. on Fuzzy Systems, 2, 3, 185193, 1994. 40. R. P. Landim, B. Rodrigues, S. R. Silva, and W. M. Matos, A Neo-FuzzyNeuron with Real Time Training Applied to Flux Observer for an Induction Motor, Proceedings of the 5th IEEE Brazilian Symposium on Neural Networks, Belo Horizonte, MG, Brazil, December 911, 1998, pp.16. 41. R. P. Landim, B. Rodrigues, S. R. Silva, and W. M. Matos, On-Line NeoFuzzy-Neuron State Observer, Proceedings of the 6th IEEE Brazilian Symposium on Neural Networks, Rio de Janeiro, Brazil, January 2225, 2000, pp 196201. 42. S. C. Lee and E. T. Lee, Fuzzy Neural Networks, Mathematical Biosciences, 23, 151177, 1975. 43. R.-P. Li, M. Mukaidono and I. B. Turksen, A Fuzzy Neural Network for Pattern Classification and Feature Selection, Fuzzy Sets and Systems, 130, 1, 101108, 2002. 154 Hybrid Systems 44. C.-T. Lin, F.-B. Duh, and D.-J. Liu, A Neural Fuzzy Network for Word Information Processing, Fuzzy Sets and Systems, 127, 1, 3748, 2002. 45. C.-T. Lin and C. S. G. Lee, Neural Fuzzy Systems, Prentice Hall PTR, New York, 1996. 46. C.-J. Lin and C.-T. Lin, An ART-Based Fuzzy Adaptive Learning Control Network, IEEE Trans. Fuzzy Systems, 5, 4, 477496, 1997. 47. F.-J. Lin, C.-H. Lin, and P.-H. Shen, Self-Constructing Fuzzy Neural Network Speed Controller for Permanent-Magnet Synchronous Motor Drive, IEEE Trans. Fuzzy Systems, 9, 5, 751759, 2001. 48. Z.-Q. Liu and F. Yan, Fuzzy Neural Network in Case-Based Diagnostic System, IEEE Trans. Fuzzy Systems, 5, 2, 209222, 1997. 49. G.N. Marichal L. Acosta, L. Moreno, J.A. Mndez, J.J. Rodrigo, and M. Sigut, Obstacle Avoidance for a Mobile Robot: A Neuro-Fuzzy Approach, Fuzzy Sets and Systems, 124, 2, 171179, 2001. 50. J. Nie and D. Linkens, Fuzzy-Neural Control: Principles, algorithms and applications, Prentice Hall, New York, 1995. 51. S. K. Pal and S. Mitra, Multilayer Perceptron, Fuzzy Sets, and Classifications, IEEE Trans. on Neural Networks, 3, 683697, 1992. 52. W. Pedrycz, Fuzzy Control and Fuzzy Systems, John Wiley & Sons, New York, 1993. 53. V. B. Rao and H. V. Rao, C++ Neural Networks & Fuzzy Logic, MIS Press, New York, 1995. 54. P. K. Simpson, Fuzzy Min-Max Neural Networks - Part 1: Classification, IEEE Trans. on Neural Networks, 3, 776786, 1992. 55. P. K. Simpson, Fuzzy Min-Max Neural Networks - Part 2: Clustering, IEEE Trans. on Neural Networks, 1, 3245, 1993. 56. H. Takagi and I. Hayashi, NN-Driven Fuzzy Reasoning, International Journal of Approximate Reasoning, 3,5, 191212, 1991. 57. L. H. Tsoukalas and R. E. Uhrig, Fuzzy and Neural Approaches in Engineering, John Wiley & Sons, New York, 1997. 155 Chapter 7 Web Resources 1. First IF/THEN for neural systems faculty.washington.edu/chudler/papy.html www.eoa.org.eg/edwintxt.htm#TOP These two sites provide a description of an ancient Egyptian papyrus that contains a surgery database in the form of an if/then rule. The papyrus has the first use of the neuro words recorded in history. 2. Neural Fuzzy Systems citeseer.nj.nec.com/64350.html A 1995, 250-page report by Robert Fullr on Neural Fuzzy Systems. The report consists of three chapters and an appendix. The first chapter discusses fuzzy systems, the second highlights basic concepts of neural networks, and the third introduces some aspects of fuzzy neural networks. The appendix provides a case study and exercise. Also, www.abo.fi/~rfuller/nfs.html Lecture notes on neural fuzzy systems in both Web and PDF formats by the same author. 3. Neural Networks and Fuzzy Systems http://nn.uidaho.edu/pap/2002\Mechatronics%20Handbook.pdf Chapter 32 authored by Bogdan Wilamowski in Mechatronics Handbook edited by Robert R. Bishop, CRC Press, pp. 331 to 3226, 2002. 4. Evolving Fuzzy Neural Networks divcom.otago.ac.nz/infosci/kel/CBIIS/pubs/pdf/ajiips-si98.pdf A paper that appeared in the Australian Journal of Intelligent Information Processing Systems, 5,154-160,1998 by Nikola Kasabov, University of Otage, New Zealand. It is a 6-page paper with 29 references available in PDF format. The complete title is Evolving Fuzzy Neural Networks: Theory and Applications for On-line Adaptive Prediction, Decision Making in Control. citeseer.nj.nec.com/kasabov01evolving.html A 67-page paper with 84 references by the same author. The full title is Evolving Fuzzy Neural Networks for Supervised/unsupervised On-line Knowledge-Based Learning. It appeared in IEEE Transactions of Systems Man and Cybertics, 131, 6, 2001. 156 Hybrid Systems 5. Adaptive Neuro-Fuzzy Inference System, ANFIS www.control.hut.fi/Kurssit/AS-74.115/Material/FVAnfis2.pdf A 25-page tutorial by Heikki Koivo on the Adaptive Neuro-Fuzzy Inference System. It has MATLAB examples, copyrighted 2000. Also, lecture notes on Neuro Fuzzy Computing in Automation by Prof. Koivo, 2001, available at www.control.hut.fi/Kurssit/AS-74.115/Material/ 6. Neural Fuzzy Motion Estimation and Compensation sipi.usc.edu/~kosko/MotionEstimation.pdf A paper by Hyum Kim and Bart Kosko, IEEE Transactions on Signal Processing, 45, 10, 2515-2532, 1997. Provided in PDF format. The paper discusses the use of neural-fuzzy systems to improve motion estimation and compensation for video compression, and it has 29 references. 7. Soft Negotiation www.iis.ee.ic.ac.uk/~frank/surp98/report/mgm1/ A report in Web format by Myles MacRae and Marcus Pickering, the Department of Computing, Imperial College of Science, Technology and Medicine, University of London. The report covers several topics including a simple, short account of soft computing. The account addresses neural networks, fuzzy logic, genetic algorithms and their combinations. 8. Neural Fuzzy Techniques scholar.lib.vt.edu/theses/available/etd-5733142539751141/unrestricted/ ETD.PDF A 187-page Ph.D. dissertation by Somkiat Sampan, Virginia Polytechnical Institute and State University, 1997. It is provided in PDF format. The dissertation discusses neural fuzzy techniques in vehicle acoustic signal classification. 9. Neural Fuzzy Inference Network www.di.uniba.it/~castella/papers/IJCNN2000.pdf A paper by Castellano and Fanelli, Universit degli Studi di Bari, Italy. It presents a self-organizing neural fuzzy inference network. It is 6 pages long, with 13 references. It is available in PDF format. 10. Fuzzy ARTMAP vs. MLP www.cairo.utm.my/publications/yhtay_airtc97.pdf A paper available in PDF format: Tay and Kalid, Comparison of fuzzy ARTMAP and MLP Neural Networks for Handwritten Character Recognition, IFAC Symposium on AI in Real-Time Control, Kuala Lumpur, Malaysia, 1997. 157 Chapter 7 The paper presents results of comparisons between Fuzzy ARTMAP and backpropagation based multilayer perceptrons, MLP. Fuzzy ARTMAP out performed MLP in both learning convergence and recognition accuracy. 11. Fuzzy ARTMAP Applications www.cairo.utm.my/publications/ksyap_wec99.pdf Slah et al., Vehicle Licence Plate Recognition by Fuzzy ARTMAP Neural Network, World Engineering Congress, WEC99, Universiti Putra Malaysia, 1999. It is a six page report with seven references available in PDF format. It reports on the prototype of a system being developed at the Center for Artifical Intelligence and Robotics in Cairo and the Universiti Teknologi Malaysia. 12. Simplified Fuzzy ARTMAP Applications medusa.sdsu.edu/Robotics/ Neuromuscular%20Control/Fuzzy_ARTMAP.pdf A paper by Sajasekarn and Pai, PSG College of Technology titled: Image Recognition using Simplified Fuzzy ARTMAP, 15 pages, 12 references. 13. Combining Neural Networks and Fuzzy Controllers citeseer.nj.nec.com/nauck93combining.html A paper by Nauck, Klawonn, and Kruse that appeared in Fuzzy Logic in Artificial Intelligence, FLAI93; it has 13 pages and 21 references, and is available in PDF format. 14. Modified Fuzzy Neural Network Classifier www.ee.iitb.ac.in/uma/~ncc2002/proc/NCC-2002/pdf/n010.pdf This paper discusses neural networks based on modifying the Kwan-Cai Networks. Authored by Kularni and Sontakke, SGGS College of Engineering and Technology, India. 15. Comparative Study of NN Structures mecha.ee.boun.edu.tr/~efe/PDF/JMechatronics.pdf Efe and Kayank of Bogazci Univesity, Turkey present a 16-page report that presents the results of investigating the identification of nonlinear systems by neural networks. Feedforward Neural Networks, Radial Basis Function Neural Networks, Rung-Kutta Neural Networks, and Adaptive Neuro Fuzzy Inference Systems are evaluated and their performance compared. 158 Hybrid Systems 16. Soft Computing Resources www.cs.nthu.edu.tw/~jang/nfsc.htm A web site maintained by Prof. Jang, Computer Science Department, Tsing Hua University, Taiwan. It provides Web links to numerous resources including FTP sites, newsgroups, neurofuzzy research sites, journals, technical reports, and more. 17. Computational Intelligence Links www.emsl.pnl.gov:2080/proj/neuron/ci/what.html This Web sites provides a large number of Web links to mainly introductory information on Computational Intelligence (Neural, Fuzzy, and Genetic). It is maintained by Pacific Northwest National Laboratory, operated by Battelle, an industry located in Ohio, USA. 18. Neuro Fuzzy Resources elve.le.ttu.ee/mesel_www_home/R&D/NEUROFUZ/Ressourc.htm A large collection of Web links to resources including tutorials, courses, journals, commercial companies, and much more. It is maintained by Martin Brown, Department of Electronics, Tallinn Technical University, Estonia. 19. Neuro Fuzzy Systems: State-of-the-art Modeling Techniques citeseer.nj.nec.com/abraham01neuro.html A document from Lecture Notes in Computer Science by Ajith Abraham of Monash University, Australia. It is an eight page paper with eleven references that suggest a modeling approach for neuro-fuzzy systems. 159 [This is a blank page.] CHAPTER 8 Hardware Implementations 8.1 Introduction Embedded systems and their realizations were discussed in Chapter 1. The general features, design metrics, processor technologies, and the IC technologies which were discussed in general terms for embedded systems are applicable to embedded fuzzy and neural systems. Specifically, the approaches that have been used to implement embedded fuzzy logic systems and neural networks include: Using general-purpose processors and depending fully on software in the realization of the system. As discussed earlier, such an approach is relatively easy and low-cost, but also leads to relatively slow operation. Adapting a general-purpose processor to perform dedicated fuzzy instructions. The approach is a trade-off between speed and generality. Using a coprocessor or exclusive hardware to perform the fuzzy operations is a closely related approach, and the speed is limited by the interfacing between the processor and the co-processor. Using dedicated fuzzy circuits or Application Specific Integrated Circuits, (ASICs). The approach leads to relatively high-speed operation, but is more costly. In Chapter 9, we will discuss software tools, including those for the implementation of fuzzy and neural systems on general-purpose processors. Also discussed are software tools that enable the conversion of a given intuitively designed system to code in a selected language such as C, C++, Java, etc., or to an assembly language of a particular general-purpose, or dedicated, processor. The following sections discuss the hardware aspect of the embedded-system implementation. The techniques used for hardware implementation include: Digital techniques Continuous-time analog techniques including: Current mode circuits Voltage mode circuits Mixed mode (current and voltage) circuits 161 Chapter 8 Discrete-time analog techniques Mixed signal (analog and digital) techniques The discussion is meant to give the reader an overview of the basic ideas of hardware implementation using digital and analog techniques. Since new ICs are proposed, designed, and produced at a rapid pace, the reader is advised to refer to the relevant parts of the Web resources section if specific up-to-date information is needed. 8.2 Digital Techniques Although fuzzy chips may have limited input/output capabilities, they have particularly useful applications in real-time control systems. Analog fuzzy values must be converted to binary digital signals. Analog-to-digital conversion can lead to quantization errors in both input signals and membership values. Thus, a deterioration in the fuzzy processing may occur if an insufficient number of bits is used to represent the analog signals. On the other hand, using a large number of bits can slow down the process. This is the trade-off between precision and speed. Fuzzy dedicated circuits are characterized by: The number of inputs and outputs. The number and shapes of membership functions. Inference techniques including operators, consequences, and size of the premises. Defuzzification method. The number of fuzzy logic inferences per second, FLIPS. Physical size. Power consumption. Software available to support the design. Numerous approaches to digital fuzzy ICs have been reported. Example First Digital Fuzzy IC Togai et al. proposed the first fuzzy logic digital integrated circuit implementation in the mid-1980s. In the first prototype, they emphasized simplicity, making it particularly useful as a starting point in understanding custom digital IC implementation of fuzzy logic inference. The implementation is based on the analysis summarized here. 162 Hardware Implementations A fuzzy rule is defined by a relation between observation (or antecedent) and action (or conclusion). For a given set of fuzzy rules, the action is inferred from observation and a fuzzy relation. Now, suppose A1,A2,,AN, are fuzzy subsets of a universe of discourse U, and B1,B2,,BN are fuzzy subsets of the universe of discourse V. A fuzzy relation is defined by rules such as IF IF A1 THEN B1, A2 THEN B2, . . . IF AN THEN BN. All the rules are combined to give an overall fuzzy relation R. The fuzzy relation Ri is constructed from Rule i and linguistic values Ai and Bi. Consider the ith rule of a set of N rules. Given an observation A and a rule Ri, the action Bi is inferred by: Bi = A Ri ; A U , Bi V , and Ri U V Bi ( v ) = max min Ri (u ), Ai (u, v ) = min i , Bi ( v ) ( = min max min A (u ), Ai (u ) , Bi ( v ) ( ( ) where i = max min A (u ), Ai (u ) ( ) ) ) Then, the maximum of B1,B2,,BN determines the overall resulting decision (action) B , i.e. B = Bi . i The inference mechanism analyzed lends itself to VLSI realization by a logical architecture with two-level hierarchy using min and max operations. Ordinary OR and AND gates were used and a custom CMOS technology was employed. The prototype implemented 16 rules with one antecedent and one consequent. It represented each fuzzy label by 31 elements with 16 possible values of membership. The maximum inference speed was 80,000 FLIPS. 163 Chapter 8 8.3 Analog Techniques The variables in fuzzy and neural systems are analog by nature. Thus, analog implementation eliminates the need for analog-to-digital and digital-to-analog conversions. The systems also require massive parallelism, making analog circuits particularly suited for their implementation. Further, the physical characteristics of transistors can be utilized in realizing the nonlinear functions required, whether it is a fuzzy operation, a membership function, or a neuron activation function. Analog implementations, however, have typically very restricted possibilities for programmability. Analog implementation techniques include voltage mode and current mode realizations, in addition to mixed mode (current and voltage) realizations. 8.3.1 Voltage mode VLSI voltage-mode implementation has the merit of the simplicity of signal voltage distribution in various parts of the circuit. However, the signals in the circuit are sensitive to changes in supply voltages. Further, charge delays of various parasitic capacitances limit the speed. Numerous circuits and architectures have been suggested and implemented; some are listed in the bibliography and the Web resources section. The following example illustrates the concept of repressing fuzzy operations using hardware by giving simple circuit examples of fuzzy INVERTER, AND, and OR operations. Example Fuzzy INVERTER The standard fuzzy complement was defined in Chapter 2 by: A (x ) = 1 A (x ) This equation can be translated into a circuit with an input-output reaction given by: Vo = Vref Vi where Vref and Vi are a reference and input voltage, respectively. The input-output relation can be expressed normalized, dividing by Vref, as: Vo = 1 Vi Thus, A ( x ) is represented by Vo and A ( x ) is represented by Vi. The circuit shown in Figure 8.1 can achieve the required input-output relation. 164 Hardware Implementations R1 Vi Vref R1 _ + R2 V0 R2 Figure 8.1: A circuit to illustrate fuzzy implementation. INVERTER hardware In the circuit of Figure 8.1, we can write: R2 R1 + R2 R2 Vo = Vref Vi R1 + R2 R1 R1 This leads to: R Vo = 2 (Vref Vi ) R1 Then, with Vref = 1, we obtain: Vo = K (1 Vi ) where K is a constant. Fuzzy OR The standard fuzzy OR operation was also defined in Chapter 1 by: A B ( x ) = max A ( x ), B ( x ) This function can be realized by a circuit that has an output voltage, Vo, equal to the value of the largest of the two input voltages. The circuit shown in Figure 8.2 is a generalization from a simple realization of a Boolean OR. 165 Chapter 8 D1 VA V0 D2 VB RL Figure 8.2: A circuit to illustrate fuzzy OR hardware implementation, where D1 and D2 represent precision rectifier circuits. In the circuit of Figure 8.2, the output voltage, Vo, is given by: Vo = VA , if VA > VB Vo = VA , if VB > VA In Boolean logic realization, the inputs and output may assume one of two possible valuese.g., 0V and 5V. Diodes can be used without considering the voltage needed to turn on the diode, VD, since it is about 0.7V. In fuzzy logic, the inputs and outputs may assume any value in the range 0V to 5V, or normalized range 0 to 1. A diode with VD = 0 (or close to zero) would be needed. This is the reason for designating D1 and D2 as precision rectifiers in Figure 8.2. Fuzzy AND An expression for a fuzzy AND operation can be expressed using OR and INVERTER operations through the applications of DeMorgans theorem as follows: A B = ( A + B ) INVERTER Thus, a fuzzy AND can be implemented using inverting precision rectifier and circuits. Prez and Bauelos presented the details and experimental results of implementing basic fuzzy logic gates along similar lines to the above discussion (see Electronic Models of Fuzzy Gates in the Web Resources section at the end of this chapter). 166 Hardware Implementations 8.3.2 Current mode This technique is becoming more and more popular in VLSI implementations because of its numerous advantages. Current mode circuits do not require resistors in their implementations. This is advantageous because integrated resistors are typically inaccurate and involve a significant amount of parasitic capacitance. Furthermore, numerous operations, including summation and subtraction, can be achieved easily using current mode circuits. In addition, these circuits exhibit low sensitivity to supply voltage variations and have lower power consumption compared to the voltage mode circuits. In the following discussion, some basic concepts and building blocks are presented. These should help visuale how fuzzy circuits can be implemented in current mode. They should also help the reader to follow and appreciate the numerous implementations cited at the end of the chapter. Current mirrors Current mirror circuits are used to distribute the signal current to various parts of the circuit. They produce a copy, or copies, of a given current signal. They are used in the design of current sources for biasing as well as operating as active loads. Numerous current mirror circuit configurations that been suggested and implemented in both MOS and bipolar technologies. Example Bipolar Junction Transistor, BJT, Current Mirror Circuit The circuit shown in Figure 8.3 acts as a current mirror. It copies an input, or reference current, Iref, to an output current, Io. It uses two matching transistors Q1 and Q2 which implies that they have the same vBE. Transistor Q1 is connected as a diode by short-circuiting its collector to its base. In the circuit shown: Io = I E , and +1 Iref = +2 IE +1 leading to: Io = 1 Iref = I 2 ref +2 1+ 167 Chapter 8 Then, for >> 1, I o = Iref Iref I +1 E 2IE +1 Io = I +1 E Q1 IE I +1 E I +1 E Q2 IE vEE Figure 8.3: A basic BJT current mirror circuit. The circuit discussed in this example can be generalized to become an N-output current mirror as shown in Figure 8.4. Iref Io1 Io2 IoN Q1 Q2 Q3 QN vEE Figure 8.4: An N-output current mirror circuit. 168 Hardware Implementations Example A Basic MOSFET Current Mirror Circuit The basic MOSFET current mirror consists of two enhancement MOSFETs as shown in Figure 8.5. For Q1, the input (reference) current is given by: Iref = K1 (VGS Vth ) Io = K2 (VGS Vth ) 2 2 The output current is given by: where K1 and K2 are constants that depend on the geometry of the devices (the ratio of the width of the channel to its length), and Vth is the threshold voltage. The above equations lead to: Io = KIref , where K = K1 K2 Iref Io Q1 + VGS Q2 Figure 8.5: A basic MOSFET current mirror circuit. 169 Chapter 8 Example Improved MOSFET Current Mirror Circuit An ideal current source has an infinite output resistance and the output current is independent of the output voltage. Thus, increasing the output resistance of a current mirror improves its performance. This could be achieved using the so-called cascode mirror (stacked mirror) shown in Figure 8.6. Iref Io Q4 Q3 Q1 Q2 VSS Figure 8.6: Cascode current mirror. The output resistance of the basic current mirror is ro2 which is the output resistance of transistor Q2. The output resistance of the cascode mirror can be calculated using the equivalent circuit shown in Figure 8.7. 170 Hardware Implementations ix Q3 ro2 + vx Q2 Ro = vx ix ix + G3 gm vgs ro3 vgs S3 ro2 + vx Figure 8.7: Calculating the output resistance of the cascode current mirror. 171 Chapter 8 The output resistance is given by vx = ro3 + ro2 + gmro3ro2 ix ( gm 3ro 3 ) ro 2 Thus, the output resistance in the cascode configuration is higher by a factor of (gmro3 ) over that of the basic current mirror. Example The Wilson Current Mirror Circuit Another current mirror circuit configuration, referred to as the Wilson current mirror, is shown in Figure 8.8. This configuration leads to increased output resistance as in the cascode configuration. However, the drain voltages of the two transistors are not equal, and hence their currents are not equal either. The problem is overcome by the so-called Modified-Wilson circuit, where a diodeconnected transistor Q4 as shown in Figure 8.9. Iref Io Q3 Q1 Q2 VSS Figure 8.8: Wilson current mirror. 172 Hardware Implementations Iref Io Q4 Q3 Q1 Q2 VSS Figure 8.9: Modified-Wilson current mirror. Example BiCMOS Double Cascode Current Mirror The current mirror circuit shown in Figure 8.10 is referred to as a double cascode BiCMOS. It uses both bipolar and MOSFET transistors. The output resistance of the circuit is given by Ro = ( gm 3ro 3 )(2ro 2 ) , indicating that this current mirror has extremely high output resistance. 173 Chapter 8 Iref Io Q6 Q3 Q5 Q2 Q4 Q1 Figure 8.10: The circuit of a BiCMOS double cascode current mirror. Fuzzy Operators I I ,I I IZ = A B A B 0, I A < I B Numerous bipolar and MOSFET-based circuits have been devised to implement various fuzzy operations. The following are examples of circuit realizations of some basic fuzzy logic functions. Bounded Difference The bounded difference operation may be defined by: Z = A B , A B B = A 0, A < B 174 Hardware Implementations A bounded difference circuit suggested by Yamakawa is shown in Figure 8.11. The circuit consists of a current mirror and a diode. The currents IA, IB, and IZ represent the membership values A , B and Z , respectively. The current IB is applied to the current mirror, and the current IB is wire-subtracted. Thus, the current at the anode of the anode of the diode is (IA + IB). The output current , IZ, will be: IA IAIB IB >>1 IZ IB Figure 8.11: Yamakawa bipolar realization of a bounded difference operation. The bounded difference operation can also be realized using MOSFETs as shown in Figure 8.12. IA IB IZ Figure 8.12: A MOSFET realization of the bounded difference operation. If the circuit is cascaded by a current mirror, the diode-connected transistor can be omitted. The input of the current mirror of the next stage achieve the task of the diode. 175 Chapter 8 Fuzzy Inverter Circuit A basic fuzzy complement operation is defined by Z = A = 1 A The circuit shown in Figure 8.13 shows a possible realization of that operation. 1 IZ IA IA Figure 8.13: A possible bipolar realization of a fuzzy inverter; the currents represent the membership values. Fuzzy AND (MIN circuit) The fuzzy AND operation can be defined as , B Z = A B = B A A , A < B = B ( B A ) = A ( A B ) As before, the membership values are represented by current values and hence the above description can be realized by cascading two boundeddifference circuits as shown in Figure 8.14. The diode of the first stage is omitted as mentioned earlier, since the input characteristics of the following stage has rectification characteristics. 176 Hardware Implementations IB IB IB IB IZ IA IA IB IA Figure 8.14: A bipolar realization of a fuzzy AND. Fuzzy Logic OR (MAX circuit) The fuzzy logic OR may be expressed as: , B Z = A B = A A B , A < B = ( A B ) + B = ( B A ) + A This description leads to the circuit realization shown in Figure 8.15. IB IB IB IZ IB IA Figure 8.15: A bipolar realization of a fuzzy OR. 177 Chapter 8 8.3.3 Mixed Mode Both current-mode and voltage-mode circuits can be implemented on the same chip in a hybrid configuration. Such a configuration takes advantage of the merits of the two circuit types. A linear and accurate conversion between the two modes is essential in such implementations. One possible approach is to use Operational Transconductance Amplifiers, OTAs. A differential input OTA is shown in Figure 8.16 along with its circuit implementation. The output current, Io, is related to the input voltages by: Io = gm (V1 V2 ) where gm is a gain parameter that represents the transfer conductance of the amplifier (referred to as transconductance). An ideal transconductance amplifier has an infinite input resistance and an infinite output resistance. a) vA + gm IO vB _ Iss b) IO vA + Iss vB 178 Hardware Implementations c) vB vA IO Iss Figure 8.16: An OTA and some possible implementations. a) Circuit Symbol. b) Circuit of a simple differential input OTA. c) Circuit of a balanced OTA. An OTA-based bounded difference circuit is shown in Figure 8.17. The first OTA has an output current of gm (VA VB ) which realizes the bounded difference for A B , with the voltages VA and VB representing the membership functions A and B . The unidirectional characteristic of the diode enforces the condition A < B as well. The second OTA converts the current at its input into a proportional output voltage, leading to a voltage-mode bounded difference. VA VB + gm1 _ + gm2 _ VO Iss1 Vref Iss2 Figure 8.17: OTA-based bounded difference. 179 Chapter 8 Once a bounded difference operation is realized, all fuzzy logic functions can be implemented using the bounded-difference circuit as a building block. Examples of fuzzy AND, fuzzy OR, and fuzzy NOT circuit realizations are shown in Figures 8.18, 8.19, and 8.20. VA + gm1 _ + gm2 _ VO Iss1 VB Vref3 + gm2 _ Iss3 Vref2 Iss2 Figure 8.18: An OTA-based fuzzy AND circuit. VA + gm1 _ + gm3 _ VO Iss1 VB Vref3 + gm2 _ Iss3 Vref2 Iss2 OR Figure 8.19: An OTA-based fuzzy circuit. 180 Hardware Implementations 1 + + _ vA VO Iss1 Vref2 _ Iss2 circuit. Figure 8.20: An OTA-based fuzzy INVERTER 8.4 Fuzzy Analog Memory One difficult aspect of analog circuit implementations is devising a reliable analog memory module. There is no long-term accurate way to store analog fuzzy values. Programmability and multistage sequential processing are thus relatively difficult tasks in analog fuzzy processors. Hirota et al. have carried out extensive studies presenting the concept of fuzzy logic flip-flops, then implementing them along with fuzzy registers. Example Hirota et al. extended the binary J-K flip-flop to a fuzzy flip-flop, the operation of which is described by: Q ( t + 1) = {J (1 K )} ( J Q ) {(1 K ) (1 K )} Based on the above description, a fuzzy flip-flop was implemented using fuzzy gates (min, max, and inverter circuits). Example Hirota et al. explained that by using a fuzzy flip-flop circuit, one element of fuzzy information can be stored. Thus, a membership function can be stored using N fuzzy flip-flops and N fuzzy AND gates as illustrated in Figure 8.21. In that circuit, the membership value A ( x ) is defined for x1,x2,,xN. The operations of the control inputs are summarized as follows: State Control Input Reset Shift Reset 1 0 Write 0 0 Shift irrelevant 1 181 Chapter 8 A(x1) FF1 A(x2) FF2 A(xn) FFn SHIFT RESET Figure 8.21: Storing membership functions using fuzzy flip-flops. 8.5 Neurons The discussion in the previous sections focused on the implementation of fuzzy logic functions. Neural networks follow a similar path in their hardware implementations. Digital and analog implementations including voltage and current mode are employed. A hardware implementation of a neural network would be ideal if it works for any training algorithm, provides global interconnection with a large number of neurons, and has programmable weights, in addition to modular structure, low power consumption, and low price. In this section only the concept of implementation of the neuron model using operational amplifier based circuits is presented as an example of analog implementation. Figure 8.22 shows a zero-bias configuration of the basic neuron model. The output, Z, is given by: Z = xi ( R f Gi ) i =1 n where Gi = 1 , Ri x1,x2,,xn are inputs, with the associated weights wi given by wi = RfGi. 182 Hardware Implementations A bias can be introduced by connecting the non-inverting input terminal of the op-amp to a bias voltage, or fixing one of the input voltages to a constant value. Positive and negative weights can be implemented as shown in the circuit of Figure 8.23. Rf x1 x2 xn R1 R2 _ + Z Rn Figure 8.22: An op-amp based implementation of the basic neuron model. x1 x2 xn y1 y2 ym R1 R2 Rn r1 r2 rm Rf _ + Z r0 Figure 8.23: A circuit to implement both positive and negative weights. Although the circuits of Figures 8.22 and 8.23 illustrate the basic concept of neuron implementation, they are of limited practical significance. Such neurons cannot learn, because the weights are fixed. Further, the modification possibilities of the transfer function characteristics are limited. Also, the circuit size and its power consumption are relatively large. 183 Chapter 8 MOSFETs can be used as voltage-controlled resistors to achieve weight controllability as illustrated in Figure 8.24. The resistance of the channel, Rds, between the drain and the source of each transistor is controlled by the voltage between the gate and the source, Vgs. x1 + Rf vgs1 x _ + x2 vgs2 Z + xn + vgsn Figure 8.24: Neuron model implementation with voltage-controlled weights. Weights can be stored using a capacitive circuit element. The elementary circuit shown in Figure 8.25 illustrates the concept. A weight could be stored as a voltage across the gate capacitance, since no current is needed to drive the gate. A capacitance could be added to the circuit to enhance the storage capability of the intrinsic gate capacitance. The weight control voltage, vc, is sampled through transistor QS/H which provides the action of an on/off switch. The resistance of transistor Q1 is controlled by the voltage Vgs1, as indicated earlier. The weights could, of course, be stored digitally, in which case an analog-to-digital conversion would be required. More elaborate implementations have been put forward by numerous researchers; for example see the extensive work reported by Wilamowski et al. included in the Web resources. 184 Hardware Implementations x1 Q1 x QS/H + + vc v + gs2 vg C vgs1 Figure 8.25: Elementary analog weight storage. Concluding Remarks The increasing interest in embedded systems applications of fuzzy logic has led to a growing interest in VLSI implementations of fuzzy logic functions, controllers, and systems in general. Both analog and digital implementations have received attention; each has its merits and demerits. Hybrid (analog/digital) systems have proven to be useful as well. Researchers have put forward numerous techniques for analog implementations along with their hybridizations. The discussion in this chapter was meant to introduce the reader to the fundamentals of a wide spectrum of hardware implementation techniques. This information provides the basic building blocks that appear, as is or modified, in various VLSI realizations of fuzzy systems. The reader may consult the available literature for implementation details of a specific system. Fundamental circuits for VLSI implementation of neural models were presented as well, the objective being the illustration of the way simple neuron models can be implemented and to facilitate further exploration in that area. 185 Chapter 8 Bibliography General 1. I. Batroune, A. Barriaga, S. Snchez-Solano, G. J. Jimnez-Fernndez, and D. R. Lpez, Microelectronic Design of Fuzzy Logic-Based Systems, CRC Proess, New York, 2000. 2. P R. Gray and R. G. Mayer, Analysis and Design of Analog Integrated Circuits, John Wiley & Sons, Toronto, 1984. 3. M. M. Gupta and T. Yamakawa, Editors, Fuzzy Computing: Theory, Hardware, and Applications, North-Holand, 1988. 4. A. M. Ibrahim, Introduction to Applied Fuzzy Electronics, Prentice Hall, Upper Saddle River, NJ, 1997. 5. M. Jamshidi, N. Vadiee, T. Rose, Fuzzy Logic and Control: Software and Hardware Applications, Prentice Hall PTR/Sun Microsystems Press,1998. 6. A. Kandel and G. Langholz, Editors, Fuzzy Hardware: Architectures and Applications, Kluwer Academic Publishers, Boston, 1998. 7. C. Mead, Analog VLSI and Neural Systems, Addison-Wesley, Don Mills, Ontario, 1989. 8. M. J. Patyra and D. M. Mlynek, Editors, Fuzzy Logic: Implementation and Applications, Wiley/Teubner, New York, 1996. 9. W. Pedrycz, Fuzzy Sets Engineering, CRC Press, Boca Raton, Florida, USA. 10. A. S. Sidra and K. C. Smith, Microelectronic Circuits, Saunders College Publishing, Toronto, 1989. 11. P. K. Simpson, Editor, Neural Networks Theory, Technology, and Applications, IEEE Press, New York, 1996. 186 Hardware Implementations Digital Techniques 1. M. Chiaberge, E. Miranda, and L. M. Reyneri, An HW/SW Co-Design Approach for Neuro-Fuzzy Hardware Design, Proceedings of the 7th International Conference on Microelectronics for Neural, Fuzzy, and Bio-Inspired Systems, Granada, Spain, April 79, 1999, pp. 332337. 2. A. Costa, A. De Gloria, F. Giudici, and M. Olivieri, Fuzzy Logic Microcontroller, IEEE Micro, 17, 1, 6674, 1997. 3. H. Eichfeld, T. Kunemund, and M. Menke, A 12 Bit General Purpose Fuzzy Logic Controller Chip, IEEE Transactions on Fuzzy Systems, 4, 4, 460475, 1996. 4. R. Lacoste, A Low-cost Neural Network Processor, Circuit Cellar, 114, 2635, 2000. 5. M J. Patyra, J. L. Granter, and K. Koster, Digital Fuzzy Logic Controller: Design and Implementation, IEEE transactions on Fuzzy Systems, 4, 4, 439459, 1996. 6. R. Rovatli, High Speed Implementation of Piecewise-quadratic TogaiSugeno Systems with Memory Savings, Proceedings of the World Multi-Conference on Circuits, Systems, Communications and Computers, Athens, Greece, 1999, pp. 64516454. 7. R. Rovatti and M. Vitluari, Linear and Fuzzy Piecewise-linear Signal Processing with Extended DSP Architecture, Proceedings of the IEEE International Conference on Fuzzy Systems, Anchorage, Alaska, USA, 1998, pp. 10821087. 8. V. Salapura, A Fuzzy RISC Processor, IEEE Transactions on Fuzzy Systems, 8, 6, 781790, 2000. 9. M. Sasaki, F. Ueno, and T. Inoue, 7.5 MFLIPS Fuzzy Processor Using SIMD and Logic-in-Memory Structure, Proceedings of the IEEE International Conference on Fuzzy Systems, San Francisco, California, USA, 1993, 527534. 10. M. Togai and H. Watanabe, A VLSI Implementation of Fuzzy-Inference Engine: Toward an Expert System on a Chip, Information Sciences, 38, 147163, 1986. 11. H. Watanabe and D. Chen, Evaluation of Fuzzy Instructions in a RISC Processor, Proceedings of the IEEE International Conference on Fuzzy Systems, San Francisco, California, USA, 1993, pp. 521526. 187 Chapter 8 Analog Techniques 1. F. Balteanu, I. Opris, and G. Kovacs, Current-Mode Fuzzy Memory Element, Electronics Letters, 29, 2, 236237, 1993. 2. C. Giustolisi, G. Palumbo, and S. Pennisi, Current-Mode A/D Fuzzy Converter, IEEE Transactions on Fuzzy Systems, 10, 4, 533540, 2002. 3. K. Hirota, The Concept of Fuzzy Flip-Flop, IEEE Transactions on Systems, Man and Cybernetics, 19, 5, 980997, 1989. 4. K. Hirota and K. Ozawa, Fuzzy Flip-Flop as a Basis of Fuzzy Memory Modules, Fuzzy Computing, M. M. Gupta and T. Yamakawa, Editors, Elsevier Science Publishers, 1998. 5. K. Hirota and K. Ozawa, Fuzzy Flip-Flop and Fuzzy Registers, Fuzzy Sets & Systems, 32, 139148, 1989. 6. K. Hirota and W. Pedrycz, Designing Sequential Systems with Fuzzy J-K Flip-Flops, Fuzzy Sets & Systems, 39, 261278, 1991. 7. T. Kettner, C. Heite, and K. Schumacher, Analog CMOS Realization of Fuzzy Logic Membership Functions, IEEE Journal of Solid-State Circuits, 28, 857861, 1993. 8. B. D. Liu, C. Y. Huang, and H. Y. Wu, Modular Current-Mode Defuzzification Circuit for Fuzzy Logic Controllers, Electronics Letters, 30, 16, 12871288, 1994. 9. C. Lu and B. Shi, Circuit Realization of a Programmable neuron Transfer Function and its Derivative, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks, IJCNN00, Como, Italy, July 2427, 2000, pp.40474050. 10. K. Ozawa, K. Hirota, and L. T. Koczy, Fuzzy Flip-Flop, Fuzzy Logic Implementation and Applications, M. J. Patyra and D. M. Mlynek, Editors, John Wiley & Son Ltd., Toronto, 1996, pp. 197236 11. F. J. Pelayo, I. Rojas, J. Ortega, and A. Prieto, Current-Mode Analog Defuzzifier, Electronics Letters, 29, 9, 743744, 1993. 12. A. Sanz, Analog Implementation of Fuzzy Controller, IEEE International Conference on Fuzzy Systems, Orlando, FL, USA, 1994, pp. 279283. 13. Y. Shirai and Yamakawa, A CAD-Oriented Synthesis of Fuzzy Logic Circuits, SystemsComputersControls, 15, 5, 7683, 1984. Translated from Denshi Tsushin Gakkai Ronbunshi, 67-D, 6, 708714, 1984. 188 Hardware Implementations 14. K. Tsukano and T. Inoue, Synthesis of Operationa; Transconductance Amplifier-Based Analog Fuzzy Functional Blocks and Its Application, IEEE Transactions on Fuzzy Systems, 3, 1, 6168, 1995. 15. E. Vittoz, Present and Future Industrial Applications of Bio-Inspired VLSI Systems, Proceedings of the 7th International Conference on Microelectronics for Neural, Fuzzy, and Bio-Inspired Systems, Granada, Spain, April 79, 1999, pp. 211. 16. T. Yamakawa, Fuzzy Logic Circuits in Current Mode, Analysis of Fuzzy Information, J. C. Bezdek, Editor, 1, 241261, CRC Press, Boca Raton, Florida, 1987. 17. T. Yamakawa and T. Miki, The Current Mode Fuzzy Logic Integrated Circuits Fabricated by Standard CMOS Process, IEEE Transactions on Computers, c-35, 2, 161167, 1986. 18. T. Yamakawa, H. Kabuo, A Programmable Fuzzifier Integrated CircuitSynthesis, Design, and Fabrication, Information Sciences, 45, 75112, 1988. 19. J. M. Zurada, Analog Implementation of Neural Networks, IEEE Circuit & Devices, 3641, September, 1992. 189 Chapter 8 Mixed-Signal Techniques 1. I. Baturone, S. Snches-Salano, A. Barriga, and J. L. Huertas, Flexible Fuzzy Controllers Using Mixed-Signal Current-Mode Techniques, Proceedings of the IEEE International Conference on Fuzzy Systems, Barcelona, Spain, 1997, pp. 875880. 2. I. Baturone, S. Snches-Salano, A. Barriga, and J. L. Huertas, Implementation of CMOS Fuzzy Controller as Mixed-signal ICs, IEEE trasactions on Fuzzy Systems, 5, 1, 119, 1997. 3. I. Baturone, S. Snches-Salano, A. Barriga, and J. L. Huertas, Mixed-Signal Design of a Fully Parallel Fuzzy Processor, Electronics Letters, 34, 5, 437 438, 1998. 4. S. Bouras, M. Kotronakis, K. Suyama, and Y. Tsividis, Mixed AnalogDigital Fuzzy Logic Controller with Continuous-Amplitude Fuzzy Inferences and Defuzzification, IEEE Trasactions on Fuzzy Systems, 6, 2, 205215, 1998. 5. E. Franchi, N. Manaresi, R. Rovatti, A. Bellini, and C. Bacarani, Analog Synthesis of Nonlinear Functions Based on Fuzzy Logic, IEEE International Journal of Solid-State Circuits, 33, 6, 885895, 1998. 190 Hardware Implementations Web Resources Tutorials and Lists 1. Principles of Semiconductor Devices http://ece-www.colorado.edu/~bart/book/contents.htm An e-book by B. Van Zeghbroeck, Electrical and Computer Engineering Department, University of Colorado at Boulder, USA. Topics covered include: Review of Modern Physics Semiconductor Fundamentals Metal Semiconductor Junctions p-n Junctions Bipolar Transistor MOS Capacitor MOSFETs 2. Operational Transconductance Amplifiers, OTAs http://amesp02.tamu.edu/~sanchez/689-Otas-Part1.PDF A comprehensive introduction to Operational Transconductance Amplifiers by E. Snchez-Sinencio, Texas A&M University, Texas, USA. The author introduces transconductance amplifiers and their IC implementation. Detailed circuit analysis and applications are also presented. 3. VLSI Systems: Fuzzy Logic http://vlsi.wpi.edu/webcourse/fuzzy/fuzzy.html This is Chapter 9, Fuzzy Logic Systems, of the excellent book titled Design of VLSI Systems authored by D. Mlynek (Swiss Federal Institute of Technology, Lausanne, Switzerland) and Y. Leblebici (Worcester Polytechnical Institute, Worcester, MA, USA) The complete book is available on the Web. Topics covered in this chapter include: Systems Considerations Fuzzy Logic Based Control Background Integrated Implementations of Fuzzy Logic Circuits Digital Implementations of Fuzzy Logic Circuits Analog Implementations of Fuzzy Logic Circuits Mixed Digital/Analog Implementations of Fuzzy Systems CAD Automation for Fuzzy Logic Circuits Design Neural Networks Implementing Fuzzy Systems 191 Chapter 8 4. Neural Networks Implementation www.np.edu.sg/~yck/ANS_L8.pdf A tutorial presentation by Yong Chaw Koh, NgeeAnn Polytechnic Singapore. The tutorial explains the term neurocomputing, compares the complexity of neural networks to RAM cells and CPUs, and provides neuron models based on operational amplifiers. It also provides examples of commercial VLSI neural processors and highlights some common applications. 5. Fuzzy Control Methods and their Real System Applications www.ic-tech.com/Fuzzy%20Logic/tsld001.htm A presentation by IC Tech, Inc. composed of 67 slides, among the topics presented is: How to Select A Special Purpose IC for Fuzzy Control. 6. Computational Neuroscience: Neural circuits in silicon www.nature.com/cgi-taf/DynaPage.taf?file=/nature/journal/ v405/n6789/full/405891a0_r.html http://hebb.mit.edu/people/seung/papers/ news%20and%20views%20nature%20405.pdf A two-page document authored by C. Diorioi and R. Rao, appeared in Nature, 405, 891892, 2000. It provides a simple discussion of the biological origins of artificial neural networks. It provides 12 references, some of which are Web-accessible. 7. Fuzzy and Neural Systems Implementation http://polimage.polito.it/~lmr/raccolta.html A collection of papers covering the various research activities on Neural Networks and Fuzzy Systems carried out jointly at the Department of Electronics of Politecnico di Torino, the Interdepartmental Mechatronics Laboratory of Politecnico di Torino, the Department of Information Engineering of Universita di Pisa, and at the PERCRO Lab of Scuola Superiore SantAnna di Pisa from 1992 to1998. The categories include: Hybrid Intelligent Control and Applications Function Approximation and Applications Neuro-Fuzzy Algorithms Performance Analysis of Neuro-Fuzzy Algorithms Hardware Implementations and Systems Neuro-Fuzzy VLSI Chips Pulse Stream Systems Handwriting Recognition 192 Hardware Implementations 8. VLSI Architectures and Systems www.madess.cnr.it/pubblicazioni/IMPAGINATO03.pdf A section from the Italian National Project MADNESS II, 2002. It is composed of several subsections, some related to Fuzzy logic VLSI implementation, including: A CAD environment for fuzzy systems development VLSI architectures and devices for integrated neuro-fuzzy controllers Fuzzy logic methodologies, devices and applications 9. Neural Networks in Hardware: Architectures, Products and Applications www.particle.kth.se/~lindsey/HardwareNNWCourse/home.html Lecture notes by Clark S. Lindsey available in zipped and HTML formats, updated August, 2002. Topics include: Overview, Why Hardware, NNs Applications, Hardware vs Software, Designers Dilemma, Users Dilemma, NNs Hardware, Case Studies, and References and Links. 10. Commercially Available Hardware List www.emsl.pnl.gov:2080/proj/neuron/fuzzy/systems/commercial.html This Web site provides links to sources of commercially available hardware for fuzzy logic and other computational intelligence applications. 11. Neural Chips http://neuralnets.web.cern.ch/NeuralNets/nnwInHepHard.html The site provides Web links to papers that review neural hardware and other sites related to neural hardware. The categories include: Commercially available NN Chips and Systems, NN PC accelerators and other cards, and Non-Commercial or prototype NNW Chips and Systems. 193 Chapter 8 Digital Techniques 1. A Collection of Papers on VLSI Processors www.bo.infn.it/fuzzy/papers.html A set of papers discussing advances in digital implementation of fuzzy logic for embedded systems applications from the Fuzzy Logic Research Group, Physics Department, University of Bologna, Italy. Topics include: Architecture of a 50 MFIPS Fuzzy Processor and the related 1 um VLSI CMOS digital circuits Design and realization of a 50 MFIPS fuzzy processor in 1.0 um CMOS technology Design of a 50 MFIPS digital fuzzy processor and preliminary results of 1.0 um CMOS VLSI MIN-MAX and defuzzifier circuits Design of a VLSI very high speed reconfigurable digital fuzzy processor Fuzzy Logic Oriented to Active Rule Selector and Membership Function Generator for High Speed Digital Fuzzy Processor Digital Membership Function Generators and No-contribute Rule Eliminator for High Speed Fuzzy Architectures High Speed Digital Fuzzy Processor for High Energy Physics Experiment Triggers Design of a Very High Speed Fuzzy Processor by VHDL Language Very Fast VLSI Fuzzy Processor: 2 inputs 1 output A two input fuzzy chip running at a processing rate of 30 ns realized in 0.35 um CMOS technology 2. Implementing Fuzzy Rule Based Systems on Silicon Chips www.neuro.sfc.keio.ac.jp/publications/pdf/fuzzy.pdf A paper authored by M.-H. Lim and Y. T. Takefuji, which appeared in IEEE Expert, 3145, February, 1990. The authors presented a general block architecture for realizing the approximate reasoning processor. They also showed that the reasoning systems knowledge base is easily programmable and independent of the fuzzy implication relation interpretation used for inferencing. 194 Hardware Implementations 3. Hardware/Software Co-Design of RISC Processors http://jamaica.ee.pitt.edu/Archives/ProceedingArchives/Date/Date98/papers/ 1998/date98/pdffiles/11c_2.pdf A paper by V. Salapura and M. Gschwind, Technische Universitt Wien, Austria. It appeared in the proceedings of the Design, Automation & Test in Europe Conference, DATE98, Paris, France, February 2326, 1998, pp. 875 782. The authors explained as a case study the definition and evaluation of instruction set extensions for fuzzy processing. The proposed instructions were evaluated in software and hardware to reach a balanced view of the cost and benefit of each instruction. The instruction set was added to a RISC processor core. The core was described in VHDL to enable hardware implementation using logic synthesis. 4. Fast Digital Fuzzy Controller for an Active Magnetic Bearing www.eek.ee.ethz.ch/publications/publication-10.html A paper by W. Liebert, Electrical Engineering and Design Laboratory, Department of Information Technology and Electrical Engineering of the Swiss Federal Institute of Technology in Zrich, Switzerland. The author presented the analysis and design of a controller consisting of PID and fuzzy controller components. Both controllers were simulated using MATLAB and implemented on board using the TMS320C50 fixed point DSP chip. 5. Intel MCS96 Microcontrollers: Fuzzy Logic Applications www.mmx.com/design/mcs96/designex/2363.htm An application report from Intel outlining the design of a fuzzy logic controller using the MCS96 microcontroller. 6. Microprocessor Implementation of Fuzzy Logic and Neural Networks http://nn.uidaho.edu/pap/2001/Microp_impl_IJCNN01.PDF A paper by J. Binfet and B. Wilamowski appeared in the Proceedings of the International Joint Conference on neural Networks, Washington DC, July 1519, 2001, pp. 234239. The authors present systems that were implemented on the Motorola 68HC711E9. 7. Fuzzy Logic Control with the Intel 8XC196 Embedded Microcontroller www.intel.com/design/mcs96/papers/esc_196.htm An 11-page document in PDF format authored by N. Govind of Intel Corporation. It discusses the development of a fuzzy inference unit and algorithms for fuzzification, rule evaluation and defuzzification of a fuzzy control system. Methods to generate optimized fuzzy-based real-time code in assembly and C are shown for the Intel 8XC196 microcontroller along with discussion of performance and features for fuzzy-based control. 195 Chapter 8 8. A Practical Implementation of a Fuzzy Logic Controller with Motorola 68HC11 www.elia.ro/susnea/fuzzy.html An HTML document authored by Ioan Susnea. The author presents a simple implementation of a fuzzy temperature controller, built around a single chip MC68HC11KA2 microcontroller. It is meant to be a resource for students and designers. 9. Demonstration Model of fuzzyTECH Implementation on Motorola 68HC12 MCU www.fuzzytech.com/e/e_a_mot.html A report by Philip Drake and Jim Sibigtroth of Motorola, and Constantin von Altrock and Ralph Konigbauer of Inform Software Corp. Inform Software Corp. and Motorola created the fuzzyTECH MCU-HC12 Edition of Motorola MCU-HC12 which supports both the HC12s fuzzy logic instruction set and the HC12 background debug mode. The authors demonstrate both the usage of the fuzzy logic instruction set and the use of the background debug mode with the fuzzyTECH development system. Inform and Motorola have designed an autonomously guided tank as a demonstration model. The authors discuss the fuzzy logic controller design for the tank and the fuzzyTECH implementation on the HC12 MCU. More information about fuzzyTECH is given in chapter 9. 10. Neuron www.castle.uk.co/neuron/models.htm This site provides information about embedded computers based on neural networks provided by Castle Technology Ltd, Suffolk, UK. Systems presented include: Neuron 100 ARM7500 embedded computer built around ARMs 64MHz 7500FE processor. It provides an ARM 7 central core and supports a wide range of industry standard storage and input/output computer peripherals. Neuron 200 A StrongARM embedded computer built around the 233MHz StrongARM processor. Cortex is an optional connector board for Neuron computers. It connects to a Neuron embedded computer to provide connectivity to other devices and can be fitted with a wide range of application-specific connectors and extra circuitry. 196 Hardware Implementations 11. Neuro-Fuzzy Control and DSPs http://polimage.polito.it/~marcello/articoli/microneuro.97.daniela.pdf A paper by B. Bona et al., Politecnico di Torino, Italy. It appeared in the Proceedings of the International Conference on Microelectronics for Neural Networks and Fuzzy Systems, MICRONEURO97, Dresden, September, 1997, pp. 113120. The authors presented a hybrid control system that uses neural and fuzzy techniques and DSP techniques. The TMS320C31 DSP was used for hardware implementation. 12. Fuzzy Logic and PLCs www.industrialtext.com/eBooks/Fuzzy_Logic.PDF An 18-page e-book from the Industrial Text & Video Company. Topics covered include: Introduction to Fuzzy Logic, Evolution of Fuzzy Logic Control, Fuzzy Logic Principles, Fuzzy Logic Application Example, and Guidelines for Controlling a Complex System. 13. Silicon Axon http://citeseer.nj.nec.com/163659.html An article authored by B. A. Minch. P. Hasler, and C. Mead. The authors present an axon intended as a building block for pulse-based neural computations involving temporal and spatial correlations of pulses. 14. A Neuro-Fuzzy Real Time Controller for Low Power Embedded Systems http://polimage.polito.it/~marcello/articoli/microneuro.94.cintia.pdf A paper by L.M. Reyneri et al., which appeared in the Proceedings of the International Conference on Microelectronics for Neural Networks and Fuzzy Systems, MICRONEURO94, Torino, Italy, September 1994, pp. 392 404. The authors describe a neuro-fuzzy real-time controller based on Pulse Stream techniques. The system discussed is a hybrid of two approaches: Neuro-Fuzzy Controllers (implemented using a custom neural chip) and Finite State Automata (implemented using 68Hc11 microcontroller). 15. A Neural Network chip Using CPWM Modulation http://polimage.polito.it/~marcello/articoli/iwann.93.pdf A paper by M. Chiaberge et al., appeared in the Proceedings of the International Workshop on Artificial Neural Networks, IWANN93, Sitges (E), June 1993, pp. 420425. The authors described the development and testing of s chip that implemented a neural network using the Coherent Pulse Width Modulation, CPWM, technique. 197 Chapter 8 16. A Comparison between Analog and Pulse Stream VLSI Hardware for Neural Networks and Fuzzy Systems http://polimage.polito.it/~marcello/articoli/microneuro.94.comp.pdf A Paper by L. M. Reyneri et al. This appeared in the Proceedings of the International Conference on Microelectronics for Neural Networks and Fuzzy Systems, Torino, Italy, September 1994, pp. 7786. The authors used Coherent Pulse Width Modulation, CPWM, as a computing technique to implement neural systems. They compared its performance with systems implemented using analog techniques. Both systems were designed using the same 0.1 m full custom, double metal, single poly CMOS technology. The authors concluded that in general the two techniques have comparable performance. They also pointed out that the design of CWPM is simpler and multiplexing of CPWM systems is easier. 198 Hardware Implementations Analog Techniques 1. Electronic Models of Fuzzy Gates A paper authored by J. L. Prez and M. A. Banuelos which appeared in the Journal of Instrumentation and Development , October 2000, pp. 4749. In this paper, circuits using op-amps were suggested to implement the functions of fuzzy AND, OR, and NOT. A suggestion for a fuzzy neuron is presented as well. 2. Current Mode CMOS Implementation http://nn.uidaho.edu/pap/1995/wcnn95_ota.pdf A paper by Y. Ota and B. Wilamowski which appeared in the Proceedings of World Congress of Neural Networks, Washington DC, USA, July 1721, 1995, pp. 480483. The authors presented a programmable fuzzy system built using current-mode CMOS circuits that implement a Gaussian-type membership function, min-max operators, and a defuzzifier circuit. The membership function is represented by the relation between an input voltage and the output current. The appearance of the Gaussian membership function and position can be controlled by adjusting a reference voltage. 3. Analog Fuzzy Inference Processor http://members.fortunecity.com/scienziatopazzo/fuzzy.htm A paper by C.T.P. Song, S.F. Quigley, and S. Pammu, published at the IEEE International Symposium for Circuits and Systems, Monterey, California, USA, May 31June 3, 1998, pp. 247250. The authors presented a design for an analog fuzzy inference processor. It was implemented using CMOS technology. 4. Analog Processing Based on Fuzzy Algorithms www.cinstrum.unam.mx/revista/pdfv4n5/art2.pdf A paper by J. Castillo et al. This appeared in the Journal of the Mexican Society of Instrumentation, 4, 5, 1218, 2002. The authors presented a fuzzy logic controller circuit on analog CMOS technology. They presented the design and simulation of circuits for membership functions, fuzzy AND, fuzzy OR, and defuzzification. 199 Chapter 8 5. VLSI Implementation of Neural Networks http://nn.uidaho.edu/pap/2001/VLSIimpl.pdf A paper authored by B. M. Wilamowski, J. Binfet, and M. O. Kaynak, which appeared in International Journal of Neural Systems, 3, 10, 191197, 2000. The authors presented the design and simulation results of two CMOS circuits that implement neural networks. MATLAB was used to develop the training algorithms. The circuits presented were two-input, but the authors indicated that they could be generalized to multidimensional circuits. 6. VLSI Implementation of Fuzzy ART http://bach.ece.jhu.edu/pub/papers/aicsp01.pdf A paper authored by J. Lubin and G. Cauwenberghs, which appeared in Analog Integrated Circuits and Signal Processing, 23, 110, 2001.The authors presented a mixed-mode VLSI chip performing unsupervised clustering and classification, implementing models of fuzzy ART. 7. Analog Neuro-Fuzzy Networks for System Modeling and Control www.micro.ea.unian.it/Staff/sim/PAPERS/1997ISIS_%20fuzzy.pdf M. Conti, S. Orcioni, C. Turchetti, Advances in Intelligent Systems, F. C. Morabito (Ed.), IOS Press, Netherlands, 1997. The authors discussed a new type of neural network based on the so-called approximate identities functions. A fuzzy-neural architecture based on this neural network type is then introduced. The authors explained further the reasons for the suitability of the suggested architectures for analog CMOS implementation, along with its SPICE simulations. The authors presented the circuit layout and more details in: A Current-Mode Neuro-Fuzzy Network, IEEE International Conference on Electronics Circuits and Systems ICECS98, Lisboa, September, 1998. This may be viewed at: www.micro.ea.unian.it/Staff/sim/PAPERS/1999ISCASfuzzy.pdf Implementation results and further details were given in: A Current-Mode Circuit for Fuzzy Partition Membership Functions, IEEE International Symposium on Circuits and Systems ISCAS99, Orlando, FL, May 30June 2, 1999. (www.micro.ea.unian.it/Staff/sim/PAPERS/ 1999ISCASfuzzy.pdf) Further implementation details are given in: CMOS Analog Implementation of Multidimensional Membership Functions of Neuro-Fuzzy Networks, IEEE European Conference on Circuit Theory and Design ECCTD99, Italy, August 29Sept. 2, 1999. www.micro.ea.unian.it/Staff/sim/PAPERS/1997ISIS_%20fuzzy.pdf 200 Hardware Implementations 8. Adaptation, Learning and Storage in Analog VLSI http://bach.ece.jhu.edu/gert/papers/asic96/ASIC96.html A paper by Gert Cauwenberghs, Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, USA. It appeared in the Proceedings of the 9th Annual IEEE International ASIC Conference, Rochester, NY, September 1996, pp. 237278. The author presented circuit modules for adaptation and memory, gradient descent learning, and stochastic error-descent learning for analog VLSI implementations. 201 Chapter 8 Mixed Signal Techniques 1. On Designing Mixed Signal Programmable Fuzzy Logic Controllers as Embedded Subsystems in Standard CMOS Technologies www.dice.ucl.ac.be/~verleyse/papers/sbcci01cd.pdf A paper by C. Dualibe, P. Jespers, and M. Verleysen which appeared in the Proceedings of the 14th Symposium on Integrated Circuits and System Design, Pirenopolis, Brazil, September 1015, 2001, pp. 194200. The author presented a digitally-programmable analog fuzzy logic controller. It is intended for low-power embedded subsystems for applications with bandwidths below 8 MHz. 2. A Mixed Signal Neuro-Fuzzy Processor for Embedded Applications www.micro.ea.unian.it/Staff/sim/PAPERS/1999CSCCfuzzy.pdf A paper by M. Conti, S. Orcioni, and C. Turchetti, appeared in the Proceedings of the IEEE Circuits, Systems, Communications and Computers, CSCC 99, July 48, 1999, Athens. The authors presented a neuro-fuzzy controller implemented using 0.8 m CMOS analog technology. The architecture is based on an analog device implementing fuzzy rules and a digital interface that allows the device to work as an analog peripheral of a microcontroller. 3. CMOS Design of Adaptive Fuzzy ASICs Using Mixed-Signal Circuits www.el.uma.es/RafaelNavas/pub_PDF/iscas96.pdf F. Vidal-Verd, R. Navas, and A. Rodrhuez-Vzquez which appeared in the Proceedings of the IEEE International Symposium on Circuits and Systems, Atlanta, USA, 1996, pp. 430433. The authors present a methodology and circuit blocks to realize fuzzy controllers in the form of analog CMOS chips. The authors also provide measurements from a prototype of a fuzzy controller chip. 4. Neuro-Fuzzy Architecture for CMOS Implementation http://nn.uidaho.edu/pap/1999/TIE_neurofuzzy2.pdf A paper by B. M. Wilamowski, R. C. Jaeger and M. O. Okyay which appeared in the IEEE Transactions on Industrial Electronics, 46, 6, 11321136, 1999. A non-conventional structure for a fuzzy controller was proposed. It combines fuzzification, MIN operators, normalization, and weighted sum blocks. The structure was implemented as a VLSI chip using n-well CMOS technology. 202 Hardware Implementations 5. A 5.26 Mflips Programmable Analogue Fuzzy Logic Controller www.dice.ucl.ac.be/~verleyse/papers/iscas00cd.pdf A paper by C. Dualibe, P. Jespers, and M. Verleysen which appeared in the Proceedings of the IEEE Symposium on Circuits and Systems, Geneva, Switzerland, May 2831, 2000, 377380. The authors presented circuit design and simulation results of a digitally-programmable analog fuzzy logic controller. A nine-rule, two-input, and one-output prototype was fabricated and characterized. 6. On-Chip Fuzzy Temperature Control www.ims.fhg.de/datenblaetter/fuzzy_systems/fuztem/fuztem_e.pdf A two-page HTML document from Fraunhofer Institut Mikroelektronische Schaltungen und Systeme. It outlines the characteristics of an on-chip fuzzy temperature control that can be used in adaptive fuzzy microsystems. 7. A Modular Programmable CMOS Analog Fuzzy Controller Chip www.el.uma.es/RafaelNavas/pub_PDF/tcas99.pdf A paper by A. Rodrguez-Vzquez et al. This appeared in the IEEE Transactions on Circuits and systems-II: Analog and Digital Signal Processing, 46, 3, 251265, 1999. The authors presented a modular fuzzy inference analog MOS chip architecture with on-chip digital programmability. The prototype presented has 16 rules and an operation speed of 2.5 MFLIPS with power consumption of 8.6 mW. It was fabricated using 1-m CMOS technology. The authors compared the characteristics of the reported prototype with those reported in: N. Manaresi et al., A Silicon Compiler of Analog Fuzzy Controllers: From Behavioral Specifications to Layout, IEEE Transactions on Fuzzy Systems, 4, 418428, 1996, and S. Guo et al., Design and Applications of an Analog Fuzzy Logic Controller, IEEE Transactions on Fuzzy Systems, 4, 429438, 1996. 8. On Designing Mixed-Signal Programmable Fuzzy Logic Controllers as Embedded Subsystems in Standard CMOS Technologies www.dice.ucl.ac.be/~verleyse/papers/sbcci01cd.pdf A paper by C. Dualibe et al. This appeared in the Proceedings of the 14th Symposium on Integrated Circuits and System Design, Pirenopolis, Brazil, 1015 September, 2001, pp. 194200. The authors presented a prototype of a digitally-programmable analog fuzzy logic controller with a nine-rule, twoinput, and one output design. It was implemented using 2.4 m CMOS Technology with speed up to 5.26 MFLIP. 203 [This is a blank page.] CHAPTER 9 Software Tools 9.1 Introduction In this chapter, a selection of software tools for fuzzy systems, neural networks, and neuro-fuzzy systems is presented. The selection can be loosely divided into the following categories: executable files with graphical user interface, executable files that have no graphical user interface, and source codes in C, C++, Java, MATLAB files, etc. The software can also be divided into commercial, shareware, and public domain. They can be also categorized as educational or productivity tools. The objectives of the software presented here differ. The tools meant for educational purposes illustrate the basic concepts of fuzzy logic and neural networks and help in the visualization of the operation of simple systems. The ones for productivity are meant for system design and analysis. They are all included here since they add educational value and enable the reader to experiment and make judgments based on first-hand experience with a large number of software tools with minimum investment of time and money. The selection criteria of the software listed can be summarized as follows: Relevance to fuzzy systems, neural networks, or both. Availability of an English version. Documentation is available, or the software can be used intuitively, or both. The software is either free or a demo of educational value is available. The software and documentation are available through the Web. The tools are arranged alphabetically for ease of reference, but it would be useful for the reader to examine them starting with the ones that demonstrate general concepts followed by simple productivity tools such as FuzzGen. Fuzzy and neural source codes do, of course, require some knowledge of MATLAB, C, Java, Perl, etc. Commercial productivity tools, such as fuzzyTECH, FIDE and Neuro Solutions, can be useful both for education and the design of large-scale projects. 205 Chapter 9 It goes without saying that the software selected here is believed to be reliable and of value. However, inclusion here does not constitute an endorsement or a guarantee for its fitness for a particular purpose. 9.2 Software Overview 1. C and C++ Source Codes The ART Gallery cns-web.bu.edu/pub/laliden/WWW/nnet.frame.html The ART Gallery is a free set of procedures developed by Lars Lidn to be used within other code for implementing many of the ART style neural networks. It supports C procedure calls on both the Unix and DOS platforms, as well as compilation as a dynamic linked library for Windows environments such as Visual Basic. The Binary Hopfield Net www.geocities.com/ResearchTriangle/Facility/7620/ The site provides a Java applet developed by Matt Hill. It demonstrates the works of a Hopfield network and the source code is provided as public domain software. One may use the mouse to enter a pattern by clicking squares inside the rectangle shown. The network stores the pattern; up to approximately 33 random patterns can be stored. After storing some patterns you can then try entering a new pattern (one of the stored patterns, but slightly corrupted) to be recognized, and then watch the network settle. Neural Networks at your Fingertips www.geocities.com/CapeCanaveral/1624/ The site provides software simulators developed by Karsten Kutza for eight of the most popular neural network architectures, coded in portable, self-contained ANSI C. The software available for free downloading include: ADALINE Simulation of the Adaline network, which is a single-layer backpropagation network. It is trained on a pattern recognition task, to classify a bitmap representation of the digits 0-9 into the corresponding classes. The network recognizes only the exact training patterns. When the application is ported into the multi-layer backpropagation network, fault-tolerance can be achieved. Figure 9.1 shows an example of the patterns to be recognized. 206 Software Tools Figure 9.1: Patterns from 0 to 9 for the ADALINE network. BPN (Backpropagation Network) This is an implementation of a multi-layer backpropagation network with bias terms and momentum. The network is used to detect structure in time-series. It is presented to the network using a simple tapped delay-line memory. The program can predict future sunspot activity by learning from historical data collected over the past three centuries. HOPFIELD This program is based on the Hopfield model. It is used as an auto-associative memory to store and recall a set of bitmap images. Images are stored by calculating a corresponding weight matrix. Then, on presentation of a corrupted image, the network settles on a stored image which is nearest to the input image in terms of Hamming distance. BAM (Bidirectional Associative Memory) The bidirectional associative memory allows for a heteroassociative memory to be implemented. It can be viewed as a generalization of the Hopfield model. The software gives, as an example, the association between names and corresponding phone numbers. After training, the network is able to recall the corresponding phone number when presented with a name and vice versa. A degree of fault-tolerance in case of corrupted input patterns is allowed. 207 Chapter 9 BOLTZMAN (Boltzmann Machine) This is a software implementation of the Boltzmann Machine network. It is used to solve an optimization problem. The weight matrix is chosen such that the global minimum of the energy function corresponds to a solution of a particular instance of the traveling salesman problem. CPN (Counterpropagation Network) The counterpropagation network is a competitive network, designed to function as a self-programming lookup table with the additional ability to interpolate between entries. The application is to determine the angular rotation of a rocket-shaped object, images of which are presented to the network as a bitmap pattern. The performance of the network is a little limited due to the low resolution of the bitmap. SOM (Self-Organizing Map) In this program the network learns to balance a pole by applying forces at the base of the pole. The behavior of the pole is simulated by numerically integrating the differential equations for its law of motion using Eulers method. The task of the network is to establish a mapping between the state variables of the pole and the optimal force to keep it balanced. ART1 This program demonstrates the basic feature of the adaptive resonance theory network, which is its ability to adapt when presented with new input patterns while remaining stable at previously learned patterns. Pittnet www.pitt.edu/~aesmith/postscript/guide.pdf Pittnet is a neural network program designed for educational purposes by Brian Carnahan and Alice Smith of the University of Pittsburgh. The compiled C++ source code, the uncompiled C++ source code, and the Users Guide are available for downloading. The software has the following neural network paradigms: backpropagation, Kohonen self-organizing, ART1, and the radial basis function, RBF. The input and output are files of Pittnet ASCII text files. UNFuzzy http://ohm.ingsala.unal.edu.co/ogduarte/Software.htm A program developed by Oscar Duarte, Universidad National de Colombia. It is available in several languages, including English and Spanish. MS-DOS NN freeware www.simtel.net/pub/msdos/neurlnet/ This site provides description and links for free downloads of a large number of programs for neural networks, which run under MS-DOS. Some of the programs include: 208 Software Tools ann110; a multi-option NN training program bam; a Bidirectional Associative Memory simulation bpnn132; a backpropagation neural net with adaptive learning rate bps100; a backpropagation simulator brain12; a backpropagation neural net simulator et; a perceptron simulator nerves; Nervous System Construction Kit with C++ source code neurfuzz; a program to train NN and fuzzy-engines and generate C source code nlmos; an educational simulation for perceptrons, backpropagation, and self-organizing maps nnutl101; a C-source code library and tutorial for neural networks. StarFLIP++ www.dbai.tuwien.ac.at/proj/StarFLIP/ A reusable iterative optimization family of C++ libraries for combinatorial problems with fuzzy constraints designed by the Database and Expert Systems Group, Institute of Information Systems, Vienna University of Technology, Austria. StarFLIP++ is progressing to include the following layered sub-libraries: the fuzzy logic inference processor library, FLIP++ the fuzzy constraint library, ConFLIP++ the dynamic constraint generation library, DynaFLIP++ the domain knowledge representation library, DomFLIP++ several heuristic optimizing libraries, OptiFLIP++ the user interface, InterFLIP++ the HTML documentation, DocuFLIP++ the knowledge-change consistency checker library, CheckFLIP++ the version-control and test environment for the complete library set, TestFLIP++ the simulation toolkit, SimFLIP++ the reactive optimizer, ReaFLIP++ the neural network extension that allows automatic tuning of fuzzy membership functions, NeuroFLIP++ . 209 Chapter 9 2. A Collection of Fuzzy Logic Based Tools for Analog Circuits Design www.gte.us.es/usr/chavez/collect.pdf A paper authored by A. Chvez and L. G. Franquelo appeared in IEEE Micro, 6068, August 1996. It presents a collection of fuzzy logic-based tools in the three main phases of the analog design: topology selection, modeling and optimization, and testing. 3. EazyNN www.easynn.com/easynnbase.html EasyNN is a shareware program provided by Stephen Wolstenholme. It can generate multi-layer neural networks from text files or grids. The networks can then be trained, validated and queried. Network diagrams, graphs, input/output data and all the network details can be displayed and printed. Neurons (nodes) can be added or deleted while the network is learning. The software has a stepby-step tutorial, example files, and a help file. A screen shot from the software is shown in Figure 9.2. Figure 9.2: EasyNN Help file. 210 Software Tools 4. FCM, Fuzzy Control Manager www.transfertech.de/www/soft_e.htm This is a Windows program that can be used intuitively to define fuzzy systems of any complexity graphically. All relevant data can be displayed while developing, debugging, or optimizing the system. In constructing the system, there is no limit on the number of variables or rules to be defined. The membership functions can be defined by an unlimited number of inflection points (or Singletons for outputs, if desired). Codes for implementing the constructed fuzzy controller can be generated in C-language, assembler, or binary, depending on the FCM version used. FCM versions include: FCM-3000 which operates on an FP-3000 adapter board and generates special C code which performs an appropriate access to the FP-3000 processor FCM-SOFT which supports the online operation of a VMEbus board containing an FP-3000 processor FCM-TEAM which supports a microcontroller board with many analog and digital I/O lines FCM-SPS which is used for direct programming of a fuzzy module of a programmable controller containing an FP-3000 The FCM-Modicon which supports fuzzy programming of an AEG-Modicon FCM-I3HD-CINT which generates codes for the NEC 78K0,78K3, V series microcontrollers FCM-I3HD-75X which generates code for microcontrollers of the NEC 75X family FCM-I3HD-17K which generates code for microcontrollers of the NEC 17K family FCM-PX-1500 which generates code for the Panasonic MN1500 microcontroller family FCM-HC11 which generates code for 68HC11 microcontrollers FCM-8051 which generates code for microcontrollers of the 8051 family. A demo with example files is available for downloading and screen shots are available on the Web. 211 Chapter 9 5. FIDE, Fuzzy Inference Development Environment www.aptronix.com/fide/ FIDE is a development environment for fuzzy logic systems from Aptronix, Inc., Santa Clara, USA. The software can automatically generate fuzzy algorithms in Java, ANSI C, MATLAB M-file, and assembly code for a variety of microcontrollers. Chips supported include: Motorola: 68HC05, 6805, 68HC08, 68HC11, 68HC12, 68HC33x, Intel 80C196 and 80C296 architectures, Siemens: SAE81C99A, and Omron Electronics: FP-3000, FP-5000. A free demo of FIDE and example files are available for downloading. The demo allows the creation and simulation of fuzzy logic models and it helps to learn about the fuzzy inference process. The demo is useful for educational purposes and as a preview of the full version. It is, of course, not meant for use in a production environment; the compiler and code generator are disabled. 6. FLDE, Fuzzy Logic Development Environment www.flde.com/flde/index.htm FLDE is a software tool for developing complex embedded control applications. It produces C-code for various target hardware architectures from natural language specifications of the behavior of the controller. Two examples are provided. The first is Building Embedded Automotive Applications with Fuzzy Logic: A NiCd Battery Charger, by Stylianos S. Sbyrakis, Syndesis Ltd, Athens, Greece. The second is Inverted Pendulum Fuzzy Control. 7. Fuzzy CLIPS http://ai.iit.nrc.ca/IR_public/fuzzy/fuzzyClips/fuzzyCLIPSIndex.html FuzzyCLIPS is an extension of the CLIPS (C-Language Integrated Production System) expert system shell from NASA. It was developed by the Integrated Reasoning Group of the Institute for Information Technology of the National Research Council of Canada. FuzzyCLIPS can deal with exact, fuzzy, and combined reasoning, allowing fuzzy and normal terms to be freely mixed in the rules and facts of an expert system. For non-commercial use, it is available for downloading free of charge, but a licence is required for commercial use. 8. Fuzzy Control Demo www.csee.wvu.edu/~esazonov/loadswayindex.htm An interactive example provided by Eduard Sazonov, West Virginia University, USA. It illustrates the use of fuzzy control of a loading crane. In this demonstration, the fuzzy rules controlling the crane can be changed at any time. Three linguistic variables have been defined: angle, distance and power. The linguistic 212 Software Tools variables and membership functions cannot be changed interactively. The author of the demo points out that unlike the FuzzyTech example, which works only in one direction of motion, or the Omron example that utilizes complicated anti-sway rules, the given model employs two very simple rules that compensate load sway at the distances close to the destination. For each swing of the load, the cranes trolley is moved in the swing direction to reduce the amplitude. The systems behavior without the compensation for the load sway can be observed by removing the last two rules in the set. A screenshot from the demo is shown in Figure 9.3. Figure 9.3: Prevention of load sway by a fuzzy controller. 9. FuzzyCOPE http://kel.otago.ac.nz/software/FuzzyCOPE3/ FuzzyCOPE is a software provided by Knowledge Engineering Laboratory, KEL, University of Otago, New Zealand. It is a hybrid connectionist software environment. It has a graphical user interface to the functionality of the system. The Dynamic Link Libraries that form the core of the system may be used to develop Windows based intelligent systems through the use of the provided programming libraries. Command line tools allow for the creation, training and validation of connectionist structures from the DOS prompt. The software is available for free download. Figure 9.4 shows the contents of the help file in order to get an idea about the potential of the software. 213 Chapter 9 Figure 9.4: The components of the help file of FuzzyCOPE. 10. Fuzzy Decision Tree, FID www.cs.umsl.edu/~janikow/fid/ FID is a program developed by Cezary Janikow, Mathematics and Computer Science, UMSL St. Louis, MO, USA. It generates a fuzzy logic based decision tree from fuzzy data. The tree can then be used to classify data with unknown classification using several different methods of inference. The ideas used were partially described in a paper by the developer: Fuzzy Decision Trees: Issues and Methods, IEEE Transactions on Systems, Man, and Cybernetics, Vol. 28, Issue 1, pp 114, 1998. 11. FuzzGen www.programmersheaven.com/zone22/cat167/1244.htm This is a freeware program, and it is probably the earliest fuzzy logic code generator. It allows the user to put together full fuzzy decision-making algorithms graphically, then allows the user to generate code in Pascal, Basic, or C/ C++ that will implement the decision process. The software could be used intuitively, but it also has a help file. Figures 9.5 to 9.9 give illustrations from the program to show its capabilities and ease of use. 214 Software Tools Figure 9.5: FuzzGen opening screen. Figure 9.6: Some of the FuzzGen menus. Figure 9.7: Membership selection in FuzzGen. 215 Chapter 9 Figure 9.8: Graphical definition of membership functions in FuzzGen. Figure 9.9: Editing fuzzy rules in FuzzGen. 216 Software Tools 12. Fuzzy Logic Control in VHDL www.ee.ualberta.ca/~elliott/ee552/studentAppNotes/2000f/vhdl/fuzzyController/ An article by Steve Dillen and Farrah Rashid presented by Duncan Elliott, University of Alberta, Canada. The article is in HTML format with links to source codes. 13. fuzzyTECH www.fuzzytech.com/ fuzzyTECH is a leading family of software development tools for fuzzy logic and and neural-fuzzy systems. The software supports both English and German languages. Moreover, the documentation is available in the English, German, and Mandarin Chinese languages. fuzzyTECH families of particular interest here include: fuzzyTECH Editions for General Target Hardware including fuzzyTECH Professional and fuzzyTECH online editions. fuzzyTECH MCU Editions for Embedded Control including: fuzzyTECH MCU-HC05/08 Edition which supports all Motorola 68HC05xx and 68HC08xx families of microcontrollers fuzzyTECH MCU-ST6 Edition which supports all STMicroelectronics ST6 family microcontrollers fuzzyTECH MCU-MP Edition which supports all Microchips microcontrollers fuzzyTECH MCU-51 Edition which supports all 8051 and 80251 microcontrollers fuzzyTECH MCU-374xx Edition which supports all 8-bit Mitsubishi microcontrollers fuzzyTECH MCU-HC11/12 Edition which supports all Motorola 68HC11xx and 68HC12xx families of microcontrollers. fuzzyTECH MCU-96 Edition which supports all Intel MCS-96 microcontrollers fuzzyTECH MCU-320 Edition which supports Texas Instruments Digital Signal Processors fuzzyTECH IA Editions for Industrial Automation including fuzzyTECH IA-S5/7 Edition for Siemens SIMATIC Programmable Logic Controllers S5. Application examples are discussed in numerous reports by C. von Altrock including: Fuzzy Logic in Automotive Engineering, Circuit Cellar, 88, 19, 1997, available at: www.circuitcellar.com/pastissues/articles/misc/88constantin.pdf 217 Chapter 9 Practical Fuzzy Logic Design, Circuit Cellar, 75, 15, 1996, available at: www.circuitcellar.com/pastissues/articles/misc/75constantin1.pdf A Fuzzy-Logic Thermostat, Circuit Cellar, 75, 15, 1996, available at: www.circuitcellar.com/pastissues/articles/misc/75constantin2.pdf A fuzzyTECH library of technical applications is also available online. It includes reports on topics such as: Practical Design, Industrial Automation, Coal Power Plant, Refuse Incineration Plant, Complex Chilling Systems, Water Treatment System, Monitoring Glaucoma, AC Induction Motor, Truck Speed Limiter, Medical Shoe, Fuzzy in Appliances, Automotive Engineering, Antilock Braking System, Aircraft Flight Path, Nuclear Fusion, Motorola 68HC12 MCU, Traffic Control, Fuzzy Logic Standardization, and Sonar Systems, and Fuzzy Logic Design: Methodology, Standards, and Tools. Further information, discussions and illustration of fuzzyTECH applications are available at: www.neurotech.com.sg/html/archive/supply_archive_files/fuzzy.html A demo of fuzzyTECH along with example files is available for downloading. The following are some illustrative screen shots from fuzzyTECH. Figure 9.10: Opening one of the fuzzyTECH example files. 218 Software Tools Figure 9.11: Defining inputs for the Container Crane Controller example. 219 Chapter 9 Figure 9.12: Defining the output of the Container Crane Controller example. Figure 9.13: fuzzyTECH Fuzzy rules editor. 220 Software Tools Figure 9.14: Some of the menus of fuzzyTECH showing important features of the software. 14. havBpETT www.hav.com/default.html havBpETT demonstrates the use of a DLL version of the havBpNet++ C++ neural network class library. The demo allows the user to describe, train, and save a simple feed-forward, recurrent or sequential neural network. The downloadable demos are the same as the fully functional simulator except that the network-save function is disabled. Screen-shots to illustrate the main user interface screens of the havBpETT demo are also available on the Web site. 221 Chapter 9 15. HyperLogic Demos www.hyperlogic.com/demos.html The site offers several demo programs, some of which run under Windows and some under MS-DOS. The main software is CubiCalc. A complete working model of that software is available with all project save and data export capabilities disabled. The demo has several example files that can be opened and examined as shown in Figure 9.15. The problems are described in detail and an example of the results of running the simulation of driving in a circle is shown in Figure 9.16. Figure 9.15: Opening one of the examples provided with CubiCalc. 222 Software Tools Figure 9.16: Example of results that could be displayed using CubiCalc. The site also provides an MS-DOS file that explains how the OWL Neural Network Library of C-subroutines works. It includes examples using Backpropagation and Competitive Learning. The source code for a sample program is also provided. In addition to Fuzzy Engineering Examples, numerous MS-DOS and Windows files that illustrate some of the neural networks operations and fuzzy logic applications are provided, including a truck backer-upper. 16. Jans Fuzzy System, JFL http://inet.uni2.dk/~jemor/jfs.htm JFs is a development environment for the programming language JFL, presented by Jan Mortensen, Frederiksberg, Denmark as freeware. The environment has tools to compile, run, improve and convert JFL programs. JFL combines features from traditional programming languages like fuzzy logic and machine learning. A compiled JFs-program can be executed by a command line from C-programs, or from a DLL. It can be converted to an HTML file with the program converted to JavaScript. It can also be converted to a source code file and included in C-programs. 223 Chapter 9 17. Java Related Joon NN http://joone.sourceforge.net/ Joone is a free neural net framework to create, train and test neural nets. It is based on Java technologies. Joon NN is composed by a central engine that is the pivot of all existing applications and those that will be developed. Applets for Neural Networks and Artificial Life www.aist.go.jp/NIBH/~b0616/Lab/Links.html A collection of Java applets provided by Akio Utsugi, National Institute of Advanced Industrial Science and Technology, Japan. The applets are classified into several categories, including: Competitive Learning Vector Quantizer, VQ Gaussian Mixture Model Multinomial Mixture Model Several Self-Organizing Maps, SOM Interactive SOM Pattern Recognition using Kohonen Featuremap Several nets for TSP Neural Competitive Models Demo Comparison between various kinds of competitive learning with topology reformation. Bayesian Self-Organizing Maps Backpropagation Learning Learning of Function Approximation Interactive Tutorials on Artificial Neural Learning Animated Neural Network Learn 3-D plane Artificial Neural Network Handwriting Recognizer OCHRE - Optical Character Recognition Neural Nets for Constraint Satisfaction and Optimization Hopfield Net 8-Queen Problem Boltzmann Machine Content Addressing Memory by Hopfield Net Brain Wave Pattern reconstruction and more. 224 Software Tools 18. LabVIEW Related LabVIEW is short for Laboratory Virtual Instrument Engineering Workbench. It is a graphical program development application from National Instruments. LabVIEW programs are called virtual instruments, VIs, because their appearance and operation mimic actual instruments. A free evaluation version of LabVIEW is available from National Instruments at: www.ni.com/downloads/ Useful LabVIEW resources are available at: www.ni.com/downloads/ A list of all off-the-shelf VIs for a specified purpose is available at: www.mooregoodideas.com/FAVIs/ Free VIs are available from REAL Controls, Inc. at: www.realcontrols.com/download.htm Commercial fuzzy VIs are available at: www.dataengine.de/english/ A summary report of a PhD thesis titled Control of a Pneumatic Servosystem Using Fuzzy Logic by H. M. Llagostera, UPC, Barcelona, Spain,where LabVIEW is used in the fuzzy control scheme is available at: http://fluid.power.net/techbriefs/papers/proc_moreno.pdf ARGE Simulation News, ARGESIM, is a non-profit working group that provides the infrastructure and administration for dissemination of information on modeling and simulation in Europe. It is located at Vienna University of Technology, Simulation Department. Several comparisons have been defined in Simulation News Europe. Comparison # 9 titled Fuzzy Control of a Two Tank System includes the use of LabVIEW. It is available at: http://argesim.tuwien.ac.at/comparisons/ 19. Linguistic Fuzzy Logic Controller, LFLC http://ac030.osu.cz/irafm/lflc/lflc.html Linguistic Fuzzy Logic Controller is provided by the Institute for Research and Applications of Fuzzy Modeling, University of Ostrava, Czech Republic. The software has two different basic inference methods: one based on the interpretation of the IF-THEN rules as linguistically described logical implications, and the other is the standard max t-norm rule which is the interpolation of an unknown function. A demo of the software is available for download. The demo has example files and a detailed help file. 225 Chapter 9 20. Matlab Related MATLAB is a commercially available (www.mathworks.com) interactive environment and programming language for scientific and technical computation; it also has a student version at a reduced cost. It allows solution of many numeric problems in much less time than it would have taken to write a program in C or C++ (for example). MATLAB allows users to build their own reusable tools. One can create ones own special functions and programs to run in a MATLAB environment; such programs are referred to as M-files. MATLAB functions created to deal with a certain class of problems could be grouped together, leading to the concept of a toolbox. A toolbox in this context refers to a specialized collection of M-files for working on a particular class of problems. Numerous toolboxes are available commercially, for example: COMMUNICATIONS TOOLBOX, for the design and analysis of communication systems CONTROL SYSTEM TOOLBOX, for the design and analysis of feedback control systems CURVE FITTING TOOLBOX, for model fitting and analysis DATA ACQUISITION TOOLBOX, for acquiring and sending out data from plug-in data acquisition boards DATABASE TOOLBOX, for exchanging data with relational databases DATAFEED TOOLBOX, for acquiring real-time financial data from data service providers EXTENDED SYMBOLIC MATH TOOLBOX, for performing computations using symbolic mathematics and variable precision arithmetic FILTER DESIGN TOOLBOX, for the design and analysis of advanced floating-point and fixed-point filters FUZZY LOGIC TOOLBOX, to support the design and analysis of fuzzy logic based systems. It supports all phases of the process, including development, research, design, simulation, and real-time implementation. It uses graphical user interfaces, GUIs, to provide an intuitive environment to guide the user through the steps of fuzzy inference system design. Functions are provided for many fuzzy logic methods, such as fuzzy clustering and adaptive neuro-fuzzy learning. Demo tutorials are provided for: Fuzzy Logic Controller in Simulink, Graphical Editors, Noise Cancellation, and Subtractive Clustering. Tutorial information is provided by Albert Oller i Pujol, Escola Tcnica Superior dEnginyeria (ETSE) at: www.etse.urv.es/~aoller/fuzzy/fuzzy_logic.htm#4. 226 Software Tools A tutorial based on the fuzzy logic toolbox is provided in PDF format by N. H. Koivo, Systeemitekniikan laboratorion kurssit. It is available at: www.control.hut.fi/Kurssit/AS-74.115/Material/fuzzy2.pdf IMAGE PROCESSING TOOLBOX, for performing image processing, analysis, and algorithm development INSTRUMENT CONTROL TOOLBOX, to control and communicate with test and measurement instruments LMI CONTROL TOOLBOX, to design robust controllers using convex optimization techniques MAPPING TOOLBOX, to analyze and visualize geographically based information MODEL PREDICTIVE CONTROL TOOLBOX, to control large, multivariable processes in the presence of constraints MODEL-BASED CALIBRATION TOOLBOX, to calibrate complex power-train systems -ANALYSIS AND SYNTHESIS TOOLBOX to design multivariable feedback controllers for systems with model uncertainty NEURAL NETWORK TOOLBOX, to provide tools for the design, implementation, visualization, and simulation of neural networks. It provides comprehensive support for many proven network paradigms, as well as a graphical user interface. Demo tutorials are available for Classification Using a Probabilistic Neural Network, Function Approximation with Radial Basis Networks, Introduction to the Neural Network Toolbox, and Signal Prediction OPTIMIZATION TOOLBOX, to solve standard and large-scale optimization problems PARTIAL DIFFERENTIAL EQUATION TOOLBOX, to solve and analyze partial differential equations ROBUST CONTROL TOOLBOX, to design robust multivariable feedback control systems SIGNAL PROCESSING TOOLBOX, to perform signal processing, analysis, and algorithm development STATISTICS TOOLBOX, to apply statistical algorithms and probability models SYMBOLIC MATH TOOLBOX, to perform computations using symbolic mathematics and variable precision arithmetic SYSTEM IDENTIFICATION TOOLBOX, to create linear dynamic models from measured input-output data 227 Chapter 9 VIRTUAL REALITY TOOLBOX, to animate and visualize Simulink systems in three-dimensions WAVELET TOOLBOX, to analyze, compress, and de-noise signals and images using wavelet techniques. There are even more commercially available toolboxes provided by a third party. In addition, and most importantly, there are toolboxes and M-files developed by top researchers and placed in the public domain. Moreover, an environment similar to MATLAB has also been developed and placed in the public domain. The following sections give a short overview of some of these public domain tools. GNU Octave www.octave.org/ GNU Octave is a high-level language similar to MATLAB but available freely. It was written by John W. Eaton with contributions from numerous other individuals listed on the Web site. A GNU Octave Repository is accessible at: http://octave.sourceforge.net/ It is a central location for custom scripts, functions and extensions for GNU Octave. Useful relevant files are also available from MatLinks/Chorus at: http://sourceforge.net/projects/matlinks This is an open source project that provides GNU Octave and MATLAB/Simulink toolboxes available in the public domain. NNSYSID Toolbox http://kalman.iau.dtu.dk/research/control/nnsysid.html This is a set of MATLAB tools for neural network based identification of nonlinear dynamic systems. The set is composed of M- and MEX-files for training and evaluation of multilayer perceptrons. There are functions for training of ordinary feedforward networks as well as for identification of nonlinear dynamic systems and for time-series analysis. The toolbox requires MATLAB 5.3 or higher, but it is completely independent of the Neural Network Toolbox and the System Identification Toolbox. SOM Toolbox for MATLAB www.cis.hut.fi/projects/somtoolbox/ SOM is a freeware package for the MATLAB environment. Numerous individuals have contributed to the SOM Toolbox. They are or were employed by the Laboratory of Information and Computer Science at the Helsinki University of Technology. The toolbox comes with several demos, including: 228 Software Tools som_demo1 som_demo2 som_demo3 som_demo4 SOM Toolbox demo 1: basic properties SOM Toolbox demo 2: basic usage SOM Toolbox demo 3: visualization SOM Toolbox demo 4: data analysis To illustrate some of the capabilities of the toolbox, illustrations from parts of the code provided in the first demo and the resulting output are shown in the following. som_demo1 A self-organized map, SOM, maps the training data. It consists of neurons located on a regular map grid. The lattice of the grid can be either hexagonal or rectangular. The diagram illustrated in Figure 9.17 results from running the following segment in the MATLAB environment: subplot(1,2,1) som_cplane(hexa,[10 15],none) title(Hexagonal SOM grid) subplot(1,2,2) som_cplane(rect,[10 15],none) title(Rectangular SOM grid) Figure 9.17: The generated SOM grid. Then, running the segment: subplot(1,3,1) som_grid(sMap) axis([0 11 0 11]), view(0,-90), title(Map in output space) subplot(1,3,2) plot(D(:,1),D(:,2),+r), hold on som_grid(sMap,Coord,sMap.codebook) title(Map in input space) results in the arrangement shown in Figure 9.18. 229 Chapter 9 Figure 9.18: The black dots show positions of map units (neurons), and the gray lines show the connections. The map was initialized randomly and hence the positions in the input space are completely disorganized. The crosses are training data. The training is based on: competitive learning, where the prototype vector most similar to a data vector is modified so that it becomes more similar to it, and cooperative learning, where not only the most similar prototype vector, but also its neighbors on the map are moved towards the data vector. This leads to the results shown in Figure 9.19. 230 Software Tools Figure 9.19: The map organizes and folds to the training data. M-files for Fuzzy Membership Optimization http://academic.csuohio.edu/simond/fuzzyopt/ This site makes available numerous M-files developed by Dan Simon, Department of Electrical Engineering, Cleveland State University, USA. They demonstrate fuzzy membership function optimization using gradient descent and Kalman filters. The task which is considered by these M-files is an automotive cruise control system. The M-files are downloadable in zip format. In addition, the site provides links to references in the area of optimization of fuzzy logic systems. The files include: VehicleControl.m, a fuzzy vehicle cruise control program. Running this file will simulate a fuzzy cruise controller for the situation where a vehicle suddenly goes from a road grade of 0% to 10%. One can change any of the details of the simulation (step size, simulation length, vehicle mass, road grade, etc.). Most of the parameters in the file are self-explanatory. Figure 9.20 shows an example of the result of running the file. 231 Chapter 9 Figure 9.20: An Example of the result obtained from running the file VehicleControl.m. FuzzCalc.m, a general-purpose fuzzy logic engine that computes defuzzified outputs corresponding to fuzzy inputs. This file can also compute the derivatives of the outputs with respect to the membership function parameters. This routine uses triangular membership functions. FuzzInit.m, initializes the rule base and membership functions for a fuzzy logic system. GradeCalc.m, a general-purpose routine that computes the membership grade of a number in a triangular fuzzy set. PlotMem.m, plots triangular membership functions based on the membership parameters stored in a file. VehicleGrad.m, optimizes a fuzzy cruise control system using gradient descent. VehicleKalman.m, optimizes a fuzzy cruise control system using a Kalman filter. Dmatrix.m, is an auxiliary routine that creates a matrix that is used when the optimization is carried out with some normal constraints. 232 Software Tools Engineering Applications of Soft Computing www.fmt.vein.hu/softcomp/software.html Numerous M-files provided by the Department of Process Engineering, Folyamatmrnki Tanszk. They relate to the work presented in: J. Abonyi and J.A. Roubos, M. Oosterom, F. Szeifert, Compact TS-Fuzzy Models through Clustering and OLS plus FIS Model Reduction, FUZZ-IEEE01, Sydney, Australia, 2001. J. Abonyi, R. Babuska, F. Szeifert, Fuzzy Modeling with Multidimensional Membership Functions: Constrained Identification and Control Design, IEEE Systems, Man and Cybernetics, Part B, Oct, 2001. J. Abonyi and R. Babuska and M. Ayala Botto and F. Szeifert and N. Lajos, Identification and Control of Nonlinear Systems Using Fuzzy Hammerstein Models, Industrial and Engineering Chemistry Research, 39, 4302-4314, 2000. Type-2 Fuzzy Logic Software http://sipi.usc.edu/~mendel/software/ This is a freeware collection of M-files provided by Nilesh N. Karnik, Qilian Liang and Jerry M. Mendel, University of Southern California. The collection has four sections: general type-2 fuzzy logic systems, interval type-2 fuzzy logic systems, type-1 fuzzy logic systems, and NEW type-reduction. NEFCON for MATLAB http://fuzzy.cs.Uni-Magdeburg.de/nefcon/nef_mat.html NEFCON is an implementation of a neural fuzzy controller based on a neuro-fuzzy controller using a fuzzy-perceptron. It requires a MATLAB/Simulink environment. NEFCON is able to learn fuzzy sets and rules by a reinforcement learning algorithm. The model was developed by the Fuzzy Systems research group at the Technical University of Braunschweig, Germany. The software is free for non-commercial use. It has online documentation with screen shots. MATLAB Software Tool for Neuro-Fuzzy Identification and Data Analysis http://iridia.ulb.ac.be/~gbonte/software/Local/FIS.html This software was developed by Gianluca Bontempi and Mauro Birattari, ULB Brussels, Belgium. The software trains a fuzzy architecture on the basis of a training set of N (single) output-(multi) input samples. The software is free. 233 Chapter 9 21. Motorola Motorola Fuzzy logic engine for the 68HC11 www.programmersheaven.com/zone5/cat26/1288.htm e-book from Motorola http://faculty.petra.ac.id/resmana/private/fuzzy/ Screen shots from the Motorola e-book are shown in Figures 9.21 and 9.22. Figure 9.21: Opening Screen of the Motorola Tutorial. 234 Software Tools Figure 9.22: Menu selection of the Motorola Tutorial. 22. NEFCLASS, Neurofuzzy Classification http://fuzzy.cs.uni-magdeburg.de/nefclass/nefclass.html NEFCLASS is a software provided by Detlef Nauck, Chief Research Scientist and Team Leader in the Intelligent Systems Lab of BTexact Technologies Advanced Research Department, Adastral Park, United Kingdom. It is freely available for scientific and personal use. The software is intended to be used for data analysis by neuro-fuzzy models. It can learn fuzzy rules and fuzzy sets by supervised learning. It can represent a fuzzy classification system, learn fuzzy classification rules incrementally, and learn fuzzy sets by using simple heuristics. 235 Chapter 9 23. Neural Networks and Fuzzy Systems Software fuzzy.cs.uni-magdeburg.de/software.html This site provides links to download numerous excellent software programs developed by the research group on Neural Networks and Fuzzy Systems at the Institute of Knowledge Processing and Language Engineering, Faculty of Computer Science, University of Magdeburg, Germany. The programs available relate to: Neuro-Fuzzy Control, Neuro-Fuzzy Function Approximation, Fuzzy Clustering, Neuro-Fuzzy Data Analysis, Multilayer Perceptron Training Visualization, Learning Vector Quantization Visualization, and Self-Organizing Map Training Visualization. For example, one of the available programs is the Multilayer Perceptron Training Visualization, named XMLP. The software helps to visualize the training process of a simple multi-layer perceptron for two logical and two simple real value functions. Figures 9.23 and 9.24 show screen shots from that program. Figure 9.23: Multilayer perceptron training. 236 Software Tools Figure 9.24: Illustration of some menu selections from XMLP. 24. NeuroDesigner www.neurodesigner.com/ NeuroDesigner is a comprehensive family of Java-based programs for neural network applications. A trial version of NeuroDesigner Editor is available for download. This version is operational for 30 days or 100 times, depending on usage. Moreover, the trial version has limited capabilities such as: the network cannot have more than 10 input and 25 output neurons, or have more than one hidden layer with four hidden neurons. Among the software capabilities highlighted are: embedding neural networks into existing systems and fetching data from various sources. 237 Chapter 9 25. Neuro Solutions www.nd.com/ NeuroSolutions is a graphical neural network development tool from NeuroDimension in Gainesville, Florida, USA. The Web page has links to a free trial version and a tour of the software complete with screen shots. The following are highlights from the tour. NeuroSolutions enables the user to create a neural network model for a given set of data. It supports numerous network architectures including: CANFIS Network (Fuzzy Logic), Multilayer Perceptron (MLP), Self-Organizing Map Network (SOM), Generalized Feed Forward, Generalized Recurrent Network , Learning Vector Quantization (LVQ), and more. The software has two separate wizards that enable the user to automatically build a neural network: NeuralExpert, where the design specifications are centered on the type of problem in which the user is interested, such as Classification, Prediction, Function Approximation, or Clustering, and NeuralBuilder, where the design specifications are centered on the specific neural network architecture the user wishes to construct. The software allows probes that provide real-time access to all internal network parameters and data, including inputs/outputs, errors, gradients, weights, etc. It also allows the user to optimize attributes such as the learning rates, weights, number of hidden neurons, etc. The Professional and Developer levels of NeuroSolutions allow the automatic generation of C++ source code for the neural network designed. 26. NICO Artificial Neural Network Toolkit www.speech.kth.se/NICO/index.html The NICO (Neural Inference COmputation) toolkit was developed by Nikko Strm at the Department for Speech, Music, and Hearing at KTH, Stockholm, Sweden. It is a general purpose toolkit for constructing artificial neural networks and training with the back-propagation learning algorithm. Units (neurons) are organized in groups and the group is a hierarchical structure; groups can have sub-groups or other objects as members. This makes it easy to specify multi-layer networks with arbitrary connection structure and to build modular networks. There is no support for viewing the networks graphically. 238 Software Tools 27. Rigel Corporation www.rigelcorp.com/8051soft.htm This site provides a variety of 8051 software. The software, help files, examples, and related text files are offered at no charge for non-commercial use. The software available includes Reads51, which is an integrated application software development system. It allows writing, compiling, assembling, debugging, downloading, and running applications software in the MCS-51 language. It contains a C-compiler, relative assembler, linker/locator, editor, chip simulator, assembly language debugger, and host-to-board communications in a menu-driven environment. Information about Fuzzy Logic LabPac is provided, which is a package for fuzzy logic...