ref_book.pdf - CONTENTS xvii PREFACE 1 2 1 INTRODUCTION 1.1...

This preview shows page 1 out of 726 pages.

Unformatted text preview: CONTENTS xvii PREFACE 1 2 1 INTRODUCTION 1.1 DEFINITION OF A DISTRIBUTED SYSTEM 2 1.2 GOALS 3 1.2.1 Making Resources Accessible 3 1.2.2 Distribution Transparency 4 1.2.3 Openness 7 1.2.4 Scalability 9 1.2.5 Pitfalls 16 1.3 TYPES OF DISTRIBUTED SYSTEMS 17 1.3.1 Distributed Computing Systems 17 1.3.2 Distributed Information Systems 20 1.3.3 Distributed Pervasive Systems 24 1.4 SUMMARY 30 33 ARCHITECTURES 2.1 ARCHITECTURAL STYLES 34 2.2 SYSTEM ARCHITECTURES 36 2.2.1 Centralized Architectures 36 2.2.2 Decentralized Architectures 43 2.2.3 Hybrid Architectures 52 2.3 ARCHITECTURES VERSUS MIDDLEWARE 54 2.3.1 Interceptors 55 2.3.2 General Approaches to Adaptive Software 57 2.3.3 Discussion 58 vii viii 3 CONTENTS 2.4 SELF-MANAGEMENT IN DISTRIBUTED SYSTEMS 59 2.4.1 The Feedback Control Model 60 2.4.2 Example: Systems Monitoring with Astrolabe 61 2.4.3 Example: Differentiating Replication Strategies in Globule 63 2.4.4 Example: Automatic Component Repair Management in Jade 65 2.5 SUMMARY 69 PROCESSES 3.1 3.2 3.3 THREADS 70 3.1.1 Introduction to Threads 70 3.1.2 Threads in Distributed Systems 75 VIRTUALIZATION 79 3.2.1 The Role of Virtualization in Distributed Systems 3.2.2 Architectures of Virtual Machines 80 CLIENTS 82 3.3.1 Networked User Interfaces 82 3.3.2 Client-Side Software for Distribution Transparency 3.4 SERVERS 88 3.4.1 General Design Issues 88 3.4.2 Server Clusters 92 3.4.3 Managing Server Clusters 98 3.5 CODE MIGRATION 1033.5.1 Approaches to Code Migration 103 3.5.2 Migration and Local Resources 107 3.5.3 Migration in Heterogeneous Systems 3.6 -4 66 SUMMARY 4.2 87 110 112 115 COMMUNICATION 4.1 79 FUNDAMENTALS 116 4.1.1 Layered Protocols 116 4.1.2 Types of Communication 124 REMOTE PROCEDURE CALL 125 4.2.1 Basic RPC Operation 126 4.2.2 Parameter Passing 130 ix CONTENTS 4.2.3 Asynchronous RPC 134 4.2.4 Example: DCE RPC 135 4.3 4.4 5 MESSAGE-ORIENTED COMMUNICATION 140 4.3.1 Message-Oriented Transient Communication 141 4.3.2 Message-Oriented Persistent Communication 145 4.3.3 Example: ffiM's WebSphere Message-Queuing System STREAM-ORIENTED COMMUNICATION ,4.4.1 Support for Continuous Media 158 4.4.2 Streams and Quality of Service 160 4.4.3 Stream Synchronization 163 157 4.5 MULTICAST COMMUNICATION 166 4.5 .1 Application-Level Multicasting 166 4.5.2 Gossip-Based Data Dissemination 170 4.6 SUMMARY 175 179 NAMING 5.1 NAMES, IDENTIFIERS, AND ADDRESSES 5.2 FLAT NAMING 182 5.2.1 Simple Solutions 183 5.2.2 Home-Based Approaches 1?6 5.2.3 Distributed Hash Tables 188 5.2.4 Hierarchical Approaches 191 5.3 STRUCTURED NAMING 195 5.3.1 Name Spaces 195 5.3.2 Name Resolution 198 5.3.3 The Implementation of a Name Space 202 5.3.4 Example: The Domain Name System 209 5.4 ATTRIBUTE-BASED NAMING 217 5.4.1 Directory Services 217 5.4.2 Hierarchical Implementations: LDAP 5.4.3 Decentralized Implementations 222 5.5 SUMMARY 152 180 218 x 6 7 CONTENTS 231 SYNCHRONIZATION 6.1 CLOCK SYNCHRONIZATION 232 6.1.1 Physical Clocks 233 6.1.2 Global Positioning System 236 6.1.3 Clock Synchronization Algorithms 238 6.2 LOGICAL CLOCKS 244 6.2.1 Lamport's Logical Clocks 244 6.2.2 Vector Clocks 248 6.3 MUTUAL EXCLUSION 252 6.3.1 Overview 252 6.3.2 A Centralized Algorithm 253 6.3.3 A Decentralized Algorithm 254 6.3.4 A Distributed Algorithm 255 6.3.5 A Token Ring Algorithm 258 6.3.6 A Comparison of the Four Algorithms 259 6.4 GLOBAL POSITIONING OF NODES 260 6.5 ELECTION ALGORITHMS 263 6.5.1 Traditional Election Algorithms 264 6.5.2 Elections in Wireless Environments 267 6.5.3 Elections in Large-Scale Systems 269 6.6 SUMMARY 270 CONSISTENCY AND REPLICATION 7.1 INTRODUCTION 274 7.1.1 Reasons for Replication 274 7.1.2 Replication as Scaling Technique 275 7.2 DATA-CENTRIC CONSISTENCY MODELS 276 7.2.1 Continuous Consistency 277 7.2.2 Consistent Ordering of Operations 281 7.3 CLIENT-CENTRIC CONSISTENCY MODELS 288 7.3.1 Eventual Consistency 289 7.3.2 Monotonic Reads 291 7.3.3 Monotonic Writes 292 7.3.4 Read Your Writes 294 7.3.5 Writes Follow Reads 295 273 - CONTENTS 8 7A REPLICA MAi'iAGEMENT 296 704.1 Replica-Server Placement 296 704.2 Content Replication and Placement 298 704.3 Content Distribution 302 7.5 CONSISTENCY PROTOCOLS 306 7.5.1 Continuous Consistency 306 7.5.2 Primary-Based Protocols 308 7.5.3 Replicated-Write Protocols 311 7.5 A Cache-Coherence Protocols 313 7.5.5 Implementing Client-Centric Consistency 315 7.6 SUMMARY 317 FAULT TOLERANCE 8.1 INTRODUCTION TO FAULT TOLERANCE 322 8.1.1 Basic Concepts 322 8.1.2 Failure Models 324 8.1.3 Failure Masking by Redundancy 326 8.2 PROCESS RESILIENCE 328 8.2.1 Design Issues 328 8.2.2 Failure Masking and Replication 330 8.2.3 Agreement in Faulty Systems 331 8.204 Failure Detection 335 8.3 RELIABLE CLIENT-SERVER COMMUNICATION 336 8.3.1 Point-to-Point Communication 337 8.3.2 RPC Semantics in the Presence of Failures 337 804 RELIABLE GROUP COMMUNICATION 343 804.1 Basic Reliable-Multicasting Schemes 343 804.2 Scalability in Reliable Multicasting 345 804.3 Atomic Multicast 348 8.5 DlSTRIBUTED COMMIT 355 8.5.1 Two-Phase Commit 355 8.5.2 Three-Phase Commit 360 8.6 RECOVERY 363 8.6.1 Introduction 363 8.6.2 Checkpointing 366 xi 321 xii CONTENTS 8.6.3 Message Logging 369 8.6.4 Recovery-Oriented Computing 372 8.7 9 10 SUMMARY 373 SECURITY 9.1 INTRODUCTION TO SECURITY 378 9.1.1 Security Threats, Policies, and Mechanisms 378 9.1.2 Design Issues 384 9.1.3 Cryptography 389 9.2 SECURE CHANNELS 396 9.2.1 Authentication 397 9.2.2 Message Integrity and Confidentiality 405 9.2.3 Secure Group Communication 408 9.2.4 Example: Kerberos 411 9.3 ACCESS CONTROL 413 9.3.1 General Issues in Access Control 414 9.3.2 Firewalls 418 9.3.3 Secure Mobile Code 420 9.3.4 Denial of Service 427 9.4 SECURITY MANAGEMENT 428 9.4.1 Key Management 428 9.4.2 Secure Group Management 433 9.4.3 Authorization Management 434 9.5 SUMMARY 439 DISTRIBUTED OBJECT-BASED SYSTEMS 10.1 ARCHITECTURE 443 10.1.1 Distributed Objects 444 10.1.2 Example: Enterprise Java Beans 446 10.1.3 Example: Globe Distributed Shared Objects 448 10.2 PROCESSES 451 10.2.1 Object Servers 451 10.2.2 Example: The Ice Runtime System 454 377 443 CONTENTS xiii 10.3 COMMUNICATION 456 10.3.1 Binding a Client to an Object 456 10.3.2 Static versus Dynamic Remote Method Invocations 458 10.3.3 Parameter Passing 460 10.3.4 Example: Java RMI 461 10.3.5 Object-Based Messaging 464 10.4 NAMING 466 10.4.1 CORBA Object References 467 10.4.2 Globe Object References 469 10.5 SYNCHRONIZATION 470 10.6 CONSISTENCY AND REPLICATION 472 10.6.1 Entry Consistency 472 10.6.2 Replicated Invocations 475 10.7 FAULT TOLERANCE 477 10.7.1 Example: Fault-Tolerant CORBA 477 10.7.2 Example: Fault-Tolerant Java 480 10.8 SECURITY 481 10.8.1 Example: Globe 482 10.8.2 Security for Remote Objects 486 10.9 SUMMARY 487 11 DISTRIBUTED FILE SYSTEMS 11.1 ARCHITECTURE 491 11.1.1 Client-Server Architectures 491 11.1.2 Cluster-Based Distributed File Systems 496 11.1.3 Symmetric Architectures 499 11.2 PROCESSES 501 11.3 COMMUNICATION 502 11.3.1 RPCs in NFS 502 11.3.2 The RPC2 Subsystem 503 11.3.3 File-Oriented Communication in Plan 9 505 11.4 NAMING 506 11.4.1 Naming in NFS 506 11.4.2 Constructing a Global Name Space 512 491 xiv CONTENTS 11.5 SYNCHRONIZATION 513 ] .5.] Semantics of File Sharing 513 1.5.2 File Locking 5] 6 1.5.3 Sharing Files in Coda 518 1.6 CONSISTENCY AND REPLICATION 5] 9 11.6.1 Client-Side Caching 520 11.6.2 Server-Side Replication 524 11.6.3 Replication in Peer-to-Peer File Systems 526 11.6.4 File Replication in Grid Systems 528 11.7 FAULT TOLERANCE 529 11.7.1 Handling Byzantine Failures 529 11.7.2 High Availability in Peer-to-Peer Systems 531 11.8 SECURITY 532 11.8.] Security in NFS 533 11.8.2 Decentralized Authentication 536 1] .8.3 Secure Peer-to-Peer File-Sharing Systems 539 11.9 SUMMARY 541 12 DISTRIBUTED WEB-BASED SYSTEMS 12.1 ARCHITECTURE 546 12.1.1 Traditional Web-Based Systems 546 12.1.2 Web Services 551 12.2 PROCESSES 554 12.2.1 Clients 554 12.2.2 The Apache Web Server 556 12.2.3 Web Server Clusters 558 12.3 COMMUNICATION 560 12.3.1 Hypertext Transfer Protocol 560 12.3.2 Simple Object Access Protocol 566 12.4 NAMING 567 12.5 SYNCHRONIZATION 569 12.6 CONSISTENCY AND REPLICATION 570 12.6.1 Web Proxy Caching 571 12.6.2 Replication for Web Hosting Systems 573 12.6.3 Replication of Web Applications 579 545 xv CONTENTS 12.7 FAULT TOLERANCE 13 12.8 SECURITY 584 12.9 SUMMARY 585 582 DISTRIBUTED COORDINATION-BASED SYSTEMS 13.1 INTRODUCTION TO COORDINATION MODELS 13.2 ARCHITECTURES 591 13.2.1 Overall Approach 592 13.2.2 Traditional Architectures 593 13.2.3 Peer-to-Peer Architectures 596 13.2.4 Mobility and 'Coordination 599 13.3 PROCESSES 13.4 COMMUNICATION 601 13.4.1 Content-Based Routing 601 13.4.2 Supporting Composite Subscriptions 603 NAMING 604 13.5.1 Describing Composite Events 604 13.5.2 Matching Events and Subscriptions 606 13.6 SYNCHRONIZATION 13.7 CONSISTENCY AND REPLICATION 13.7.1 Static Approaches 608 13.7.2 Dynamic Replication 611 13.8 FAULT TOLERANCE 613 13.8.1 Reliable Publish-Subscribe Communication 13.8.2 Fault Tolerance in Shared Dataspaces 616 607 SECURITY 617 13.9.1 Confidentiality 618 13.9.2 Secure Shared Dataspaces 13.10 SUMMARY -589 601 13.5 13.9 589 621 620 607 613 xvi 14 CONTENTS SUGGESTIONS FOR FURTHER READING 623 AND BIBLIOGRAPHY4.1 SUGGESTIONS FOR FURTHER READING 623 14.1.1 Introduction and General Works 6234.1.2 Architectures 624 14.1.3 Processes 625 14.1.4 Communication 626 14.1.5 Naming 626 14.1.6 Synchronization 627 14.1.7 Consistency and Replication 628 14.1.8 Fault Tolerance 629 14.1.9 Security 630 14.1.10 Distributed Object-Based Systems 631 14.1.11 Distributed File Systems 632 14.1.12 Distributed Web-Based Systems 632 14.1.13 Distributed Coordination-Based Systems 633 14,2 ALPHABETICAL BIBLIOGRAPHY 634 INDEX 669 PREFACE Distributed systems form a rapidly changing field of computer science. Since the previous edition of this book, exciting new topics have emerged such as peerto-peer computing and sensor networks, while others have become much more mature, like Web services and Web applications in general. Changes such as these required that we revised our original text to bring it up-to-date. This second edition reflects a major revision in comparison to the previous one. We have added a separate chapter on architectures reflecting the progress that has been made on organizing distributed systems. Another major difference is that there is now much more material on decentralized systems, in particular peer-to-peer computing. Not only do we discuss the basic techniques, we also pay attention to their applications, such as file sharing, information dissemination, content-delivery networks, and publish/subscribe systems. Next to these two major subjects, new subjects are discussed throughout the book. For example, we have added material on sensor networks, virtualization, server clusters, and Grid computing. Special attention is paid to self-management of distributed systems, an increasingly important topic as systems continue to scale. Of course, we have also modernized the material where appropriate. For example, when discussing consistency and replication, we now focus on consistency models that are more appropriate for modem distributed systems rather than the original models, which were tailored to high-performance distributed computing. Likewise, we have added material on modem distributed algorithms, including GPS-based clock synchronization and localization algorithms. xvii xviii PREFACE Although unusual. we have nevertheless been able to reduce the total number of pages. This reduction is partly caused by discarding subjects such as distributed garbage collection and electronic payment protocols, and also reorganizing the last four chapters. As in the previous edition, the book is divided into two parts. Principles of distributed systems are discussed in chapters 2-9, whereas overall approaches to how distributed applications should be developed (the paradigms) are discussed in chapters 10-13. Unlike the previous edition, however, we have decided not to discuss complete case studies in the paradigm chapters. Instead, each principle is now explained through a representative case. For example, object invocations are now discussed as a communication principle in Chap. 10 on object-based distributed systems. This approach allowed us to condense the material, but also to make it more enjoyable to read and study. Of course. we continue to draw extensively from practice to explain what distributed systems are all about. Various aspects of real-life systems such as WebSphere MQ, DNS, GPS, Apache, CORBA, Ice, NFS, Akamai, TIBlRendezvous. Jini, and many more are discussed throughout the book. These examples illustrate the thin line between theory and practice, which makes distributed systems such an exciting field. A number of people have contributed to this book in various ways. We would especially like to thank D. Robert Adams, Arno Bakker, Coskun Bayrak, Jacques Chassin de Kergommeaux, Randy Chow, Michel Chaudron, Puneet Singh Chawla, Fabio Costa, Cong Du, Dick Epema, Kevin Fenwick, Chandan a Gamage. Ali Ghodsi, Giorgio Ingargiola, Mark Jelasity, Ahmed Kamel, Gregory Kapfhammer, Jeroen Ketema, Onno Kubbe, Patricia Lago, Steve MacDonald, Michael J. McCarthy, M. Tamer Ozsu, Guillaume Pierre, Avi Shahar, Swaminathan Sivasubramanian, Chintan Shah, Ruud Stegers, Paul Tymann, Craig E. Wills, Reuven Yagel, and Dakai Zhu for reading parts of the manuscript, helping identifying mistakes in the previous edition, and offering useful comments. Finally, we would like to thank our families. Suzanne has been through this process seventeen times now. That's a lot of times for me but also for her. Not once has she said: "Enough is enough" although surely the thought has occurred to her. Thank you. Barbara and Marvin now have a much better idea of what professors do for a living and know the difference between a good textbook and a bad one. They are now an inspiration to me to try to produce more good ones than bad ones (AST). Because I took a sabbatical leave to update the book, the whole business of writing was also much more enjoyable for Marielle, She is beginning to get used to it, but continues to remain supportive while alerting me when it is indeed time to redirect attention to more important issues. lowe her many thanks. Max and Elke by now have a much better idea of what writing a book means, but compared to what they are reading themselves, find it difficult to understand what is so exciting about these strange things called distributed systems. I can't blame them (MvS). 1 INTRODUCTION , Computer systems are undergoing a revolution. From 1945, when the modem c;omputerera began, until about 1985, computers were large and expensive. Even minicomputers cost at least tens of thousands of dollars each. As a result, most organizations had only a handful of computers, and for lack of a way to connect them, these operated independently from one another. Starting around the the mid-1980s, however, two advances in technology began to change that situation. The first was the development of powerful microprocessors. Initially, these were 8-bit machines, but soon 16-, 32-, and 64-bit CPUs became common. Many of these had the computing power of a mainframe (i.e., large) computer, but for a fraction of the price. The amount of improvement that has occurred in computer technology in the past half century is truly staggering and totally unprecedented in other industries. From a machine that cost 10 million dollars and executed 1 instruction per second. we have come to machines that cost 1000 dollars and are able to execute 1 billion instructions per second, a price/performance gain of 1013.If cars had improved at this rate in the same time period, a Rolls Royce would now cost 1 dollar and get a billion miles per gallon. (Unfortunately, it would probably also have a 200-page manual telling how to open the door.) The second development was the invention of high-speed computer networks. Local-area networks or LANs allow hundreds of machines within a building to be connected in such a way that small amounts of information can be transferred between machines in a few microseconds or so. Larger amounts of data can be 1 2 INTRODUCTION CHAP. moved between machines at rates of 100 million to 10 billion bits/sec. Wide-area networks or WANs allow miJIions of machines all over the earth to be connected at speeds varying from 64 Kbps (kilobits per second) to gigabits per second. The result of these technologies is that it is now not only feasible, but easy, to put together computing systems composed of large numbers of computers connected by a high-speed network. They are usually caned computer networks or distributed systems, in contrast to the previous centralized systems (or singleprocessor systems) consisting of a single computer, its peripherals, and perhaps some remote terminals. 1.1 DEFINITION OF A DISTRIBUTED SYSTEM Various definitions of distributed systems have been given in the literature, none of them satisfactory, and none of them in agreement with any of the others. For our purposes it is sufficient to give a loose characterization: A distributed system is a collection of independent computers that appears to its users as a single coherent system. This definition has several important aspects. The first one is that a distributed system consists of components (i.e., computers) that are autonomous. A second aspect is that users (be they people or programs) think they are dealing with a single system. This means that one way or the other the autonomous components need to collaborate. How to establish this collaboration lies at the heart of developing distributed systems. Note that no assumptions are made concerning the type of computers. In principle, even within a single system, they could range from high-performance mainframe computers to small nodes in sensor networks. Likewise, no assumptions are made on the way that computers are interconnected. We will return to these aspects later in this chapter. Instead of going further with definitions, it is perhaps more useful to concentrate on important characteristics of distributed systems. One important characteristic is that differences between the various computers and the ways in which they communicate are mostly hidden from users. The same holds for the internal organization of the distributed system. Another important characteristic is that users and applications can interact with a distributed system in a consistent and uniform way, regardless of where and when interaction takes place. In principle, distributed systems should also be relatively easy to expand or scale. This characteristic is a direct consequence of having independent computers, but at the same time, hiding how these computers actually take part in the system as a whole. A distributed system will normally be continuously available, although perhaps some parts may be temporarily out of order. Users and applications should not notice that parts are being replaced or fixed, or that new parts are added to serve more users or applications .. SEC. 1.1 DEFINITION OF A DISTRIBUTED SYSTEM 3 In order to support heterogeneous computers and networks while offering a single-system view, distributed systems are often organized by means of a layer of software-that is, logically placed between a higher-level layer consisting of users and applications, and a layer underneath consisting of operating systems and basic communication facilities, as shown in Fig. 1-1 Accordingly, such a distributed system is sometimes called middleware. Figure I-I. A distributed system organized as middleware. The middleware layer extends over multiple machines, and offers each application the same interface. Fig. 1-1 shows four networked computers and three applications, of which application B is distributed across computers 2 and 3. Each application is offered the same interface. The distributed system provides the means for compone...
View Full Document

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture