end-to-end - End-To-End Arguments in System Design J H...

Info icon This preview shows pages 1–12. Sign up to view the full content.

Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

Image of page 2
Image of page 3

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

Image of page 4
Image of page 5

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

Image of page 6
Image of page 7

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

Image of page 8
Image of page 9

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

Image of page 10
Image of page 11

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

Image of page 12
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: End-To-End Arguments in System Design J. H. SALTZER, D. P. REED, and D. D. CLARK Massachusetts Institute of Technology Laboratory for Computer Science This paper presents a design principle that helps guide placement of functions among the modules of a distributed computer system. The principle, called the end-to-end argument, suggests that functions placed at low levels of a system may be redundant or of little value when compared with the cost of providing them at that low level. Examples discussed in the paper include bit-error recovery, security using encryption, duplicate message suppression, recovery from system crashes, and delivery acknowl- edgment. Low-level mechanisms to support these functions are justified only as performance enhance- ments. CR Categories and Subject Descriptors: C.0 [General] Computer System Organization—system architectures; 0.2.2 [Computer-Communication Networks}: Network Protocols—protocol archi- tecture; C.2.4 [Computer-Communication Networks]: Distributed Systems; D.4.7 [Operating Systems]: Organization and Design—distributed systems General Terms: Design Additional Key Words and Phrases: Data communication, protocol design, design principles 1. INTRODUCTION Choosing the proper boundaries between functions is perhaps the primary activity of the computer system designer. Design principles that provide guidance in this choice of function placement are among the most important tools of a system designer. This paper discusses one class of function placement argument that has been used for many years with neither explicit recognition nor much convic- tion. However, the emergence of the data communication network as a computer system component has sharpened this line of function placement argument by making more apparent the situations in which and the reasons why it applies. This paper articulates the argument explicitly, so as to examine its nature and to see how general it really is. The argument appeals to application requirements and provides a rationale for moving a function upward in a layered system closer to the application that uses the function. We begin by considering the commu- nication network version of the argument. This is a revised version of a paper adapted from End-to-End Arguments in System Design by J. H. Saltzer, D.P. Reed, and DD. Clark from the 2nd International Conference on Distributed Systems (Paris, France, April 8'10) 1981, pp. 509—512. © IEEE 1981 This research was supported in part by the Advanced Research Projects Agency of the US. Department of Defense and monitored by the Office of Naval Research under contract N00014-75- C-0661. Authors’ addres: J. H. Saltzer and D. D. Clark, M.I.T. Laboratory for Computer Science, 545 Technology Square, Cambridge, MA 02139. D. P. Reed, Software Arts, Inc., 27 Mica Lane, Wellesley, MA 02181. Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee and/or specific permission. © 1984 ACM 0734-2071/84/1100-0277 $00.75 ACM Transactions on Computer Systems, Vol. 2, No. 4, November 1984, Pages 277—288. 278 - J. H. Seltzer, D. P. Reed, and D. D. Clark In a system that includes communications, one usually draws a modular boundary around the communication subsystem and defines a firm interface between it and the rest of the system. When doing so, it becomes apparent that there is a list of functions each of which might be implemented in any of several ways: by the communication subsystem, by its client, as a joint venture, or perhaps redundantly, each doing its own version. In reasoning about this choice, the requirements of the application provide the basis for the following class of arguments: The function in question can completely and correctly be implemented Only with the knowledge and help of the application standing at the endpoints of the communication system. Therefore, providing that questioned function as a feature of the communication system itself is not possible. (Sometimes an incomplete version of the function provided by the communication system may be useful as a performance enhancement.) We call this line of reasoning against low-level function implementation the end-to-end argument. The following sections examine the end-to-end argument in detail, first with a case study of a typical example in which it is used—the function in question is reliable data transmission—and then by exhibiting the range of functions to which the same argument can be applied. For the case of the data communication system, this range includes encryption, duplicate mes- sage detection, message sequencing, guaranteed message delivery, detecting host crashes, and delivery receipts. In a broader context, the argument seems to apply to many other functions of a computer operating system, including its file system. Examination of this broader context will be easier, however, if we first consider the more specific data communication context. 2. CAREFUL FILE TRANSFER 2.1 End-to-End Caretaking Consider the problem of careful file transfer. A file is stored by a file system in the disk storage of computer A. Computer A is linked by a data communication network with computer B, which also has a file system and a disk store. The object is to move the file from computer A’s storage to computer B’s storage without damage, keeping in mind that failures can occur at various points along the way. The application program in this case is the file transfer program, part of which runs at host A and part at host B. In order to discuss the possible threats to the file’s integrity in this transaction, let us assume that the following specific steps are involved: (1) At host A the file transfer program calls upon the file system to read the file from the disk, where it resides on several tracks, and the file system passes it to the file transfer program in fixed-size blocks chosen to be disk format independent. (2) Also at host A, the file transfer program asks the data communication system to transmit the file using some communication protocol that involves splitting the data into packets. The packet size is typically different from the file block size and the disk track size. ACM Transactions on Computer Systems, Vol. 2, No. 4, November 1984. End-to—End Arguments in System Design - 279 (3) The data communication network moves the packets from computer A to computer B. (4) At host B, a data communication program removes the packets from the data communication protocol and hands the contained data to a second part of the file transfer application that operates within host B. (5) At host B, the file transfer program asks the file system to write the received data on the disk of host B. With this model of the steps involved, the following are some of the threats to the transaction that a careful designer might be concerned about: (1) The file, though originally written correctly onto the disk at host A, if read now may contain incorrect data, perhaps because of hardware faults in the disk storage system. (2) The software of the file system, the file transfer program, or the data communication system might make a mistake in buffering and copying the data of the file, either at host A or host B. (3) The hardware processor or its local memory might have a transient error while doing the buffering and copying, either at host A or host B. (4) The communication system might drop or change the bits in a packet or deliver a packet more than once. (5) Either of the hosts may crash part way through the transaction after performing an unknown amount (perhaps all) of the transaction. How would a careful file transfer application then cope with this list of threats? One approach might be to reinforce each of the steps along the way using duplicate copies, time-out and retry, carefully located redundancy for error detection, crash recovery, etc. The goal would be to reduce the probability of each of the individual threats to an acceptably small value. Unfortunately, systematic countering of threat (2) requires writing correct programs, which is quite difficult. Also, not all the programs that must be correct are written by the file transfer—application programmer. If we assume further that all these threats are relatively low in probability—low enough for the system to allow useful work to be accomplished—brute force countermeasures, such as doing everything three times, appear uneconomical. The alternate approach might be called end-to-end check and retry. Suppose that as an aid to coping with threat (1), stored with each file is a checksum that has sufficient redundancy to reduce the chance of an undetected error in the file to an acceptably negligible value. The application program follows the simple steps above in transferring the file from A to B. Then, as a final additional step, the part of the file transfer application residing in host B reads the transferred file copy back from its disk storage system into its own memory, recalculates the checksum, and sends this value back to host A, where it is compared with the checksum of the original. Only if the two checksums agree does the file transfer application declare the transaction committed. If the comparison fails, something has gone wrong, and a retry from the beginning might be attempted. If failures are fairly rare, this technique iwill normally work on the first try; occasionally a second or even third try might be required. One would probably consider two or more failures on the same file transfer attempt as indicating that some part of this system is in need of repair. ACM Transactions on Computer Systems, Vol. 2, No. 4, November 1984. 280 - J. H. Saltzer, D. P. Reed,and D. D.Clark Now let us consider the usefulness of a common proposal, namely, that the communication system provide, internally, a guarantee of reliable data transmis- sion. It might accomplish this guarantee by providing selective redundancy in the form of packet checksums, sequence number checking, and internal retry mechanisms, for example. With sufficient care, the probability of undetected bit errors can be reduced to any desirable level. The question is whether or not this attempt to be helpful on the part of the communication system is useful to the careful file transfer application. The answer is that threat (4) may have been eliminated, but the careful file transfer application must still counter the remaining threats; so it should still provide its own retries based on an end-to—end checksum of the file. If it does, the extra effort expended in the communication system to provide a guarantee of reliable data transmission is only reducing the frequency of retries by the file transfer application; it has no effect on inevitability or correctness of the outcome, since correct file transmission is ensured by the end-to-end checksum and retry whether or not the data transmission system is especially reliable. Thus, the argument: In order to achieve careful file transfer, the application program that performs the transfer must supply a file-transfer-specific, end-to- end reliability guarantee—in this case, a checksum to detect failures and a retry- commit plan. For the data communication system to go out of its way to be extraordinarily reliable does not reduce the burden on the application program to ensure reliability. 2.2 A Too-Real Example An interesting example of the pitfalls that one can encounter turned up recently at the Massachusetts Institute of Technology. One network system involving several local networks connected by gateways used a packet checksum on each hop from one gateway to the next, on the assumption that the primary threat to correct communication was corruption of bits during transmission. Application programmers, aware of this checksum, assumed that the network was providing reliable transmission, without realizing that the transmitted data were unpro- tected while stored in each gateway. One gateway computer developed a transient error: while copying data from an input to an output buffer a byte pair was interchanged, with a frequency of about one such interchange in every million bytes passed. Over a period of time many of the source files of an operating system were repeatedly transferred through the defective gateway. Some of these source files were corrupted by byte exchanges, and their owners were forced to the ultimate end-to-end error check: manual comparison with and correction from old listings. 2.3 Performance Aspects However, it would be too simplistic to Conclude that the lower levels should play no part in obtaining reliability. Consider a network that is somewhat unreliable, dropping one message of each hundred messages sent. The simple strategy outlined above, transmitting the file and then checking to see that the file has arrived correctly, would perform more poorly as the length of the file increased. The probability that all packets of a file arrive correctly decreases exponentially ACM Transactions on Computer Systems, Vol. 2, No. 4, November 1984. End-to—End Arguments in System Design - 281 with the file length, and thus the expected time to transmit the file grows exponentially with file length. Clearly, some effort at the lower levels to improve network reliability can have a significant effect on application performance. But the key idea here is that the lower levels need not provide “perfect” reliability. Thus the amount of effort to put into reliability measures within the data communication system is seen to be an engineering trade-off based on perform- ance, rather than a requirement for correctness. Note that performance has several aspects here. If the communication system is too unreliable, the file transfer application performance will suffer because of frequent retries following failures of its end-to-end checksum. If the communcation system is beefed up with internal reliability measures, those measures also have a performance cost, in the form of bandwidth lost to redundant data and added delay from waiting for internal consistency checks to complete before delivering the data. There is little reason to push in this direction very far, when it is considered that the end- to-end check of the file transfer application must still be implemented no matter how reliable the communication system becomes. The proper trade-off requires careful thought. For example, one might start by designing the communication system to provide only the reliability that comes with little cost and engineering effort, and then evaluate the residual error level to ensure that it is consistent with an acceptable retry frequency at the file transfer level. It is probably not important to strive for a negligble error rate at any point below the application level. Using performance to justify placing functions in a low-level subsystem must be done carefully. Sometimes, by examining the problem thoroughly, the same or better performance enhancement can be achieved at the high level. Performing a function at a low level may be more efficient, if the function can be performed with a minimum perturbation of the machinery already included in the low-level subsystem. But the opposite situation can occur—that is, performing the function at the lower level may cost more—for two reasons. First, since the lower level subsystem is common to many applications, those applications that do not need the function will pay for it anyway. Second, the low-level subsystem may not have as much information as the higher levels, so it cannot do the job as efficiently. Frequently, the performance trade-off is quite complex. Consider again the careful file transfer on an unreliable network. The usual technique for increasing packet reliability is some sort of per-packet error check with a retry protocol. This mechanism can be implemented either in the communication subsystem or in the careful file transfer application. For example, the receiver in the careful file transfer can periodically compute the checksum of the portion of the file thus far received and transmit this back to the sender. The sender can then restart by retransmitting any portion that has arrived in error. The end-to-end argument does not tell us where to put the early checks, since either layer can do this performance-enhancement job. Placing the early retry protocol in the file transfer application simplifies the communication system but may increase overall cost, since the communication system is shared by other applications and each application must now provide its own reliability enhance- ment. Placing the early retry protocol in the communication system may be more ACM Transactions on Computer Systems, Vol. 2, No. 4, November 1984. 282 - J. H. Saltzer, D. P. Reed, and D. D. Clark efficient, since it may be performed inside the network on a hop-by-hop basis, reducing the delay involved in correcting a failure. At the same time there may be some application that finds the cost of the enhancement is not worth the result, but it now has no choice in the matter.1 A great deal of information about system implementation is needed to make this choice intelligently. 3. OTHER EXAMPLES OF THE END-TO-END ARGUMENT 3.1 Delivery Guarantees The basic argument that a lower level subsystem that supports a distributed application may be wasting its effort in providing a function that must, by nature, be implemented at the application level anyway can be applied to a variety of functions in addition to reliable data transmission. Perhaps the oldest and most widely known form of the argument concerns acknowledgment of delivery. A data communication network can easily return an acknowledgment to the sender for every message delivered to a recipient. The ARPANET, for example, returns a packet known as Request For Next Message (RFNM) [1] whenever it delivers a message. Although this acknowledgment may be useful within the network as a form of congestion control (originally the ARPANET refused to accept another message to the same target until the previous RFNM had returned), it was never found to be very helpful for applications using the ARPANET. The reason is that knowing for sure that the message was delivered to the target host is not very important. What the application wants to know is whether or not the target host acted on the message; all manner of disaster might have struck after message delivery but before completion of the action requested by the message. The acknowledgment that is really desired is an end-to-end one, which can be originated only by the target application—“I did it,” or “I didn’t.” Another strategy for obtaining immediate acknowledgments is to make the target host sophisticated enough that when it accepts delivery of a message it also accepts responsibility for guaranteeing that the message is acted upon by the target application. This approach can eliminate the need for an end-to-end acknowledgment in some, but not all, applications. An end-to-end acknowledg- ment is still required for applications in which the action requested of the target host should be done only if similar actions requested of other hosts are successful. This kind of application requires a two-phase commit protocol [5, 10, 15], which is a sophisticated end-to-end acknowledgment. Also, if the target application either fails or refuses to do the requested action, and thus a negative acknowl- edgment is a possible outcome, an end-to-end acknowledgment may still be a requirement. 3.2 Secure Transmission of Data Another area in which an end-to-end argument can be applied is that of data encryption. The argument here is threefold. First, if the data transmission system perfoms encryption and decryption, it must be trusted to securely mana...
View Full Document

  • Fall '08
  • MuhamedMudawar

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern