{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

spaper02 - International Journal of Parallel and...

Info iconThis preview shows pages 1–13. Sign up to view the full content.

View Full Document Right Arrow Icon
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Background image of page 2
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Background image of page 4
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Background image of page 6
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Background image of page 8
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Background image of page 10
Background image of page 11

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Background image of page 12
Background image of page 13
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: International Journal of Parallel and Distributed Systems and Networks, Vol. 4, No. 2, 2001 a SUPPORTING MULTIMEDIA COMMUNICATION OVER A GIGABIT ETHERNET NETWORK B. Daines,‘ J.C.L. Lin," and K. Sivalingam‘“ f5. hstract The need for gigabit networks is driven by the explosive growth of the internet and the World Wide Web. Applications such as streaming multimedia, virtual reality, high-performance distributed :oz‘riputing, and distance learning will all drive the bandwidth M-tm‘ements into the multi-gigabit range. This article introduces w. Joint effort to support multimedia communication using a set It first describes the ilavrlware/firmware design on the gigabit network interface card, of gigabit Ethernet network components. :-. buffered hub, then a gigabit routing switch for providing the backbone. The switch can route packets based on layer 2 MAC addresses, layer 3 (IP or IPX) addresses, and layer 4 and above policy-based schemes. The design principles common to these different components are: (1) simplicity, to ensure the network ' imponents reach the performance beyond the gigabit speed, and (2) ., lvto-end solutions, to eventually integrate the gigabit component 2.31.1 the emerging gigabit network. With the accomplished network hardware/firmware, we bring in our unique multimedia delivery software to investigate the support for multimedia applications over the gigabit Ethernet network. The main challenges of retrieving streaming videos are large data size, real-time constraint, and supporting concurrent accesses. Performance measurements using netperf benchmark, our own TOP-based benchmark, and video server applications are presented in the article. The results indicate a promising result with maximum throughput of 190—200 Mbps achieved at the TCP level using Linux (even on low-end Pentium machines). With faster PCI buses and more powerful processors such as UltraSPARC and DEC Alpha, higher TCP throughput in the range of 400 to 500 Mbps was achieved. The video server experiments indicated that eight to nine simultaneous video streams at 16 Mbps each can be handled by the server. 1. Introduction The motivation for gigabit network speeds can hardly be overstated. The need for gigabit networks is driven by the ‘ World Wide Packets, Veradale, WA 99037 USA ” Department of Computer, Information Science and Engineer- ing, University of Florida, Gainesville, FL 32611 USA; e—mail jcliu©cise.ufl.edu “‘ School of Electrical Engineering and Computer Science, Wash- ington State University, Pullman, WA 99164 USA (paper no. 204-0333) 102 explosive growth of the Internet and the World Wide Web. There have been a number of gigabit research testbeds built in the past few years, some of which are described in [1]. Gigabit networks are beginning to leave the research domain and enter the enterprise network domain. The national backbone network speeds are ranging between 155 Mbps and 622 Mbps, as in the case of vBNS [2]. It will not be long before the wide—area backbone networks move to gigabit speeds. The increasing multimedia content of web traffic, among others reasons, will ensure that the gigabit bandwidth is significantly utilized. Local area network bandwidth is following the same trend. Applications such as streaming multimedia, virtual reality, high-performance distributed computing, and dis- tance learning will all drive the bandwidth requirements into the multi-gigabit range. For instance, a video server can generate 2.88 Gbps traffic from the storage subsystems [3]. In addition, the nature of traffic distribution at the enterprise network has changed. Traditionally, 80% of the trafiic traversed local subnets, and only 20% traversed the enterprise backbone. Now the situation has reversed, with an increasing percentage of traffic traversing the backbone. It is essential to provide gigabit speeds to both the individ- ual workgroups and subnets, and the enterprise backbone. This article introduces our joint effort to support mul- timedia communication using a set of gigabit Ethernet net- work components. It first describes the hardware/ firmware design on the gigabit network interface card, a. buffered hub, then a gigabit routing switch for providing the back— bone. These three hardware/firmware components are critical for the gigabit Ethernet deployment. We foresee that the first step is the deployment of small “power” or pilot workgroups within a local area network (LAN). This requires the development of gigabit interface cards, point— to—point optical links, and a gigabit hub. Examples of such workgroups include specialized research laboratories (like ours). Therefore, the development of multimedia and high-speed file server environment can be provided within the LAN. The next step is the integration of these servers through gigabit switches and aggregation of these gigabit switches. Thus, the deployment of gigabit capability in the building and campus backbones (i.e., MAN) can be accomplished .1 There are many design issues and challenges in the hardware / firmware level to achieve high—perforrnance Eth- ernet that can perform beyond the gigabit speeds. We present the common challenges faced in MAC level design, among these different network components. Gigabit net— work interface cards (GNIC) is presented in Section 2.2. The particular design issues that we discussed include (1) 802.3 frame compatibility, (2) design challenge of the MAC ASIC to operate the GNIC at the line speed, and (3) tech— niques for reducing host CPU utilization with descriptor- based direct memory access (DMA) and interrupt filtering. Section 2.3 presents the gigabit hubs that will scale up the number of hosts that can communicate with each other. Unique design features of the buffered gigabit hub include (1) full—duplex for eliminating the CSMA/ CD collision is- sues, (2) congestion control to avoid frame dropping, and (3) round—robin scheduling to prevent “packet clumping.” The design of a gigabit routing switch is then presented in Section 2.4 with design descriptions on (1) architecture is- sues, (2) novel parallel access shared memory architecture, and (3) priority queue design. The switch can route pack— ets based on layer 2 MAC addresses, layer 3 (IP or IPX) addresses, and layer 4 and above policy-based schemes. The common design principles across these different components are ( 1) simplicity, to ensure the network com- ponents reach the performance beyond the gigabit speed, and (2) end~to—end solutions, to eventually integrate the gigabit component into the emerging gigabit network. We extend the design considerations for supporting emerging distributed multimedia applications. The main character- istics of supporting concurrent streaming videos are intro- duced. Then the multimedia-friendly features associated with the different network components are described. Re— lated new IEEE draft standards (e.g., 802.1p and 802.1Q) are introduced as an ongoing investigation. Possible ap- proaches are considered in port-level and box-level to even— tually ensure an end-to—end solution for distributed multi- media applications. Gigabit Ethernet testbeds have been established at University of Florida and Washington State University. One Pentium Pro and one Pentium 11 computer, each equipped with a GNIC card, are connected via an optical link. A testbed connecting up to four clients and a server using the gigabit hub (FDR) has been set up. Performance measurements using netperf [4] benchmark, [5] benchmark, our own TCP—based benchmark, and video server applica— tions are presented. The results indicate that a maximum throughput of 200 Mbps was achieved at the TCP level using Linux on Pentium. With faster PCI buses and more powerful processors such as UltraSPARC and DEC Alpha, higher throughput in the range of 400 to 500 Mbps was achieved. The video server experiments indicated that eight to nine high—quality streaming videos at 16 Mbps each can he handled by the server over the gigabit Ethernet network. \ 1 The University of Florida has recently adopted the gigabit Ethernet as the campus backbone. 2. Hardware /Firmware Design Considerations We begin this section by addressing network interfaces required to support gigabit speeds. Before introducing the network interface design, it is first necessary to describe the IEEE 802.32 standard. 2.1 IEEE 802.3z Standard The IEEE 802.3z committee made the standard the MAC and PHY (physical) layers for Gigabit Ethernet. The PHY interface is defined for fiber operating at 1000 Mbps— 1000 BASE-8X and 1000 BASE-LX. The 1000 BASE—SX standard operates at 800 nm, uses multi-mode fiber [6], and is defined for a maximum link length of 260m at 62.5 pm and 440m at 50 ,um core diameter. The 1000 BASE- LX standard for single—mode fiber [6] operates at 1300 nm and can traverse a maximum link length of 3Km. Typically, single-mode fiber is more expensive than multi- mode fiber. In addition, a copper-based physical interface called 1000 BASE—CX using twinax cable has been defined for a distance of 25m. The MAC frame format is shown in Fig. 1. <—46_15m ————-—> r IDLE flan-I DATA I mamflfl- SA 7 l 6 5 1 C 11 7 l I MINIMUM FRAME (64 m) lFG .1616 Minimum pneumonia-l) 7mm a byln pun-mu: m sun in: .096u5 1.; [and inner: l byI: Minimum fnmexuhym Muimumfnmc=1518by1u IPG=lmaPncl¢|Gup-O.96us=l2byvn-3doch:31.15m-lz lPG:lnlflaneGlp= 0.16as220bylulsdockISSIlsMflz Figure 1. IEEE 802.3z medium access control frame for- mat. The frame format defined for 1 Gbps Ethernet is iden- tical to that defined in IEEE 802.3 for 10 Mbps Ethernet and IEEE 802.3u for 100 Mbps Ethernet. Born a timing perspective, the Inter Frame Gap, which is the equiva— lent of 16 bytes, is reduced to 0.16ps, compared to 1.6,us in 100 Mbps Ethernet and 16 [is in 10 Mbps Ethernet. This places a stricter requirement on synchronization and preamble collision. The frame consists of the following fields: DA, SA: The destination and sender fields, each repre sented using the IEEE 802.3 48-bit MAC address format. LEN: The length of the data field, in bytes—2 bytes allocated for this field. DATA: Variable length data field with a minimum of 46 bytes and a maximum of 1500 bytes. FCS: Frame check sum requiring 4 bytes. With the common frame standard introduced, we will introduce the detailed design for the gigabit GNIC in the next subsection. W_,MA, 2.2 Gigabit Network Interface Card This subsection describes the Gigabit Network Interface Card (GNIC) we have developed. The original GNIC is designed to operate with the 64-bit, 33-Mhz PCI bus and the SBus from SUN Microsystems. The GNIC is also com— patible with current 32-bit, 33—Mhz PCI buses. The next generation card, denoted GNIC—II, is designed to operate with 64-bit, 66-Mhz PCI buses such as those available on Ultra-30 workstations from SUN Microsystems. The theoretical bandwidth of a 32—bit, 33—Mhz PCI bus is about 1 Gbps; practical limits are around 800 Mbps. Similarly, the theoretical and practical limits of the 64-bit, 66 Mhz PCI bus are around 4 Gbps and 3 Gbps, respectively. This is an indication that high-end systems are currently capable of handling gigabit speeds. As processors and buses get more MIPS and bandwidth, more systems can avail themselves of the network’s gigabit speeds.2 The GNIC consists of an application—specific integrated circuit (ASIC) chip, packet buffer memory (512 KBytes), serializer/deserializer drip, and physical layer components. One important feature of the GNIC is that its data—path is designed to operate at line speeds to utilize the potential gigabit bandwidths from the 32-bit and 64-bit PCI buses. Figure 2. Block diagram of a gigabit network interface card. The block diagram of the ASIC chip is shown in Fig. 2. The ASIC chip has a PCI bus interface to the host bus. On the other side of the bus interface, a pair of DMA (direct memory access) controllers—one each for transmit and receive—are provided to reduce host CPU utilization. A dual—burst FIFO of 512 bytes is provided—one each for Transmit and Receive. This memory is interconnected to the on-card memory through an external FIFO interface. The GMAC ASIC block implements the IEEE 802.32 stan— dard and the 802.3x full-duplex flow control. The GMAC is a single entity and is used on a number of other giga- bit interface cards. The GMAC is connected to the seri- alizer/deserializer, the fiber transceiver, and the physical interface (not shown in figure). 2 We are in the process of investigating the next-level design of 10—Gbps Ethernet networks. 104 One of the main design objectives was to reduce the host CPU utilization, and a number of optimizing tech- niques have been used. Some of them are described below: 0 The PCI Bus Master in a PCI 32- or 64—bit bus op- erating at 33 Mhz accesses host memory directly to offload the host CPU. On—board Independent Receive (RX) and Transmit (TX) Descriptor-Based DMA Pro- cessors (DBDMA) reduce host utilization by streaming RX and TX data to and from host memory without host intervention. A descriptor is similar to a CPU instruction that is executed by the DMA processors. 0 Transmit chaining reduces host CPU utilization by transferring an arbitrary number of packets from host memory to the GNIC without host intervention. 0 The GNIC will not interrupt the host when the host is currently servicing packets for transmission or recep- tion. This allows the host to adapt the interrupt rate to network load. An enhanced data path is provided on the ASIC to achieve gigabit speeds. The dual—burst FIFO—one each for RX and TX—allows simultaneous transfer of data from the host memory to the PCI FIFOs and from the PCI FIFOs to the wire (and vice versa). Also, bursting multiple descriptors reduces PCI overhead, as the PCI protocol overhead is incurred each time the master arbitrates the bus. Hence, the more data that are burst over the bus for each arbitration, the higher the efficiency of bus bandwidth usage. The provision of large packet buffers on the card, of size between 512 KB and 2 MB, reduces the probability of dropped packets during extended 1 Gbps bursts. This buffer is partitioned for RX and TX purposes. For example, when the buffer size is 512 MB, it is partitioned so that the RX buffer varies from a minimum of 480 KB to a maximum of 512 KB. On the other hand, the TX buffer is limited to a maximum of 32 KB. The rationale is to accommodate bursts during reception without dropping packets. We have observed a sustained throughput up to 800 Mbps achieved in the hardware/firmware level using this GNIC (excluding the protocol stack and operating system overhead). Without our unique design, a network inter— face card for gigabit communication might occupy all the host CPU utilization, making it unfeasible to achieve the sustained 800 Mbps throughput. With our GNIF design, the efficiency of the CPU utilization has been improved 6.8 times compared to conventional Ethernet network interface cards. 2.3 Full Duplex Repeater for LAN Interconnection of more than one GNIC card is accom- plished by the Full Duplex Repeater (FDR) considered in this subsection. As the FDR provides the MAC function- ality, we first discuss the current Ethernet environment, and a solution to provide medium access at gigabit speeds. In most current 10/100 Mbps Ethernet networks, the network interface cards (NICs) are connected to each other via a hub. The network uses the Carrier Sense Multi- ple Access / Collision Detection (CSMA / CD) mechanism to Table 1 Comparison of Key Features of Typical Hub, FDR, and Switch Complexity Full/Half—Duplex Low Network Diameter Performance CSMA/CD Collisions MAC Checks Frames Store & Forward Buffers Yes No N o N o No Congestion Control Fair Bandwidth Allocation COST provide shared access to the medium. The performance of CSMA / CD is highly dependent on the ratio of end-to- end propagation delay (time for electro—magnetic waves to travel in the medium) to the average packet transmission time. Let us denote this ratio as a. The higher the value . of a, lower the efficiency and overall utilization of the net- work, and vice versa [7]. The performance dependence on a occurs mainly because of the time required to sense carrier and detect collisions. Let us consider a 10 Mbps Ethernet with a 2 Km span and 64-byte minimum packet length. The ratio a z 0.2 does not cause significant performance degradation. How- ever, when the speed is increased to 1 Gbps, the value a = 20 for the same distance span, which significantly decreases performance [7]. Thus, increasing transmission speed by itself does not necessarily imply higher perfor- mance. There are two ways to combat the performance degradation: fncrease the minimum packet length, or de— crease the distance span. As the IEEE 802.32 standard maintains the minimum packet length of 64 bytes, the network distance has to be reduced to 200 m to maintain the lower value of a. A technique called virtual collisions has been proposed [8] that increases the network distance to 400 In. For a typical hub, multiple transmissions arriving simultaneously results in all packets being lost. In the virtual collision solution, the hub is able to correctly broadcast exactly one of the collided packets. Thus, despite collision, one packet can be successfully transmitted. An alternative to the CSMA/ CD shared medium architecture is to use a switched medium such as Switched Ethernet. Although switching significantly improves performance, it is associated with higher costs. Therefore, we prefer an intermediate between fully shared and fully switched solution. The Full Duplex Repeater (FDR) uses a distinct ap— proach to overcome the limitations of CSMA/ CD and at the same time achieve better utilization and fairness of service. The term repeater is actually a misnomer, as the FDR operates at the data—link layer and processes MAC and link—level frames. A repeater, however, operates at the physical layer and is not aware of link-level frames. The FDR Switch Half—Duplex (CSMA/CD) 200 m (CSMA/CD dependent) PHY dependent < 1000 Mbps 105 Moderate Full-Duplex High F ull—Duplex PHY dependent > 1000 Mbps No Yes Yes Yes Scheduling Alg. Dependent High 1000 Mbps N 0 Yes Yes Yes Yes Low FDR architecture combines switching and shared design concepts to achieve switch-like performance while main- taining the cost and ease-of—use advantages of a shared hub. The FDR provides maximum throughput with full—duplex ports, collisionless frame forwarding, and congestion con- trol. A fair scheduling algorithm (currently round-robin) is used to fairly allocate forwarding bandwidth to all the ports. A comparison shown in Table 1 highlights the differ— ences between a typical hub, the FDR, and a switch. The FDR architecture, illustrated in Fig. 3, consists of multiple ports based on the IEEE 802.32 Gigabit Ethernet standard, a 1000 Mbps forwarding bus, and a frame for— warder. Each of the ports includes a 802.32 physical layer device (PHY), an 802.32 media access controller (MAC), and buffers for incoming and outgoing frames, and uses a 802.3x flow-control logic. The forwarding bus is controlled by a flame forwarder. Forwarding Bus W xv- Figure 3. Architecture of the full duplex repeater. To illustrate how frames travel through an FDR, Fig. 4 shows the logical flow of frames passing through the phys- ical architecture: input, forwarding path, and output. An incoming frame enters through Port 1. During the input stage, the frame passes through the PHY and the MAC, and is then queued in the buffer on Port 1, which is controlled by the frame forwarder. The frame then moves into the forwarding path stage, when the frame forwarder selects the buffer on Port 1. The frame forwarder then con- 802.3x Flow-Control Frame \ Port 1 802.32 Phy/MAC I Pon 2 802.32 I’M/ME ------------- Port 3 802.32 Ply/ME ............... Forwarding Path Input Output Figure 4. An example of frame passing in the full duplex repeater. figures this port for transmission onto the bu...
View Full Document

{[ snackBarMessage ]}