sit - UsenetDHT: A low-overhead design for Usenet Emil Sit,...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
UsenetDHT: A low-overhead design for Usenet Emil Sit, Robert Morris, and M. Frans Kaashoek MIT CSAIL Abstract Usenet is a popular distributed messaging and ±le sharing ser- vice: servers in Usenet ²ood articles over an overlay network to fully replicate articles across all servers. However, replication of Usenet’s full content requires that each server pay the cost of receiving (and storing) over 1 Tbyte/day. This paper presents the design and implementation of UsenetDHT, a Usenet system that allows a set of cooperating sites to keep a shared, distributed copy of Usenet articles. UsenetDHT consists of client-facing Usenet NNTP front-ends and a distributed hash table (DHT) that provides shared storage of articles across the wide area. This design allows participating sites to partition the storage burden, rather than replicating all Usenet articles at all sites. UsenetDHT requires a DHT that maintains durability despite transient and permanent failures, and provides high storage per- formance. These goals can be dif±cult to provide simultane- ously: even in the absence of failures, verifying adequate repli- cation levels of large numbers of objects can be resource inten- sive, and interfere with normal operations. This paper intro- duces Passing Tone, a new replica maintenance algorithm for DHash [7] that minimizes the impact of monitoring replication levels on memory and disk resources by operating with only pairwise communication. Passing Tone’s implementation pro- vides performance by using data structures that avoid disk ac- cesses and enable batch operations. Microbenchmarks over a local gigabit network demonstrate that the total system throughput scales linearly as servers are added, providing 5.7 Mbyte/s of write bandwidth and 7 Mbyte/s of read bandwidth per server. UsenetDHT is currently deployed on a 12-server network at 7 sites running Passing Tone over the wide-area: this network supports our research laboratory’s live 2.5 Mbyte/s Usenet feed and 30.6 Mbyte/s of synthetic read traf- ±c. These results suggest a DHT-based design may be a viable way to redesign Usenet and globally reduce costs. 1 Introduction For decades, the Usenet service has connected users world-wide. Users post articles into newsgroups which are propagated widely by an overlay network of servers. Users host lively discussions in newsgroups and also, be- cause articles can represent multi-media ±les, coopera- tively produce a large shared pool of ±les. A major at- This research was supported by the National Science Founda- tion under Cooperative Agreement No. ANI-0225660, http:// project-iris.net/ . traction of Usenet is the incredible diversity and volume of content that is available. Usenet is highly popular and continues to grow: one Usenet provider sees upwards of 40,000 readers reading at an aggregate 20 Gbit/s [35]. Several properties contribute to Usenet’s popularity. Because Usenet’s design [1, 19] aims to replicate all articles to all interested servers, any Usenet user can publish highly popular content without the need to personally provide a server and bandwidth.
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 14

sit - UsenetDHT: A low-overhead design for Usenet Emil Sit,...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online