: The design of a large-scale event notification
, Anne-Marie Kermarrec
, Miguel Castro
, and Peter Druschel
7 J J Thomson Avenue, Cambridge, CB3 0FB, UK.
Rice University MS-132, 6100 Main Street,
Houston, TX 77005-1892, USA.
This paper presents Scribe, a large-scale event notification infrastruc-
ture for topic-based publish-subscribe applications. Scribe supports large num-
bers of topics, with a potentially large number of subscribers per topic. Scribe is
built on top of Pastry, a generic peer-to-peer object location and routing substrate
overlayed on the Internet, and leverages Pastry’s reliability, self-organization and
locality properties. Pastry is used to create a topic (group) and to build an ef-
ficient multicast tree for the dissemination of events to the topic’s subscribers
(members). Scribe provides weak reliability guarantees, but we outline how an
application can extend Scribe to provide stronger ones.
Publish-subscribe has emerged as a promising paradigm for large-scale, Internet based
distributed systems. In general, subscribers register their interest in a topic or a pattern
of events and then asynchronously receive events matching their interest, regardless
of the events’ publisher. Topic-based publish-subscribe [1–3] is very similar to group-
based communication; subscribing is equivalent to becoming a member of a group. For
such systems the challenge remains to build an infrastructure that can scale to, and
tolerate the failure modes of the general Internet.
Techniques such as SRM (Scalable Reliable Multicast Protocol)  or RMTP (Re-
liable Message Transport Protocol)  have added reliability to network-level IP mul-
ticast [6, 7] solutions. However, tracking membership remains an issue in router-based
multicast approaches and the lack of wide deployment of IP multicast limits their ap-
plicability. As a result, application-level multicast is gaining popularity. Appropriate
algorithms and systems for scalable subscription management and scalable, reliable
propagation of events are still an active research area [8–11].
Recent work on peer-to-peer overlay networks offers a scalable, self-organizing,
fault-tolerant substrate for decentralized distributed applications [12–15]. Such systems
Appears in the proceedings of 3rd International Workshop on Networked Group Communica-
tion (NGC2001), UCL, London, UK, November 2001.