Hunt - ZooKeeper: Wait-free coordination for Internet-scale...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ZooKeeper: Wait-free coordination for Internet-scale systems Patrick Hunt and Mahadev Konar Yahoo! Grid { phunt,mahadev } @yahoo-inc.com Flavio P. Junqueira and Benjamin Reed Yahoo! Research { fpj,breed } @yahoo-inc.com Abstract In this paper, we describe ZooKeeper, a service for co- ordinating processes of distributed applications. Since ZooKeeper is part of critical infrastructure, ZooKeeper aims to provide a simple and high performance kernel for building more complex coordination primitives at the client. It incorporates elements from group messaging, shared registers, and distributed lock services in a repli- cated, centralized service. The interface exposed by Zoo- Keeper has the wait-free aspects of shared registers with an event-driven mechanism similar to cache invalidations of distributed file systems to provide a simple, yet pow- erful coordination service. The ZooKeeper interface enables a high-performance service implementation. In addition to the wait-free property, ZooKeeper provides a per client guarantee of FIFO execution of requests and linearizability for all re- quests that change the ZooKeeper state. These design de- cisions enable the implementation of a high performance processing pipeline with read requests being satisfied by local servers. We show for the target workloads, 2:1 to 100:1 read to write ratio, that ZooKeeper can handle tens to hundreds of thousands of transactions per second. This performance allows ZooKeeper to be used exten- sively by client applications. 1 Introduction Large-scale distributed applications require different forms of coordination. Configuration is one of the most basic forms of coordination. In its simplest form, con- figuration is just a list of operational parameters for the system processes, whereas more sophisticated systems have dynamic configuration parameters. Group member- ship and leader election are also common in distributed systems: often processes need to know which other pro- cesses are alive and what those processes are in charge of. Locks constitute a powerful coordination primitive that implement mutually exclusive access to critical re- sources. One approach to coordination is to develop services for each of the different coordination needs. For exam- ple, Amazon Simple Queue Service [3] focuses specif- ically on queuing. Other services have been devel- oped specifically for leader election [25] and configura- tion [27]. Services that implement more powerful prim- itives can be used to implement less powerful ones. For example, Chubby [6] is a locking service with strong synchronization guarantees. Locks can then be used to implement leader election, group membership, etc. When designing our coordination service, we moved away from implementing specific primitives on the server side, and instead we opted for exposing an API that enables application developers to implement their own primitives. Such a choice led to the implementa- tion of a coordination kernel that enables new primitives...
View Full Document

This note was uploaded on 12/08/2011 for the course CS 525 taught by Professor Gupta during the Spring '08 term at University of Illinois, Urbana Champaign.

Page1 / 14

Hunt - ZooKeeper: Wait-free coordination for Internet-scale...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online