gfs-sosp2003 - The Google File System Sanjay Ghemawat...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
The Google File System Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung Google ABSTRACT We have designed and implemented the Google File Sys- tem, a scalable distributed fle system ±or large distributed data-intensive applications. It provides ±ault tolerance while running on inexpensive commodity hardware, and it delivers high aggregate per±ormance to a large number o± clients. While sharing many o± the same goals as previous dis- tributed fle systems, our design has been driven by obser- vations o± our application workloads and technological envi- ronment, both current and anticipated, that reflect a marked departure ±rom some earlier fle system assumptions. This has led us to reexamine traditional choices and explore rad- ically di²erent design points. The fle system has success±ully met our storage needs. It is widely deployed within Google as the storage plat±orm ±or the generation and processing o± data used by our ser- vice as well as research and development e²orts that require large data sets. The largest cluster to date provides hun- dreds o± terabytes o± storage across thousands o± disks on over a thousand machines, and it is concurrently accessed by hundreds o± clients. In this paper, we present fle system inter±ace extensions designed to support distributed applications, discuss many aspects o± our design, and report measurements ±rom both micro-benchmarks and real world use. Categories and Subject Descriptors D[ 4 ]: 3— Distributed file systems General Terms Design, reliability, per±ormance, measurement Keywords Fault tolerance, scalability, data storage, clustered storage The authors can be reached at the ±ollowing addresses: { sanjay,hgobiof,shuntak } @google.com . Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for proFt or commercial advantage and that copies bear this notice and the full citation on the Frst page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior speciFc permission and/or a fee. SOSP’03, October 19–22, 2003, Bolton Landing, New York, USA. Copyright 2003 ACM 1-58113-757-5/03/0010 . .. $ 5.00. 1. INTRODUCTION We have designed and implemented the Google File Sys- tem (GFS) to meet the rapidly growing demands o± Google’s data processing needs. GFS shares many o± the same goals as previous distributed fle systems such as per±ormance, scalability, reliability, and availability. However, its design has been driven by key observations o± our application work- loads and technological environment, both current and an- ticipated, that reflect a marked departure ±rom some earlier fle system design assumptions. We have reexamined tradi- tional choices and explored radically di²erent points in the design space.
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 03/01/2011 for the course CSSE 250 taught by Professor Dr.yingwuzhu during the Spring '11 term at UH Clear Lake.

Page1 / 15

gfs-sosp2003 - The Google File System Sanjay Ghemawat...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online