mazieres - USENIX Association Proceedings of the 2001...

USENIX Association
Proceedings of the
2001 USENIX Annual
Technical Conference
Boston, Massachusetts, USA
June 25–30, 2001
THE ADVANCED COMPUTING SYSTEMS ASSOCIATION
© 2001 by The USENIX Association All Rights Reserved For more information about the USENIX Association:
Phone: [phone] FAX: [phone] Email: [email protected] WWW: http://www.usenix.org
Rights to individual papers remain with the author or the author's employer.
Permission is granted for noncommercial reproduction of the work for educational or research purposes.
This copyright notice must be included in the reproduced paper. USENIX acknowledges all trademarks herein.

A toolkit for user-level file systems
David Mazi`eres
Department of Computer Science, NYU
[email protected]
Abstract
This paper describes a C++ toolkit for easily extending
the Unix file system. The toolkit exposes the NFS in-
terface, allowing new file systems to be implemented
portably at user level. A number of programs have im-
plemented portable, user-level file systems. However,
they have been plagued by low-performance, deadlock,
restrictions on file system structure, and the need to re-
boot after software errors. The toolkit makes it easy to
avoid the vast majority of these problems. Moreover, the
toolkit also supports user-level access to existing file sys-
tems through the NFS interface—a heretofore rarely em-
ployed technique. NFS gives software an asynchronous,
low-level interface to the file system that can greatly ben-
efit the performance, security, and scalability of certain
applications. The toolkit uses a new asynchronous I/O
library that makes it tractable to build large, event-driven
programs that never block.
1 Introduction
Many applications could reap a number of benefits from
a richer, more portable file system interface than that of
Unix. This paper describes a toolkit for portably ex-
tending the Unix file system—both facilitating the cre-
ation of new file systems and granting access to existing
ones through a more powerful interface. The toolkit ex-
ploits both the client and server sides of the ubiquitous
Sun Network File System [15]. It lets the file system
developer build a new file system by emulating an NFS
server. It also lets application writers replace file system
calls with networking calls, permitting lower-level ma-
nipulation of files and working around such limitations
as the maximum number of open files and the synchrony
of many operations.
We used the toolkit to build the SFS distributed file
system [13], and thus refer to it as the SFS file system
development toolkit. SFS is relied upon for daily use
by several people, and thus shows by example that one
can build production-quality NFS loopback servers. In
addition, other users have picked up the toolkit and built
functioning Unix file systems in a matter of a week. We
have even used the toolkit for class projects, allowing
students to build real, functioning Unix file systems.