Caching Web File System
Jonathan Ledlie Matt McCormick
Outline
Motivation - why design a new file system? Current state of affairs Design of web caching file system Performance comparison - WCFS to Unix Future work Conclusions
Two points
Optimizing on invariants Impending I/O bottleneck
Motivation
Disks are slow Communication rates increasing rapidly Web cache anomalies
only write to files when they are created permissions stay constant for files all files are have copies at original server
Web Caching File System vs. Unix File System
1.2 1 cfs create 0.8 unix cfs read unix cfs delete unix 0.2
Time
0.6
0.4
50,000 of each operation
0 cfs create unix cfs read Operation unix cfs delete unix
CFS is using one thread. Unix file I/O is synchronous 500 mHz PIII, 8.5 G disk
Current State of Affairs Internet Topology
Client Client Client Client Side Cache
Client Client Client Client Side Cache
N E T W O R K
Server Side Cache
S E R V E R
Current State of Affairs Unix File System
Life of file in web cache
create, write, close open, read, close (multiple times) delete lots of flexibility that is not needed extra access to disk for each file reference
Using i-nodes
Directory structure and name lookup
Design of WCFS Specializations
Life of file
create read (multiple times) delete faster create and file access
No i-nodes or permanent file status data In memory hash table stores file locations
faster file lookup and delete faster reads and writes
All file data written to consecutive blocks
Design of WCFS Object Diagram
CacheDisk getNewCacheObject
Disk
Cache
Cache
Cache
FileTable
RequestQueue Request Request Request
BitMap
Design of WCFS Disk Initialization
CacheDisk
Disk
First create cache disk object
creates disk object to represent physical disk starts a disk thread running utilize an SGI raw I/O patch for Linux bypass kernel and kernel buffers
Disk object and physical disk
Design of WCFS Disk Object
Disk
FileTable
RequestQueue
BitMap
FileTable stores names and locations of files on disk MD5 conversion of url RequestQueue stores read and write requests from process threads whenever anything in queue, disk thread runs BitMap keeps status of each block on disk locates and marks spot on disk for files to be placed
Design of WCFS Request Objects
RequestQueue Request Request Request
Request
write starting block, length, buffer to write from read starting block, length, buffer write to to (implies files must be smaller than virtual memory)
Currently queued by FIFO (soon to be one-way elevator)
Design of WCFS Cache Objects for Threading
CacheDisk
Cache
Cache
Cache
Multiple threads for handling clients Each thread gets a single Cache object Cache Object
Thread calls sync to guarantee its operations are done
create, read, remove, length, sync Thread create and read Asynchronous turned into request objects placed in request queue for disk
Design of CFS Code Snippet
Common web caching operations:
create("url", buffer, size); read("url", buffer); remove("url"); sync();
Equivalent Operations in Unix:
fd = creat("url", permissions); write(fd, buffer, size); close(fd); fd = open("url", mode); read(fd, buffer, size); close(fd); unlink("url");
Design of WCFS Basic File System Layout
CacheDisk getNewCacheObject
Disk
Cache
Cache
Cache
FileTable
RequestQueue Request Request Request
BitMap
Design of WCFS Feature Recap
Raw I/O Multi-threading Asynchronous I/O Quick name lookup File data on consecutive blocks
Time (usec) 1000 1500 2000 2500 3000 3500 4000 4500 500
0 0 600 1200 1800 2400 3000 3600 4200 4800 5400 6000 6600 7200 7800 8400 9000 9600 10200 10800 11400 12000 12600 13200 13800 14400 Operations
Performance Comparisons Trace
cfs ext2
Performance Comparisons Create
4500 4000 3500 3000 2500 2000 1500 1000 500 0
15 0 35 0 10 0 20 0 25 0 30 0 40 0 45 0 0 5...