cluster04-rfs - RFS: Efcient and Flexible Remote File...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
RFS: Efficient and Flexible Remote File Access for MPI-IO Jonghyun Lee *‡ Xiaosong Ma †§ Robert Ross * Rajeev Thakur * Marianne Winslett * Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, IL 60439, U.S.A. Department of Computer Science, North Carolina State University, Raleigh, NC 27695, U.S.A. Department of Computer Science, University of Illinois, Urbana, IL 61801, U.S.A. § Computer Science and Mathematics Division, Oak Ridge National Laboratory, Oak Ridge, TN 37381, U.S.A. { jlee, rross, thakur }, [email protected], [email protected] Abstract Scientific applications often need to access remote file systems. Because of slow networks and large data size, however, remote I/O can become an even more serious per- formance bottleneck than local I/O performance. In this work, we present RFS, a high-performance remote I/O fa- cility for ROMIO, which is a well-known MPI-IO imple- mentation. Our simple, portable, and flexible design elim- inates the shortcomings of previous remote I/O efforts. In particular, RFS improves the remote I/O performance by adopting active buffering with threads (ABT), which hides I/O cost by aggressively buffering the output data using available memory and performing background I/O using threads while computation is taking place. Our experimen- tal results show that RFS with ABT can significantly reduce the remote I/O visible cost, achieving up to 92% of the theoretical peak throughput. The computation slowdown caused by concurrent I/O activities was 0.2–6.2%, which is dwarfed by the overall performance improvement in ap- plication turnaround time. 1 Introduction The emergence of fast processors and high-bandwidth, low-latency interconnects has made high-performance commodity-based clusters widely available. These clusters are gaining popularity as an affordable, yet powerful, paral- lel platform compared to commercial supercomputers, be- cause of the clusters’ excellent cost-performance ratio. For many computational scientists, clusters are an attractive op- tion for running parallel scientific codes that require a large number of computing resources. Scientific applications are typically I/O intensive. For example, most simulation codes periodically write out the intermediate simulation data to local secondary storage as snapshots for future time-dependent visualization or analy- sis. Checkpoint files also need to be written in case of sys- tem crash or application failure. Many visualization tools read large amounts of data from disks into memory for vi- sualization. Despite recent improvements in disk perfor- mance, local I/O performance is still a serious performance bottleneck for these data-intensive applications. Many re- search efforts have addressed this slow I/O problem through utilizing I/O parallelism [5, 11, 12, 17, 20]. In addition to their local I/O needs, scientific simulations
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 11/12/2011 for the course CE 726 taught by Professor Staf during the Spring '11 term at SUNY Buffalo.

Page1 / 11

cluster04-rfs - RFS: Efcient and Flexible Remote File...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online