MicroGrid: The Using Online Simulation to Predict Application Performance in Diverse Grid Network Environments
Huaxia Xia* , Holly Dail , Henri Casanova* , Andrew A. Chien*
*
Department of Computer Science and Engineering University of California at San Diego San Diego Supercomputer Center University of California at San Diego
Abstract
Improvements in networking and middleware technology are enabling large-scale grids that aggregate resources over wide-area networks to support applications at unprecedented levels of scale and performance. Unfortunately, existing middleware and tools provide little information to users as to the suitability of a given grid topology for a specic grid application. Instead, users generally use ad-hoc performance models to evaluate mappings of their applications to resource and network topologies. Grid application behavior alone is complex, and adding resource and network behavior makes the situation even worse. As a result, users typically employ nearly blind experimentation to nd good deployments of their applications in each new grid environment. Only through actual deployment and execution can a user discovers if the mapping was a good one. Further, even after nding a good conguration, there is no basis to determine if a much better conguration has been missed. This approach slows effective grid application development and deployment. We present a richer methodology for evaluating grid software and diverse grid environments based on the MicroGrid grid online simulator. With the MicroGrid, users, grid researchers, or grid operators can dene and simulate arbitrary collections of resources and networks. This allows study of an existing grid testbed under controlled conditions
This material is based upon work supported in part by the National Science Foundation under awards NSF EIA-9975020 Grads, NSF ACI-0103759, NSF Cooperative Agreement ANI-0225642 (OptIPuter), NSF CCR-0331645 (VGrADS), NSF ACI-0305390, and NSF Research Infrastructure Grant EIA-0303622. Support from HewlettPackard, BigBangwidth, Microsoft, and Intel is also gratefully acknowledged.
hxia,hdail,casanova,achien @cs.ucsd.edu
or even to study the efcacy of higher performance environments than are available today. Further, the MicroGrid supports direct execution of grid applications unchanged. These application can be written with MPI, C, C++, Perl, and/or Python and use the Globus middleware. This enables detailed and accurate study of application behavior. This paper presents: (1) the rst validation of the MicroGrid for studying whole-program performance of MPI Grid applications and (2) a demonstration of the MicroGrid as a tool for predicting the performance of applications on a range of grid resources and novel network topologies.
1. Introduction
Rapid improvements in the performance of wide-area networks and the pervasive deployment of commodity resources provide us grid computing infrastructures with tremendous potential [4]. Today, this potential is being widely tapped, and many grid middleware projects such as Globus [16], Condor [29], and NetSolve [1] have been pursued to provide uniform, secure and efcient access to remote resources. Unfortunately, there are a paucity of tools that assist a user in predicting if their application will obtain suitable performance on a particular platform. Instead, with little information on likely performance, users must invest signicant time to obtain accounts on the new grid, adapt their application to new middleware, debug their grid executions, and nally run experiments to determine the best deployment of their application on the new grid. At last the user knows if the blind date was a good one: did the application run efciently on the grid? If not, it is time to set up a new blind date and start over. Two major problems with this ap-
proach are that it is both labor and resource intensive, and does not provide any assurance of the quality of results. As resource environments and application performance structure continues to increase in complexity, it is likely the distance between achieved results and optimal will in...
clade04xia