See discussions, stats, and author profiles for this publication at: A Survey on Load Testing of Large-Scale Software SystemsArticleinIEEE Transactions on Software Engineering · November 2015DOI: 10.1109/TSE.2015.2445340CITATIONS41READS1,8922 authors, including:Some of the authors of this publication are also working on these related projects:Bounties on Technical Q&A Sites: A Case Study of Stack Overflow BountiesView projectMining Online Gaming StoresView projectAhmed E. HassanQueen's University388PUBLICATIONS9,847CITATIONSSEE PROFILEAll content following this page was uploaded by Ahmed E. Hassan on 19 July 2016.The user has requested enhancement of the downloaded file.
A Survey on Load Testing of Large-ScaleSoftware SystemsZhen Ming Jiang,Member, IEEEand Ahmed E. Hassan,Member, IEEEAbstract—Many large-scale software systems must service thousands or millions of concurrent requests. These systems must beload tested to ensure that they can function correctly under load (i.e., the rate of the incoming requests). In this paper, we survey thestate of load testing research and practice. We compare and contrast current techniques that are used in the three phases of a loadtest: (1) designing a proper load, (2) executing a load test, and (3) analyzing the results of a load test. This survey will be useful for loadtesting practitioners and software engineering researchers with interest in the load testing of large-scale software systems.Index Terms—Software testing, load testing, software quality, large-scale software systems, surveyÇ1INTRODUCTIONMANYlarge-scale systems ranging from e-commercewebsites to telecommunication infrastructures mustsupport concurrent access from thousands or millions ofusers. Studies show that failures in these systems tend to becaused by their inability to scale to meet user demands,as opposed to feature bugs , . The failure to scaleoften leads to catastrophic failures and unfavorable mediacoverage (e.g., the meltdown of the Firefox website ,thebotchedlaunchofApple’sMobileMeandUSGovernment’s Health Care Website ). To ensure the qual-ity of these systems, load testing is a required testingprocedure in addition to conventional functional testingprocedures, like unit testing and integration testing.This paper surveys the state of research and practices inthe load testing of large-scale software systems. This paperwill be useful for load testing practitioners and softwareengineering researchers with interests in testing and analyz-ing large-scale software systems. Unlike functional testing,where we have a clear objective (pass/fail criteria), loadtesting can have one or more functional and non-functionalobjectives as well as different pass/fail criteria. As illus-trated in Fig. 1, we propose the following three researchquestions on load testing based on the three phases of tradi-tional software testing (test design, test execution and testanalysis ):1)How is a proper load designed?TheLoad Designphase