DBO06 - ESTIMATING DIRECT RETURN ON INVESTMENT OF...

This preview shows page 1 - 2 out of 6 pages.

ESTIMATING DIRECT RETURN ON INVESTMENT OF INDEPENDENT VERIFICATION AND VALIDATION USING COCOMO-II James B. Dabney Systems Engineering Program University of Houston – Clear Lake Houston, Texas [email protected] Gary Barber and Don Ohi L3 Communications Titan Group NASA IV&V Facility Fairmont, West Virginia {gary.barber, don.ohi}@L-3com.com ABSTRACT We define direct return on investment (ROI) as the ratio of reduction in development cost arising from early issue detection by independent verification and validation (IV&V) to the cost of IV&V. This paper describes a methodology to compute direct ROI for projects that don’t maintain detailed cost-to-fix records. The method is used in a case study in which IV&V was applied to a mission-critical NASA software project. For this project, direct IV&V ROI was 11.8, demonstrating that IV&V was cost effective. KEYWORDS Verification and validation, return on investment, defect leakage, cost modeling. 1 Introduction A standard management measure for determining the worth of an investment is return on investment (ROI), also known as benefit/cost ratio [1]. For software independent verification and validation (IV&V) [2], [3], we believe that there are many benefits and therefore many components of ROI. For example, benefits include reduced development cost, increased confidence in the final product, improved quality, reduced risk, and improved safety. Unfortunately, all of these benefits are difficult to measure. Consequently, IV&V ROI is inherently difficult to calculate. Among these benefits, reduced development cost is the least difficult to quantify. We refer to ROI based solely on reduced development cost as direct ROI . A previous paper [4] presented a methodology to compute direct ROI for projects that maintain detailed records of the cost to fix each discovered defect. This paper extends the methodology to the more common (in our experience) situation in which detailed cost-to-fix records are not maintained. The method exploits the COCOMO-II [5] model, calibrated to actual project results, to estimate cost-to-fix. The method is illustrated using a case study from a mission-critical NASA software project. Additionally, the sensitivities of IV&V ROI to variations in IV&V scheduling and developer defect removal efficiency are studied. We define direct ROI as the ratio (C x -C i )/C IVV , where C x is the project cost without IV&V, C i is the project cost with IV&V, and therefore the difference δ C r = (C x -C i ) is reduction in development cost due to early issue identification by the IV&V team. C IVV is the cost of the IV&V effort. Cost can be expressed in any consistent unit. Typically, equivalent person-months (EPM) or equivalent person-hours (EPH) are convenient. The denominator of the ROI ratio is usually fairly easy to obtain from IV&V project records. The numerator, on the other hand, can only be estimated.

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture