Measuring software product quality

Measuring software product quality - Pittsburgh, PA...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Pittsburgh, PA 15213-3890 Measuring Software Product Quality: the ISO 25000 Series and CMMI European SEPG June 14, 2004 Dave Zubrow Sponsored by the U.S. Department of Defense © 2004 by Carnegie Mellon University Carnegie Mellon Software Engineering Institute Objectives Provide status on a new Software Product Quality Measurement standard and its connection to CMMI Provide ideas on how to get started with Software Product Quality Measurement today page 2 © 2004 by Carnegie Mellon University Carnegie Mellon Software Engineering Institute Outline Background and Overview Concepts and Models Software Product Quality Measurement Summary page 3 © 2004 by Carnegie Mellon University Carnegie Mellon Software Engineering Institute Achieving Quality Software Requires planning and intentional design More than achieving the desired functionality Must explicitly attend to both functional and non-functional requirements Need to verify all requirements are being met throughout the life cycle page 4 © 2004 by Carnegie Mellon University Carnegie Mellon Software Engineering Institute CMMI Definition for Quality Requirements The phrase “quality and process-performance objectives” covers objectives and requirements for product quality, service quality, and process performance. Process performance objectives include product quality. page 5 © 2004 by Carnegie Mellon University Carnegie Mellon Software Engineering Institute Requirements Development This process area describes three types of requirements: • customer requirements (quality in use) • product requirements (external quality attributes) • product-component requirements (internal quality attributes) Taken together, these requirements address the needs of relevant stakeholders, including those pertinent to various product life-cycle phases (e.g., acceptance testing criteria) and product attributes (e.g., safety, reliability, maintainability). Requirements also address constraints caused by the selection of design solutions (e.g., integration of commercial off-the-shelf products). page 6 © 2004 by Carnegie Mellon University Carnegie Mellon Software Engineering Institute Requirements Development Goals SG 1 Develop Customer Requirements Stakeholder needs, expectations, constraints, and interfaces are collected and translated into customer requirements. SG 2 Develop Product Requirements Customer requirements are refined and elaborated to develop product and productcomponent requirements. SG 3 Analyze and Validate Requirements The requirements are analyzed and validated, and a definition of required functionality is developed. page 7 © 2004 by Carnegie Mellon University Carnegie Mellon Software Engineering Institute Process Management and Performance The organization’s process needs and objectives cover aspects that include the following: • characteristics of the processes • process performance objectives, such as time to market and product quality • process effectiveness A quantitatively managed process is institutionalized by doing the following: • controlling the process using statistical and other quantitative techniques such that product quality, service quality, and process performance attributes are measurable and controlled throughout the project (internal and external quality measures and criteria) page 8 © 2004 by Carnegie Mellon University Carnegie Mellon Software Engineering Institute Key Points in Relationship of CMMI and ISO 9126/25000 - 1 CMMI takes a total life cycle view and is inclusive in its approach to requirements development. Requirements development explicitly seeks to have the developer consider quality requirements. Project and Process Management processes explicitly consider product quality as process performance objectives. Neither the standard nor CMMI endorses a unidimensional view of quality. page 9 © 2004 by Carnegie Mellon University Carnegie Mellon Software Engineering Institute Key Points in Relationship of CMMI and ISO 9126/25000 - 2 Product Quality Requirements are transformed into designs and implemented via the Technical Solution and Product Integration process areas. The implementation of Product Quality Requirements are monitored and confirmed via the Project Management, Verification, and Validation process areas. CMMI acknowledges the need for interaction and perhaps iteration among the related process areas to satisfactorily identify, specify, and address Product Quality Requirements. page 10 © 2004 by Carnegie Mellon University Carnegie Mellon Software Engineering Institute Relating Requirements, Evaluation, and Measurement Customer User Development Organization provide Acquirer Evaluator Developer guidance Product Quality Requirements determines guidance perform Product Quality Evaluation supports supports Product Quality Measurement page 11 © 2004 by Carnegie Mellon University Carnegie Mellon Software Engineering Institute Outline Background and Overview Concepts and Models Software Product Quality Measurement Summary page 12 © 2004 by Carnegie Mellon University Carnegie Mellon Software Engineering Institute SQuaRE: Architecture ISO/IEC 2503n ISO/IEC 2503n Quality Quality Requirement Requirement Division Division ISO/IEC 2501n ISO/IEC 2501n Quality Model Quality Model Division Division ISO/IEC 2500n ISO/IEC 2500n Product Quality Product Quality General Division General Division General Overview and General Overview and Guide to the SQuaRE Guide to the SQuaRE Planning and Planning and Management Management ISO/IEC 2502n ISO/IEC 2502n Quality Metrics Quality Metrics Division Division © 2004 by Carnegie Mellon University ISO/IEC 2504n ISO/IEC 2504n Quality Quality Evaluation Evaluation Division Division page 13 Carnegie Mellon Software Engineering Institute Needs and Requirements Stakeholders’ Needs in their Minds Select & Specify Solicit & Identify Stated, Implied or Unaware Needs Selected and Specified Needs & QIU Requirements Collected and Identified Stakeholders’ (Business) Needs Functional Requirements External Quality Requirements Functional Design & Internal Quality Requirements Non Functional Design & Internal Quality Requirements Internal and External Quality Requirements may be stated in coding standards, project quality goal statements, process descriptions (e.g., exit criteria), test case descriptions, etc. They need not be explicitly identified as requirements. page 14 © 2004 by Carnegie Mellon University Carnegie Mellon Software Engineering Institute The Product Quality Measurement Reference Model page 15 © 2004 by Carnegie Mellon University Carnegie Mellon Software Engineering Institute Quality In Use Model (ISO/IEC 9126) Quality In Use Quality In Use Effectiveness Effectiveness Productivity Productivity Safety Safety Satisfaction Satisfaction page 16 © 2004 by Carnegie Mellon University Carnegie Mellon Software Engineering Institute Internal and External Software Quality Model (ISO/IEC 9126) Quality Characteristics •Functionality Suitability Subcharacteristics Accuracy Interoperability Security Compliance •Reliability Maturity Fault tolerance Recoverability Compliance •Usability Understandability Learnability Operability Comp Attractiveness •Efficiency Time behavior Resource utilization Compliance •Maintainability Analyzability Changeability Stability Testability Compliance •Portability Adaptability Installability Co-existence Replaceability Comp page 17 © 2004 by Carnegie Mellon University Carnegie Mellon Software Engineering Institute Outline Background and Overview Concepts and Models Software Product Quality Measurement Summary page 18 © 2004 by Carnegie Mellon University Carnegie Mellon Software Engineering Institute Quality Model Elements and Measurement Model Elements Conceptual Model Characteristic Operationalization represents One or more produce comprise Subcharacteristic represents Quality Measure One or more produce comprise Attribute Quality Measure represents Measurement Primitive page 19 © 2004 by Carnegie Mellon University Carnegie Mellon Software Engineering Institute Relating the Quality and Measurement Models Software Product Quality Quality Evaluation Report Assessment Analysis Rating Quality Quality Characteristics Characteristics Quality Quality Sub Characteristics -characteristics Quality Measures Function (Formula) Measurement Primitives Quality Quality Attribute Attributes Measurement Method Quality Attribute © 2004 by Carnegie Mellon University page 20 Carnegie Mellon Software Engineering Institute Relating the Quality Measurement Model to the ISO Software Measurement Process (15939) Quality Evaluation Report Information Product Assessment Analysis Rating Interpretation Indicator Quality Measures Analysis 25030 25040 Analysis Model Function (Formula) Measurement Primitives Measurement Method Quality Attribute © 2004 by Carnegie Mellon University Definition & Measurement Function Collection 2502n Base Measures Derived Measures Measurement Method Attribute page 21 Carnegie Mellon Software Engineering Institute CMMI Measurement & Analysis Process Area Goals Align Measurement and Analysis Activities Provide Measurement Results Institutionalize a Managed Process page 22 © 2004 by Carnegie Mellon University Carnegie Mellon Software Engineering Institute Activities for Goal 1 Align Measurement and Analysis Activities • Establish Measurement Objectives • Specify Measures • Specify Data Collection and Storage Procedures • Specify Analysis Procedures Note: The first two practices directly address the need to translate from the conceptual to the operational. page 23 © 2004 by Carnegie Mellon University Carnegie Mellon Software Engineering Institute Activities for Goal 2 Provide Measurement Results • Collect Measurement Data • Analyze Measurement Data • Store Data and Results • Communicate Results page 24 © 2004 by Carnegie Mellon University Carnegie Mellon Software Engineering Institute Mapping of M&A Practices to Indicator Template Store Data & Results Communicate Results INDICATOR TEMPLATE INDICATOR TEMPLATE Measurement GoalTEMPLATE INDICATOR #_____: Measurement Goal #_____: Objective Measurement Goal #_____: Questions Objective Visual Display Questions Objective Visual Questions Display 100 Visual Display 80 100 60 80 40 100 60 20 80 40 60 20 40 Input(s) 20 Input(s) Data Elements Input(s) Data Elements Responsibility for Reporting Data Elements Responsibility Form(s) for Reporting Responsibility AlgorithmForm(s) for Reporting AlgorithmForm(s) Assumptions Algorithm Assumptions Interpretation Assumptions Interpretation X-reference Probing Interpretation X-referenceQuestions Evolution Probing Questions X-reference Evolution Probing Questions Evolution Establish Measurement Objectives Specify Measures Specify Data Collection C Procedures ollect Data Specify Analysis Procedures Analyze Data page 25 © 2004 by Carnegie Mellon University Carnegie Mellon Software Engineering Institute Measuring External Quality to Manage Software Development Quality Characteristic/Subcharacteric: Efficiency/Time Behavior Operational Measure: Response Time Objective: Track satisfaction of user requirement for system response time. Questions: What is the system response time with respect to common transaction? What is the variability in response time? 3.50 Response Time 3.00 2.50 2.00 1.50 1.00 0.50 Obs 29 27 Te st 25 Te st 23 Te st 21 Te st 19 Te st 17 Avg Te st 15 Te st 13 Te st 11 Te st 7 5 3 9 Te st Te st Te st Te st Te st Te st 1 0.00 Target page 26 © 2004 by Carnegie Mellon University Carnegie Mellon Software Engineering Institute Outline Background and Overview Concepts and Models Software Product Quality Measurement Summary page 27 © 2004 by Carnegie Mellon University Carnegie Mellon Software Engineering Institute Summary Measurement links the specification of requirements to acceptance criteria Quality is conceptual; measurement is operational. GQ(I)M provides a means for moving from the conceptual to the operational. The ISO 25000 series and the GQ(I)M Indicator Template together can help with your implementation of CMMI Requirements Development, Verification, and Validation. page 28 © 2004 by Carnegie Mellon University Carnegie Mellon Software Engineering Institute Contact Information Dave Zubrow 3118 SEI 4500 Fifth Ave Pittsburgh, Pa 15213 USA +1-412-268-5243 (v) +1-412-268-5758 (f) dz@sei.cmu.edu page 29 © 2004 by Carnegie Mellon University ...
View Full Document

Ask a homework question - tutors are online