CoverageSummary-lastlecture

CoverageSummary-lastlecture - Introduction to Software...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Introduction to Software Testing Chapters 1-5 Coverage Summary Paul Ammann & Jeff Offutt http://www.cs.gmu.edu/~offutt/softwaretest/ From Chapter 1 Criteria Based on Four Structures Structures : Ways to model software 1. Graphs 2. Logical Expressions (not X or not Y) and A and B 3. Input Domain Characterization Characterization A: {0, 1, >1} B: {600, 700, 800} C: {swe, cs, isa, infs} {swe cs isa infs} 4. Syntactic Structures Introduction to Software Testing (Ch 1-5) if (x > y) z = x - y; else z = 2 * x; © Ammann & Offutt 2 1 Test Coverage Criteria A tester’s job is simple : Define a model of the software, then find ways to cover it • Test Requirements: Specific things that must be satisfied or covered during testing • Test Criterion: A collection of rules and a process that define test requirements Testing researchers have defined dozens of criteria, but they are all really just a few criteria on four types of structures. Introduction to Software Testing (Ch 1-5) © Ammann & Offutt 3 Coverage Overview Four Structures for Modeling Software Graphs Logic Input Space Syntax Applied to Applied to Source Specs Source Specs Design Introduction to Software Testing (Ch 1-5) Applied to FSMs DNF Source Models Integ Use cases © Ammann & Offutt Input 4 2 Coverage Overview Four Structures for Modeling Software Graphs Logic Input Space Syntax Applied to Applied to Source Specs Source Specs Design Applied to FSMs DNF Source Integ Use cases Introduction to Software Testing (Ch 1-5) Models Input © Ammann & Offutt 5 Ch. 2 : Graph Coverage Criteria (1) • Concepts for graphs • • • • • Visiting, touring Tests and test paths Structural and data flow coverage Prime Paths Defs and uses • Simple graph coverage criteria Node Coverage (NC) : TR contains each reachable node in G. Edge Coverage (EC) : TR contains each reachable path of length up to 1 in G. Introduction to Software Testing (Ch 1-5) © Ammann & Offutt 6 3 Graph Coverage Criteria (2) • Covering multiple edges Edge-Pair Coverage (EPC) : TR contains ereachable path of length up to 2 in G. each Complete Path Coverage (CPC) : TR contains all paths in G. Specified Path Coverage (SPC) : TR contains a set S of test paths, where S is supplied as a parameter. parameter. Introduction to Software Testing (Ch 1-5) © Ammann & Offutt 7 Graph Coverage Criteria (3) • Handling loops Simple Round Trip Coverage (SRTC) : TR contains at least one round-trip path for each reachable node in roundG that begins and ends a round-trip path. roundpath. Complete Round Trip Coverage (CRTC) : TR contains all round-trip paths for each reachable node in G. roundPrime Path Coverage (PPC) : TR contains each prime path in G. Introduction to Software Testing (Ch 1-5) © Ammann & Offutt 8 4 Graph Coverage Criteria (4) • Data flow coverage All-defs coverage (ADC) : For each set of du-paths S = du (n, v), TR contains at least one path d in S. AllAll-uses coverage (AUC) : For each set of du-paths duto uses S = du (ni, nj, v), TR contains at least one path d in S. All-duAll-du-paths coverage (ADUPC) : For each set S = du (ni, nj, v), TR contains every path d in S. Introduction to Software Testing (Ch 1-5) © Ammann & Offutt 9 Graph Coverage Criteria Subsumption Complete Path Coverage CPC All-DU-Paths Coverage ADUP All-uses Coverage AUC All-defs Coverage ADC Introduction to Software Testing (Ch 1-5) Prime Path Coverage PPC Edge-Pair Coverage EPC Complete Round Trip Coverage CRTC Edge Coverage EC Simple Round Trip Coverage SRTC Node Coverage NC © Ammann & Offutt 10 5 Coverage Overview Four Structures for Modeling Software Graphs Logic Input Space Syntax Applied to Applied to Source Specs Source Specs Design Applied to FSMs DNF Source Integ Use cases Introduction to Software Testing (Ch 1-5) Models Input © Ammann & Offutt 11 Ch. 3 Logic Coverage Criteria (1) • Concepts in logical expressions • Active clauses • Determination of the predicate • Simple logical expression criteria logical expression criteria Predicate Coverage (PC) : For each p in P, TR contains two TR requirements: p evaluates to true, and p evaluates to false. Clause Coverage (CC) : For each c in C, TR contains two TR requirements: c evaluates to true, and c evaluates to false. Introduction to Software Testing (Ch 1-5) © Ammann & Offutt 12 6 Logic Coverage Criteria (2) • Testing multiple clauses at once Combinatorial Coverage (CoC) : For each p in P, TR has test Coverage (CoC) For each in TR has test requirements requirements for the clauses in Cp to evaluate to each possible combination of truth values. Active Clause Coverage (ACC) : For each p in P and each major clause ci in Cp, choose minor clauses cj, j != i, so that ci ci Cp cj != ci determines p. TR has two requirements for each ci : ci TR evaluates evaluates to true and ci evaluates to false. ci Introduction to Software Testing (Ch 1-5) © Ammann & Offutt 13 Logic Coverage Criteria (3) General Active Clause Coverage (GACC) : For each p in P and each major clause ci in Cp, choose minor clauses cj, j != i, ci Cp cj != so that ci determines p. TR has two requirements for each ci : ci ci ci evaluates to true and ci evaluates to false The values ci evaluates to true and ci evaluates to false. The values chosen chosen for the minor clauses cj do not need to be the same cj when ci is true as when ci is false, that is, cj(ci = true) = cj(ci = ci ci cj(ci false) for all cj OR cj(ci = true) != cj(ci = false) for all cj. cj cj(ci Restricted Active Clause Coverage (RACC) : For each p in P and each major clause ci in Cp, choose minor clauses cj, j != i, Cp so that ci determines p. TR has two requirements for each ci: ci ci ci ci evaluates to true and ci evaluates to false. The values ci chosen for the minor clauses cj must be the same when ci is cj ci true as when ci is false, that is, it is required that cj(ci = true) = ci cj(ci cj(ci = false) for all cj. cj Introduction to Software Testing (Ch 1-5) © Ammann & Offutt 14 7 Logic Coverage Criteria (4) • Practical version Correlated Active Clause Coverage (CACC) : For each p in P Cl (CACC) and each major clause ci in Cp, choose minor clauses cj, j != i, ci Cp cj != so that ci determines p. TR has two requirements for each ci: ci ci ci ci evaluates to true and ci evaluates to false. The values ci chosen for the minor clauses cj must cause p to be true for one cj value of the major clause ci and false for the other, that is, it is ci required that p(ci true) != p(ci false) required that p(ci = true) != p(ci = false). Introduction to Software Testing (Ch 1-5) © Ammann & Offutt 15 Logic Coverage Criteria (5) • Inactive clauses Inactive Clause Coverage (ICC) : For each p in P and each major clause ci ci in Cp, choose minor clauses cj, j != i, so that ci does not determine p. TR Cp cj != ci has four requirements for each ci (1) ci evaluates to true with true (2) ci has four requirements for each ci: (1) ci evaluates to true with p true, (2) ci evaluates to false with p true, (3) ci evaluates to true with p false, and (4) ci ci ci evaluates to false with p false. General Inactive Clause Coverage (GICC) : For each p in P and each major clause ci in Cp, choose minor clauses cj, j != i, so that ci does not ci Cp cj != ci determine p. The values chosen for the minor clauses cj do not need to be cj the same when ci is true as when ci is false, that is, cj(ci = true) = cj(ci = ci ci cj(ci false) for all cj OR cj(ci true) != cj(ci false) for all cj false) for all cj OR cj(ci = true) != cj(ci = false) for all cj. Restricted Inactive Clause Coverage (RICC) : For each p in P and each major clause ci in Cp, choose minor clauses cj, j != i, so that ci does not ci Cp cj != ci determine p. The values chosen for the minor clauses cj must be the same cj when ci is true as when ci is false, that is, it is required that cj(ci = true) = ci ci cj(ci cj(ci = false) for all cj. cj Introduction to Software Testing (Ch 1-5) © Ammann & Offutt 16 8 Logic Coverage Criteria Subsumption Combinatorial Clause Coverage COC Restricted Active Cl Clause Coverage RACC Restricted Inactive Cl Clause Coverage RICC Correlated Active Clause Coverage CACC General Inactive Clause Coverage GICC General Active Active Clause Coverage GACC Clause Coverage CC Predicate Coverage PC Introduction to Software Testing (Ch 1-5) © Ammann & Offutt 17 Coverage Overview Four Structures for Modeling Software Graphs Logic Input Space Syntax Applied to Applied to Source Specs Source Specs Design Introduction to Software Testing (Ch 1-5) Applied to FSMs DNF Source Models Integ Use cases © Ammann & Offutt Input 18 9 Ch. 4 : Input Space Criteria (1) • Concepts for input space criteria • • • • Input domains No difference with level of software abstraction (unit, integration, etc.) Partitioning, domains, blocks domains, blocks Constraints among partitions • Simple input space criterion Each Choice (EC) : One value from each block for each partition must be used in at least one test case partition must be used in at least one test case. Introduction to Software Testing (Ch 1-5) © Ammann & Offutt 19 Input Space Criteria (2) • Combining values PairPair-Wise (PW) : A value from each block for each partition must be combined with a value from every block for each other partition. t-Wise (TW) : A value from each block for each group of t partitions must be combined. Introduction to Software Testing (Ch 1-5) © Ammann & Offutt 20 10 Input Space Criteria (2) • Using domain knowledge of the program Base Choice (BC) : A base choice block is chosen for each partition, and a base test is formed by using the base choice for each and partition. Subsequent tests are chosen by holding all but one base choice constant and using each non-base choice in each other nonparameter. Multiple Base Choice (MBC) : One or more base choice blocks are chosen for each partition, and base tests are formed by using each base choice for each partition. Subsequent tests are chosen by holding all but one base choice constant for each base test and using each non-base choices in each other parameter. nonIntroduction to Software Testing (Ch 1-5) © Ammann & Offutt 21 ISP Coverage Criteria Subsumption All Combinations Coverage AC T-Wise Coverage TW Multiple Base Choice Coverage MBC Pair-Wise Coverage PW Base Choice Coverage BC Each Choice Coverage EC Introduction to Software Testing (Ch 1-5) © Ammann & Offutt 22 11 Coverage Overview Four Structures for Modeling Software Graphs Logic Input Space Syntax Applied to Applied to Source Specs Source Specs Design Applied to FSMs DNF Source Integ Use cases Introduction to Software Testing (Ch 1-5) Models Input © Ammann & Offutt 23 Ch. 5 : Syntax-based Criteria (1) • Concepts for syntax-based testing • Grammars • Regular expressions, finite state machines • Mutation testing • Simple syntax-based criteria syntax criteria Terminal Symbol Coverage (TSC) : TR contains each terminal symbol t in the grammar G. Production Coverage (PC) : TR contains each production p in th the grammar G. Derivation Coverage (DC) : TR contains every possible string that can be derived from the grammar G. Introduction to Software Testing (Ch 1-5) © Ammann & Offutt 24 12 Mutation Testing • • • • Grammars describe both valid and invalid strings Both types can be produced as mutants A mutant is a variation of a valid string Mutation is based on “mutation operators” and “ground is based on operators and strings” • Ground string: A string in the grammar • The term “ground” is used as a reference to algebraic ground terms • Mutation Operator : A rule that specifies syntactic variations of Operator rule that specifies syntactic variations strings generated from a grammar • Mutant : The result of one application of a mutation operator • A mutant is a string Introduction to Software Testing (Ch 1-5) © Ammann & Offutt 25 Syntax-based Criteria (2) • Mutation criteria Mutation Coverage (MC) : For each m M, TR contains exactl one requirement, to kill m. exactly kill Mutation Operator Coverage (MOC) : For each mutation operator, TR contains exactly one requirement, to create a mutated string m that is derived using the mutation operator. Mutation Production Coverage (MPC) : For each mutation operator, TR contains several requirements, to create one mutated string m that includes every production that can be mutated by that operator. Introduction to Software Testing (Ch 1-5) © Ammann & Offutt 26 13 Summary of Criteria • Chapter 2 – Eleven criteria, five recommended • Chapter 3 – Eight criteria, three recommended • Chapter 4 – Six criteria, two recommended • Chapter 5 – Six criteria, three recommended • Total: Thirty-one criteria, thirteen recommended Introduction to Software Testing (Ch 1-5) © Ammann & Offutt 27 14 ...
View Full Document

This note was uploaded on 01/17/2012 for the course SWE 637 taught by Professor Offutt,j during the Fall '08 term at George Mason.

Ask a homework question - tutors are online