improve-testsuite-slides

improve-testsuite-slides - ImprovingTestSuitesvia...

Info iconThis preview shows pages 1–12. Sign up to view the full content.

View Full Document Right Arrow Icon
Michael Ernst, page 1 Improving Test Suites via Operational Abstraction Michael Ernst MIT Lab for Computer Science http://pag.lcs.mit.edu/~mernst/ Joint work with Michael Harder, Jeff Mellen, and Benjamin Morse
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Michael Ernst, page 2 Creating test suites Goal: small test suites that detect faults well Larger test suites are usually more effective Evaluation must account for size Fault detection cannot be predicted Use proxies, such as code coverage
Background image of page 2
Michael Ernst, page 3 Test case selection Example: creating a regression test suite Assumes a source of test cases Created by a human Generated at random or from a grammar Generated from a specification Extracted from observed usage
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Michael Ernst, page 4 Contributions Operational difference technique for selecting test cases, based on observed behavior Outperforms (and complements) other techniques (see paper for details) No oracle, static analysis, or specification Stacking and area techniques for comparing test suites Corrects for size, permitting fair comparison
Background image of page 4
Michael Ernst, page 5 Outline Operational difference technique for selecting test cases Generating operational abstractions Stacking and area techniques for comparing test suites Evaluation of operational difference technique Conclusion
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Michael Ernst, page 6 Operational difference  technique Idea: Add a test case c to a test suite S if c exercises behavior that S does not Code coverage does this in the textual domain We extend this to the semantic domain Need to compare run-time program behaviors Operational abstraction: program properties x > y a[] is sorted
Background image of page 6
Michael Ernst, page 7 Test suite generation or augmentation Idea: Compare operational abstractions induced by different test suites Given: a source of test cases; an initial test suite Loop: Add a candidate test case If operational abstraction changes, retain the case Stopping condition: failure of a few candidates
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Michael Ernst, page 8 The operational difference  technique is effective Operational difference suites are smaller have better fault detection than branch coverage suites (in our evaluation; see paper for details)
Background image of page 8
Michael Ernst, page 9 Example of test suite  generation Program under test: abs (absolute value) Test cases: 5, 1, 4, -1, 6, -3, 0, 7, -8, 3, … Suppose an operational abstraction contains: var = constant var ≥ constant var ≤ constant var = var property property
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Michael Ernst, page 10 Considering test case 5 Initial test suite: { } Initial operational abstraction for { }: Ø Candidate test case: 5 New operational abstraction for { 5 }: Precondition: arg = 5 Postconditions: arg = return New operational abstraction is different , so retain the test case
Background image of page 10
Michael Ernst, page 11 Considering test case 1 Operational abstraction for { 5 }: Pre: arg = 5 Post: arg = return Candidate test case: 1 New operational abstraction for { 5, 1 }: Pre: arg ≥ 1 Post: arg = return Retain the test case
Background image of page 11

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 12
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 45

improve-testsuite-slides - ImprovingTestSuitesvia...

This preview shows document pages 1 - 12. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online