schuler-issta-2009 - Efficient Mutation Testing by Checking...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Efficient Mutation Testing by Checking Invariant Violations David Schuler, Valentin Dallmeier, and Andreas Zeller Saarland University, Saarbrcken, Germany { ds, dallmeier, zeller } @cs.uni-saarland.de ABSTRACT Mutation testing measures the adequacy of a test suite by seeding artificial defects (mutations) into a program. If a mutation is not detected by the test suite, this usually means that the test suite is not adequate. However, it may also be that the mutant keeps the programs semantics unchanged and thus cannot be detected by any test. Such equivalent mutants have to be eliminated manually, which is tedious. We assess the impact of mutations by checking dynamic invariants. In an evaluation of our JAVALANCHE framework on seven industrial-size programs, we found that mutations that violate invariants are significantly more likely to be de- tectable by a test suite. As a consequence, mutations with impact on invariants should be focused upon when improv- ing test suites. With less than 3% of equivalent mutants, our approach provides an efficient, precise, and fully automatic measure of the adequacy of a test suite. Categories and Subject Descriptors D.2.5 [ Software ]: Software Engineering Testing and De- bugging General Terms Experimentation Keywords Dynamic Invariants, Mutation Testing 1. INTRODUCTION How do we know a test suite is adequate in finding de- fects? Among the best ways is mutation testing: seeding defects into a program and checking whether the test suite finds them. Such defects can be created automatically, us- ing a set of mutation operators to change (mutate) random program parts. A mutation that is not detected (killed) by the test suite indicates that the test suite was unable to Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ISSTA09, July 1923, 2009, Chicago, Illinois, USA. Copyright 2009 ACM 978-1-60558-338-9/09/07 ...$5.00. detect the seeded defectand therefore is likely to miss sim- ilar, true defects in the code. Test managers can use such results to improve their test suites such that they detect these mutants. Mutation testing has been shown to be an effective mea- surement for the quality of a test suite [ 2 ] and superior to commonplace assessments such as coverage metrics [ 27 , 8 ]. However, mutation testing is known to be very expensive. A well-known issue is its large usage of computing resources....
View Full Document

Page1 / 11

schuler-issta-2009 - Efficient Mutation Testing by Checking...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online