annotation-study-fse2002

annotation-study-fse2002 - Invariant Inference for Static...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Invariant Inference for Static Checking: An Empirical Evaluation Jeremy W. Nimmer Michael D. Ernst MIT Lab for Computer Science 545 Technology Square Cambridge, MA 02139 USA { jwnimmer,mernst } @lcs.mit.edu http://sdg.lcs.mit.edu/daikon/ Abstract Static checking can verify the absence of errors in a program, but often requires written annotations or specifications. As a result, static checking can be difficult to use effectively: it can be difficult to determine a specification and tedious to annotate programs. Automated tools that aid the anno- tation process can decrease the cost of static checking and enable it to be more widely used. This paper describes an evaluation of the effectiveness of two techniques to assist the annotation process: inference via static analysis and inference via dynamic invariant de- tection. We quantitatively and qualitatively evaluate 33 users in a program verification task over three small pro- grams, using ESC/Java as the static checker, Houdini for static inference, and Daikon for dynamic detection. With a well-constructed test suite, Daikon produces fully-verifiable annotations; therefore, we supplied Daikon with poor test suites to study its effectiveness in suboptimal circumstances. Statistically significant results show that Daikon enables users to express more correct invariants; Houdini users do not take full advantage its capabilities; and both tools im- prove task completion. Interviews suggest that beginning users found Daikon to be helpful; Houdini to be neutral; static checking to be of potential practical use; and both assistance tools to have benefits. 1. Introduction Static analysis is a useful technique for detecting and checking properties of programs. A static analysis can re- veal properties that would otherwise have be detected only during testing or even deployment. This is valuable be- cause the earlier in the development process that problems can be identified, the less costly they are to correct. Sim- ple static analyses like type-checking are widely applied and successful; more complicated analyses like theorem-proving Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Copyright 2001 ACM X-XXXXX-XX-X/XX/XX ... $ 5.00. and correctness-checking are still topics of research. Anno- tations that are checked by analyses such as type-checkers and theorem-provers are useful in their own right: they serve as a machine-verified form of documentation....
View Full Document

This note was uploaded on 02/24/2012 for the course CSE 503 taught by Professor Davidnotikin during the Spring '11 term at University of Washington.

Page1 / 10

annotation-study-fse2002 - Invariant Inference for Static...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online