chapter10

chapter10 - Chapter 10 Testing and Debugging Goals Learn...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Chapter 10 Testing and Debugging Goals Learn techniques to test your code Learn to carry out unit tests Understand principles of test case selection and evaluation Learn to use logging Become familiar with a debugger Unit Testing The single most important testing tool Checks a single method or a set of cooperating methods You don't test the complete program that you are developing; you test the classes in isolation Unit Testing Avoids confusion of interactions More startup cost, but less work in the end Analogy: Testing car Test Harness For each test, you provide a simple class called a test harness Test harness feeds parameters to the methods being tested Example To compute the square root of a use a common algorithm: Guess a value x that might be somewhat close to the desired square root (x = a is ok) Actual square root lies between x and a/x Take midpoint (x + a/x) / 2 as a better guess Repeat the procedure. Stop when two successive approximations are very close to each other Example Method converges rapidly. Square root of 100: Guess #1: 50.5 Guess #2: 26.24009900990099 Guess #3: 15.025530119986813 Guess #4: 10.840434673026925 Guess #5: 10.032578510960604 Guess #6: 10.000052895642693 Guess #7: 10.000000000139897 Guess #8: 10.0 Testing RootApproximator class getRoot() nextGuess() Numeric class approxEqual(double, double) 06:public class RootApproximatorTester 07:{ 08: public static void main(String args) 09: { 10: System.out.print("Enter a number: "); 11: Scanner in = new Scanner(System.in); 12: double x = in.nextDouble(); 13: RootApproximator r = new RootApproximator(x); 14: final int MAX_TRIES = 10; 15: for (int tries = 1; tries <= MAX_TRIES; tries++) 16: { 17: double y = r.nextGuess(); 18: System.out.println("Guess #"+tries+ ": " + y); 19: } 20: System.out.println("Square root: " + r.getRoot()); 21: } 22:} Unit Testing What is needed to make the testing more robust? More tests Limitation: Hard to replicate tests when bugs are fixed Solution: Use test harness to repeat values Providing Test Input Solution #1: Hardwire series of tests No need to memorize series of tests, already taken care of. Just run tester class. public class RootApproximatorHarness1 { public static void main(String args) { double testInputs = { 100, 4, 2, 1, 0.25, 0.01 }; for (int x = 0; x < testInputs.length; x++) { RootApproximator r = new RootApproximator(testInputs[x]); double y = r.getRoot(); System.out.println("square root of " + testInputs[x] + " = " + y); } } } Instead of hardcoding array of values 1. Generate Test Cases Automatically Loop through a sample range of values for (int x = MIN; x <= MAX; x += INCREMENT){ RootApproximator r = new RootApproximator(x); double y = r.getRoot(); System.out.println("square root of " + x + " = " + y); } Generate Test Cases Automatically Instead of hardcoding array of values Use random number generator 1. final int SAMPLES = 100; Random generator = new Random(); for (int i = 0; i < SAMPLES; i++){ double x = 1000 * generator.nextDouble(); RootApproximator r = new RootApproximator(x); double y = r.getRoot(); System.out.println("square root of " + x + " = " + y); } What to Test for? Good testing requires testing good cases Reduces debugging time and ensures better product Test all of the features of the method Positive tests normal behavior of method Boundary test cases (e.g. x = 0), make sure end conditions are correct Negative cases program should reject How do you know whether the output is correct? Test Case Evaluation Calculate correct values by hand E.g., for a payroll program, compute taxes manually Supply test inputs for which you know the answer E.g., square root of 4 is 2 and square root of 100 is 10 Verify that the output values fulfill certain properties E.g., square root squared = original value for (int i = 1; i <= SAMPLES; i++) { double x = 1000 * generator.nextDouble(); RootApproximator r = new RootApproximator(x); double y = r.getRoot(); if (Numeric.approxEqual(y * y, x)){ System.out.print("Test passed: "); passcount++; } else { System.out.print("Test failed: "); failcount++; } System.out.println("x = " + x 37: + ", root squared = " + y * y); } Test Case Evaluation Use an Oracle: a slow but reliable method to compute a result for testing purposes E.g., use Math.pow to slower calculate x1/2 (equivalent to the square root of x) for (int i = 1; i <= SAMPLES; i++) { double x = 1000 * generator.nextDouble(); RootApproximator r = new RootApproximator(x); double y = r.getRoot(); double oracleValue = Math.pow(x, 0.5); if (Numeric.approxEqual(y,oracleValue)){ System.out.print("Test passed: "); passcount++; } else { System.out.print("Test failed: "); failcount++; } } Regression Testing and Test Coverage Save test cases Particularly tests that reveal bugs Use saved test cases in subsequent versions A test suite is a set of tests for repeated testing Why keep a test case? Very common for bugs to show up later We often think we fixed a bug, but just covered it up Cycling = bug that is fixed but reappears in later versions Regression testing: repeating previous tests to ensure that known failures of prior versions do not appear in new versions Test Coverage Test coverage: measure of how many parts of a program have been tested Good testing tests all parts of the program E.g. every line of code is executed in at least one test Example: Test all possible branches inside of an if statement In Tax code program, 6 tests (3 $ brackets * 2 types of status) Test Coverage Blackbox testing: test functionality without consideration of internal structure of implementation Useful since this is what the user interacts with Whitebox testing: take internal structure into account when designing tests Unit testing Why? Cannot prove absence of bugs, only presence Testing Levels The part of the program that is being tested. System Unit System Level Testing Test interactions of the various parts of the application. Completed after the unit tests pass. Unit Level Testing Test individual units (classes) in isolation. Test features of the unit. Multiple test cases for each feature. Black Box Testing a.k.a. behavioral, functional, closed against the specification does not need the program source internal implementation is unknown Black Box Testing Advantages: designed from the specifications user's point of view less biased can discover if part of the specification has not been fulfilled. Black Box Testing Disadvantages: may be redundant can be difficult to design. testing every possible input stream is unrealistic White Box Testing a.k.a. structural, clear box, open tests against the implementation every line of source code is executed at least once. tests exception handling code. White Box Testing Advantages can discover if any part is faulty test coverage (test all paths) Disadvantages will not discover if part is missing only if the programmer knows what the program is supposed to do code must also be visible to the tester. "Fully" Tested To fully test a software product, the following are required: unit testing and system level testing black and white box testing Does this ensure there are no bugs? Testing Tips Develop cases to take into account end conditions Tip: write first test cases before program is written completely gives insight into what program should do Build up as program goes along and bugs are found Logging To figure out where your program "goes", use program trace statements Help determine the "flow" of execution if (status == SINGLE) { System.out.println("status is SINGLE"); . . . } Problem When done debugging, have to remove all System.out.println trace messages What if we find a bug again? Solution: Create a separate class for the purpose of logging Logging Logging messages can be deactivated when testing is complete Use global object Logger.global in place of System.out Log a message Logger.global.info("status is SINGLE"); Turn off/on How does this class help? Can easily switch logging on and off Default: On To turn off, insert at top of main(): Logger.global.setLevel(Level.OFF); Common to need to know enter/exit conditions of methods What to print out public TaxReturn(double anIncome, int aStatus){ Logger.global.info("Parameters: anIncome = " + anIncome + " aStatus = " + aStatus); . . . } //Exit public double getTax(){ . . . Logger.global.info("Return value = " + tax); return tax; } Logging Obviously other output is needed Bug is that certain condition isn't true when it is supposed to be track values as you go along Logging class has many other options (see API) Disadvantages: Time consuming Debuggers To avoid logging problems, most professionals use a debugger Programs rarely (never) run perfectly the first time Think of perfect first draft to paper The larger your programs, the harder to debug them simply by logging Debuggers Most current development environments contain a debugger Debugger program to run your program and analyze its runtime behavior A debugger lets you stop and restart your program, see contents of variables, and step through it Debuggers Debuggers can be part of your IDE (Eclipse, BlueJ) or separate programs (JSwat) Three key concepts: Breakpoints Singlestepping Inspecting variables Breakpoint Breakpoint the debugger doesn't stop running the program until it hits a defined breakpoint Setup by programmer Once stopped, you can see the state of all the variables Stepthrough Once a breakpoint is hit, two options Step through the next few statements carefully inspecting Slow, but useful in heavy calculations Singlestep line by line execution Step into doesn't just skip line to line, but goes into method calls within each line Step over Execute the enter statement and stop Run at full speed until the next breakpoint Example Current line: String input = in.next(); Word w = new Word(input); int syllables = w.countSyllables(); System.out.println("Syllables in " + input + ": " + syllables); Step Over Next step: String input = in.next(); Word w = new Word(input); int syllables = w.countSyllables(); System.out.println("Syllables in " + input + ": " + syllables); Step Into public int countSyllables() { int count = 0; int end = text.length() - 1; . . . } Which to choose If the method is suspect (it may be the cause of a problem) Step into If you are sure the method works correctly step over What if a test fails? It's time to debug! We will want (need) a strategy. Debugging Strategy A systematic way to find and fix bugs. Debug Strategy 1: Trial & Error 1. 2. 3. Write code Run program Fix bugs Advantages no planning required Disadvantages Spend more time trying to find and fix bugs than you did writing the program. 1. 2. 3. Write code Run program Use Debugger program to fix bugs Advantages Debug Strategy 2: Trial & Error with a Debugger no planning required Disadvantages Spend lots of time in the debugger program trying to find and fix the bugs Incremental Development with Testing 1. 2. 3. 4. Debug Strategy 3: Write small amount of code with testing and debugging in mind. Document and write tests for each new piece of code. Run tests on new code Fix any bugs before adding new code. Disadvantage: Sometimes you have to debug code that you didn't write. Incremental Development with Testing Debug Strategy 3: Disadvantages Takes more time (harder) to design the solution Must write more code (tests) Only possible if you're the one who wrote the program Advantages A small amount of code to review and fix. Bug is fixed before it get hidden by some other bug. If small parts work, the bigger parts will usually work too. If not, it's only the public interfaces that have to be debugged. 1. 1. 1. 1. 1. 1. Identify the bug. Must be repeatable and testable. Write test(s) that fails for the bug. Locate the source of the bug. HOW? Understand the problem before you fix it. Try your fix and run all tests. Repeat until all tests pass. Debugging Strategy 4: Test and use a Debugger Pick a point to start looking How do we find the source of the bug? Beginning (main method) Divide and conquer Trace the code. (Look for variables with unexpected values) Manual trace Trace messages Debugger program System.out.println(...) Java 1.5: Use java.util.logging.Logger class ...
View Full Document

Ask a homework question - tutors are online