#7 Lab Testing Due In Lab, October 22, 2003 Name: Lab Time: Grade: Testing Today you will be testing a program to expose failures. In Homework 7, you familiarized yourself with the operation of a quick sort program, quick-okay. Now that you know the type of input the program takes, and the output it produces, you will be writing test cases for it. Two executables can be found in ~csci3308/arch/$ARCH/bin. Their names are quick-okay for the program without bugs, and quick-bugs for the program with bugs. We are providing the bug-free program as a reference so you can generate the expected output for your test cases. The buggy program is the one you will be testing. The Testing Directory Structure Get the le ~csci3308/src/quick-bugs.tar and unpack it in your source directory. It does not contain the source code for the quick-bugs program, but it does contain the README le and a test case. In the quick-bugs directory you will nd a directory named test. All of the test cases will be placed in here. Inside of test there is a directory named ts1. This stands for test set 1. A test set is a large group of test cases. There may be several test sets for a particular program. Inside ts1 there is a directory named tc1. This stands for test case 1. Each test case has the same les and so has to be placed in its own directory. Go into tc1 and look at the les. There is a README le explaining what this test case tests for. There is an input le, and there is an output.expected le. Run the quick-okay program on the input le. Did the test pass or fail? How do you know? (The answer to the second question is not because we told you the program is correct.) /10
1
Create four more test cases. These can include the test cases from homework 7, if you want. Create the directories tc2, tc3, tc4, and tc5 for the new test cases, and create the les README, input, and output.expected for each one. Write your test cases below (all three parts for each test case). You can use the back of this page if you need extra space.
Running Test Cases from the Build Directory Now Im going to discuss some software engineering philosophy behind testing. You should think of test cases as a type of source code. As such, test cases should be placed within a subdirectory of a source directory, as we have done here for this lab. When running a test case, the output and comparisions should be done in a build directory, since the output les are automatically generated. A test case is like source code because it is generated by a human. A person has to decide what input should be tested for each Normally, case. the expected output is also generated by a person who knows what the program is supposed to do. We are cheating by using a correct version to tell us the expected output, but normally you will not have access to such a program. A test run, which is the actual output of a test case, is automatically generated from the input and the program. It can be recreated at any time and thus can be safely deleted. You should be very careful when deleting anything in a source directory. Things that can be deleted and easily re-created go in the build directory. Another dierence is that the test run changes, but the 2
test case does not. If you x a bug, the test run may produce dierent output, but the expected output, the output the program is supposed to have, stays the same. There is one nal reason to run tests in the build directory. The build directory is architecture specic. It is not inconceivable that a program would pass a test on one architecture, but fail on another. Sometimes the program requires dierent code for dierent architectures. In the C programming language, such code can contain #ifdef statements to use dierent code for dierent architectures. Since dierent architectures often require dierent assumptions about a programs operating environment, there may be a bug in the code that only fails on one architecture but not others. Hopefully, I have convinced you to leave your test cases in the source directory and run (build) them in the build directory. Think about a test run as a compiler whose output is a si...