Testing Software Notebook Worksheet #1 Functional Testing Due: October 31, 2003 Name: Lab Time: Grade: /75
On my honor, as a University of Colorado at Boulder student, I have neither given nor received unauthorized assistance on this work. Signature:
The Program Under Test You will start the testing notebook by getting familiar with the program you will be testing, the Easy Pay System, or ezpay. The tar le ~csci3308/src/ezpay.tar contains the les for worksheet 1. Unpack this tar le in your src directory. In the ezpay directory, you should nd the following: spec.txt tests/ ezpays requirements specication A directory containing les that represent two test cases for the ezpay program
The program itself is located at ~csci3308/bin/ezpay. This shell script is a wrapper for a java program contained in the le: <~csci3308/bin/ezpay-bugs.jar>. You do not need to copy these les to your account. If you do, be sure to update the path in the shell script to point to your version of the jar le. To sanity test the program, try typing: ~csci3308/bin/ezpay --help The program should print usage information. To make it easier to work with this program, you should add ~csci3308/bin to your path. Once this is done, type rehash and make sure that the program is in your path. The second thing you should do is move the test case les into a test case structure simliar to the one we used in lab 7. That is, create a directory called test within ezpays src directory and within that directory create a directory called ts1. Then create directories for the two test cases tc1 and tc2 within the ts1 directory and move the test case les to the appropriate directory. You 1
should rename the les such that the test case directories contain three les, one called ezpay.in, one called output.expected, and one called documentation. You can delete the tests directory once you have nished creating the tc1 and tc2 directories. The third thing you should do is create an architecture-independent build directory for ezpay (even though we dont need to actually build the program this week) and create a testing structure within it similar to the one created in step 2 above. (Note: you may need to create a build directory within your ~/csci3308 directory. Then create an ezpay directory within this newly created build directory.) Thus, your architecture-independent build directory should contain a test directory which contains a ts1 directory which contains the tc1 and tc2 directories. Now, similar to what we did in lab 7, run the following code within the architecture-independent build directory (indeed you should probably put this code within a script, so you can run it multiple times): set srcdir = $HOME/csci3308/src/ezpay foreach testdir (test/ts1/*) cp $srcdir/$testdir/ezpay.in $testdir cd $testdir ezpay diff $srcdir/$testdir/output.expected ezpay.out > /dev/null echo $status cd ../../.. end Recall that a 0 indicates success while a 1 indicates a failure. If all went well, then one of the sample test cases passed and one failed. You should now develop a funcational test plan (described below) that will help you create additional test cases that you can use to test the ezpay program. You are welcome to use the two test cases that we have provided but be careful, you may have to rename these test cases to match your test plan. That is, your test plan is going to develop a set of test cases and it will specify the contents for tc1, tc2, tc3, etc. So if, for whatever reason, your test plan has indicated that our tc1 needs to be called tc20, then be sure to rename it accordingly. Since this is so important, it bears repeating: Make sure that your test cases match your test plan. Functional Testing Functional testing provides a criteria to determine how many test cases to create for a test suite based on the functional specication of a program. A functional test plan is created to help identify test cases. In order to write your test plan, perform the following steps: 1. The rst step in functional testing is to analyze the programs specication and identify its functional categories. These categories are typically 2
broad (such as handling a programs command line options, sorting, detecting errors, etc.) but can sometimes be quite specic (such as testing a programs ability to calculate a particular type of value). 2. Once the functional have categories been identied, analyze the specication to determine the programs specication items. A specication item is a specic function that the program must perform. Each specication item can typically be assigned to one of the functional categories. 3. Then, for each specication item, determine its functional equivalence classes. That is, the specication item may need to behave dierently given dierent types of input. In lecture 16, an example of selecting functional equivalence classes for the GreatestCommonDivisor function is provided on page 15. (Note: most items will have only one or two equivalence classes, although more is possible.) 4. Once you have determined the functional equivalence classes for each specication item, determine test inputs for each functional equivalence class. Many equivalence classes will only require one test input (just like the GreatestCommonDivisor example), but some may require additional inputs to test boundary conditions (similar to the car database example in lecture 18) You are now ready to write your test plan. The Test Plan 1. The rst section of your test plan should contain a brief description of the ezpay program and list the functional categories that you identied. 2. The second section of your test plan should be subdivided by functional categories, where each subsection lists the specication items for a particular functional category. Each specication item should be assigned a number (e.g. S1, S2, S3, ...) and should be described as precisely as possible. Below the description, you should list the items functional equivalence classes along with the test inputs that you have selected for each class. Since each test input corresponds to a test case be sure to assign each one a testcase number (e.g. tc1, tc2, tc3, ...). These numbers need to increase sequentially across all specication items in this section of your test plan. Note that these numbers are not yet nal (see below). 3. In the third section of your test plan, identify redundant test cases. For instance, if tc1 and tc23 use the exact same input to test two dierent specication items, then you can delete tc23 and use tc1 to test both items. As a result of this analysis, your test case numbers may change (e.g. if you delete tc23, then tc24 becomes tc23, tc25 becomes tc24, etc.). As such, once you have nished identifying redundant test cases, create a nal list of test cases that identies which key specication items (which 3
have also been assigned a number) each test case covers. Your nal list should use a format similar to this: tc1: tc2: tc3: tc4: ... S1, S2, S23 S4...