UsingadefinedandmeasuredPersonalSoftwareProcess - Improved...

Unformatted text preview: Improved software processes lead to improved product quality. The Personal Software Process is a framework of techniques to help engineers improve their performance — and that of their organizations — through a step-by-step, disciplined approach to measuring and analyzing their work. This article explains how the PSP is taught and how it applies to different software- engineering tasks. The author reports some promising early results. WATTS S. HUll/lPHREY Software Engineering Institute lEEE SOFTWARE ewer code defects, better estimating and planning, enhanced productive ity — software engineers can enjoy these benefits by learning and using the disciplines of the Personal Software Process. As a learning vehicle for introducing process concepts7 the PSP framework gives engineers measurement and analysis tools to help them understand their own skills and improve personal performance. Moreover, the PSP gives engineers the process understanding they need to help improve organi- zational performance. Up to a point, process improvement can be dri— ven by senior management and process staffs. Beyond Level 3 of the Software Engineering lnstitute’s Capability Maturity Model, however, improvement requires that engineers apply process principles on an individual basis.I in fact, it was because of the difficulqu small engineering groups had in applying CNIM principles that I developed the PSP. Large and small organizations alike can benefit from CIVth practices, and I focused the original PSP research on demonstrating how individuals and small teams could apply process—improvement methods. In this article, I describe the PSP and experiences with teaching it to date. Thus far, the PSP is introduced in a one-semester graduate—level course where engineers develop 10 U740-7A59/QB/$S.OD ©1993|EEE *:.;IAB,I.,E PelasxsrclsEis; [ Jzii :' - Using a linked list, write a nrogram to calculate the mean and rogram LOC. Enhance program 2A to count total program LOC and LOC of Using a linked list, write a orogram to calculate the linear regression n a numerical integration. Enhance program 4A to calculate the linear regression parameters Using a linked list, write a program to calculate the correlation of rogram to do a chisquared test for 21 Using a li iked list, write a program to calculate the three—parameter multiple regression parameters and the prediction interval. module—sized programs and write five analysis reports. Early results are encouraging M while individual perfor mancc varies widely, data on 104 stu— dents and engineers show reductions of 58 percent in the average number of defects injected (and found in develop— ment) per 1,000 lines of code (KLOC), and reductions of 71.9 percent in the average number of defects per KLOC found in test. Estimating and planning accuracy are also improved, as is pro- ductivity —— the average improvement in LOC developed per hour is 20.8 per— CCIIT. rogram 213 to handle common user errors. Jrogram 313 to handle further user error types. Jrogram 4B to handle arrays of real numbers. rogram SE to calculate the linear regression parameters arogram 613 to calculate the linear regression parameters {Write a program to do a chi—squared test for a normal distribution Program NumberBrief Description 1A standard deviation of a set of data. 2A Write a program to count 3A functions or Objects. 4A parameters (straight line fit). 5A Write a program to perfor 6A and the prediction interval. 7A two sets of data. 8A Write a program to sort a inked list. 9A Using a linked list, write a normal distribution. 10A 113 Write a p'ogram to store and retrieve numbers in a file. 213 Enhance orogram IE to modify records in a file. 3B Enhance 4B Enhance 5B Enhance 6B Enhance from a file. 7B Enhance and the prediction interval. 813 Enhance arogram SE to sort a file. 9B \ from data stored in a file. Reports “'Rl LOG counting standard: Count logical LOC in the language you use to develop the PSP exercises. Coding standard: Provide one logical LOC per physical LOC. ‘ Defect analysis report: Analyze the defects for programs 1A through 3A. Nlidterm analysis report of process improvement. Final report of process and quality improvement and lessons learned. You can apply PSP principles to almost any soft“are—engineering task because its structure is simple and inde pendent of technology — it prescribes no specific languages, tools, or design methods. P5P OVERVIEW A software process is a sequence of steps required to develop or maintain software. The PSP is supported with a textbook and an introductory course.2 It uses a family of seven steps tailored to develop module—sized programs of 50 to 5,000 LOC. Each step has a set of asso— ciated scripts, forms, and templates. During the course, engineers use the recesses to complete the programming and report exercises shown in Table 1. As engineers learn to measure their work, analyze these measures, and set and meet improvement goals, they see the benefits of using defined methods and are motivated to consistently use them. The 10 PSP exercise programs are small: the first eight average 100 LOC and die last two average 200 and 300 LOC, respectively. Completing these programs, however, takes a good deal of work. While a knowledgeable instructor can substantially assist the students, the principal learning vehicle is the experience the students gain in doing the exercises. When properly taught, the PSP 6 demonstrates personal process principles, 0 assists engineers in making accu— rate plans, 6 determines the steps engineers can take to improve product quality, 6 establishes benchmarks to measure personal process improvement, and 0 determines the impact of process changes on an engineer’s performance. The PSP introduces process con— cepts in a series of steps. Each PSP step, shown in Figure 1, includes all the ele— ments of prior steps together with one or two additions. Introducing these con— cepts one by one helps the engineers learn disciplined personal methods. P673071 ill Alarm rcmen t (P S P 0) is ‘ where the PSP starts. In this first step, 3 engineers learn how to apply the PSP ‘ forms and scripts to their personal work. They do this by measuring development time and defects (both injected and removed). This lets engineers gather real, practical data and gives them benchmarks against which they measure progress while learning and practicing the PSP. PSPO has three phases: plan— MAY 1995 ning, development (which includes design, code, compile, and test), and postmortem. PSPO.1 adds a coding stan— dard, size measurement, and the Process Improvement Proposal form. The PIP ‘ form lets engineers record problems, issues, and ideas to use later in improv— ing their processes. They also see how forms help them to gather and use process data. Personal lemiazg (PSPl) introduces the PROBE method. Engineers use PROBE to estimate the sizes and devel— opment limes for new programs based on their personal data.Z PROBE uses linear regression to calculate estimating parameters, and it generates prediction intervals to indicate size and time esti— mate quality. Schedule and task plan— ning are added in PSP1.1. By introduc— ing planning early, the engineers gather enough data from the 10 PSP exercises to experience the benefits of the PSP statistical estimating and planning meth— : ods. Pomona] Quality (PSPZ) introduces defect management. With defect data from the PSP exercises, engineers con— struct and use checklists for design and code review. They learn why it’s impor— tant to focus on quality from the start and how to efficiently review their pro— grams. From their own data, they see how checklists can help them effectively review design and code as well as how to develop and modify these checklists as their personal skills and practices evolve PSP2.l introduces design specification and analysis techniques, along with defect prevention, process analyses, and process benchmarks. By measuring the time tasks take and the number of defects they inject and remove in each process phase, engineers learn to evalu— ate and improve their personal perfor— manee. Scaling Up (PSP3) is the final PSP step. Figure 2 illustrates how engineers can couple multiple PSP2.1 processes in a cyclic fashion to scale up to developing lEEE SOFTWARE Cyclic process // Personal quality / 1‘ Personal planning/ / / Personal measuremen ‘ F igure 1. PSP process evolution. Specificoiions l Figure 2. PSP3 prover)". modules with as many as several thou- sand LOC. At this PSP level7 engineers also explore design—verification methods as well as process—definition principles and methods. PSP/(MM reluiionship. The CMM is an organization—focused process—improve— ment framework.1'3 While the CNIM enables and facilitates good work, it does not guarantee it. The engineers must also use effective personal prac— tices. This is where the PSP comes in, with its bottom-up approach to process 3improvement. PSP demonstrates process improvement principles for indi— vidual engineers so they see how to effi— ciently produce quality products. To be fully effective, engineers need the sup— port of a disciplined and efficient envi— ronment, which means that the PSP will Eevei 5 Process change management Technology change management i Defect prevention ‘ Level A—Monoged ’3 Software quality management was: ,Quantitofive process management 1’ yet 3—Delined eer reviews o‘up coordination reduct engineering {*rwore management Eng program ,, process definition process focus +Rep’eotoble ligur‘otion management l'ty assurance trod management king and oversight elect planning at Hniiisl Figure 3. P3P elements in the Capability .Mrzz‘zn‘z'ty .l'hdel, high/glued in boldface. be most effective in software organizar tit ins near or above (.lMNl l .evel 2. The PSP and the CMNI are mutually supportive. The CIWIVT provides the orderly support environment engineers need to do superior work, and the PSP equips engineers to do highiquality work and participate in organitational process improvement. As Figure 3 shows, the PSP demonstrates the goals of 12 of the 18 CMM key process areas. The PSP demonstrates only those that can be accommodated with individual, classroom—sized exercises. P5P development. lo the initial PSP experiments. I wrote 61 Pascal and C++ pl‘Ogl‘Hl‘HS LlSlllg 3 PCYSUHQl pl‘OCCSS ‘35 near to meeting the goals of CNlM Level 5 as 1 could devise. l also applied these same principles to personal finan— cial work. technical writing, and process development. This work showed me that a defined and measured personal process could help me do better work and that programming development is a compelling vehich for introducing per— sonal process management. A more complex process than other personal activities, software development potenr tially includes many useful measures that can profidc engineers an objective evaluation of their work and the quality of their products. After the initial experiments, I need- cd to demonstrate that the PSP meth— ods could be effectively applied by other software engineers, so I had two gradu— ate students write several programs using an early PSP version. Because this early PSP was introduced all at once in one step, the students had difficulty. They tried some parts of the PSP and ignored others7 which meant they did not understand the overall process and could not measure its effect on their personal performance. This experiment convinced me that process introducticui was important; thus, the seven—step strategy evolved. Early industrial PSP experiments corroborated the importance of process introduction. Various groups were will— ing to experiment with the PSP but until these methods were introduced in an orderly phased way no engineer cone sistently used die PSP. In one 2group, for example, project engineers defined per— sonal and team processes and commit— ted to use them. Although a few gath— ered some process data and tried several methods, no one consistently used the full process. The problem appeared to be the pressure the engineers felt to complete their projects. Management had told them that using the PSP was more important than ineenng the pro- ject schedules, but they still felt pres— sured and were unwilling to use unfa— miliar methods. Learning new software methods involves trial and error, but when faced with deadlines engineers are reluctant to experiment. While they might intellec— tually agree that a new practice is an improvement, diey are reluctant to take a chance and generally fall back on familiar practices. I was thus faced with a catch—22. W'ithout data, I couldn’t convincc engi— neers to use the PSP. And unless engir neers used the PSP, I couldn’t get data. Clearly, to obtain industrial experience, l needed to first convince engineers that the PSP methods would help them do better work, so I decided to introduce the PSP with a graduate university course. By introducing PSP methods one at a time and with one or two exerr cises for each, this course would give engineers the data to demonstrate how well the PSP worked, without the pres— sure of project schedules. PSP METHODS Among the software—cuginccring methods P5P introduces are data gathering, size and resource estimat— ing, defect management, yield man— agement, cost of quality, and produc— tivity analysis. I discuss these meth— ods hero with some examples that merge data for several PSP classes and for multiple programming lan— guages. As you can see from the sta— tistical analysis box on page 81, it makes sense to pool the PSP data in this manner. MAY 1996 TA'I'ISTICAL ANALYSIS OF PSP DATA The analysis of variance test was applied to data for 88 engineers from eight PSP classes. Program sizes, devel— opment times, and numbers of defects found were all sep— arately tested. The results are shown in Table A. Since F(80, 7) at 5 percent is 3.29, the null hypothesis cannot be rejected in any of the pro— gram 1 cases. For the analy- ses in this article, the various class data are thus treated as a single set. Data for program 10, die last PSP exercise, was simi— larly examined. Here, the the null hypothesis could not be rejected. In this case, F(52, 4) at 5 percent is 5.63. Again, for this article. data from all the PSP classes are thus pooled for the analyses. The analysis of variance test also examined potential performance differences caused by six different pro— gramming languages used in the PSP classes to date. Only three had Substantial use, however: C was used by 46 engineers, C++ by 21, and Ada by 8. The other languages were Fortran, Visual Basic, and Pascal. Only data on C, C++, and Ada were tested. As Table A shows, the variances among individuals were substantiale ly greater than those among languages, so the null hypothesis cannot be rejectr ed and the data for all lan— guages are pooled. Here, F(72, 2) at 5 percent is 19.5. The VVilcoxon matched— pairs signed—rank test exam— ined the significance of the changes in the engineers7 performance between pro- grams 1 and 10. The com— parison was made for one class of 14 engineers for total defects found per KLOC, defects per KLOC found in compiling, and defects per KLOC found in testing. The T values obtained in these cases were 5, 1, and 0 respectively. For N:15 and 0.005 significance in the one tailed test, T should be less than 9. In all cases, these improvements thus had a significance of better than 0.005. A repeat— ed measures test has also been run on these same parameters and all the changes were found to be significant. population examined was 57 engineers from five courses. The smaller population was used because two of the eight courses only completed nine of the exercises and one course. had not completed at the time of the analysis. As Table A shows, the analysis ofvariance test showed that Gathering dull]. The PSP measures were defined with the GoaleQuestione Metric paradigm.4 These are the time the engineer spends in each process 3 phase, the defects introduced and found in each phase, and the developed prod— uct sizes in LOC. These data, gathered ‘ in every process phase and summarized at project completion, provide the engi— neers a family of process quality meae , ‘ tent, unbiased estimates. When engi— ‘ sures: 9 size and time estimating error, 9 cost—performance index, e defects injected and removed per hour, 9 process yield, 0 appraisal and failure cost of quality, and Q the appraisal to failure ratio. Estimating and planning. PROBE is a proxy~based estimating method I devel— oped for the PSP that lets engineers use their personal data to judge a new prov gram’s size and required development time. Size proxies, which in the PSP are objects and functions, help engineers Visualize the probable size of new pro— gram components. Other proxies — IEEE SOFTWARE Table A Analysis of Variance — F Values function points, book chapters, screens, or reports V are also possible. The PSP estimating strategy has engineers make detailed size and l resource estimates. Almough individual estimates generally have considerable error, the objective is to learn to make 3 ‘ exercises can help engineers understand unbiased estimates. By coupling a defined estimating process with histori— cal data, engineers inake more consise neers estimate a new development in multiple parts, and when they make , about as many overestimates as undercs- ‘ timates, their total project estimates are more accurate. The estimating measure 1 is the percentage by which the final size or development time differs from the original estimates. Overall, engineers’ estimating ability improved moderately during the PSP course. At the beginning, only 30.8 percent of 104 engineers estimated within 20 percent of the correct pro- 1 ‘ gram size. For program 10, 42.3 per- cent did. For time estimates, 32.7 per— cent of these 104 engineers estimated within 20 percent of the correct devel— opment time for program 1 and 49.0 ‘ percent did for program 10. In general, individual estimating i errors varied Widely. Some engineers master estimating skill more quickly than others, so it was no surprise that some engineers improved considerably while others did not. Even though 10 estimating methods, they generally need more experience both to build an ade quate personal estimating database and to gain estimating proficiency. These data suggest, however, that by using PROBE most engineers can improve their ability to estimate both program size and development time. Planning accuracy is measured by the cost—performance index, the ratio of planned to actual development cost. For 3 the PSP course, engineers track the cumulative value of their personal CPI 3 through the last six exercises. Managing defects. In the PSP, all defects are counted, including those found in compiling, testing, and desk checking. When engineers do inspec— l tions, the defects they find are also counted. The reason to count all defects is best understood by analogy with filter design in electrical engineering. If you examine only the noise output, you can— not obtain the information needed to design a better filter. Finding software defects is like filtering noise from elec— trical signals — the removal process must be designed to find each defect type. Logically, therefore7 engineers should understand the defects they inject before they can adjust their processes to find them. A key P8P tenet is that defect man— agement is a software engineers person— al responsibility. If you introduce a defect, it is your responsibility to find and fix it. if the defects are not managed like this, they are more expensive to find and fix later 011.1 “31.52.. ' Ea? DEFECT TYPES ' Description spelling, punctuatto change manageme ’ declaration, duph» procedure calls and er error messages, ina configuration, timing, me design, compile, test,,or problems As engineers learn to track and ana— lyze defects in the PSP exercises, they gather data on the plmres when the defects were injected and removed, the defect (rpm. the fix...
View Full Document