SCHEDULE AND COST BUFFER SIZING_HOW TO ACCOUNT FOR THE BIAS - Leach, Lawrence

SCHEDULE AND COST BUFFER SIZING_HOW TO ACCOUNT FOR THE BIAS - Leach, Lawrence

Info iconThis preview shows pages 1–15. Sign up to view the full content.

View Full Document Right Arrow Icon
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 2
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 4
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 6
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 8
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 10
Background image of page 11

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 12
Background image of page 13

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 14
Background image of page 15
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: AND .CosT UFFE SIZING: HOW TO ACCOUNT FR THE Bins BETWEEN PROJECT PERFORMANCE YOUR MODEL V Abstract Bias in project performnca causes saltedulennd cost to overrun baseline estimates [your model). Bios is-‘ihe one- sided. tendency of actual scheduie or to oven-tin the model. A Guidete the Project! Management .Body of Knowledge Guide] and sup- porting literature tacammend estimot» ing the vo'fidbiliiy For: all project time and cosh-estimates, nd sizing qppro— prints schedule or cost buffets- (also known as contingency or management reserve) :using Monte-Carlo: analysis 'or PERT. This paper describes a number of sources of bins in performance of proij to sdhecible and cost esti~ trusts:‘ and provides recommendations to size buffers that-ensure your projects come in; under :yotir beseiina schedule. and: budget. Keywords: buffer: contingency; management reserve: PER-T; ‘Moi'ti‘a Carlo: critioul chain; CCPM @2603 by iqu men! Institute vol. 34. No. '2. 3-447, issu mamas/03 WHY EACH. Advonced Projects Institute. 5230 South Pegasus Way. Boise, [D 83716 USA rojects frequently Overrun the baseline schedule and budget. Despite the PPMBOKW Guide recommendation to state the uncertainty of all estimates (PMI, 2000. p. 73), many projects use single-point schedule and cost task esti- mates with no estimation of variation. In reality, all project costs and activity durations are variable. in addition, project duration and cost estimates are uncertain in part due to the duration and cost variability, but also due to the estimate's limited ability to predict accurately the real variation. While sched- ule modeling tools such as PERT, Monte Carlo. and critical chain make explic- it attempts to model the variation, project performance evidence (i.e., sched- ule and cost overrun) indicates that they frequently miss something. This paper proposes several factors that cause schedules to systematically overrun estimates, predicts the contribution to these ovenuns, and proposes a method for project schedule and cost estimates to account for these causes. It also addresses how to combine the correction for bias with a buffer for normal variation in task duration and cost. The following section describes and com- pares the various models, and the subsequent section proposes a number of potential causes of bias in schedule and cost performance relative to esti- mates. The paper concludes with recommendations for sizing schedule and cost buffers to account for both variation and bias. The Models Projects use models (e.g.. (3PM) to estimate (predict) schedule and cost. Most project management literature recommends project schedule and budget esti- mates that include specific buffers as allowances for a contingency reserve (CII, 1989; PM]. 2000), or a buffer. Most of these recommendations assume using the critical path method (CPM), which generally attempts to use low- risk (pessimistic) individual task duration and cost estimates. In this discus- sion, the CPM corrections assume a CPM schedule made up of low-risk task estimates summed along a path (schedule) or budget summed for the entire project (cost). The PERT model uses optimistic. most likely, and pessimistic estimates-of task duration or cost to estimate the mean and standard deviation for each task. The PERT method assumes that the difference between the optimistic and pessimistic estimates is some multiple of the standard deviation. The method then applies the square root of the sum of the squares (SSQ) of the differences to estimate standard deviation of the sum, be it the length of the critical path or the total project cost. Some texts recommend using this with a table of the normal distribution to estimate total duration or cost versus probability. and to size the appropriate buffer. Although often illustrated for cost, the same PERT model applies to schedule estimates along the project criti- cal path. The calculations start with estimating the mini— mum or optimistic (0), most likely (M), and pessimistic (P), time or cost for each task. Optimistic means lower [duration and cost) and pessimistic means higher cost or longer duration. i take issue with these categorizations, as in a statistical or mathematical sense there should be no attri- bution of motivation or emotion on these numbers. The various estimates are merely points on a statistical distribu- tion. As points, they have equal probability: zero. Legitimate statement of a finite probability require a range for the vari- able, e.g.. "the probability that the result will be less than or equal to x is y.” Or, "the probability that the result will fall between it and y is 2." Most discussions of PERT, including those in the current version of the PMBOK® Guide [PM]. 2000) and supporting documents (Wideman, 1992), make an assumption of six standard deviations between the extreme estimates. I will clarify below how this is one cause of bias. They recom mend calculating the standard deviation, 5 as: leP—QI 6 The common recommendation for the mean activity time or Kb“ is usually stated as deriving from the beta dis— tribution and given as: (1) X1“, = O+4M+P 6 Meredith and Mantel (1995) note that the bar equation "could not be derived from the formula for the beta distri- bution without several assumptions that are not necessarily reasonable" (p. 344). The parameter assumptions for this trace back to the original PERT publication (Malcolm, Rosebloom, Clark, 8t Fazar, 195.9), and are not addressed by many recent publications (e.g., PMBOK® Guide, 2000]. This {2} deficiency is well known by researchers, and alternative “more accurate” equations have been proposed (e.g., Huang, 1999). The authors put in quotes and as noted also by some of these researchers, the uncertainty in actual task data usually far outweighs the variation introduced by the choice of mathematical model. Table 1 illustrates an example PERT calculation. When applying the standard CPM model, the individual task esti- mates (cost or duration) must use the pessimistic estimates, or P column if the proiect is to complete on time and with- in budget. If this calculation represented the critical path task durations, the total CPM proiect duration would have to be 45 days. The calculation of the total standard deviation uses the statistical method for pooling of variances. It calculates the standard deviation for the SSQ of the standard deviations For each element included in the sum. Although not stated in most PERT discussions, the nominal values used to communicate the schedule or cost should be the mean (KW) values. For example, you should put the Xbm values should be placed into the schedule for the duration estimates. The reason is that statistically, only the mean values add in a straightforward way. Thus, the schedule as a chain of tasks sums to 27.5 days. There is about a 50% chance of completing within the nominal PERT schedule of 27.5 days; so must be added some number of standard deviations to that amount to esti- mate the duration for the total project. The author recom- mends using two or three standard deviations, so you should add up to 7.5 days to estimate the project comple- tion time, for a total PERT project duration (includingI buffer) of~35 days. Monte-Carlo analysis applies a more sophisticated method to implement the idea underlying PEREZ instead of fixed formulas, estimate a distribution for each element, and instead of a defined calculation, Monte Carlo runs a simulation that randomly selects points from the individual distributions I...“— 2 3 4 5 6 1O 2 3 4 7 Table]. Example of PERT Calculation 10 12 B 15 45 To 5.33 5.83 5.00 0.67 10.33 1.33 27.50 2.50 7.50 35.00 1.33 1.50 1.78 2.25 0.44 1.78 6.25 June 2003 Proiact Management Journal 35 to create a distribution for the sum. Some Monte Carlo sim- ulations allow simulating the schedule as well as the cost. The schedule simulations should account for the schedule path merging effect: an effect not important to cost calcula- tions. Path merging means a critical path activity has addi- tional predecessors from one or more non-critical paths through the proiect. Early PERT calculations did not account for this merging (Wideman, 1992), as they only looked at the critical path. Critical chain project ntanagement (CCPM) includes buffers as essential features of the process. The CCPM buffer sizing method uses the same statistical basis as PERT, but only tw0 estimates for the task duration: a most likely and a low-risk estimate. CCPM (Leach, 2000) does not make a specific assumption about the number of standard devia- tions between the two numbers. it assumes that the same number of standard deviations is acceptable for the pooled result as was inherent in the task estimate ranges. The dif- ference between these two estimates is the uncertainty, u, in task estimate (duration or cost). Leach (2000) describes several schedule buffers and an overall project cost buffer. He recommends using the SSQ method to size buffers, along with a minimum project buffer size of 25% of the o’itical chain, and a 10% minimum cost buffer to account for bias. Table 2 shows the CCPM calculation corresponding to the PERI‘ illustration of Table l. Table 2. Example CCPM Calculation Do not attach significance to the results for PERT and CCPM being close. This is a specific (unplanned) artifact of the example, and not a general result. In general, the critical chain is not the same as the critical path, so the results can be significantly different. The CPM result always will be greater than the PERT or Monte Carlo result. The CPM schedule result with resource leveling always will be longer than the critical chain result. The sum of the mean estimate plus the buffer always is less than the sum of low-risk (pessimistic) estimates for the individual tasks. Thereforer the description of adding a buffer can be misleading. As Tables 1 and 2 illustrate, compared to the sum of pessimistic or low-risk estimates (CPM). the mean probabilistic estimate with buffer always is less. 36 Project Management Journal June 2003 CCPM buffer sizing recommendations follow the statis- tical logic used in PERT. The sizing uses two statistical facts: (1) The variance of the sum (schedule path or total cost) is the sum of the variances of the elements; i.e., task duration or cost. and (2) the central limit theorem, which states that the distribution for a collection of samples will tend toward a normal distribution, regardless of the shape of the distri- bution from which the individual samples are drawn from. When using PERT. Monte Carlo, and CCPM, should be used the mean duration and cost estimates as the baseline task estimate. The mean is the only unbiased estimate of the sum along a path or for the entire project. For these tech- niques. the bias corrections discussed next are relative to the sum of the mean task estimates along a path, or the total sum of the elemental costs for a project. Note (in Table i) that this sum may be substantially less than the sum of the pessimistic (low-risk) estimates used in basic (PM. The next section illustrates a major gap between the real world of project performance and the models addressed previ- ously. leading into the author's main hypothesis that it is bias in schedule performance to plan that causes this gap. The sub: sequent parts of the paper pose numerous potential causes of this bias, and the final section provides recommendations on how to account for them in your project plans. The Real World Real projects usually have hundreds and frequently thousands of activities, and the largest projects may have tens of thoua sands. Schedule and cost buffers estimated by this variance pooling method become a very small percentage of the total duration and cost. As the number of activities pooled in the variance grows, the variance of the pooled mean as a percent- age of the mean reduces. You can use PERT or the (ZCPM SSQ method of buffer sizing to estimate this impact. Monte Carlo simulations show the same general behavior; i.e.. decreasing relative total variation with increasing project size. Figure 1 illustrates the relative size of a pooled variance sized buffer decreasing as the number of tasks increases. The figure assumes equal size tasks, each with a low-risk estimate twice the average estimate. For CCPM project or feeding buffers, the number oftasks is the number in the respective chain of tasks (path). Because cost adds for the entire proj- ect, the cost buffer for the number of tasks includes all of the tasks in the project. For four tasks. the SSQ buffer is 50% of the sum of the average estimates. For 128 tasks. the SSQ buffer decreases significantly: to only 8% of the critical chain. Illustrations in project management books. the PMBOK® Guide, and the CII risk management booklet (CH. 1989) illustrate using the PERT method with a few activities. sometimes as few as three, and rarely more than 10. For example, the PMBOK® Guide (PMi, 2000). Figure 11-4 shows an example with only three tasks: Design, Build. and Test. The figure shows a low, most likely. and high estimate for each taskr and then illustrates how the PERT method would require a contingency of 22% above the meanr which is higher than the sum ofthe most likely (median) estimate to 128 8 16 32 64 NumberafTasks Figure 1. 550 Buffer Variation With Number of Tasks achieve 75% likelihood of success. This particular example is for cost, but the same math applies for the schedule along a path (eg, critical path or chain). For the PMBOKE Guide example. the individual activity range was on the order of the minimum estimate, or i 50% of the most likely. If considering a more realistic project case with 100 tasks, all with that same range for each activity as the PMBOK® Guide example (i.e., i 50% of the mean). the buffer (using exactly the same method) comes out to less than 4% of the mean sum for the 100 tasks. Construction projects fare better than many other types of projects, yet construction bids on well-defined projects typically range more than 120%. Large construction projects [e.g., Chunnel, Denver Airport. Boston Big Dig) have increasingly impressive cost and schedule overruns. The PERT, Monte Carlo, and SSQ buffer sizing methods reduce the relative size of the buffer, as the number of tasks grows large. This implies that larger projects should be more likely to come in on budget and time; i.e.. the trend of the 300 100 g o -- do -200- F *- . i _. o 0 Figure 2. Real Project Schedule Variation illustrates Wide Range Delta Days Control Chart Vtw . HQ —__———_.—__—__ . I\ . , . . . y _._ .,'_-..._ :_. .I . .. _ I .. E303 527 435 «a 542m; 93 535 533355 3647‘! 551552 554465 556 sets sstgtsaflgsssr gal-:32 501 537:: 599642 645603 . i . . - f-oqqo-oov-r-o-o-vo model is toward increasing project success (closer to esti- mate] with increasing project size. l-Iowever, real project data shows project success rate decreases with increasing project size. A Standish Group (Johnson. 1999) study calls it success that the average overrun (bias) on inforrntion technology projects is down to 60%, from more than 200% in 1994. It shows a strong (negative) correlation of project success with size. the opposite effect predicted by the PERT, Monte Carlo and SSQ methods. No projects ntore than 10 million dollars were successful. Similar results were obtained for other types of projects. Thus, predicted trends of the PERT, Monte Carlo and CCPM SSQ model do not cor- rectly describe the trends of reality. Figures 2 and 3 illustrate control charts for a series of [T projects completed in the year 2001 by an IT organization. This is real data for a relatively large Fl" organization in a company with m $2 billion/year revenue. These data are rep- resentative of the data reported by the Standish group. The numbers along the axis identify specific projects. Control charts plot the data versus the order of the data, and three reference lines. One reference line is the mean of the data. The other two reference lines are the upper and lower control limits representing the variation in the data. The control limits are derived directly from the data also using predetermined formulas. For a process in statistical control, one point per thousand should be outside the con— trol limits. Other indicators also illuminate points outside the range of common-cause variation. The organization that generated the results shown in Figures 2 and 3 underwent a transition at the beginning of 2001, adopting a fomtalized project delivery process. Thus. the projects were sorted by start date, and separate control limits were plotted for those before and after the introduo tion of the process which follows the PMBOK® Guide and the Software Engineering Institute's (SE1) Capability Maturity Model (CMM) (SE1, 1994]. Examination of the control chan illustrates a key point. If mean estimates are used for tasks, and if you want projects to complete at or under the estimated duration and cost, a buffer 200 II-II-II‘II'IIII‘II'II-III-I hill-lll-ii-‘I ’ .t 6 o Target-Act - -I- - UC]. -—u— Xbar - «v - LCL June 2003 Project Management Journal 37 Delta 5 (‘36) Control Chart 600.00% l 400.00% 200.00% 0.00% - 200.00% - 400.00% 600.00% -800.00% Figure 3. Real Project Cost Variation Illustrates Wide Range must be added to the mean estimate. if all projects must come under the promised schedule and cost, the lower con- trol limits must be at zero. Therefore. the average perform; ance must be above zero by at least the variation amount. The variation amount equals one half the distance between the control limits. The amount of buffer must equal the bias plus this variation amount. The bias is the amount that the average is below zero without a buffer. Note that the two example figures show significant negative bias before adopt- ing formal project management, and much less after the implementation. Data for Figures 2 and 3 cost and schedule demonstrate two distinct groupings. The left half of the chart is a process that is not in statistical control. it has wide variation, and points outside the control limits. The mean is negative, illus- trating a pattern of schedule and cost overruns. These data seem typical of the data that made up the Standish Chaos study. The data on the right half of the charts indicate that the process has been brought into statistical control, and thus the data on the control charts are useful to predict future performance. Although difficult to discern due to the large scale necessitated by the earlier data. the control limits are still relatively large: a 130 days (compared to an average duration for these projects of 162 days; thus about 1: 90%). and t 45% cost. The means {Xbflr} are a 10-day schedule overrun, and a positive 4% cost variance. The random component of this variation appears much too large, and the bias component too small, relative to the models described in this paper. We shall return to this point after discussing the causes of bias. Some of the projects in the later pans of the sample (right hand side of the figures) were beginning to use CCPM for schedule management, but none of the projects includ- ed a cost buffer. The narrowed control limits after process stabilization imply a needed schedule buffer of 150 days. and a cost buffer of 50% to ensure that most future projects complete within the estimates. as Project Management Journal June 2003 There is some confounding of the schedule data, as the projects completed in the year sorted by start date led to most of the shorter duration projects being on the right half of the control charts. The data trend agrees with the state- ments about the more global Standish Group data, in that there appears to he a trend of greater variation for longer duration (and higher cost) projects. Further improvement [narrowing of the control limits) will occur as project teams become skilled in the project planning and delivery process and as higher maturity level process elements are added. Nevertheless, a dramatic reduc— tion in the control limits has occurred from the simple introduction of a defined. repeatable process. As noted in the Standish Group information, size does matter. Factors other than random fluctuations clearly are at work. The method of pooling variances is not in line with real experience. This difference is bias in project performance relative to the estimate assumptions. The subsequent discus- sion poses some explanation of the observed real world data. Whether one uses CPM or CCPM (Leach. 2000} influ— ences how much bias adjustment may be necessary. Because specific features of CCPM work to reduce the schedule impact of bias. The unique CCPM features that impact need- ed schedule buffer bias corrections are: - Feeding buffers placed at the end of chains of activities that merge into the critical chain; - Run rules that specify work on a project task must start when the activity is available and continue at 100% until complete; 0 Assured in-project resource leveling through the definition of the critical chain; 0 Across project resource leveling through project priority setting and staggering of project starts to the capacity of the constraining resource with a capacity buffer; I Buffer management. which provides all project resources a priority tool enable focused work on project tasks. This balance of this paper identifies ll factors (there are probably more!) that influence project performance to schedule and cost baseline, evaluates the impact on neces— sary schedule and cost buffers, and provides recommenda tions for bias corrections. Bias A bias is anything that might invalidate pooling of variances of the individual tasks to size schedule or cost buffers. Since project success is judged by a single-sided (undersrunj com- parison to planned schedule and cost, we address only pos- itive bias factors that may systematically increase sched- ule or cost performance relative to plan. The following addresses 11 potential causes of overall underestimate of schedule and cost. A general rule of thumb for how many independent samples are required for statistical pooling of variance to be effective is about 25-30. With small samples, the sample mean and variance can vary widely relative to the population mean and variance. Larger samples more accurately portray the population. You need the law of large numbers (a statis- tical rule) working for you to successfully apply pooling of variance. The reason is that one low probability outcome can cause the mean of smaller groupings to vary by more than the pooled variance. This generalization depends on the rel- ative variance of the independent trials, and the sensitivity of concern. Projects with less than 30 activities must use more sophisticated statistical analysis to obtain a meaningful esti- mate of the probability of success. The following discussion applies to projects with more than 30 activities. Fewer activi- ties might demand larger relative (percent) buffers. Omissions Project plans often leave out activities essential for the com— pletion of the project, such as documentation, handling interfaces with the existing infrastructure or other projects. or premium payments to ensure schedule. Project plan reviews normally will eliminate any items not necessary to deliver the project functions but review of a plan often does not completely identify potential omissions. Some organizations use extensive checklists, prior proj- ect plan templates, and lessons learned to help avoid omis- sions. These organizations will suffer less from this bias. Such practices are common in organizations that perform fixed»price contracts and unusual in organizations that per- form cost~plus connects or internal projects. Project plan and design reviews also are more common and rigorous in the fixed-price project organizations. Potential omissions vary by project type. environment, effectiveness of process control, and project manager and team experience. Companies that perform fixed-price proj- ects often use extensive checklists and project plan reviews to minimize omissions. Consequently, omissions may range from 5% to more than 15% of the overall project cost esti— mate. The impact on schedule is more difficult to generalize, as the impact of minor omissions may be included in low- risk duration estimates, or omitted activities may not be on the critical path or critical chain. Merging The PMBOK® Guide {PM}, 2000) notes that merging project paths causes systematic increase to the expected duration of a project, as compared to performance of the individual paths. The bias occurs because the successor task at a merge point (where two or more tasks are predecessors to one task) cannot start until both the predecessor tasks are complete. Figure 4 illustrates the impact for two—10 merging paths, compared to the distribution for a single path the critical path or chain. Figure 4 calculations assumed a normal distribution to represent each path. This assumption results from applying the central limit theorem to a chain of tasks. The mean of the distribution is five (days, weeks, months, years), and the standard deviation is two. Recall that the central limit theo- rem ensures that the distribution at the end of a path of tasks will approach the normal distribution as the number of tasks in the path increases, regardless of distribution of the duration for each of the tasks in the path. Merging Effect: No Feeding Buffer 9.00 5 w, -- D ,_, m -- a s: 5.00 4 _ 400 -- g m -- g == g “’0 -- E 0.00 Number of Fun: Figure 4. The Merging Effecl Causes Lute Bios Merging With 50% Feeding Buffers Number of Paths Figure 5. Feeding Buffers Mitigate Merge Bios June 2003 Project Management Journal 39 Note that for 10 merging paths, the mean increases by 60% more than the nominal chain merge date. Even though the variance of this mean is reduced [because it is averaging multiple paths), late project delivery is a near certainty. CCPM applies feeding buffers to the paths merging into the critical chain to reduce this bias. But CCPM makes no specific allowance for the number of merging paths. In large projects, the number of merging paths may exceed the capa- bility of the feeding buffers, combined with the sizing of the project buffer, to protect the project end date. Figure 6 shows the effectiveness of the feeding buffer at reducing the increase in mean merged path completion time (or start of the successor activity on the merged path). This curve assumes feeding buffers of 50% of the path length. Smaller feeding buffers will be less effective at providing protection horn the merging effect. Even with the 50% feed- ing buffers, the feeding buffer effectiveness is waning by time there are 10 merging paths. The bias grows to nearly 20% compared to the single path mean time of five. The reduction in variance can compensate for some of the growth in the mean, if the project buffer is large enough to absorb the bias. Still. the bias is one-third that of merging paths without feeding buffers. The merging impact on schedule also may bias the cost estimate in two ways. First, often is required to pay for resources engaged on the waiting paths. For example, if the truck is late to pour concrete, you may have to pay the con- crete finishers who arrived at the job site to work on the con- crete. Second, an increase to overall project duration also increases the cost of all of the level-of-effort (LOB) support to the project. The LOE impact on cost is not as great as the direct schedule delay impact because the direct schedule impact causes the cost for resources estimated within the project tasks to increase. while the LOE cost impact is because of due to extending the time of using the LOE resources. normally budgeted outside the project task net— work (or with a pseudo-task to not impact the critical chain). Errors Errors only can increase duration or cost. All projects have some degree of error in the product produced. Few projects explicitly include time or cost for error correction. other than planned test and rework cycles. Errors detected early in the project introduce much less bias than errors discovered late in the project, placing a premium on quality assurance processes that focus on error prevention. as compared to quality inspection oriented processes. Project error cost can range easily up to 50%. Schmidt and Kiemeie (2002) discuss the cost of poor quality (COPQ). "According to quality gurus such as Deming, Juran, etc, the percentage of COPQ in most companies is 20-40% [of total revenue]." The Standish Group Chaos study (lohnson, 1999]. examined the cost of poor quality on [T projects, and noted outright project failure (i.e., 100% loss) exceeded 28%, and only about 15% of projects complete within 20% of estimate. Except for the highest quality organizations (i.e., Six Sigma 40 Project Management Journal June 2003 organizations). the cost of poor quality is at least 5%. Errors can introduce schedule bias if they impact time on the criti- cal path or chain. Overconfr'dence Extensive research into people's ability to estimate probabil- ity (Kahneman, Slovic, s Tversky, 1982: Morgan S2 I'lenron, 1990) demonstrates that people are overconfident in their ability to predict the range of outcomes. The author noted earlier that PER’T makes the unsupported assumption that when people estimate extremes for their cost or schedule duration estimates that the range of these estimates repre- sents plus or minus three standard deviations. lfso, it would mean that >99% of the results fall within these extremes. Experiments consistently show that the actual results fall outside people’s range in excess of 10% of the time and often up to 50% of the time. Some of these experiments include experts in the respective fields, who show equally poor or worse performance than novices. This means that most range estimate really are only plus or minus about one to two standard deviations. When using PERT, divide by two or four rather than six to estimate the standard deviation because your estimates of the range are more likely to correspond to plus or minus two standard deviations than to plus or minus three stanA dard deviations. Similar reasoning applies to fitting Monte Carlo distributions to cost and duration estimates. The probability calculations are at the task level, this effect is not significant on lager projects, as the random contribution to the required schedule or cost buffer is small compared to the bias elements discussed in this paper. Using critical path plans directly; i.e., without PERT or Monte Carlo, the project may be completed on time by completing each task on time. If. instead of exceeding task times or cost 1% of the time they are exceeded 10% to 50% of the time, an equivalent buffer; i.e., 10% to 50% of the path length or total cost estimate must be added. This bias does not directly affect CCPM, because no assumption is made about the number of standard devia— tions inherent in the estimate ranges. Queuing Queuing is the build up ofa line ofwork waiting to be per- formed by resources. The build up can cause very long lines if the average demand for resources approaches the average supply (called the service rate in queuing theory). Figure 6 illustrates the classic steady-state solution for a queue with Poisson arrivals and exponential service times. The vertical axis represents the number of tasks (length of the line of tasks) waiting to be worked on by a resource. The utilization is the average demand divided by the average service capacity of the resource. The surprising thing about queues is that the steady-state average line length (wait time) can be quite large, even with seemingly modest uti- lization of resources. Note that as average demand eXCeeds only 80% of capacity, the line begins to build rapidly. Many organizations seek to maintain 80% "billability" of resources to project work. This practice ensures that projects will have to wait for resources, and may cause project resources to multitask, with much more serious conse- quences (addressed next). Line Length Length of line Q: '5 Q: 9?, Q9 Q9 Utilization Figure 6. Queuing Causes Lore Bios at Modest Resource Loading An informal survey of project managers at PMI meetings around the country confirms that many CPM projects do not level resources. This means queues are likely to form for any resource in a project. Many multiproject organizations do not use resource capacity to control the release of projects into the organization. leading to the potential for resource queues with tasks from multiple projects. in both cases. nor- mal fluctuations in actual task duration will cause tempo- rary queues, even if average resource loading is as low as 80%. CCPM always levels resources within a project. This is how CCI’M determines the critical chain. CCPM also seeks to explicin avoid queues for the constraining resource in a multiproject environment [called the drum resource in CCPM) by staggering project starts using a capacity con- straint buffer between scheduled demands for the drum resource on different projects. A small queue for the drum resource maintains organization efficiency by assuring that the drum is not starved for work (at the expense of project delay). Undersizing the capacity buffer can lead to queues for the drum resource. Although other resources in the organization are, by definition, more lightly loaded than the drum resource, ran- dom variations in task performance can cause lines to build up for tasks by these non-constraint resources. This also can lead to schedule delays. It introduces a schedule bias because there is no such thing as a negative queue; resources can't work on inputs they do not have. Queuing causes a cost bias through the [DE effect described next. Multitasking Multitasking is a major cause of CPM schedule bias. Many (3PM projects plan on multitasking; that is, the duration controlling resource is planned to work less than 100% on the project tasks. Multitasking causes project delay for three reasons: I Project tasks wait while the individual works on a task in another project, or on another task in the same project. Figure 7 illustrates this effect for three project tasks. All three tasks take three times as long clue to three-task multitasking. Many people report much more multitasking. 0 Task switching eflicienql loss. Studies show there may be up to 40% efficiency loss. 0 The network delay of multitasking. Figure 8 illustrates this impact and shows how CCPM resolves it by sequencing proj— ects. The exhibit illustrates two identical simple single-path projects, assuming a finite resource of each color. In the upper case, the resource splits effort bemoan the two projects. In the lower case, the resources focus on one project, and then go to the nerd project (as illustrated by the lower case in Figure 8). Figure 7. Multitasking Causes All Taslr. Durations to Extend June 2003 Project Management Journal 41 Not accounting for the task switching efficiency, Figure Sshows that both projects complete in less time by synchro- nizing the start, (i.e., delaying the start of the second proj- ect) to enable avoiding the task switching. This is called the network effect. The schedule extension from multitasking also impacts cost, both through the task-switching efficiency loss, and through the LOE effect described next. Special-Cause Risk Events Special-cause risk events are the realm of conventional proj— ect risk management (e.g.. Wideman. CII). Special—cause risk events may cause schedule delay or cost increases In the strict sense, project risk assessment also seeks to take advan- tage of opportunities. Project plans likely would build iden— tified opportunities into the baseline plan as they are dis- covered, leading to a negative cost and schedule bias. Such special-cause events should not be pan of the variation within the individual task performance duration or cost ranges. They are due to identifiable discrete events outside the control of the resources performing the planned project tasks. A common mistake is to confound special-cause and common-cause risk, such as addressed by PERT. Monte Carlo, and CCPM (Leach, 2001). The overall approach to risk assessment seeks first to mitigate such risks by planning and executing actions to reduce the probability or potential impact of a risk event. The final step in risk management is to estimate a schedule and cost buffer to absorb the impact of the collective prob- ability of the residual risk events not identified by project assumptions or specifically excluding such items from the project scope. This is an allowance for bias. Methods used to size the buffer include PERT and Monte Carlo analysis. Risk events are sorted to a limited number of high-impact items. The discrete risk items identi- fied in these analyses are special-cause risks. it is impossible to determine all special-cause risks. especially those in the category of "Unknown unknowns.” High impact means that the range ofthe impact may be many times the baseline cost or schedule of the affected task or tasks; sometimes exceed- ing the time or cost estimate for the entire project. Thus, PERT and Monte Carlo analysis combining speciallcause variation may not suffer the SSQ effect of much reduced size as a percent of baseline. Large projects with thousands of tasks may have only tens of items in the risk assessment. Student Syndrome Student syndrome is the task-performance bias introduced when task performers delay starting a task until they feel task due date pressure. This usually occurs because they are busy on other tasks. T hen, if everything does not go right, they overrun the estimated duration. This is devastating in CPM projects, where only one late task on the critical path makes the entire project late. It is a serious problem in many 11‘ projects because lT projects tend to be massively parallel; that is. the development of many modules proceeds in par- allel. If one module is subject to this behavior, the entire project can be late. To some extent, properly applied PERT, Monte Carlo. and CCPM exen a positive influence, relative to CPM. that reduces this behavior by measuring performance to mean estimates rather than to low-risk estimates. This activates urgent work on the task sooner. CCPM seeks to remove this bias by emphasizing Relay- racer or roadnmner task performance, i.e., starting a task as soon as the inputs are available (racer has the baton), work- ing on it fully until complete, and immediately passing on the result (handing on the baton to the next racer]. To the Individual Project Schedules (Note resource contention — Synchronized Multiple Projects Accelerate and Reduce Synchronizing (Constraint) Buffer Constraint Resource Figure 8. Network Effect of Multitasking Causes All Projects to Extend; Sequencing Accelerates Completion of Both Projects 42 Froin Management Journal lune 2003 extent that behavior is not achieved, student syndrome can contribute to schedule bias. It should not contribute to cost bias because it does not add to the hours spent on the task. student syndrome even may have a positive impact on cost, as people working extra hard to catch up frequently do not charge their overtime to the project. Policy Driven Behavior The two words most frequently heard at project status meet- ings are “on schedule.” If mean estimates are used for dura- tion about half the tasks at any schedule status meeting will be ” on schedule." if you use low-risk estimates. you should hear “ahead of schedule” most of the time. [All ofthe chil- dren in Lake Woebegone are above average.) informal studies of actual completion times to schedule completion times show that up to 80% of tasks are reported as complete right on the scheduled due date. Two possible explanations of this are that (a) people make very low-risk estimates, or (b) they believe it is acceptable to hold on to work until it is "due." if (a) were true, most projects would be completed very early. Indeed, many people use the evi— dence of late projects to support the opposite assertion: i.e., certain kinds of people such as engineers and, scientists make very aggressive estimates. In most organizations. cul- ture reinforces the latter understanding and behavior. That is, management gives positive feedback for work completed on time or within budget and negative feedback for over- runs. This encourages people to try to make low-risk esti- mates. In addition, if one delivers substantially under the budget or estimated duration, management in many organ- izations exerts pressure to reduce subsequent estimates. Thus, there is a bias to achieve very little of the positive vari- ation that actually may occur. People will turn things in on time, but net significantly early. The behavior usually means that people do extra quality checks or may "polish the apple" until the due date arrives. In some circumstances. supervisors put aside work turned in early, until the due date is near. This supervisor behavior is another manifestation of the student syndrome. in cost-plus contracting arrangements, including proj- ects within matrix type organizational structures or any type of cross charging, there frequently is a requirement to keep up billab‘ility on the part ofthe resource managers. Thus. the managers are unlikely to turn work in early and lose the compensation for the authorized hours. The date-driven bias violates a major assumption in all of the probabilistic methods to estimate the required buffer. All methods assume that task performance will follow a probability distribution that includes actual duration or cost results below and above the mean. Date-driven behavior thus eliminates the left side of the statistical distributions. This causes the mean of the remaining (actual performance) distribution to move to the right adding a bias to increase both the duration and cost of the entire chain of tasks. This bias is some percentage ofthe relative standard deviation of the tasks, i.e., on the order of tens of percentage points of total duration and cost, depending on the amount of variation in the individual tasks and shape of the distributions. Simulations run with a normal distribution where the stan- dard deviation was 40% of the task mean yielded a 17% bias in the aggregate. CCPM explicitly seeks to avoid date-driven behavior by not using task due dates or authorizing specific task budgets. To the extent this is not achieved, the existing practices require accounting for bias in the cost and schedule buffers. Failure to Report Rework Requirements Reports suggest that there is a frequent bias to not report known changes that require rework. Ford and Sterman [1998] suggest that this is the primary cause of the 90% syndrome, where projects are able to achieve 90% complete (reported) in one unit of time, and require additional time to complete the final 10%. They suggest a number of implick it policies and some human behavior attributes cause this behavior, including: 0 Reluctance to report bad news, coupled with, "if I wait long enough, someone else will report a bigger slip than 1 have; " 0 Fear of the messenger being killed; 0 Temporary reduction of work loads, improving apparent schedule progress; I Reduction of paperwork associated with reporting the need for rework: I Hiding problems permits problems from other areas to take the limelight, thus allowing time to recover; 0 Maintainng adequate apparent progress reduces management intervention. Concealment behavior can lead to a bias in schedule per- formance due to delaying the knowledge of rework required and overloading the resources that are responsible for the rework. This can contribute to queuing bias. The impact can be large. In the actual project and simulation analyzed by Ford and Sterman, "the concealment policy delays the last portion of the project into the characteristic pattern of the 90% syndrome, causing the project to be completed 87% (35 weeks) later titan originally planned and 11 weeks (17%) later than the project management without concealment. This potential bias is very situation specific. It argues for spending whatever is necessary to ensure true scope verification. Level of Effort (10E) LOll causes a potential cost bias for the project. By definition, it does not impact schedule. All projects have some amount of LOF. activity. This is work that occurs at a certain rate over the duration of the project and is not associated with specific proj- ect tasks. Examples include the operation of the project ofiice, including the salaries of the project manager, administrators, and project—level technical personnel such as communication specialists, quality assurance. safety, and environmental protec- tion personnel. LOP. costs can range up to 20% of the project cost but usually are more on the order of 10%. Their unique feature is that they extend over the total actual duration of the project, so any penetration into the project buffer extends the LOE cost for the project. June 2003 Project Management Journal 43 The final sections ofthe paper summarize all of the poten- tial cost and schedule bias impacts, and provide recommenda— tions for how to account for them in your project plan. Summary of Bias Impacts Table 3 summarizes the identified potential causes of bias, and the estimated range of impact on schedule and cost buffers. The final sections of this paper describe the conclu- sion when from considering all of the arguments, how to use Table 3 to size buffers, and a path For future improve- ment of your project delivery system. Back to the Real World (Conclusion) The author illustrated why the model method and the trend implied by PERT, Monte Carlo, and CCPM SSQ models (i.e.. smaller percent buffers for larger projects) fail to account for schedule and cost overruns, especially on larger projects. We have isolated a major potential cause of project cost and Bras Factor Merging < 20% CCPM Errors Queuing Schedule Impact somEI "at to exceed coat 50th for CPM. less for Monte Cane that calculates schedule sat-25% sat-50% meson CPM 0"” WWW“ More for PERT, Monte Carlo None CCPM CPM: most. or more if no resource leveling in or across projects schedule overruns as failure to account for identified biases in the actual performance of projects compared to the models. The potential bias causes identified cover a wide range. Table 3 summarizes the biases and illustrates overall ranges. Please note: 0 There is not a firm technical basis for some of the estimates; 0 The bias range is different for every organization. depending on the organization behavior; 0 There is no theoretical method to sum the various biases. It is unlikely that one could sell project customers on buffers as large as 100%. Senior managers have been known to criticize project managers proposing a buffer of only 25% in stating, "If estimates cannot be more accurate, the proj- ect should not done.” If you bid projects competitively, you would never win a contract with buffer estimates of 100% or more. Cost impact None to small (controllable) Nth-50% CPM PERT, Monte Carlo ~30% None CCPM No dlrect impact (See LOE) CCPM: Nominal (Resource leveling 81 M ultitaskin Student Syndrome 6PM: 10%-20% (Startan late) CPM: Several hundred fit: or more GCPM‘. Small (feeding buffers) Up to 150% (efficiency loss}. plus LDE impact None to positive Less for PERT and Monte Carlo it mean estimates used in baseline Failure to Report Rework 0%v20‘lta None Total CCPM Table 3. Recommended Bios Correction Adjustments to Buffers 44 Project Management Journal June 2003 LOE 10%-25% (Project Butler) More with many parallel tasks CCPM.‘ Smll Buffer mane ement Date Driven Behavior (Not reporting CPM: 10%-20% early °°mplefi°nl CCPM: Small Run rules) Covered by errors LOE rate times schedule delay 10% 50% (Cost Buffet) 10% 60% (Cost Buffer) 1035-2536 (Cost Buffer) Fortunately, most organizations that do projects do not execute just one project. They do many. Their projecr plan- ning and delivery process may be formal or informal, it may be good or bad, and it may be in statistical control or not. Organizations can and should manage that process using the tools of process management. The most important tool is the control chart, introduced at the beginning of this paper. The control chart is the "voice ofthe process," in this case, the project delivery process. Figure 9 illustrates a control chart for an effective proj- ect delivery process, i.e., one that delivers on or under time and cost all the time. This generic chart applies to both cost and schedule. For that case, the required buffer is the sum of (-) XI)“, and the half-width of the control limits. Adding a buffer effectively moves the chart to the position indicated on the right side of Figure 9, meaning few projects fail to come in early and under the cost estimate. The earlier discussion of the control charts presented in Figure 2 and 3 identified that the apparent random contri- bution to the variation was too large and the bias contribu— tion too small compared to the theory presented herein. The author unravels the explanation in two parts. First, the ran- dom variation in the individual tasks is much larger than normally asserted. Task estimates based on historical experi« ence within a multitasking environment are a likely cause of this effect. That is, estimates are made to conform to experience. The variation introduced by multitasking can be several hun- dred percent of schedule time, not the tens of percent points usually assumed for standard deviation on individual tasks. Other uncontrolled bias factors randomly turned on and off for projects add to the observed overall variatiOn. This is how sta- tistically summed actual variance can be as large as 50% to 90% after the process has been brought into control, as demonstrat- ed hy the right side of the control charts. Second, the experientially determined bias on the right side of the charts appears too small because, contrary to assertions, bias was included in the initial estimates. instead of including a schedule buffer at the end of a chain of tasks, or adding a cost buffer to the mean cost estimate, the buffer was distributed amongst all the tasks but hidden. (This is the most common practice.) The author continued this with a separate analysis of individual projectnlike tasks that used the same estimating process. The included bias was about 30%. That is, individual task estimates must he cut to 70% of the original estimate to obtain the mean duration and cost for the tasks. Initial projects using CCPM cut project estimates to 60% and confirmed this i.e., the projects com- pleted well within the buffer. Note that control charts make no assumptions about the variance or cause of variance. They are the purely empirical (Target*-Actual) x 100 d E (D “U G D 0.) 3° H E (D U 6 D—r * Without buffer Project # Figure 9. Illustration of a Control Chart for an Effective Project DeliveryI Process June 2003 Probe! Management Journal 45 voice of the process. Initially, the variation within individual tasks can be so large as to mask the bias contributions, as in the illustrated example As process improvement (including the elimination of bad multitasking) continue to reduce the varia- tion of the individual tasks, and as estimates move toward mean estimates, the bias contribution will emerge. With buffers, as illustrated by the right side of Figure 9, projects will, on average, underrun by the bias amount and only rare projects will overrun the cost and schedule estimates. Process improvement should work to reduce both variation and bias for the long term. Variation reduction will focus on the work processes within the tasks, while bias reduction will focus on improving the project estimation and delivery process. Recommendations for Schedule Buffers Size project schedule buffers as follows: Project Schedule Buffer z Variation Buffer + Bias Buffer For basic CPM, the Variation Buffer is zero. For the other methods, sum the variation component along the critical path or critical chain. For PERT, world data on overconfi- dence suggests using a divisor of two to four instead of six for the variance range estimate. For PERT and Monte Carlo, consistency suggests setting the variation component of the buffer to two. Until the control chart in Figure 9, can be used, Table 3 must be assessed to determine which of the bias elements apply to projects and estimate an appropriate amount for the bias component. For example, for projects with morn of the tasks on the critical path or chain, the merging impact may be insignificant. For IT projects with a 100 parallel modules, the merging impact is potentially huge. Or, with an excellent-quality process, know error rates and build the correction into the plan, the error correction may be negli- gible. A formal quality process and data as the actual error rate and rework is not built into the plan, without error cor- rection bias will be very large. For organizations new to CCPM, a total schedule buffer equal to 50% of the chain length has proven successful for many projects. This applies to both feeding and project buffers, although, it is the author's experience that this method leads to frequently penetrating the feeding buffers located early in the project plan by more than l00'%. The project buffer never should be less than 25% of the total task duration along the critical chain. After 25 or more projects are completed. create and use the control chart to determine the necessary buffers. Use the control chart to identify opportunities to reduce the variance of project schedule performance. For example, improve task identification, task estimating, task performance processes. quality processes, or risk management to reduce the bias or the variance of schedule performance vs. estimate. As the variation narrows, the necessary schedule buffer size. Recommendation for Cost Buffer-(s) Point ofview determines the approach to sizing cost buffers. 46 Prefect Management Journal June 2003 For example, fixedwprice contract projects demand a differ. ent approach than internal developments. In general, size the cost buffer as follows: Project Cost Buffer = Variation Buffer + Bias Buffer Size the variation component of the buffer using the same methods described above for schedule buffer, but summed for all tasks in the project. Table 3 indicates less potential cost bias than schedule bias. Management pressure frequently urges minimizing cost buffers. When bidding projects competitively, a firm must minimize cost buffers to win the job while avoiding too small a buffer that would lose money on the job. These fac- tors suggest an alternative approach to cost buffer sizing; i.e., balancing the risk of losing the project revenue and profit vs. the potential financial loss from too small a cost buffer. The censtrttction industry, which is primarily a fixed— price project environment, most commonly uses prior expe- rience or Monte Carlo analysis to size cost buffers. The Monte Carlo analysis should account for common-cause variation, although sometimes special—cause variation is erroneously confounded in Monte Carlo analysis. it may account for the merging effect, if performed on the schedule in additi0n to the cost estimate. Some methods separately perform the Monte Carlo analysis on schedule and cost. Based on these recommendations, a minimum cost buffer on the order of 25% appears justified, if not too small. The total cosr buffer never should be less than 10% of the average cost estimate. References Cl! [Construction Industry institute). (1989). Management of project n'slzs and uncertainties ((211 Publication 6- 8). Austin, TX: University of Texas at Austin. Ford, D. N., St Sterman, J. D. (1998). Overcoming the 90% syndrome: Iteration management in concurrent develop- ment projects. Retrieved January 2002 from MIT Web site. Cambridge: MIT lluang, Y. (1999). The alternatives of estimating PERT activi- ty times and standard deviations. Retrieved March 2, 2002 from http:f/www.scisnovaedu/ruyaochin/621-p.htrn Johnson, J. (1999, December). CHAOS into success. Software Magazine. Kahneman, 0., Slovic. Paul, 81 Tversky, Amos. (1982). Judgment under Uncertainty: Heuristics and bias. New York: Cambridge tininarsity Press. Leach. L. .l‘. (2000). Critical trltuin project rnnnagement. Boston: Artech House. beach, L. P. (2001 ). Putting quality in project risk manage- ment. PM Network (Febmary), p. 53. Malcolm, D. (3., Rosebloom. John H., Clark, Charles F... «St Fazar, Willard. (1959). Application of a Technique for Research and Development Program Evaluation. Operations Research (September), p. 646. Meredith. l. R., d: Mantel, S. l. (1995). Project nutmtgement: A nmnagerfttl approach (3rd ed.) New York: John Wiley. Morgan, M. (1., s llenron, Max. (1990). Uncertainty, xi guilty to limiting with tmrertnirrn' in qualitative as}: and pain}- minty- sis: Cambridge University Press: New York. Project Management institute (PMI). (2000]. A giriric to the prajrerr murmgunmu imdy qt'lrrmwlmlge. Newtown Square, PA: I'Ml. Schmidt, S. R., tit Kicmeic. M. l. (2002). Don’t let TQM [)mm you dry udtlimtt any ROI. Retrieved December 10, 2002 from hup:[lwwwairncadcom/tqmroiinm Software Engineering Institute (Slit). (1994). Thu t‘npntiility maturity mortal- Cuiiieliries for improving the software Morass. Reading, MA: AddisnnWesley. Wideman, R. M. (1992). Pi'ojt’t't and Pi‘tigi‘fltll risk num- ugc’mrnt, A guide to nmnugiug project risies and opportunities. The PMBOK Handbook Series, Volume No. 6. Upper Darby, PA: I’Ml. LARRY LEACH, PMP, is president at the Advanced Projects Institute [API], 0 management consulting Firm. He developed and operated proiect management office for American National insurance Company [ANICO], in Galveston Texas. He worked as the vice president level in several Fortune 500 companies, where he managed programs at large and small projects at many types. He has Masters' degrees in both business management From the University at ldaho and in mechanical engineering from the University at Connecticut. He is a member at the Protect Management Institute [Pit/ti), has pub- lished many papers and books and has presented seminars on related topics. PEOJEKTMANAGEMENT GROUP W ' '2' CONSULTING pm tage ‘03: PROJECTS & EMOTIONS megm. October 29'“ to November 1", 2003 Venue: ORF-Center (Austrian Broadcasting Corporation) Vienna. Austria Emotional Project research conference: Management October 29‘". 2003 Differences in Project Teams: practice conference: A Chance and a Challenge p“,ij g. Emu-ans _ I _ October 3o" to 31". 2003 Virtual Organisations have no Emotions? expert seminars.- Programme Management Burn-out and High-life in the and Pmiecl Portfolio Project-oriented Company Management Management Auditing of Projects Passion for the Proiect and Programmes Management Profession November 1". 2003 Scientific Director: Univ.Prof.Dkfm.Dr. Roland Gareis Information: Daniela Dobrincic. do ROLAND GAREIS CONSULTING, Silbergasse 30/3. A-ngo Vienna, TEL: +43hlg67 70 22-0; Fax: +q3f1f63? 70 22-70: e-mail: pmtage®rgcat www.wu-wien.ac.at,tpmg www.rgr.at June 2003 Pruion Management Journal 47 Copyright© 2003 EBSCO Publishing ...
View Full Document

This note was uploaded on 02/02/2012 for the course MANAGEMENT 1234 taught by Professor Larry during the Spring '11 term at Rasmussen College.

Page1 / 15

SCHEDULE AND COST BUFFER SIZING_HOW TO ACCOUNT FOR THE BIAS - Leach, Lawrence

This preview shows document pages 1 - 15. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online