IJLM 1994 Caplice and Sheffi

IJLM 1994 Caplice and Sheffi - .there is a pressing need...

Info icon This preview shows pages 1–18. Sign up to view the full content.

View Full Document Right Arrow Icon
Image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 2
Image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 4
Image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 6
Image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 8
Image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 10
Image of page 11

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 12
Image of page 13

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 14
Image of page 15

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 16
Image of page 17

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 18
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ...there is a pressing need for companies to reevaluate (or to analyze for the first time) their performance measurement systems. A Review and Evaluation of Logistics Metrics Chris Caplice and Yossi Sheffi Massachusetts Institute of Technology Performance measurement in the logistics function, like all business functions. begins at the individual metric level. A performance measurement system that is well designed at the strategic level can be flawed at the individual metric level; the Achilles‘ Heel of any measurement system. The pressing need is not for the development of novel performance metrics: there is a great abundance of sufficient metrics already in existence. Rather, there is a need for a method with which to evaluate existing metrics. This paper addresses this need by suggesting a set of evaluation criteria for individual logistics performance metrics and identifying the inherent trade-offs. A taxonomy of logistics performance metrics, organized by process rather than by function, is also presented and the metrics are evaluated using the established criteria. in response to external pressures, many firms are modifying their supply chain. To properly manage these and other evolving structures, upper management needs adaptable and accurate performance metrics. All too often, though, performance metrics have not kept pace with the changing business environment and are no longer adequate (if indeed they ever were). The problem, in our opinion, is not that there is a need for developing novel performance metrics based on new physical or financial qualities. Existing metrics, if used properly, can capture the critical elements of the logistics process: time, distance, and money are still the basis of all logistics management. Rather, we feel there is a pressing need for companies to reevaluate (or to analyze for the first time} their performance measurement systems. This reevaluation should be conducted for both the individual metrics and the performance measurement system as a whole. This paper concentrates on the first component of the assessment process: the evaluation of the individual metrics. Specifically, there are three objectives: 1. Establish useful criteria which can be applied to evaluate individual logistics performance metrics, 2. Identify any trade-offs which are present in the selection of individual performance metrics, and 3. Classify and critique existing performance metrics from a process, rather than functional, orientation. The primary motivation for analyzing individual metrics separately is that they are the building blocks of a complete measurement system. If they are flawed, then regardless of how well the overall measurement system is designed, the signals sent to decision makers will be inaccurate. To use a building analogy, the structural integrity of a bridge design is only as valid as the characteristics of the raw materials used. If a design calls for a certain tensile strength of a steel component, then the bridge will most likely fail if that component does not meet the specific tensile strength standard, even if it meets other less critical standards. Similarly, if a performance measurement system relies on a specific individual metric to provide information on order cycle time, but the metric does not include a critical portion of the process (e.g., the time elapsed between a customer’s first contact and the actual generation of the purchase order), then the wrong signals are being sent and the system is flawed. Examples of improper performance metrics are widespread in practice as the following two actual, and aft too common, examples illustrate. A major health care products manufacturer uses on-time performance to Volume 5, NumberZ l994 Page i I track customer service for its overseas distributors. The metric ”percentage of on— time shipments” is reported quarterly and plays a significant role in the distribution manager’s bonus and compensation plan. The specific metric used, however, records any shipment as being ”on—time” if it leaves the company’s own distribution center (DC) during the same month that the order was received. 80, an order received on January 1 and shipped on January 31 is considered ”on-time” while an order received On January 31 and shipped on February 1 is not. While a system—wide objective of measuring performance from the custOmer’s perspective appears to be satisfied, this metric is inconsistent with the customer’s point of view and is subject to obvious tinkering by the manager. As a second example, a large discount retailer uses the distribution costs from its warehousing operations both to make strategic decisions and to evaluate the performance of logistics managers of specific product groups. Managers are rewarded for achieving lower distribution costs per each item and these cost figures are used to determine marketing and distributiOn channels. The metric ”item distribution cost,” however, is calculated as the total distribution cost (e.g., direct labor, facility cost, and overhead) divided by the number of “units" of each product group processed. No other bases (such as density, fragility, value, or demand level) are used for allocation. These examples illustrate the importance of examining the individual metrics which feed into the larger performance measurement systems. While from the system wide level, each of these systems might be acceptable in overall design, the specific metrics upon which they are based are flawed thereby fainting the information they provided. The metric in the first example led to gaming since it was not behaviorally sound, while the metric in the second example prOvided misleading information since it was neither valid nor of a sufficient level of detail. This paper does not develop new performance metrics. instead, it provides the logistics manager with a set of tools with which to evaluate and select individual performance metrics for use in a performance measurement system. While Page 12 many aspects of performance measurement are situation specific, there are several quite general guidelines that can assist the logistics manager in this task. The remainder of this paper is organized into three sections. The first section presents suggested evaluation criteria of individual performance metrics. The following section describes the three general forms of process measurement, presents a taxonomy of common metrics, and evaiuates these metrics using the criteria established in the first section. Finaliy, the paper is summarized and concluded. Evaluation Criteria The specific selection of performance metrics depends on the end user, the organizational structure, the current business environment, and numerous other factors. Some general characteristics, however, can be identified to assist in the development of “good” performance metrics. This section summarizes past research into performance metric evaluation and proposes a comprehensive set of eight evaluation criteria. Review of Literature Researchers have identified several criteria to consider when selecting individual performance measurements for logistics as well as for business functions in general. Table 1 summarizes the suggested criteria from these studies and illustrates both the commonalties and the gaps in the various studies. In a text on formal measurement theory, Mock and Grove {1] define a measurement or metric as an ”assignment process where numbers are assigned to represent some attribute of an object or event of interest” for the decision maker. The “goodness” of a metric, they c0ntinue, can be evaluated along six criteria: validity, reliability, scale type, meaningfulness, economical worth, and behavioral implications. In a survey of performance measures used in managerial accounting, Edwards [2] identifies five important keys for selecting measures: availability, consistency, usefulness, reliability, and cost-benefit analysis. Juran [3] suggests that an ideal metric must: (1) provide an agreed basis for decision making, (2) be understandable, (3) apply broadly, {4} be capable of uniform While many aspects of performance measurement are situation specific, there are several quite general guidelines that can assist the logistics manager in this task. The international journal of Logistics Management $96,: 5660...... m .2 .556th _._m.mo B 6.6.. £29.50 BE :32”. .6 _o>m_ Em_u_::m m B 25E 9: w_ w:_m.._o 232m 9: E Emimfi muorwfi m5 cos—2025 mwohum cowmcmEooo .20an ESE 9: $00 $385 2: B cozmcEcmE—u 5:990; maumamm ucm EcwconEoo Entan. 9: 5:9me .595 $9900 9:29.60 ho 2m 233E ucm €3.05 25E 9: $00 $23th $9.: 3 “822mg... 25mm 3 2 £505 mmmcémm: EmEoEmfitm ucm ”REE uEmE m5 2 £93 con: 32% w Eugen—LEE mmmEmu 3.8% :5 ‘3 >um__E_m mmmcfianom Eats: E w 3 $55.95 new umzws 0:55 9.: 2 «2.962590 new .5232 $3530: mEEmnEoO mEEmnEoo 6E: mmeum uEmnEoo m5 oEwE 9.: 50 $me on 2 mmmEEmm: . 828 cm .2 mafia a 8305 2:2: 9: $00 mE£m>w mczmfiw mac: ucm 629$ coumEEE 2 22:32.00 8v mczflxm m5 2:; Enzmano 0:65 9: m_ we 9525 9: Ea: b___n=mn_Eoo m_nm__m>m 22959 Emu was 0:85 9: $00 958 5323 228238. aflum 923::258 .5 238329.". :95... _m..o_>w:wm_ Baofifi 5* wo>ncmo£ Emma 25:. 9: =5; 9:85 Emcwm 5:25 «“2..qu m5 >Eocoom bmoo _mu_Eocoum mean B 980 9.: 3553 £555 9: on. uczow wwnb 652.5 2 umELSmcg 265323 535:3: m_ mfium 9.: __ 2938 EH 2sz 9: w. nczom $38.; an. 5. 28m afihosmsmm 83. 2me 32.56:: 8 m9: Eaton m mEm: uEmE 9: m. Btu $55532 : w. $982.8 Emu mmmEmsnoI EmEmSmmoE 222.2”. E EBB E925 5., .225". uEwE m5 mmoo «32938 mmE>=om Egg 23:2”. u__m> new 3:26 E28 9333 £29: 9: ammo 2:3: 33: 53:5 2.5: 3 :55 39:3. 3526.. nth-“Bum 3:20 :ozntomon .w 00:30 a 53:22 ._.< a x02: 9.323: 9: E nucozcms. utmutu 2322 3323.2.— EEaEQ no coats—too — mink Page 13 1994 Voiume 5, Number 2 interpretation, (5) be economical to apply, and (6) be compatible with existing data collection. in the logistics area, three studies specifically describe criteria for individual metrics. The most influential is AT Kearney [4, 5, 6}, which is a series of studies sponsored by the Council of Logistics Management (CLM). In a very detailed discussion of performance measurement, primarily at the functional level, they recommend the use of seven criteria (validity, coverage, comparability, complete- ness, usefulness, compatibility, and cost effectiveness) for selecting performance metrics. The Netherlands Association for Logistics Management (NEVEM) conducted a similar study as a European response to the CLM study. NEVEM l7] analyzed ”indicators” for logistics using seven similar criteria for metric selection: validity, covering potential, comparability, accuracy, utility, compatibility, and profitability. Finally, Mentzer and Konrad [8] stress the importance of capturing both efficiency and effectiveness in performance measurement, and identify four common problems: i. under-determination where the metric does not entirely measure all aspects of the process, 2. comparability where a measure is not readily comparable across periods, shipments, or firms, 3. measurement error where responsibility and causality are incorrectly assigned, and 4. human behavior where incentives harmful to the firm are created. As shown in Table 1, while there is a great deal of agreement between these studies on the importance of certain characteristics, no single study, prior to this one, appears to capture all aspects. The AT Kearney and NEVEM studies, for example, do not explicitly consider the behavioral implications. Additionally, the previous studies all assume that these characteristics are independent of each other and thus they do not address the inherent trade-offs between the characteristics. The next section presents the eight criteria that we believe fully capture the essential characteristics of individual performance metrics and identifies and examines their interactions. Criteria Definitions and Descriptions Eight criteria, thought to be comprehensive and succinct in their coverage of the previously identified characteristics, were selected: validity, robustness, usefulness, integration, economy, compatibility, level of detail, and behavioral soundness. The criteria are discussed in detail in this section and are defined in Table 2. Table 2 Definitions of the Eight Metric Evaluation Criteria Validity Deacrl tion The metric accurately captures the events and activities being measured and controls for any exogenous factors. The metric is interpreted similarty by the users. is comparable across time, location, & organizations, and is repeatable. Usefulness The metric is readily understandable by the decision maker and provides a guide for action to be taken. integration The metric includes all relevant aspects of the process and promotes coordination across functions and divisions. Economy The benefits of using the metric outweight the costs of data collection, analysis. and reporting. Compatibility The metric is compatible with the existing inionnation, material, and cash flows and systems in the organization. Level of Detail The metric provides a sufficient degree of granularity or aggregation for the user. Behavioral Soundness Page 14 The metric minimizes incentives for counter-productive acts or game- playing and is presented in a useful form. Eight criteria...were selected: validity, robustness, usefulness, integration, economy, compatibility, level of detail, and behavioral soundness. The international journal of Logistics Management Following the individual discussions of each criterion, there is an analysis of the interactions between them. In order to avoid confusion, for the remainder of this paper all evaluation criterion are printed in italics. Validity A metric is valid if it reflects the actual activity being performed and controls for any exogenous factors that are out of the process manager’s control. For example, if a traffic department ships product over a wide mix of haul lengths, using various modes, and responding to very different lead times, then measuring productivity as cost per ton-mile is not particularly valid. For example, any shift in the order pattern to smaller, more time sensitive modes will cause an increase in the cost per ton-mile, thus making the manager look bad regardless of actual job performance. Segmenting cost per ton-mile according to haul length, service level, or other characteristics would make the metrics more valid since these additional factors have a significant effect on transportation costs. Robustness A metric is robust if it is widely accepted, is interpreted similarly by different users, and can be used for comparisons across time, locations, and organizations. Using the same example, cost per ton-mile, while not a very valid metric, is robust because (1) tons hauled and miles driven are easy to collect, (2) ton—miles is widely accepted in the transportation industry, and (3) it is difficult to misinterpret a ton-mile. An example of a meaSure that is not very robust is the direct labor cost of logistics, often used as a measure of input. it is not comparable across firms since the definition of direct labor differs widely between firms.‘ Usefulness A metric is useful if it is readily understood by the decision maker and suggests a course of action or direction to be taken. For example, a metric tracking the use of expedited transportation, such as percentage of shipments using overnight transportation, is useful in that it is easily understood and it provides the manager with direct guidance, that is, pay attention to the modes of transportation used. In contrast, a composite metric combining several factors into a single index is not as useful to a manager since the method by which the index was calculated may be considered a ”black box” and the index, as an abstract value, does not suggest a specific action to take. Integration A metric is integrative if it incorporates all of the major components and aspects of the process being measured and promotes coordination across functions, divisions, or firms in the Supply chain. The primary thrust of this criterion is to promote coordination between the players involved in the process. For example, in an automobile assembly plant it was found that if the finished cars were sequenced for production according to the order of dealer delivery, then distribution costs would be significantly lowered. This, however, required the production manager to slightly modify his operations and since all of his performance metrics where self contained within the plant, he had no incentive to change. Thus, an opportunity to lower overall costs was lost. If a better integrative metric such as total cost of car to delivery had been used, then there would have been an incentive for the production manager to make these changes. Economy A metric is economical if the benefit of tracking it outweighs the cost to collect, process, and report it. This is more of a judgement call than a strict cost-benefit comparison so that the economy criterion should be used to Select between potential metrics rather than for the decisiOn of whether to use any metric at all. For example, an inventory control system which captures the time spent in inventory for each individual item in a pencil manufacturing plant is probably not as economical as a metric which reports aggregate dollar values of stock. Compatibility A metric is compatible with the existing data collection, information systems, and K 'For example, some firms include the order entry clerical staff in direct distribution labor while others consider this as support staff and treat them as indirect. it is an arbitrary decision. Volume 5, NumberZ 7994 Page I5 mun-W information flows of the firm if no significant additional work is required to install and use it. For example, measuring on—time performance in terms of hours early or late is not compatible with a system which only recognizes deliveries in weekly buckets. While compatibility has some overlap with the economy criterion, in that any system can be made to be compatible to a proposed metric given the needed time and money, they are not the same. A metric which is economical in terms of collecting and reporting data might not always be compatible with the existing flow of informatiOn. Level of Detail A metric has the correct level of detail if it captures and reports the data in a level of aggregation or granularity to be useful to the decision maker. For example, an inventory level measure which is taken monthly may be of insufficient detail for high value items which require daily monitoring, while hourly tracking of inventory levels of coal stockpiles at a power plant during normal operations would be overly detailed. The level of detail is very much a function of its user. For example, a warehouse manager might track crew productivity, a district manager might roll these into productivity for regional warehouses, and a national manager would most likely combine the warehouse measures into functional measures. Behavioral Soundness A metric that is behaviorally sound discourages any counter-productive actions or game—playing by those people or organizations being measured. While it is always hoped that a measure will align peoples’ actions with the organization’s overall objectives, in many cases it can provide incentives for doing the opposite. For example, the on-time performance metric used by the health products manufacturer in the introduction creates an incentive to manipulate the pattern of order arrivals so as to maximize the amount of ”lead time” for the distribution department. To the customer and to the organization, however, this is counter-productive since order cycle time will increase while the department manager will not be penalized. in fact, the manager will be rewarded since, on paper, the "on-time” percentage will increase! Fisher [9] refers to this type of behavior as ”dysfunctional activities” since the people who...
View Full Document

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern