CarrCh09v2

CarrCh09v2 - C H A P T E R 9 The Efficiency of Algorithms...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: C H A P T E R 9 The Efficiency of Algorithms CONTENTS Motivation Measuring an Algorithm’s Efficiency Big Oh Notation Formalities Picturing Efficiency The Efficiency of Implementations of the ADT List The Array-Based Implementation The Linked Implementation Comparing the Implementations PREREQUISITES Chapter Chapter Chapter 1 5 6 Java Classes List Implementations That Use Arrays List Implementations That Link Data OBJECTIVES After studying this chapter, you should be able to G G Determine the efficiency of a given algorithm Compare the expected execution times of two methods, given the efficiencies of their algorithms W ith amazing frequency, manufacturers introduce new computers that are faster and have larger memories than their recent predecessors. Yet we—and likely your computer science professors—ask you to write code that is efficient in its use of time and space (memory). We have to admit that such efficiency is not as pressing an issue as it was fifty years ago, when computers were much slower and their memory size was much smaller than they are now. (Computers had small memories, but they were physically huge, occupying entire rooms.) Even so, efficiency remains an issue—in some circumstances, a critical issue. 199 200 CHAPTER 9 The Efficiency of Algorithms This chapter will introduce you to the terminology and methods that computer scientists use to measure the efficiency of an algorithm. With this background, you not only will have an intuitive feel for efficiency, but also will be able to talk about efficiency in a quantitative way. Motivation 9.1 Perhaps you think that you are not likely to write a program in the near future whose execution time is noticeably long. You might be right, but we are about to show you some simple Java code that does take a long time to perform its computations. Imagine that we are defining a Java class to represent extremely large integers. Of course, we will need to represent the many digits possible in one of these integers, but that is not the subject of this example. We want a method for our class that adds two large integers—that is, two instances of our class—and a method that multiplies two large integers. Suppose that we have implemented the add method successfully and are about to design a multiply method. As you know, multiplication is equivalent to repeated addition. So if we want to compute the product 7562 times 423, for example, we could add 7562 to an initially zero variable 423 times. Remember that our add method works, so we can readily use it in our implementation of multiply. 9.2 Let’s write some simple Java code to test our idea. Realize that this code is simply an experiment to verify our approach of repeated addition and does not use anything that we might already have written for our class. If we use long integers in our experiment, we could write the following statements: long firstOperand = 7562; long secondOperand = 423; long product = 0; for (; secondOperand > 0; secondOperand--) product = product + firstOperand; System.out.println(product); If you execute this code, you will get the right answer of 3,198,726. Now change the second operand from 423 to 100,000,000 (that is, a 1 followed by eight zeros), and execute the code again. Again, you will get the correct answer, which this time is 756,200,000,000 (7562 followed by eight zeros). However, you should notice a delay in seeing this result. Now try 1,000,000,000, which is a 1 followed by nine zeros. Again you will get the correct answer—7562 followed by nine zeros—but you will have to wait even longer for the result. The wait might be long enough for you to suspect that something is broken. If not, try 1 followed by ten zeros! What’s our point? Our class is supposed to deal with really large integers. The integers in our little experiment, while large, are not really large. Even so, the previous simple code takes a noticeably long time to execute. Should you use a faster computer? Should you wait for an even faster computer to be invented? We have a better solution. 9.3 The delay in our example is caused by the excessive number of additions that the code must perform. Instead of adding 7562 to an initially zero variable one billion times, we could save time by adding one billion to an initially zero variable 7562 times. This approach won’t always be faster, because one operand might not be much smaller than the other. Besides, we can do much better. Motivation 201 Consider again the product 7562 * 421. The second operand, 423, has three digits: A hundreds digit 4, a tens digit 2, and a ones digit 3. That is, 423 is the sum 400 + 20 + 3. Thus, 7562 * 423 = 7562 * (400 + 20 + 3) = 7562 * 400 + 7562 * 20 + 7562 * 3 = 756200 * 4 + 75620 * 2 + 7562 * 3 Now we can replace the three previous multiplications with additions, so that the desired product is the sum (756200 + 756200 + 756200 + 756200) + (75620 + 75620) + (7562 + 7562 + 7562) There are three parenthesized terms, one for each digit in the second operand. Each of these terms is the sum of d integers, where d is a digit in the second operand. We can conveniently compute this sum from the right—involving first 7562, then 75620, and finally 756200—because 75620 is 10 * 7562, and 756200 is 10 * 75620. 9.4 We can use these observations to write pseudocode that will compute this type of sum. Let the operands of the multiplication be firstOperand and secondOperand. The following pseudocode computes the product of these operands by using addition. secondOperandLength = product = 0 number of digits in secondOperand for (; secondOperandLength > 0; secondOperandLength--) { digit = rightmost digit of secondOperand for (; digit > 0; digit--) product = product + firstOperand Drop rightmost digit of secondOperand Tack 0 onto end of firstOperand } // Assertion: product is the result If we use this pseudocode to compute 7562 * 423, firstOperand is 7562 and secondOperand is 423. The pseudocode computes the sum (7562 + 7562 + 7562) + (75620 + 75620) + (756200 + 756200 + 756200 + 756200) from left to right. The outer loop specifies the number of parenthesized groups as well as their contents, and the inner loop processes the additions within the parentheses. 9.5 Here is some Java code for you to try based on the previous pseudocode. Again, this is an experiment to verify an approach and is not the implementation of a method in our class of large integers. As such, we will implement the last two steps of the pseudocode by dividing by 10 and multiplying by 10. long firstOperand = 7562; long secondOperand = 100000000; int secondOperandLength = 9; long product = 0; 202 CHAPTER 9 The Efficiency of Algorithms for (; secondOperandLength > 0; secondOperandLength--) { long digit = secondOperand - (secondOperand/10) * 10; for (; digit > 0; digit--) product = product + firstOperand; secondOperand = secondOperand/10; // discard last digit firstOperand = 10 * firstOperand; // tack zero on right } // end for System.out.println(product); You should find that this code executes faster than the code in Segment 9.2 after you change secondOperand to 100000000. Note: As this example shows, even a simple program can be noticeably inefficient. Measuring an Algorithm’s Efficiency 9.6 The previous section should have convinced you that a program’s efficiency matters. How can we measure efficiency so that we can compare various approaches to solving a problem? In the previous section, we asked you to run two programs and observe that one was noticeably slower than the other. In general, having to implement several ideas before you can choose one requires too much work to be practical. Besides, a program’s execution time depends in part on the particular computer and the programming language used. It would be much better to measure an algorithm’s efficiency before you implement it. For example, suppose that you want to go to a store downtown. Your options are to walk, drive your car, ask a friend to take you, or take a bus. What is the best way? First, what is your concept of best? Is it the way that saves money, your time, your friend’s time, or the environment? Let’s say that the best option for you is the fastest one. After defining your criterion, how do you evaluate your options? You certainly do not want to try all four options so you can discover which is fastest. That would be like writing four different programs that perform the same task so you can measure which one is fastest. Instead you would investigate the “cost” of each option, considering the distance and the speed at which you can travel, the amount of other traffic, the number of stops at traffic lights, the weather, and so on. 9.7 The same considerations apply when deciding what algorithm is best. Again, we need to define what we mean by best. An algorithm has both time and space requirements, called its complexity, that we can measure. Typically we analyze these requirements separately and talk about an algorithm’s time complexity—the time it takes to execute—or its space complexity—the memory it needs to execute. So a “best” algorithm might be the fastest one or the one that uses the least memory. The process of measuring the complexity of algorithms is called the analysis of algorithms. When we measure an algorithm’s complexity, we are not measuring how involved or difficult it is. We will concentrate on the time complexity of algorithms, because it is usually more important than space complexity. You should realize that an inverse relationship often exists between an algorithm’s time and space complexities. If you revise an algorithm to save execution time, you usually will need more space. If you reduce an algorithm’s space requirement, it likely will require more time to execute. Sometimes, however, you will be able to save both time and space. Measuring an Algorithm’s Efficiency 203 Your measure of the complexity of an algorithm should be easy to compute, certainly easier than implementing the algorithm. You should express this measure in terms of the size of the problem. For example, if you are searching a collection of data, the problem size is the number of items in the collection. Such a measure enables you to compare the relative cost of algorithms as a function of the size of the problem. Typically, we are interested in large problems; a small problem is likely to take little time, even if the algorithm is inefficient. 9.8 9.9 Figure 9-1 Realize that you cannot compute the actual time requirement of an algorithm. After all, you have not implemented the algorithm in Java and you have not chosen the computer. Instead, you find a function of the problem size that behaves like the algorithm’s actual time requirement. That is, as the time requirement increases, the value of the function increases, and vice versa. The value of the function is said to be directly proportional to the time requirement. Such a function is called a growth-rate function because it measures how an algorithm’s time requirement grows as the problem size grows. By comparing the growth-rate functions of two algorithms, you can determine whether one algorithm is faster than the other. You could estimate the maximum time that an algorithm could take—that is, its worst-case time. If you can tolerate this worst-case time, your algorithm is acceptable. You also could estimate the minimum or best-case time. If the best-case time is still too slow, you need another algorithm. For many algorithms, the worst and best cases rarely occur. A more useful measure is the averagecase time requirement of an algorithm. This measure, however, is usually harder to find than the best and worst cases. Typically, we will find the worst-case time. Example. Consider the problem of computing the sum 1 + 2 + . . . + n for any positive integer n. Figure 9-1 contains pseudocode showing three ways to solve this problem. Algorithm A computes the sum 0 + 1 + 2 + . . . + n from left to right. Algorithm B computes 0 + (1) + (1 + 1) + (1 + 1 + 1) + . . . + (1 + 1 + . . . 1). Finally, Algorithm C uses an algebraic identity to compute the sum. Three algorithms for computing the sum 1 + 2 + . . . + n for an integer n > 0 Algorithm A sum = 0 for i = 1 to n sum = sum + i Algorithm B sum = 0 for i = 1 to n { for j = 1 to i sum = sum + 1 } Algorithm C sum = n * (n + 1) / 2 Which algorithm—A, B, or C—is fastest? We can begin to answer this question by considering both the size of the problem and the effort involved. The integer n is a measure of the problem size: As n increases, the sum involves more terms. To measure the effort, or time requirement, of an algorithm, we must find an appropriate growth-rate function. To do so, we begin by counting the number of operations required by the algorithm. Figure 9-2 tabulates the number of assignments, additions, multiplications, and divisions that Algorithms A, B, and C require. These counts do not include the operations that control the loops. We have ignored these operations here to make counting easier, but as you will see later in Segment 9.14, doing so will not affect our final conclusion about algorithm speed. 204 CHAPTER 9 The Efficiency of Algorithms Figure 9-2 The number of operations required by the algorithms in Figure 9-1 Algorithm A Assignments Additions Multiplications Divisions Total operations Question 1 Algorithm B nϩ1 n 1 ϩ n (n ϩ 1) / 2 n (n ϩ 1) / 2 2n ؉ 1 n2 ؉ n ؉ 1 Algorithm C 1 1 1 1 4 For any positive integer n, the identity 1 + 2 + . . . + n = n(n + 1) / 2 is one that you will encounter while analyzing algorithms. Can you derive it? If you can, you will not need to memorize it. Hint: Write 1 + 2 + . . . + n. Under it write n + (n - 1) + . . . + 1. Then add terms from left to right. Question 2 Can you derive the values in Figure 9-2? Hint: For Algorithm B, use the identity given in Question 1. Note: Useful identities 1 + 2 + . . . + n = n(n + 1) / 2 1 + 2 + . . . + (n - 1) = n(n - 1) / 2 9.10 The various operations listed in Figure 9-2 probably take different amounts of time to execute. However, if we assume that they each require the same amount of time, we will still be able to determine which algorithm is fastest. For example, Algorithm A requires n + 1 assignments and n additions. If each assignment takes no more than t= time units and each addition takes no more than t+ time units, Algorithm A requires no more than (n + 1) t= + n t+ time units. If we replace t= and t+ with the larger of the two values and call it t, Algorithm A requires no more than (2n + 1) t time units. Whether we look at a time estimate such as (2n + 1) t or the total number of operations 2n + 1, we can draw the same conclusion: Algorithm A requires time directly proportional to 2n + 1 in the worst case. Thus, Algorithm A’s growth-rate function is 2n + 1. Using similar reasoning, we can conclude that Algorithm B requires time directly proportional to n2 + n + 1, and Algorithm C requires time that is constant and independent of the value of n. Figure 9-3 plots these time requirements as a function of n. You can see from this figure that as n grows, Algorithm B requires the most time. 9.11 Typical growth-rate functions are algebraically simpler than the ones you have just seen. Why? Recall that since you are not likely to notice the effect of an inefficient algorithm when the problem is small, you should focus on large problems. Thus, if we care only about large values of n when comparing the algorithms, we can consider only the dominant term in each growth-rate function. For example, n2 + n + 1 behaves like n2 when n is large because n2 is much larger than n + 1 in that case. In other words, the difference between the value of n2 + n + 1 and that of n2 is relatively small and can be ignored when n is large. So instead of using n2 + n + 1 as Algorithm B’s growthrate function, we can use n2—the term with the largest exponent—and say that Algorithm B requires time proportional to n2. Likewise, Algorithm A requires time proportional to n. On the other hand, Algorithm C requires time that is independent of n. Measuring an Algorithm’s Efficiency The number of operations required by the algorithms in Figure 9-1 as a function of n 32 28 Number of operations Figure 9-3 205 Algorithm B: n2 ϩ n ϩ 1 operations 24 20 16 Algorithm A: 2n ϩ 1 operations 12 8 4 0 Algorithm C: 4 operations 1 2 3 4 5 n Big Oh Notation 9.12 Computer scientists use a notation to represent an algorithm’s complexity. Instead of saying that Algorithm A has a worst-case time requirement proportional to n, we say that A is O(n). We call this notation Big Oh since it uses the capital letter O. We read O(n) as either “Big Oh of n” or “order of at most n.” Similarly, since Algorithm B has a worst-case time requirement proportional to n2, we say that B is O(n2). Algorithm C always requires four operations. Regardless of the problem size n, this algorithm requires the same time, be it worst case, best case, or average case. We say that Algorithm C is O(1). We will discuss Big Oh notation more carefully and introduce other notations in the next section. 9.13 Example. Imagine that you are at a wedding reception, seated at a table of n people. In preparation for the toast, the waiter pours champagne into each of n glasses. That task is O(n). Someone makes a toast. It is O(1), even if the toast seems to last forever, because it is independent of the number of guests. If you clink your glass with everyone at your table, you have performed an O(n) operation. If everyone at your table does likewise, they collectively have performed an O(n2) operation. 9.14 Example. In Segment 9.9, we ignored the operations that control the loops in the algorithms. Obviously this simplification affects the total number of operations, but even if we counted them, we would get the same growth-rate functions that you saw in the previous segment. For example, Algorithm A contains the for statement for i = 1 to n This statement represents an assignment to i, additions to i, and comparisons with n. In total, the loop-control logic requires 1 assignment, n additions, and n + 1 comparisons, for a total of 2n + 2 additional operations. So Algorithm A actually requires 4n + 3 operations instead of only 2n + 1. What is important, however, is not the exact count of operations, but the general behavior of the algorithm. The functions 4n + 3 and 2n + 1 are each directly proportional to n. For large values of n, the difference between these two functions is negligible. We do not have to count every operation to see that Algorithm A requires time that increases linearly with n. Thus, Algorithm A is O(n). 206 CHAPTER 9 The Efficiency of Algorithms 9.15 The growth-rate functions that you are likely to encounter grow in magnitude as follows when n > 10: O(1) < O(log log n) < O(log n) < O(log2 n) < O(n) < O(n log n) < O(n2) < O(n3) < O(2n) < O(n!) The logarithms given here are base 2. As you will see later in Segment 9.20, the choice of base does not matter. Figure 9-4 tabulates the magnitudes of these functions for increasing values of the problem size n. From this data you can see that O(log log n), O(log n), and O(log2 n) algorithms are much faster than O(n) algorithms. Although O(n log n) algorithms are significantly slower than O(n) algorithms, they are markedly faster than O(n2) algorithms. Figure 9-4 Typical growth-rate functions evaluated at increasing values of n n 10 102 103 104 105 106 log (log n) log n (log n)2 2 3 3 4 4 4 3 7 10 13 17 20 11 44 99 177 276 397 n n log n n2 n3 2n n! 10 100 1000 10,000 100,000 1,000,000 33 664 9966 132,877 1,660,964 19,931,569 102 104 106 108 1010 1012 103 106 109 1012 1015 1018 103 1030 10301 103010 1030,103 10301,030 105 1094 101435 1019,335 10243,338 102,933,369 Note: When analyzing the time efficiency of an algorithm, consider large problems. For small problems, the difference between the execution times of two solutions to the same problem is usually insignificant. 9.16 Example. Segments 9.2 and 9.5 showed you two ways to perform a computation. One way was noticeably slow. You should now be able to predict this behavior without actually running the code. The first way—call it Method 1—used this loop: long product = 0; for (; secondOperand > 0; secondOperand--) product = product + firstOperand; How many times is firstOperand added to product? If secondOperand contains an integer n, then n such additions occur. When n is large, so is the number of additions required to compute the solution. From our discussion in the previous section, we can say that Method 1 is O(n). The second way—call it Method 2—used the following code, where secondOperandLength is the number of digits in secondOperand: long product = 0; for (; secondOperandLength > 0; secondOperandLength--) { long digit = secondOperand - (secondOperand/10) * 10; for (; digit > 0; digit--) product = product + firstOperand; secondOperand = secondOperand/10; firstOperand = 10 * firstOperand; } // end for Measuring an Algorithm’s Efficiency 207 How many times is firstOperand added to product this time? Each digit of the second operand can be at most 9, so the inner for loop adds firstOperand to product at most nine times. The loop, in fact, executes once for each digit in secondOperand. Since secondOperand contains secondOperandLength digits, the loop adds firstOperand to product at most 9 * secondOperandLength times. If secondOperand contains an integer n, as it does in Method 1, how does n affect secondOperandLength here? The data in Figure 9-5 can help us answer this question. The figure tabulates log10 n truncated to an integer—which we denote as log10 n —for two-digit, three-digit, and four-digit values of the integer n. (Note that 4.9 , for example, is 4.) You can see that the number of digits in n is 1 + log10 n . Therefore, the number of additions that Method 2 requires—at most 9 * secondOperandLength—is at most 9 * (1 + log10 n ). Figure 9-5 The number of digits in an integer n compared with the integer portion of log10 n n Number of Digits log10 n 10 – 99 100 – 999 1000 – 9999 2 3 4 1 2 3 Now we need to compare our analyses of Methods 1 and 2. Method 1 requires n additions. Method 2 requires at most 9 * (1 + log10 n ) additions. Which approach requires the fewest additions? Let’s see if we can answer this question by looking at the data in Figure 9-6. For values of n greater than 18, n > 9 * (1 + log10 n ) Figure 9-6 The values of two logarithmic growth-rate functions for various ranges of n n log10 n 10 – 99 100 – 999 1000 – 9999 10,000 – 99,999 100,000 – 999,999 1 2 3 4 5 9 * (1 ϩ log10 n) 18 27 36 45 54 As n increases, n is much larger than 9 * (1 + log10 n ). Since Method 1 is O(n), it requires many more additions than Method 2 when n is large, and so it is much slower. In fact, Method 2 is O(log n) in its worst case. Figure 9-4 shows that an O(n) algorithm is slower than an O(log n) algorithm. Note: Floors and ceilings The floor of a number x, denoted as x , is the largest integer less than or equal to x. For example, 4.9 is 4. When you truncate a real number to an integer, you actually are computing the number’s floor by discarding any fractional portion. The ceiling of a number x, denoted as x , is the smallest integer greater than or equal to x. For example, 4.1 is 5. 208 CHAPTER 9 The Efficiency of Algorithms Formalities 9.17 Big Oh notation has a formal mathematical meaning that can justify some of the sleight-of-hand we used in the previous sections. You saw that an algorithm’s actual time requirement is directly proportional to a function f of the problem size n. For example, f(n) might be n2 + n + 1. In this case, we would conclude that the algorithm is of order at most n2—that is, O(n2). We essentially have replaced f(n) with a simpler function—let’s call it g(n). In this example, g(n) is n2. What does it really mean to say that a function f(n) is of order at most g(n)—that is, that f(n) = O(g(n))? In simple terms, it means that Big Oh provides an upper bound on a function’s growth rate. More formally, we have the following mathematical definition. Note: Formal definition of Big Oh An algorithm’s time requirement f(n) is of order at most g(n)—that is, f(n) = O(g(n))—in case a positive real number c and positive integer N exist such that f(n) ≤ c g(n) for all n ≥ N. Figure 9-7 illustrates this definition. You can see that when n is large enough—that is, when n ≥ N— f(n) does not exceed c g(n). The opposite is true for smaller values of n. An illustration of the definition of Big Oh Value of growth-rate function Figure 9-7 25 c g(n) 20 f(n) 15 10 5 0 0 5 N 10 15 20 25 30 n 9.18 Example. In Segment 9.14, we said that an algorithm that uses 4n + 3 operations is O(n). We now can show that 4n + 3 = O(n) by using the formal definition of Big Oh. When n ≥ 3, 4n + 3 ≤ 4n + n = 5n. Thus, if we let f(n) = 4n + 3, g(n) = n, c = 5, and N = 3, we have shown that 4n + 3 = O(n). That is, if an algorithm requires time directly proportional to 4n + 3, it is O(n). Other values for the constants c and N will also work. For example, 4n + 3 ≤ 4n + 3n = 7n when n ≥ 1. Thus by choosing c = 7 and N = 1, we have shown that 4n + 3 = O(n). You need to be careful when choosing g(n). For example, we just found that 4n + 3 ≤ 7n when n ≥ 1. But 7n < n2 when n ≥ 7. So why wouldn’t we let g(n) = n2 and conclude that our algorithm is O(n2)? Although we could draw this conclusion, it is not as good—or tight—as it can be. You want the upper bound on f(n) to be as small as possible, and you want it to involve simple functions like the ones given in Figure 9-4. Formalities 9.19 209 Example. Show that 4n2 + 50n - 10 = O(n2). It is easy to see that 4n2 + 50n - 10 ≤ 4n2 + 50n for any n Since 50n ≤ 50n2 for n ≥ 50, 4n2 + 50n - 10 ≤ 4n2 + 50n2 = 54n2 for n ≥ 50 Thus, with c = 54 and N = 50, we have shown that 4n2 + 50n - 10 = O(n2). Note: To show that f(n) = O(g(n)), replace the smaller terms in f(n) with larger terms until only one term is left. Question 3 9.20 Show that 3n2 + 2n = O(2n). What values of c and N did you use? Example. Show that logb n = O(log2 n). Let L = logb n and B = log2 b. From the meaning of a logarithm, we can conclude that n = bL and b = 2B. Combining these two conclusions, we have n = bL = (2B)L = 2BL Thus, log2 n = BL = B logb n or, equivalently, logb n = (1/B) log2 n for any n ≥ 1. Taking c = 1/B and N = 1 in the definition of Big O, we reach the desired conclusion. It follows from this example that the general behavior of a logarithmic function is the same regardless of its base. Often the logarithms used in growth-rate functions are base 2. But since the base really does not matter, we typically omit it. Note: The base of a log in a growth-rate function is usually omitted, since O(loga n) = O(logb n). 9.21 Identities. The following identities hold for Big Oh notation: O(k f(n)) = O(f(n)) O(f(n)) + O(g(n)) = O(f(n) + g(n)) O(f(n)) O(g(n)) = O(f(n) g(n)) By using these identities and ignoring smaller terms in a growth-rate function, you can usually determine the order of an algorithm’s time requirement with little effort. For example, if the growth-rate function is 4n2 + 50n - 10, O(4n2 + 50n - 10) = O(4n2) by ignoring the smaller terms = O(n2) by ignoring the multiplier 9.22 The complexities of program constructs. The time complexity of a sequence of statements in an algorithm or program is the sum of the statements’ individual complexities. However, it is sufficient to take instead the largest of these complexities. Thus, if S1, S2, . . . , Sk is a sequence of statements, and if fi is the growth-rate function for statement Si, the complexity is O(max(f1, f2,..., fk)). The time complexity of the if statement if (condition) S1 else S2 is the sum of the complexity of the condition and the complexity of S1 or S2, whichever is largest. 210 CHAPTER 9 The Efficiency of Algorithms The time complexity of a loop is the complexity of its body times the number of times the body executes. Thus, the complexity of a loop such as for i = 1 S to n is O(n f(n)), where f is the growth-rate function for S. But O(n f(n)) is O(f(n)) according to the identities given in Segment 9.21. 9.23 Other notations. Although we will use Big Oh notation most often in this book, other notations are sometimes useful when describing an algorithm’s time requirement f(n). We mention them here primarily to expose you to them. Beginning with the definition of Big Oh that you saw earlier, we define Big Omega and Big Theta. G G G Big Oh. f(n) is of order at most g(n)—that is, f(n) = O(g(n))—in case positive constants c and N exist such that f(n) ≤ c g(n) for all n ≥ N. The time requirement f(n) is not larger than c g(n). Thus, an analysis that uses Big Oh produces a maximum time requirement for an algorithm. Big Omega. f(n) is of order at least g(n)—that is, f(n) = Ω(g(n))—in case g(n) = O(f(n)). In other words, in case positive constants c and N exist such that f(n) ≥ c g(n) for all n ≥ N. The time requirement f(n) is not smaller than c g(n). Thus, a Big Omega analysis produces a minimum time requirement for an algorithm. Big Theta. f(n) is of order g(n)—that is, f(n) = Θ(g(n))—in case f(n) = O(g(n)) and g(n) = O(f(n)). Another way to say the same thing is f(n) = O(g(n)) and f(n) = Ω(g(n)). The time requirement f(n) is the same as g(n). A Big Theta analysis assures that the time estimate is as good as possible. Even so, Big Oh is the more common notation. Picturing Efficiency 9.24 Much of an algorithm’s work occurs during its repetitive phases, that is, during the execution of loops or as a result of recursive calls. In this section, we will illustrate the time efficiency of several examples. We begin with the loop in Algorithm A of Figure 9-1, which appears in pseudocode as follows: for i = 1 to n sum = sum + i The body of this loop requires a constant amount of execution time, and so it is O(1). Figure 9-8 represents that time with one icon for each repetition of the loop, and so a row of n icons represents the loop’s total execution time. This algorithm is O(n): Its time requirement grows as n grows. Figure 9-8 An O(n) algorithm for i = 1 to n sum = sum + i ... 1 2 3 O(n) n Picturing Efficiency 9.25 211 Algorithm B in Figure 9-1 contains nested loops, as follows: for i = 1 to n { for j = 1 to i sum = sum + 1 } When loops are nested, you examine the innermost loop first. Here, the body of the inner loop requires a constant amount of execution time, and so it is O(1). If we again represent that time with an icon, a row of i icons represents the time requirement for the inner loop. Since the inner loop is the body of the outer loop, it executes n times. Figure 9-9 illustrates the time requirement for these nested loops, which is proportional to 1 + 2 + . . . + n. Question 1 asked you to show that 1 + 2 + . . . + n = n(n + 1) / 2 Thus, the computation is O(n2). Figure 9-9 An O(n2) algorithm for i = 1 to n { for j = 1 to i sum = sum + 1 } i=1 i=2 i=3 . . i=n 1 9.26 O(1 + 2 + ... + n) = O(n2) ... 2 3 n The body of the inner loop in the previous segment executes a variable number of times that depends on the outer loop. Suppose we change the inner loop so that it executes the same number of times for each repetition of the outer loop, as follows: for i = 1 to n { for j = 1 to n sum = sum + 1 } 212 CHAPTER 9 The Efficiency of Algorithms Figure 9-10 illustrates these nested loops and shows that the computation is O(n2). Figure 9-10 Another O(n2) algorithm for i = 1 to n { for j = 1 to n sum = sum + 1 } i=1 ... i=2 ... i=3 ... . . . O(n * n) = O(n2) ... i=n 1 2 3 n Question 4 Using Big Oh notation, what is the order of the following computation’s time requirement? for i = 1 to n { for j = 1 to 5 sum = sum + 1 } 9.27 Let’s get a feel for the growth-rate functions in Figure 9-4. As we mentioned, the time requirement for an O(1) algorithm is independent of the problem size n. We can apply such an algorithm to larger and larger problems without affecting the execution time. This situation is ideal, but not typical. For other orders, what happens if we double the problem size? The time requirement for an O(log n) algorithm will change, but not by much. An O(n) algorithm will need twice the time, an O(n2) algorithm will need four times the time, and an O(n3) algorithm will need eight times the time. Doubling the problem size for an O(2n) algorithm squares the time requirement. Figure 9-11 tabulates these observations. Question 5 Suppose that you can solve a problem of a certain size on a given computer in time T by using an O(n) algorithm. If you double the size of the problem, how fast must your computer be to solve the problem in the same time? Question 6 Repeat Question 5, but instead use an O(n2) algorithm. Picturing Efficiency Figure 9-11 The effect of doubling the problem size on an algorithm’s time requirement Growth-Rate Function Growth-Rate Function for Size n Problems for Size 2n Problems 1 log n n n log n n2 n3 2n 9.28 Figure 9-12 213 1 1 ϩ log n 2n 2n log n ϩ 2n (2n)2 (2n)3 22n Effect on Time Requirement No effect Negligible Doubles Doubles and then adds 2n Quadruples Multiplies by 8 Squares Now suppose that your computer can perform one million operations per second. How long will it take an algorithm to solve a problem whose size is one million? We cannot answer this question exactly without knowing the algorithm, but the computations in Figure 9-12 will give you a sense of how the algorithm’s Big Oh would affect our answer. An O(log n) algorithm would take a fraction of a second, whereas an O(2n) algorithm would take billions of years! The time to process one million items by algorithms of various orders at the rate of one million operations per second Growth-Rate Function f log n n n log n n2 n3 2n f(106)/106 0.0000199 seconds 1 second 19.9 seconds 11.6 days 31,709.8 years 10301,016 years Note: You can use O(n2), O(n3), or even O(2n) algorithms as long as your problem size is small. For example, at the rate of one million operations per second, an O(n2) algorithm would take one second to solve a problem whose size is 1000. An O(n3) algorithm would take one second to solve a problem whose size is 100. And an O(2n) algorithm would take about one second to solve a problem whose size is 20. Question 7 The following algorithm determines whether an array contains duplicates within its first n items: Algorithm hasDuplicates(array, n) for (index = 0 to n-2) for (rest = index+1 to n-1) if (array[index] equals array[rest]) return true return false What is the Big Oh of this algorithm in the worst case? 214 CHAPTER 9 The Efficiency of Algorithms The Efficiency of Implementations of the ADT List We now consider the time efficiency of two implementations of the ADT list that we discussed in previous chapters. The Array-Based Implementation One of the implementations of the ADT list given in Chapter 5 used a fixed-size array to represent the list’s entries. We can now determine the efficiency of the list operations when implemented in this way. 9.29 Adding to the end of the list. Let’s begin with the operation that adds a new entry to the end of the list. Segment 5.6 provided the following implementation for this operation: public boolean add(Object newEntry) { boolean isSuccessful = true; if (!isFull()) { entry[length] = newEntry; length++; } else isSuccessful = false; return isSuccessful; } // end add Each step in this method—determining whether the list is full, assigning a new entry to an array element, and incrementing the length—is an O(1) operation. By applying your knowledge of the material presented in Segments 9.21 and 9.22, you can show that this method is O(1). Intuitively, since you know that the new entry belongs at the end of the list, you know what array element should contain the new entry. Thus, you can make this assignment independently of any other entries in the list. 9.30 Adding to the list at a given position. The ADT list has another method that adds a new entry to a list, but this one adds the entry at a position that the client specifies: public boolean add(int newPosition, Object newEntry) { boolean isSuccessful = true; if (!isFull() && (newPosition >= 1) && (newPosition <= length+1)) { makeRoom(newPosition); entry[newPosition-1] = newEntry; length++; } else isSuccessful = false; return isSuccessful; } // end add The Efficiency of Implementations of the ADT List 215 The method’s general form is similar to the previous add method in that it determines whether the list is full, assigns the new entry to an array element, and increments the length. Once again, these are all O(1) operations, as are the initial comparisons that check the value of newPosition. What is different here is making room in the array for the new entry. This task is accomplished by the private method makeRoom: private void makeRoom(int newPosition) { for (int index = length; index >= newPosition; index--) entry[index] = entry[index-1]; } // end makeRoom The worst case occurs when newPosition is 1 because the method must shift all of the list elements. If the list contains n entries, the body of the loop is repeated n times in the worst case. Therefore, the method makeRoom is O(n). This observation implies that the method add is also O(n). Question 8 What is the Big Oh of the method remove in the worst case? (See Segment 5.8.) Assume an array-based implementation, and use an argument similar to the one we just made for add. Question 9 5.9.) What is the Big Oh of the method replace in the worst case? (See Segment Question 10 What is the Big Oh of the method getEntry in the worst case? (See Segment 5.9.) Question 11 What is the Big Oh of the method contains in the worst case? (See Segment 5.10.) Question 12 What is the Big Oh of the method 5.5.) display in the worst case? (See Segment The Linked Implementation 9.31 Adding to the end of the list. Now consider the linked implementation of the ADT list as given in Chapter 6. Let’s begin with the method in Segment 6.26 that adds to the end of the list: public boolean add(Object newEntry) { Node newNode = new Node(newEntry); if (isEmpty()) firstNode = newNode; else { Node lastNode = getNodeAt(length); lastNode.next = newNode; } // end if length++; return true; } // end add Except for the call to getNodeAt, the statements in this method are all O(1) operations. 216 CHAPTER 9 The Efficiency of Algorithms We need to examine getNodeAt: private Node getNodeAt(int givenPosition) { Node currentNode = firstNode; for (int counter = 1; counter < givenPosition; counter++) currentNode = currentNode.next; return currentNode; } // end getNodeAt Except for the loop, the method contains all O(1) operations. In the worst case, givenPosition is n, so the loop, and therefore the method, is O(n). We can conclude that the method add is also O(n). This result makes intuitive sense since, to add at the end of the list, the method must traverse the chain of linked nodes to locate the last one. 9.32 Adding to the list at a given position. The analysis of the second add method is essentially the same as that of the one in the previous segment. To add a new entry at a given position, the method must traverse the chain of linked nodes to determine the point of insertion. In the worst case, the traversal goes to the end of the chain. If you look at the method’s implementation in Segment 6.30 of Chapter 6, you will see that it calls the method getNodeAt, which you just saw is O(n). Reaching the conclusion that the second add method is O(n) is straightforward. Question 13 What is the Big Oh of the linked implementation of the method remove in the worst case? (See Segment 6.37.) Use an argument similar to the one we just made for add. 9.33 Retrieving an entry. Consider the method within a list: getEntry that retrieves the item at a given position public Object getEntry(int givenPosition) { Object result = null; // result to return if (!isEmpty() && (givenPosition >= 1) && (givenPosition <= length)) result = getNodeAt(givenPosition).data; return result; } // end getEntry This method uses the method getNodeAt to locate the desired entry in the chain of linked nodes. We saw that getNodeAt is O(n), so getEntry is also O(n). Question 14 What is the Big Oh of the method 6.38.) replace in the worst case? (See Segment Question 15 What is the Big Oh of the method contains in the worst case? (See Segment 6.40.) Question 16 What is the Big Oh of the method display in the worst case? (See Appendix E for the answer to Question 11 of Chapter 6.) Comparing the Implementations 9.34 Figure 9-13 summarizes the orders of the ADT list’s operations for the implementations that use an array and a chain of linked nodes. These orders represent the worst-case behaviors of these operations. The Efficiency of Implementations of the ADT List 217 For an array-based implementation of the ADT list, the operations add by position, and and display are each O(n). The other operations are each O(1). For a linked implementation, all operations are O(n) except for clear, getLength, isEmpty, and isFull. These four operations are each O(1). As you can see, many of the operations have the same Big Oh for both implementations. However, operations that add to the end of a list, replace an entry, or retrieve an entry are much faster when you use an array to represent a list than when you use linked nodes. If your application uses these particular operations frequently, an array-based implementation could be attractive. Adding a tail reference to the linked implementation, as was done in Segments 6.43 through 6.48 of Chapter 6, makes adding to the end of the list an O(1) operation. Exercise 5 at the end of this chapter asks you to analyze this implementation. remove, contains, Figure 9-13 The time efficiencies of the ADT list operations for two implementations, expressed in Big Oh notation Operation add(newEntry) add(newPosition, newEntry) remove(givenPosition) replace(givenPosition, newEntry) getEntry(givenPosition) contains(anEntry) display() clear(), getLength(), isEmpty(), isFull() Array Linked O(1) O(n) O(n) O(1) O(1) O(n) O(n) O(1) O(n) O(n) O(n) O(n) O(n) O(n) O(n) O(1) Programming Tip: When choosing an implementation for an ADT, you should consider the operations that your application requires. If you use a particular operation frequently, you want its implementation to be efficient. Conversely, if you rarely use an operation, you can afford to use a class that has an inefficient implementation of that operation. C HAPTER S UMMARY G An algorithm’s complexity is described in terms of the time and space required to execute it. G An algorithm’s time requirement f(n) is of order at most g(n)—that is, f(n) = O(g(n))—in case positive constants c and N exist such that f(n) ≤ c g(n) for all n ≥ N. G The relationships among typical growth-rate functions are as follows: G O(1) < O(log log n) < O(log n) < O(log2 n) < O(n) < O(n log n) < O(n2) < O(n3) < O(2n) < O(n!) G For an array-based implementation of the ADT list, the operations add by position, and remove, contains, and display are each O(n). The other operations are each O(1). For a linked implementation, all operations are O(n) except for clear, getLength, isEmpty, and isFull. These four operations are each O(1). 218 CHAPTER 9 The Efficiency of Algorithms P ROGRAMMING T IP G E XERCISES 1. Using Big Oh notation, indicate the time requirement of each of the following tasks in the worst case. When choosing an implementation for an ADT, you should consider the operations that your application requires. If you use a particular operation frequently, you want its implementation to be efficient. Conversely, if you rarely use an operation, you can afford to use a class that has an inefficient implementation of that operation. a. After arriving at a party, you shake hands with each person there. b. Each person in a room shakes hands with everyone else in the room. c. You climb a flight of stairs. d. You slide down the banister. e. After entering an elevator, you press a button to choose a floor. f. You ride the elevator from the ground floor up to the nth floor. g. You read a book twice. 2. Describe a way to climb from the bottom of a flight of stairs to the top in O(n2) time. 3. Using Big Oh notation, indicate the time requirement of each of the following tasks in the worst case. a. b. c. d. Display all the integers in an array of integers. Display all the integers in a chain of linked nodes. Display the nth integer in an array of integers. Compute the sum of the first n even integers in an array of integers. 4. Chapter 5 describes an implementation of the ADT list that uses an array that can expand dynamically. Using Big Oh notation, what is the time efficiency of the method doubleArray, as given in Segment 5.16? 5. Suppose that you alter the linked implementation of the ADT list to include a tail reference, as described in Segments 6.43 through 6.48. The time efficiencies of what methods, if any, are affected by this change? Use Big Oh notation to describe the efficiencies of any affected methods. 6. By using the definition of Big Oh, show that a. b. c. d. 6n2 + 3 is O(n2) n2 + 17n + 1 is O(n2) 5n3 + 100 n2 - n - 10 is O(n3) 3n2 + 2n is O(2n) 7. In the worst case, Algorithm X requires n2 + 9n + 5 operations and Algorithm Y requires 5n2 operations. What is the Big Oh of each algorithm? 8. Plot the number of operations required by Algorithms X and Y of Exercise 7 as a function of n. What can you conclude? 9. Show that O(loga n) = O(logb n) for a, b > 1. Hint: loga n = logb n/logb a. The Efficiency of Implementations of the ADT List 219 10. Suppose that your implementation of a particular algorithm appears in Java as follows: for (int pass = 1; pass <= n; pass++) { for (int index = 0; index < n; index++) { for (int count = 1; count < 10; count++) { . . . } // end for } // end for } // end for The algorithm involves an array of n items. The previous code shows the only repetition in the algorithm, but it does not show the computations that occur within the loops. These computations, however, are independent of n. What is the order of the algorithm? 11. Repeat Exercise 10, but replace 10 with n in the inner loop. 12. Using Big Oh notation, indicate the time requirement of the methods in IteratorInterface when implemented as an internal iterator of the ADT list. The implementation begins at Segment 7.9. 13. Using Big Oh notation, indicate the time requirement of the methods in IteratorInterface when implemented as an external iterator of the ADT list. The implementation begins at Segment 7.19. Consider both linked and array-based implementations of the list. 14. Using Big Oh notation, indicate the time requirement of the methods in Java’s interface Iterator, as implemented in Chapter 8. The linked implementation begins at Segment 8.6, and the array-based implementation begins at Segment 8.10. 15. Using Big Oh notation, indicate the time requirement of the methods in Java’s interface ListIterator, as implemented in Chapter 8. The array-based implementation begins at Segment 8.24. 16. If f(n) is O(g(n)) and g(n) is O(h(n)), use the definition of Big Oh to show that f(n) is O(h(n)). 17. Segment 9.15 and the chapter summary showed the relationships among typical growthrate functions. Indicate where the following growth-rate functions belong in this ordering: a. n2 log n b. n c. n2/log n 18. Show that 7n2 + 5n is not O(n). 19. Suppose that you have a dictionary whose words are not sorted in alphabetical order. As a function of the number, n, of words, what is the efficiency of searching for a particular word in this dictionary? 20. Repeat Exercise 19 for a dictionary whose words are sorted alphabetically. Compare your results with those for Exercise 19. 220 CHAPTER 9 The Efficiency of Algorithms 21. Consider a football player who runs windsprints on a football field. He begins at the 0yard line and runs to the 1-yard line and then back to the 0-yard line. Then he runs to the 2-yard line and back to the 0-yard line, runs to the 3-yard line and back to the 0-yard line, and so on until he has reached the 10-yard line and returned to the 0-yard line. a. How many total yards does he run? b. How many total yards does he run if he reaches the n-yard line instead of the 10-yard line? c. How does his total distance run compare to that of a sprinter who simply starts at the 0-yard line and races to the n-yard line? 22. Exercise 3 in Chapter 4 asked you to write Java statements at the client level that return the position of a given object in a list. Using Big Oh notation, compare the time requirement of these statements with that of the method getPosition in Exercise 3 of Chapter 5 and Exercise 3 of Chapter 6. P ROJECTS For the following projects, you should know how to time a section of code in Java. One approach is to use the class java.util.Date. A Date object contains the time at which it was constructed. This time is stored as a long integer equal to the number of milliseconds that have passed since 00:00:00.000 GMT on January 1, 1970. By subtracting the starting time in milliseconds from the ending time in milliseconds, you get the run time—in milliseconds— of a section of code. For example, suppose that thisMethod is the name of a method you wish to time. The following statements will compute the number of milliseconds that thisMethod requires to execute: Date current = new Date(); long startTime = current.getTime(); thisMethod(); current = new Date(); long stopTime = current.getTime(); long elapsedTime = stopTime - startTime; // get current time // code to be timed // get current time // milliseconds 1. Write a Java program that implements the three algorithms in Figure 9-1 and times them for various values of n. The program should display a table of the run times of each algorithm for various values of n. 2. Consider the following two loops: // Loop A for (i = 1; i <= n; i++) for (j = 1; j <= 10000; j++) sum = sum + j; // Loop B for (i = 1; i <= n; i++) for (j = 1; j <= n; j++) sum = sum + j; Although loop A is O(n) and loop B is O(n2), loop B can be faster than loop A for small values of n. Design and implement an experiment to find the value of n for which loop B is faster. The Efficiency of Implementations of the ADT List 3. Repeat Project 2, but use the following for loop B: // Loop B for (i = 1; i <= n; i++) for (j = 1; j <= n; j++) for (k = 1; k <= n; k++) sum = sum + k; 221 ...
View Full Document

This note was uploaded on 04/29/2010 for the course CS 5503 taught by Professor Kaylor during the Spring '10 term at W. Alabama.

Ask a homework question - tutors are online