This preview shows page 1. Sign up to view the full content.
Unformatted text preview: C H A P T E R 9 The Efﬁciency of
Algorithms CONTENTS Motivation
Measuring an Algorithm’s Efficiency
Big Oh Notation
Formalities
Picturing Efficiency
The Efficiency of Implementations of the ADT List
The ArrayBased Implementation
The Linked Implementation
Comparing the Implementations
PREREQUISITES Chapter
Chapter
Chapter 1
5
6 Java Classes
List Implementations That Use Arrays
List Implementations That Link Data OBJECTIVES After studying this chapter, you should be able to
G
G Determine the efficiency of a given algorithm
Compare the expected execution times of two methods, given the efficiencies of their algorithms W ith amazing frequency, manufacturers introduce new computers that are faster
and have larger memories than their recent predecessors. Yet we—and likely your
computer science professors—ask you to write code that is efﬁcient in its use of time
and space (memory). We have to admit that such efﬁciency is not as pressing an issue
as it was ﬁfty years ago, when computers were much slower and their memory size
was much smaller than they are now. (Computers had small memories, but they were
physically huge, occupying entire rooms.) Even so, efﬁciency remains an issue—in
some circumstances, a critical issue.
199 200 CHAPTER 9 The Efﬁciency of Algorithms
This chapter will introduce you to the terminology and methods that computer scientists use to
measure the efﬁciency of an algorithm. With this background, you not only will have an intuitive
feel for efﬁciency, but also will be able to talk about efﬁciency in a quantitative way. Motivation
9.1 Perhaps you think that you are not likely to write a program in the near future whose execution time
is noticeably long. You might be right, but we are about to show you some simple Java code that
does take a long time to perform its computations.
Imagine that we are deﬁning a Java class to represent extremely large integers. Of course, we
will need to represent the many digits possible in one of these integers, but that is not the subject of
this example. We want a method for our class that adds two large integers—that is, two instances of
our class—and a method that multiplies two large integers. Suppose that we have implemented the
add method successfully and are about to design a multiply method.
As you know, multiplication is equivalent to repeated addition. So if we want to compute
the product 7562 times 423, for example, we could add 7562 to an initially zero variable 423
times. Remember that our add method works, so we can readily use it in our implementation of
multiply. 9.2 Let’s write some simple Java code to test our idea. Realize that this code is simply an experiment to
verify our approach of repeated addition and does not use anything that we might already have written for our class.
If we use long integers in our experiment, we could write the following statements:
long firstOperand = 7562;
long secondOperand = 423;
long product = 0;
for (; secondOperand > 0; secondOperand)
product = product + firstOperand;
System.out.println(product); If you execute this code, you will get the right answer of 3,198,726. Now change the second operand from 423 to 100,000,000 (that is, a 1 followed by eight zeros), and execute the code again.
Again, you will get the correct answer, which this time is 756,200,000,000 (7562 followed by eight
zeros). However, you should notice a delay in seeing this result. Now try 1,000,000,000, which is a
1 followed by nine zeros. Again you will get the correct answer—7562 followed by nine zeros—but
you will have to wait even longer for the result. The wait might be long enough for you to suspect
that something is broken. If not, try 1 followed by ten zeros!
What’s our point? Our class is supposed to deal with really large integers. The integers in our
little experiment, while large, are not really large. Even so, the previous simple code takes a noticeably long time to execute. Should you use a faster computer? Should you wait for an even faster
computer to be invented? We have a better solution. 9.3 The delay in our example is caused by the excessive number of additions that the code must perform. Instead of adding 7562 to an initially zero variable one billion times, we could save time
by adding one billion to an initially zero variable 7562 times. This approach won’t always be
faster, because one operand might not be much smaller than the other. Besides, we can do much
better. Motivation 201 Consider again the product 7562 * 421. The second operand, 423, has three digits: A hundreds
digit 4, a tens digit 2, and a ones digit 3. That is, 423 is the sum 400 + 20 + 3. Thus,
7562 * 423 = 7562 * (400 + 20 + 3)
= 7562 * 400 + 7562 * 20 + 7562 * 3
= 756200 * 4 + 75620 * 2 + 7562 * 3
Now we can replace the three previous multiplications with additions, so that the desired product is
the sum
(756200 + 756200 + 756200 + 756200) + (75620 + 75620) + (7562 + 7562 + 7562)
There are three parenthesized terms, one for each digit in the second operand. Each of these
terms is the sum of d integers, where d is a digit in the second operand. We can conveniently compute this sum from the right—involving ﬁrst 7562, then 75620, and ﬁnally 756200—because 75620
is 10 * 7562, and 756200 is 10 * 75620. 9.4 We can use these observations to write pseudocode that will compute this type of sum. Let the
operands of the multiplication be firstOperand and secondOperand. The following pseudocode
computes the product of these operands by using addition.
secondOperandLength =
product = 0 number of digits in secondOperand for (; secondOperandLength > 0; secondOperandLength)
{
digit = rightmost digit of secondOperand
for (; digit > 0; digit)
product = product + firstOperand Drop rightmost digit of secondOperand
Tack 0 onto end of firstOperand }
// Assertion: product is the result If we use this pseudocode to compute 7562 * 423, firstOperand is 7562 and secondOperand
is 423. The pseudocode computes the sum
(7562 + 7562 + 7562) + (75620 + 75620) + (756200 + 756200 + 756200 + 756200)
from left to right. The outer loop speciﬁes the number of parenthesized groups as well as their contents, and the inner loop processes the additions within the parentheses. 9.5 Here is some Java code for you to try based on the previous pseudocode. Again, this is an experiment to verify an approach and is not the implementation of a method in our class of large integers.
As such, we will implement the last two steps of the pseudocode by dividing by 10 and multiplying
by 10.
long firstOperand = 7562;
long secondOperand = 100000000;
int secondOperandLength = 9;
long product = 0; 202 CHAPTER 9 The Efﬁciency of Algorithms
for (; secondOperandLength > 0; secondOperandLength)
{
long digit = secondOperand  (secondOperand/10) * 10;
for (; digit > 0; digit)
product = product + firstOperand;
secondOperand = secondOperand/10; // discard last digit
firstOperand = 10 * firstOperand; // tack zero on right
} // end for
System.out.println(product); You should ﬁnd that this code executes faster than the code in Segment 9.2 after you change secondOperand to 100000000. Note: As this example shows, even a simple program can be noticeably inefﬁcient. Measuring an Algorithm’s Efﬁciency
9.6 The previous section should have convinced you that a program’s efﬁciency matters. How can we
measure efﬁciency so that we can compare various approaches to solving a problem? In the previous section, we asked you to run two programs and observe that one was noticeably slower than the
other. In general, having to implement several ideas before you can choose one requires too much
work to be practical. Besides, a program’s execution time depends in part on the particular computer and the programming language used. It would be much better to measure an algorithm’s efﬁciency before you implement it.
For example, suppose that you want to go to a store downtown. Your options are to walk, drive
your car, ask a friend to take you, or take a bus. What is the best way? First, what is your concept of
best? Is it the way that saves money, your time, your friend’s time, or the environment? Let’s say
that the best option for you is the fastest one. After deﬁning your criterion, how do you evaluate
your options? You certainly do not want to try all four options so you can discover which is fastest.
That would be like writing four different programs that perform the same task so you can measure
which one is fastest. Instead you would investigate the “cost” of each option, considering the distance and the speed at which you can travel, the amount of other trafﬁc, the number of stops at trafﬁc lights, the weather, and so on. 9.7 The same considerations apply when deciding what algorithm is best. Again, we need to deﬁne what
we mean by best. An algorithm has both time and space requirements, called its complexity, that we
can measure. Typically we analyze these requirements separately and talk about an algorithm’s time
complexity—the time it takes to execute—or its space complexity—the memory it needs to execute. So a “best” algorithm might be the fastest one or the one that uses the least memory.
The process of measuring the complexity of algorithms is called the analysis of algorithms.
When we measure an algorithm’s complexity, we are not measuring how involved or difﬁcult it is.
We will concentrate on the time complexity of algorithms, because it is usually more important
than space complexity. You should realize that an inverse relationship often exists between an algorithm’s time and space complexities. If you revise an algorithm to save execution time, you usually
will need more space. If you reduce an algorithm’s space requirement, it likely will require more
time to execute. Sometimes, however, you will be able to save both time and space. Measuring an Algorithm’s Efﬁciency 203 Your measure of the complexity of an algorithm should be easy to compute, certainly easier than
implementing the algorithm. You should express this measure in terms of the size of the problem. For
example, if you are searching a collection of data, the problem size is the number of items in the collection. Such a measure enables you to compare the relative cost of algorithms as a function of the
size of the problem. Typically, we are interested in large problems; a small problem is likely to take
little time, even if the algorithm is inefﬁcient. 9.8 9.9 Figure 91 Realize that you cannot compute the actual time requirement of an algorithm. After all, you have
not implemented the algorithm in Java and you have not chosen the computer. Instead, you ﬁnd a
function of the problem size that behaves like the algorithm’s actual time requirement. That is, as
the time requirement increases, the value of the function increases, and vice versa. The value of the
function is said to be directly proportional to the time requirement. Such a function is called a
growthrate function because it measures how an algorithm’s time requirement grows as the problem size grows. By comparing the growthrate functions of two algorithms, you can determine
whether one algorithm is faster than the other.
You could estimate the maximum time that an algorithm could take—that is, its worstcase
time. If you can tolerate this worstcase time, your algorithm is acceptable. You also could estimate
the minimum or bestcase time. If the bestcase time is still too slow, you need another algorithm.
For many algorithms, the worst and best cases rarely occur. A more useful measure is the averagecase time requirement of an algorithm. This measure, however, is usually harder to ﬁnd than the
best and worst cases. Typically, we will ﬁnd the worstcase time.
Example. Consider the problem of computing the sum 1 + 2 + . . . + n for any positive integer n.
Figure 91 contains pseudocode showing three ways to solve this problem. Algorithm A computes
the sum 0 + 1 + 2 + . . . + n from left to right. Algorithm B computes 0 + (1) + (1 + 1) + (1 + 1 + 1)
+ . . . + (1 + 1 + . . . 1). Finally, Algorithm C uses an algebraic identity to compute the sum. Three algorithms for computing the sum 1 + 2 + . . . + n for an integer n > 0
Algorithm A
sum = 0
for i = 1 to n
sum = sum + i Algorithm B
sum = 0
for i = 1 to n
{ for j = 1 to i
sum = sum + 1
} Algorithm C
sum = n * (n + 1) / 2 Which algorithm—A, B, or C—is fastest? We can begin to answer this question by considering
both the size of the problem and the effort involved. The integer n is a measure of the problem size:
As n increases, the sum involves more terms. To measure the effort, or time requirement, of an
algorithm, we must ﬁnd an appropriate growthrate function. To do so, we begin by counting the
number of operations required by the algorithm. Figure 92 tabulates the number of assignments,
additions, multiplications, and divisions that Algorithms A, B, and C require. These counts do not
include the operations that control the loops. We have ignored these operations here to make counting easier, but as you will see later in Segment 9.14, doing so will not affect our ﬁnal conclusion
about algorithm speed. 204 CHAPTER 9 The Efﬁciency of Algorithms
Figure 92 The number of operations required by the algorithms in Figure 91
Algorithm A
Assignments
Additions
Multiplications
Divisions
Total operations Question 1 Algorithm B nϩ1
n 1 ϩ n (n ϩ 1) / 2
n (n ϩ 1) / 2 2n ؉ 1 n2 ؉ n ؉ 1 Algorithm C
1
1
1
1
4 For any positive integer n, the identity 1 + 2 + . . . + n = n(n + 1) / 2
is one that you will encounter while analyzing algorithms. Can you derive it? If you can, you
will not need to memorize it. Hint: Write 1 + 2 + . . . + n. Under it write n + (n  1) + . . . + 1.
Then add terms from left to right.
Question 2 Can you derive the values in Figure 92? Hint: For Algorithm B, use the identity given in Question 1. Note: Useful identities 1 + 2 + . . . + n = n(n + 1) / 2
1 + 2 + . . . + (n  1) = n(n  1) / 2 9.10 The various operations listed in Figure 92 probably take different amounts of time to execute.
However, if we assume that they each require the same amount of time, we will still be able to
determine which algorithm is fastest. For example, Algorithm A requires n + 1 assignments and n
additions. If each assignment takes no more than t= time units and each addition takes no more than
t+ time units, Algorithm A requires no more than (n + 1) t= + n t+ time units. If we replace t= and t+
with the larger of the two values and call it t, Algorithm A requires no more than (2n + 1) t time
units. Whether we look at a time estimate such as (2n + 1) t or the total number of operations 2n +
1, we can draw the same conclusion: Algorithm A requires time directly proportional to 2n + 1 in
the worst case. Thus, Algorithm A’s growthrate function is 2n + 1.
Using similar reasoning, we can conclude that Algorithm B requires time directly proportional
to n2 + n + 1, and Algorithm C requires time that is constant and independent of the value of n.
Figure 93 plots these time requirements as a function of n. You can see from this ﬁgure that as n
grows, Algorithm B requires the most time. 9.11 Typical growthrate functions are algebraically simpler than the ones you have just seen. Why?
Recall that since you are not likely to notice the effect of an inefﬁcient algorithm when the problem
is small, you should focus on large problems. Thus, if we care only about large values of n when
comparing the algorithms, we can consider only the dominant term in each growthrate function.
For example, n2 + n + 1 behaves like n2 when n is large because n2 is much larger than n + 1 in
that case. In other words, the difference between the value of n2 + n + 1 and that of n2 is relatively
small and can be ignored when n is large. So instead of using n2 + n + 1 as Algorithm B’s growthrate function, we can use n2—the term with the largest exponent—and say that Algorithm B
requires time proportional to n2. Likewise, Algorithm A requires time proportional to n. On the
other hand, Algorithm C requires time that is independent of n. Measuring an Algorithm’s Efﬁciency The number of operations required by the algorithms in Figure 91 as a function of n
32
28 Number of operations Figure 93 205 Algorithm B:
n2 ϩ n ϩ 1 operations 24
20
16 Algorithm A:
2n ϩ 1 operations 12
8
4
0 Algorithm C: 4 operations
1 2 3 4 5 n Big Oh Notation
9.12 Computer scientists use a notation to represent an algorithm’s complexity. Instead of saying that
Algorithm A has a worstcase time requirement proportional to n, we say that A is O(n). We call
this notation Big Oh since it uses the capital letter O. We read O(n) as either “Big Oh of n” or
“order of at most n.” Similarly, since Algorithm B has a worstcase time requirement proportional
to n2, we say that B is O(n2). Algorithm C always requires four operations. Regardless of the problem size n, this algorithm requires the same time, be it worst case, best case, or average case. We
say that Algorithm C is O(1). We will discuss Big Oh notation more carefully and introduce other
notations in the next section. 9.13 Example. Imagine that you are at a wedding reception, seated at a table of n people. In preparation
for the toast, the waiter pours champagne into each of n glasses. That task is O(n). Someone makes
a toast. It is O(1), even if the toast seems to last forever, because it is independent of the number of
guests. If you clink your glass with everyone at your table, you have performed an O(n) operation.
If everyone at your table does likewise, they collectively have performed an O(n2) operation. 9.14 Example. In Segment 9.9, we ignored the operations that control the loops in the algorithms.
Obviously this simpliﬁcation affects the total number of operations, but even if we counted them,
we would get the same growthrate functions that you saw in the previous segment. For example,
Algorithm A contains the for statement
for i = 1 to n This statement represents an assignment to i, additions to i, and comparisons with n. In total, the
loopcontrol logic requires 1 assignment, n additions, and n + 1 comparisons, for a total of 2n + 2
additional operations. So Algorithm A actually requires 4n + 3 operations instead of only 2n + 1.
What is important, however, is not the exact count of operations, but the general behavior of the
algorithm. The functions 4n + 3 and 2n + 1 are each directly proportional to n. For large values of n,
the difference between these two functions is negligible. We do not have to count every operation to
see that Algorithm A requires time that increases linearly with n. Thus, Algorithm A is O(n). 206 CHAPTER 9 The Efﬁciency of Algorithms 9.15 The growthrate functions that you are likely to encounter grow in magnitude as follows when
n > 10:
O(1) < O(log log n) < O(log n) < O(log2 n) < O(n) < O(n log n) < O(n2) < O(n3) < O(2n) < O(n!)
The logarithms given here are base 2. As you will see later in Segment 9.20, the choice of base does
not matter.
Figure 94 tabulates the magnitudes of these functions for increasing values of the problem
size n. From this data you can see that O(log log n), O(log n), and O(log2 n) algorithms are much
faster than O(n) algorithms. Although O(n log n) algorithms are signiﬁcantly slower than O(n)
algorithms, they are markedly faster than O(n2) algorithms. Figure 94 Typical growthrate functions evaluated at increasing values of n n
10
102
103
104
105
106 log (log n) log n (log n)2
2
3
3
4
4
4 3
7
10
13
17
20 11
44
99
177
276
397 n n log n n2 n3 2n n! 10
100
1000
10,000
100,000
1,000,000 33
664
9966
132,877
1,660,964
19,931,569 102
104
106
108
1010
1012 103
106
109
1012
1015
1018 103
1030
10301
103010
1030,103
10301,030 105
1094
101435
1019,335
10243,338
102,933,369 Note: When analyzing the time efﬁciency of an algorithm, consider large problems. For small
problems, the difference between the execution times of two solutions to the same problem is
usually insigniﬁcant. 9.16 Example. Segments 9.2 and 9.5 showed you two ways to perform a computation. One way was
noticeably slow. You should now be able to predict this behavior without actually running the code.
The ﬁrst way—call it Method 1—used this loop:
long product = 0;
for (; secondOperand > 0; secondOperand)
product = product + firstOperand; How many times is firstOperand added to product? If secondOperand contains an integer n, then
n such additions occur. When n is large, so is the number of additions required to compute the solution. From our discussion in the previous section, we can say that Method 1 is O(n).
The second way—call it Method 2—used the following code, where secondOperandLength is
the number of digits in secondOperand:
long product = 0;
for (; secondOperandLength > 0; secondOperandLength)
{
long digit = secondOperand  (secondOperand/10) * 10;
for (; digit > 0; digit)
product = product + firstOperand;
secondOperand = secondOperand/10;
firstOperand = 10 * firstOperand;
} // end for Measuring an Algorithm’s Efﬁciency 207 How many times is firstOperand added to product this time? Each digit of the second operand can be at most 9, so the inner for loop adds firstOperand to product at most nine times.
The loop, in fact, executes once for each digit in secondOperand. Since secondOperand contains
secondOperandLength digits, the loop adds firstOperand to product at most 9 * secondOperandLength times.
If secondOperand contains an integer n, as it does in Method 1, how does n affect secondOperandLength here? The data in Figure 95 can help us answer this question. The ﬁgure tabulates
log10 n truncated to an integer—which we denote as log10 n —for twodigit, threedigit, and
fourdigit values of the integer n. (Note that 4.9 , for example, is 4.) You can see that the number
of digits in n is 1 + log10 n . Therefore, the number of additions that Method 2 requires—at most
9 * secondOperandLength—is at most 9 * (1 + log10 n ).
Figure 95 The number of digits in an integer n compared with the integer portion of log10 n
n Number of Digits log10 n 10 – 99
100 – 999
1000 – 9999 2
3
4 1
2
3 Now we need to compare our analyses of Methods 1 and 2. Method 1 requires n additions.
Method 2 requires at most 9 * (1 + log10 n ) additions. Which approach requires the fewest additions? Let’s see if we can answer this question by looking at the data in Figure 96. For values of n
greater than 18,
n > 9 * (1 + log10 n )
Figure 96 The values of two logarithmic growthrate functions for various ranges of n
n log10 n 10 – 99
100 – 999
1000 – 9999
10,000 – 99,999
100,000 – 999,999 1
2
3
4
5 9 * (1 ϩ log10 n)
18
27
36
45
54 As n increases, n is much larger than 9 * (1 + log10 n ). Since Method 1 is O(n), it requires many
more additions than Method 2 when n is large, and so it is much slower. In fact, Method 2 is O(log n)
in its worst case. Figure 94 shows that an O(n) algorithm is slower than an O(log n) algorithm. Note: Floors and ceilings
The ﬂoor of a number x, denoted as x , is the largest integer less than or equal to x. For example, 4.9 is 4. When you truncate a real number to an integer, you actually are computing the
number’s ﬂoor by discarding any fractional portion. The ceiling of a number x, denoted as x ,
is the smallest integer greater than or equal to x. For example, 4.1 is 5. 208 CHAPTER 9 The Efﬁciency of Algorithms Formalities
9.17 Big Oh notation has a formal mathematical meaning that can justify some of the sleightofhand
we used in the previous sections. You saw that an algorithm’s actual time requirement is directly
proportional to a function f of the problem size n. For example, f(n) might be n2 + n + 1. In
this case, we would conclude that the algorithm is of order at most n2—that is, O(n2). We
essentially have replaced f(n) with a simpler function—let’s call it g(n). In this example, g(n) is n2.
What does it really mean to say that a function f(n) is of order at most g(n)—that is, that f(n) =
O(g(n))? In simple terms, it means that Big Oh provides an upper bound on a function’s growth
rate. More formally, we have the following mathematical deﬁnition. Note: Formal deﬁnition of Big Oh An algorithm’s time requirement f(n) is of order at most g(n)—that is, f(n) = O(g(n))—in case a
positive real number c and positive integer N exist such that f(n) ≤ c g(n) for all n ≥ N.
Figure 97 illustrates this deﬁnition. You can see that when n is large enough—that is, when
n ≥ N— f(n) does not exceed c g(n). The opposite is true for smaller values of n.
An illustration of the deﬁnition of Big Oh Value of growthrate function Figure 97 25 c g(n) 20
f(n)
15
10
5
0 0 5 N 10 15 20 25 30 n 9.18 Example. In Segment 9.14, we said that an algorithm that uses 4n + 3 operations is O(n). We now
can show that 4n + 3 = O(n) by using the formal deﬁnition of Big Oh.
When n ≥ 3, 4n + 3 ≤ 4n + n = 5n. Thus, if we let f(n) = 4n + 3, g(n) = n, c = 5, and N = 3, we
have shown that 4n + 3 = O(n). That is, if an algorithm requires time directly proportional to 4n + 3,
it is O(n).
Other values for the constants c and N will also work. For example, 4n + 3 ≤ 4n + 3n = 7n when
n ≥ 1. Thus by choosing c = 7 and N = 1, we have shown that 4n + 3 = O(n).
You need to be careful when choosing g(n). For example, we just found that 4n + 3 ≤ 7n when
n ≥ 1. But 7n < n2 when n ≥ 7. So why wouldn’t we let g(n) = n2 and conclude that our algorithm is
O(n2)? Although we could draw this conclusion, it is not as good—or tight—as it can be. You want
the upper bound on f(n) to be as small as possible, and you want it to involve simple functions like
the ones given in Figure 94. Formalities 9.19 209 Example. Show that 4n2 + 50n  10 = O(n2). It is easy to see that
4n2 + 50n  10 ≤ 4n2 + 50n for any n
Since 50n ≤ 50n2 for n ≥ 50,
4n2 + 50n  10 ≤ 4n2 + 50n2 = 54n2 for n ≥ 50
Thus, with c = 54 and N = 50, we have shown that 4n2 + 50n  10 = O(n2). Note: To show that f(n) = O(g(n)), replace the smaller terms in f(n) with larger terms until
only one term is left.
Question 3 9.20 Show that 3n2 + 2n = O(2n). What values of c and N did you use? Example. Show that logb n = O(log2 n). Let L = logb n and B = log2 b. From the meaning of a logarithm, we can conclude that n = bL and b = 2B. Combining these two conclusions, we have
n = bL = (2B)L = 2BL
Thus, log2 n = BL = B logb n or, equivalently, logb n = (1/B) log2 n for any n ≥ 1. Taking c = 1/B
and N = 1 in the deﬁnition of Big O, we reach the desired conclusion.
It follows from this example that the general behavior of a logarithmic function is the same
regardless of its base. Often the logarithms used in growthrate functions are base 2. But since the
base really does not matter, we typically omit it. Note: The base of a log in a growthrate function is usually omitted, since O(loga n) = O(logb n).
9.21 Identities. The following identities hold for Big Oh notation:
O(k f(n)) = O(f(n))
O(f(n)) + O(g(n)) = O(f(n) + g(n))
O(f(n)) O(g(n)) = O(f(n) g(n))
By using these identities and ignoring smaller terms in a growthrate function, you can usually
determine the order of an algorithm’s time requirement with little effort. For example, if the
growthrate function is 4n2 + 50n  10,
O(4n2 + 50n  10) = O(4n2) by ignoring the smaller terms
= O(n2) by ignoring the multiplier 9.22 The complexities of program constructs. The time complexity of a sequence of statements in an
algorithm or program is the sum of the statements’ individual complexities. However, it is sufﬁcient
to take instead the largest of these complexities. Thus, if S1, S2, . . . , Sk is a sequence of statements,
and if fi is the growthrate function for statement Si, the complexity is O(max(f1, f2,..., fk)).
The time complexity of the if statement
if (condition) S1
else
S2 is the sum of the complexity of the condition and the complexity of S1 or S2, whichever is largest. 210 CHAPTER 9 The Efﬁciency of Algorithms
The time complexity of a loop is the complexity of its body times the number of times the body
executes. Thus, the complexity of a loop such as
for i = 1 S to n is O(n f(n)), where f is the growthrate function for S. But O(n f(n)) is O(f(n)) according to the identities given in Segment 9.21. 9.23 Other notations. Although we will use Big Oh notation most often in this book, other notations are
sometimes useful when describing an algorithm’s time requirement f(n). We mention them here primarily to expose you to them. Beginning with the deﬁnition of Big Oh that you saw earlier, we
deﬁne Big Omega and Big Theta.
G G G Big Oh. f(n) is of order at most g(n)—that is, f(n) = O(g(n))—in case positive constants c
and N exist such that f(n) ≤ c g(n) for all n ≥ N. The time requirement f(n) is not larger than
c g(n). Thus, an analysis that uses Big Oh produces a maximum time requirement for an
algorithm.
Big Omega. f(n) is of order at least g(n)—that is, f(n) = Ω(g(n))—in case g(n) = O(f(n)). In
other words, in case positive constants c and N exist such that f(n) ≥ c g(n) for all n ≥ N. The
time requirement f(n) is not smaller than c g(n). Thus, a Big Omega analysis produces a minimum time requirement for an algorithm.
Big Theta. f(n) is of order g(n)—that is, f(n) = Θ(g(n))—in case f(n) = O(g(n)) and g(n) =
O(f(n)). Another way to say the same thing is f(n) = O(g(n)) and f(n) = Ω(g(n)). The time
requirement f(n) is the same as g(n). A Big Theta analysis assures that the time estimate is as
good as possible. Even so, Big Oh is the more common notation. Picturing Efﬁciency
9.24 Much of an algorithm’s work occurs during its repetitive phases, that is, during the execution of
loops or as a result of recursive calls. In this section, we will illustrate the time efﬁciency of several
examples.
We begin with the loop in Algorithm A of Figure 91, which appears in pseudocode as follows:
for i = 1 to n
sum = sum + i The body of this loop requires a constant amount of execution time, and so it is O(1). Figure 98
represents that time with one icon for each repetition of the loop, and so a row of n icons represents the loop’s total execution time. This algorithm is O(n): Its time requirement grows as n
grows.
Figure 98 An O(n) algorithm
for i = 1 to n
sum = sum + i
...
1 2 3 O(n)
n Picturing Efﬁciency 9.25 211 Algorithm B in Figure 91 contains nested loops, as follows:
for i = 1 to n
{
for j = 1 to i
sum = sum + 1
} When loops are nested, you examine the innermost loop ﬁrst. Here, the body of the inner loop
requires a constant amount of execution time, and so it is O(1). If we again represent that time with
an icon, a row of i icons represents the time requirement for the inner loop. Since the inner loop is
the body of the outer loop, it executes n times. Figure 99 illustrates the time requirement for these
nested loops, which is proportional to 1 + 2 + . . . + n. Question 1 asked you to show that
1 + 2 + . . . + n = n(n + 1) / 2
Thus, the computation is O(n2).
Figure 99 An O(n2) algorithm
for i = 1 to n
{ for j = 1 to i
sum = sum + 1
}
i=1 i=2 i=3
.
. i=n
1 9.26 O(1 + 2 + ... + n) = O(n2) ...
2 3 n The body of the inner loop in the previous segment executes a variable number of times that
depends on the outer loop. Suppose we change the inner loop so that it executes the same number of
times for each repetition of the outer loop, as follows:
for i = 1 to n
{
for j = 1 to n
sum = sum + 1
} 212 CHAPTER 9 The Efﬁciency of Algorithms
Figure 910 illustrates these nested loops and shows that the computation is O(n2).
Figure 910 Another O(n2) algorithm
for i = 1 to n
{ for j = 1 to n
sum = sum + 1
}
i=1 ... i=2 ... i=3 ... .
.
.
O(n * n) = O(n2) ... i=n
1 2 3 n Question 4 Using Big Oh notation, what is the order of the following computation’s time
requirement?
for i = 1 to n
{
for j = 1 to 5
sum = sum + 1
} 9.27 Let’s get a feel for the growthrate functions in Figure 94. As we mentioned, the time requirement for an O(1) algorithm is independent of the problem size n. We can apply such an algorithm
to larger and larger problems without affecting the execution time. This situation is ideal, but not
typical.
For other orders, what happens if we double the problem size? The time requirement for an
O(log n) algorithm will change, but not by much. An O(n) algorithm will need twice the time, an
O(n2) algorithm will need four times the time, and an O(n3) algorithm will need eight times the
time. Doubling the problem size for an O(2n) algorithm squares the time requirement. Figure 911
tabulates these observations.
Question 5 Suppose that you can solve a problem of a certain size on a given computer in
time T by using an O(n) algorithm. If you double the size of the problem, how fast must
your computer be to solve the problem in the same time?
Question 6 Repeat Question 5, but instead use an O(n2) algorithm. Picturing Efﬁciency Figure 911 The effect of doubling the problem size on an algorithm’s time requirement GrowthRate Function GrowthRate Function
for Size n Problems
for Size 2n Problems
1
log n
n
n log n
n2
n3
2n 9.28 Figure 912 213 1
1 ϩ log n
2n
2n log n ϩ 2n
(2n)2
(2n)3
22n Effect on Time
Requirement
No effect
Negligible
Doubles
Doubles and then adds 2n
Quadruples
Multiplies by 8
Squares Now suppose that your computer can perform one million operations per second. How long will it
take an algorithm to solve a problem whose size is one million? We cannot answer this question
exactly without knowing the algorithm, but the computations in Figure 912 will give you a sense
of how the algorithm’s Big Oh would affect our answer. An O(log n) algorithm would take a fraction of a second, whereas an O(2n) algorithm would take billions of years! The time to process one million items by algorithms of various orders at the rate of one million operations per second
GrowthRate
Function f
log n
n
n log n
n2
n3
2n f(106)/106
0.0000199 seconds
1 second
19.9 seconds
11.6 days
31,709.8 years
10301,016 years Note: You can use O(n2), O(n3), or even O(2n) algorithms as long as your problem size is
small. For example, at the rate of one million operations per second, an O(n2) algorithm would
take one second to solve a problem whose size is 1000. An O(n3) algorithm would take one second to solve a problem whose size is 100. And an O(2n) algorithm would take about one second
to solve a problem whose size is 20. Question 7 The following algorithm determines whether an array contains duplicates within
its ﬁrst n items:
Algorithm hasDuplicates(array, n)
for (index = 0 to n2)
for (rest = index+1 to n1)
if (array[index] equals array[rest])
return true
return false What is the Big Oh of this algorithm in the worst case? 214 CHAPTER 9 The Efﬁciency of Algorithms The Efﬁciency of Implementations of the ADT List
We now consider the time efﬁciency of two implementations of the ADT list that we discussed in
previous chapters. The ArrayBased Implementation
One of the implementations of the ADT list given in Chapter 5 used a ﬁxedsize array to represent
the list’s entries. We can now determine the efﬁciency of the list operations when implemented in
this way. 9.29 Adding to the end of the list. Let’s begin with the operation that adds a new entry to the end of the
list. Segment 5.6 provided the following implementation for this operation:
public boolean add(Object newEntry)
{
boolean isSuccessful = true;
if (!isFull())
{
entry[length] = newEntry;
length++;
}
else
isSuccessful = false;
return isSuccessful;
} // end add Each step in this method—determining whether the list is full, assigning a new entry to an array
element, and incrementing the length—is an O(1) operation. By applying your knowledge of the
material presented in Segments 9.21 and 9.22, you can show that this method is O(1). Intuitively,
since you know that the new entry belongs at the end of the list, you know what array element
should contain the new entry. Thus, you can make this assignment independently of any other
entries in the list. 9.30 Adding to the list at a given position. The ADT list has another method that adds a new entry to a
list, but this one adds the entry at a position that the client speciﬁes:
public boolean add(int newPosition, Object newEntry)
{
boolean isSuccessful = true;
if (!isFull() && (newPosition >= 1)
&& (newPosition <= length+1))
{
makeRoom(newPosition);
entry[newPosition1] = newEntry;
length++;
}
else
isSuccessful = false;
return isSuccessful;
} // end add The Efﬁciency of Implementations of the ADT List 215 The method’s general form is similar to the previous add method in that it determines whether
the list is full, assigns the new entry to an array element, and increments the length. Once again,
these are all O(1) operations, as are the initial comparisons that check the value of newPosition.
What is different here is making room in the array for the new entry. This task is accomplished by
the private method makeRoom:
private void makeRoom(int newPosition)
{
for (int index = length; index >= newPosition; index)
entry[index] = entry[index1];
} // end makeRoom The worst case occurs when newPosition is 1 because the method must shift all of the list elements. If the list contains n entries, the body of the loop is repeated n times in the worst case.
Therefore, the method makeRoom is O(n). This observation implies that the method add is also O(n).
Question 8 What is the Big Oh of the method remove in the worst case? (See Segment
5.8.) Assume an arraybased implementation, and use an argument similar to the one we just
made for add.
Question 9
5.9.) What is the Big Oh of the method replace in the worst case? (See Segment Question 10 What is the Big Oh of the method getEntry in the worst case? (See Segment
5.9.)
Question 11 What is the Big Oh of the method contains in the worst case? (See Segment
5.10.)
Question 12 What is the Big Oh of the method
5.5.) display in the worst case? (See Segment The Linked Implementation
9.31 Adding to the end of the list. Now consider the linked implementation of the ADT list as given in
Chapter 6. Let’s begin with the method in Segment 6.26 that adds to the end of the list:
public boolean add(Object newEntry)
{
Node newNode = new Node(newEntry);
if (isEmpty())
firstNode = newNode;
else
{
Node lastNode = getNodeAt(length);
lastNode.next = newNode;
} // end if
length++;
return true;
} // end add Except for the call to getNodeAt, the statements in this method are all O(1) operations. 216 CHAPTER 9 The Efﬁciency of Algorithms
We need to examine getNodeAt:
private Node getNodeAt(int givenPosition)
{
Node currentNode = firstNode;
for (int counter = 1; counter < givenPosition; counter++)
currentNode = currentNode.next;
return currentNode;
} // end getNodeAt Except for the loop, the method contains all O(1) operations. In the worst case, givenPosition is n,
so the loop, and therefore the method, is O(n). We can conclude that the method add is also O(n).
This result makes intuitive sense since, to add at the end of the list, the method must traverse the
chain of linked nodes to locate the last one. 9.32 Adding to the list at a given position. The analysis of the second add method is essentially the
same as that of the one in the previous segment. To add a new entry at a given position, the method
must traverse the chain of linked nodes to determine the point of insertion. In the worst case, the
traversal goes to the end of the chain. If you look at the method’s implementation in Segment 6.30
of Chapter 6, you will see that it calls the method getNodeAt, which you just saw is O(n). Reaching
the conclusion that the second add method is O(n) is straightforward.
Question 13 What is the Big Oh of the linked implementation of the method remove in the
worst case? (See Segment 6.37.) Use an argument similar to the one we just made for add. 9.33 Retrieving an entry. Consider the method
within a list: getEntry that retrieves the item at a given position public Object getEntry(int givenPosition)
{
Object result = null; // result to return
if (!isEmpty() && (givenPosition >= 1) && (givenPosition <=
length))
result = getNodeAt(givenPosition).data;
return result;
} // end getEntry This method uses the method getNodeAt to locate the desired entry in the chain of linked nodes. We
saw that getNodeAt is O(n), so getEntry is also O(n).
Question 14 What is the Big Oh of the method
6.38.) replace in the worst case? (See Segment Question 15 What is the Big Oh of the method contains in the worst case? (See Segment
6.40.)
Question 16 What is the Big Oh of the method display in the worst case? (See Appendix E
for the answer to Question 11 of Chapter 6.) Comparing the Implementations
9.34 Figure 913 summarizes the orders of the ADT list’s operations for the implementations that use an
array and a chain of linked nodes. These orders represent the worstcase behaviors of these operations. The Efﬁciency of Implementations of the ADT List 217 For an arraybased implementation of the ADT list, the operations add by position, and
and display are each O(n). The other operations are each O(1). For a linked
implementation, all operations are O(n) except for clear, getLength, isEmpty, and isFull. These
four operations are each O(1).
As you can see, many of the operations have the same Big Oh for both implementations. However, operations that add to the end of a list, replace an entry, or retrieve an entry are much faster
when you use an array to represent a list than when you use linked nodes. If your application uses
these particular operations frequently, an arraybased implementation could be attractive.
Adding a tail reference to the linked implementation, as was done in Segments 6.43 through
6.48 of Chapter 6, makes adding to the end of the list an O(1) operation. Exercise 5 at the end of
this chapter asks you to analyze this implementation.
remove, contains, Figure 913 The time efﬁciencies of the ADT list operations for two implementations, expressed in Big
Oh notation
Operation
add(newEntry)
add(newPosition, newEntry)
remove(givenPosition)
replace(givenPosition, newEntry)
getEntry(givenPosition)
contains(anEntry)
display()
clear(), getLength(), isEmpty(), isFull() Array Linked O(1)
O(n)
O(n)
O(1)
O(1)
O(n)
O(n)
O(1) O(n)
O(n)
O(n)
O(n)
O(n)
O(n)
O(n)
O(1) Programming Tip: When choosing an implementation for an ADT, you should consider the operations that your application requires. If you use a particular operation frequently, you want
its implementation to be efﬁcient. Conversely, if you rarely use an operation, you can afford to
use a class that has an inefﬁcient implementation of that operation. C HAPTER S UMMARY G An algorithm’s complexity is described in terms of the time and space required to execute it. G An algorithm’s time requirement f(n) is of order at most g(n)—that is, f(n) = O(g(n))—in case positive
constants c and N exist such that f(n) ≤ c g(n) for all n ≥ N. G The relationships among typical growthrate functions are as follows: G O(1) < O(log log n) < O(log n) < O(log2 n) < O(n) < O(n log n) < O(n2) < O(n3) < O(2n) < O(n!) G For an arraybased implementation of the ADT list, the operations add by position, and remove, contains, and display are each O(n). The other operations are each O(1). For a linked implementation,
all operations are O(n) except for clear, getLength, isEmpty, and isFull. These four operations
are each O(1). 218 CHAPTER 9 The Efﬁciency of Algorithms P ROGRAMMING T IP G E XERCISES 1. Using Big Oh notation, indicate the time requirement of each of the following tasks in
the worst case. When choosing an implementation for an ADT, you should consider the operations that your application requires. If you use a particular operation frequently, you want its implementation to be efﬁcient.
Conversely, if you rarely use an operation, you can afford to use a class that has an inefﬁcient implementation of that operation. a. After arriving at a party, you shake hands with each person there.
b. Each person in a room shakes hands with everyone else in the room.
c. You climb a ﬂight of stairs.
d. You slide down the banister.
e. After entering an elevator, you press a button to choose a ﬂoor.
f. You ride the elevator from the ground ﬂoor up to the nth ﬂoor.
g. You read a book twice.
2. Describe a way to climb from the bottom of a ﬂight of stairs to the top in O(n2) time.
3. Using Big Oh notation, indicate the time requirement of each of the following tasks in
the worst case.
a.
b.
c.
d. Display all the integers in an array of integers.
Display all the integers in a chain of linked nodes.
Display the nth integer in an array of integers.
Compute the sum of the ﬁrst n even integers in an array of integers. 4. Chapter 5 describes an implementation of the ADT list that uses an array that can
expand dynamically. Using Big Oh notation, what is the time efﬁciency of the method
doubleArray, as given in Segment 5.16?
5. Suppose that you alter the linked implementation of the ADT list to include a tail
reference, as described in Segments 6.43 through 6.48. The time efﬁciencies of what
methods, if any, are affected by this change? Use Big Oh notation to describe the
efﬁciencies of any affected methods.
6. By using the deﬁnition of Big Oh, show that
a.
b.
c.
d. 6n2 + 3 is O(n2)
n2 + 17n + 1 is O(n2)
5n3 + 100 n2  n  10 is O(n3)
3n2 + 2n is O(2n) 7. In the worst case, Algorithm X requires n2 + 9n + 5 operations and Algorithm Y requires
5n2 operations. What is the Big Oh of each algorithm?
8. Plot the number of operations required by Algorithms X and Y of Exercise 7 as a
function of n. What can you conclude?
9. Show that O(loga n) = O(logb n) for a, b > 1. Hint: loga n = logb n/logb a. The Efﬁciency of Implementations of the ADT List 219 10. Suppose that your implementation of a particular algorithm appears in Java as follows:
for (int pass = 1; pass <= n; pass++)
{
for (int index = 0; index < n; index++)
{
for (int count = 1; count < 10; count++)
{
. . .
} // end for
} // end for
} // end for The algorithm involves an array of n items. The previous code shows the only repetition
in the algorithm, but it does not show the computations that occur within the loops.
These computations, however, are independent of n. What is the order of the algorithm?
11. Repeat Exercise 10, but replace 10 with n in the inner loop.
12. Using Big Oh notation, indicate the time requirement of the methods in
IteratorInterface when implemented as an internal iterator of the ADT list. The
implementation begins at Segment 7.9.
13. Using Big Oh notation, indicate the time requirement of the methods in
IteratorInterface when implemented as an external iterator of the ADT list. The
implementation begins at Segment 7.19. Consider both linked and arraybased
implementations of the list.
14. Using Big Oh notation, indicate the time requirement of the methods in Java’s interface
Iterator, as implemented in Chapter 8. The linked implementation begins at Segment
8.6, and the arraybased implementation begins at Segment 8.10.
15. Using Big Oh notation, indicate the time requirement of the methods in Java’s interface
ListIterator, as implemented in Chapter 8. The arraybased implementation begins at
Segment 8.24.
16. If f(n) is O(g(n)) and g(n) is O(h(n)), use the deﬁnition of Big Oh to show that f(n) is
O(h(n)).
17. Segment 9.15 and the chapter summary showed the relationships among typical growthrate functions. Indicate where the following growthrate functions belong in this
ordering:
a. n2 log n
b. n
c. n2/log n
18. Show that 7n2 + 5n is not O(n).
19. Suppose that you have a dictionary whose words are not sorted in alphabetical order. As
a function of the number, n, of words, what is the efﬁciency of searching for a particular
word in this dictionary?
20. Repeat Exercise 19 for a dictionary whose words are sorted alphabetically. Compare
your results with those for Exercise 19. 220 CHAPTER 9 The Efﬁciency of Algorithms
21. Consider a football player who runs windsprints on a football ﬁeld. He begins at the 0yard line and runs to the 1yard line and then back to the 0yard line. Then he runs to the
2yard line and back to the 0yard line, runs to the 3yard line and back to the 0yard
line, and so on until he has reached the 10yard line and returned to the 0yard line.
a. How many total yards does he run?
b. How many total yards does he run if he reaches the nyard line instead of the 10yard
line?
c. How does his total distance run compare to that of a sprinter who simply starts at the
0yard line and races to the nyard line?
22. Exercise 3 in Chapter 4 asked you to write Java statements at the client level that return
the position of a given object in a list. Using Big Oh notation, compare the time
requirement of these statements with that of the method getPosition in Exercise 3 of
Chapter 5 and Exercise 3 of Chapter 6. P ROJECTS For the following projects, you should know how to time a section of code in Java. One
approach is to use the class java.util.Date. A Date object contains the time at which it was
constructed. This time is stored as a long integer equal to the number of milliseconds that
have passed since 00:00:00.000 GMT on January 1, 1970. By subtracting the starting time in
milliseconds from the ending time in milliseconds, you get the run time—in milliseconds—
of a section of code.
For example, suppose that thisMethod is the name of a method you wish to time. The following statements will compute the number of milliseconds that thisMethod requires to execute:
Date current = new Date();
long startTime = current.getTime();
thisMethod();
current = new Date();
long stopTime = current.getTime();
long elapsedTime = stopTime  startTime; // get current time
// code to be timed
// get current time
// milliseconds 1. Write a Java program that implements the three algorithms in Figure 91 and times them
for various values of n. The program should display a table of the run times of each
algorithm for various values of n.
2. Consider the following two loops:
// Loop A
for (i = 1; i <= n; i++)
for (j = 1; j <= 10000; j++)
sum = sum + j;
// Loop B
for (i = 1; i <= n; i++)
for (j = 1; j <= n; j++)
sum = sum + j; Although loop A is O(n) and loop B is O(n2), loop B can be faster than loop A for small
values of n. Design and implement an experiment to ﬁnd the value of n for which loop B
is faster. The Efﬁciency of Implementations of the ADT List 3. Repeat Project 2, but use the following for loop B:
// Loop B
for (i = 1; i <= n; i++)
for (j = 1; j <= n; j++)
for (k = 1; k <= n; k++)
sum = sum + k; 221 ...
View
Full
Document
This note was uploaded on 04/29/2010 for the course CS 5503 taught by Professor Kaylor during the Spring '10 term at W. Alabama.
 Spring '10
 Kaylor
 Computer Science

Click to edit the document details