9
Searching 1 and Sorting
1
2008 Pearson Education, Inc. All rights reserved.
2
With sobs and tears he sorted out Those of the largest size... Attempt the end, and never stand to doubt; Nothing's so hard, but search will find it out.
-- Robert Herrick -- Lewis Carroll
`Tis in my memory lock'd, And you yourself shall keep the key of it.
-- William Shakespeare
It is an immutable law in business that words are words, explanations are explanations, promises are promises -- but only performance is reality
-- Harold S. Green
2008 Pearson Education, Inc. All rights reserved.
3
OBJECTIVES
In this chapter you'll learn: To search for a given value in a vector using binary search. To use Big O notation to express the efficiency of an algorithm and to compare the performance of algorithms. To review the efficiency of the selection sort and insertion sort algorithms. To sort a vector using the recursive merge sort algorithm. To determine the efficiency of various searching and sorting algorithms. To enumerate the searching and sorting algorithms discussed in this text. To understand the nature of algorithms of constant, linear and quadratic runtime.
2008 Pearson Education, Inc. All rights reserved.
4
19.1 19.2
19.3
19.4
Introduction Searching Algorithms 19.2.1 Efficiency of Linear Search 19.2.2 Binary Search Sorting Algorithms 19.3.1 Efficiency of Selection Sort 19.3.2 Efficiency of Insertion Sort 19.3.3 Merge Sort (A Recursive Implementation) Wrap-Up
2008 Pearson Education, Inc. All rights reserved.
5
19.1 Introduction
Searching data
Determine whether a value (the search key) is present in the data Algorithms
Linear search Simple Slow for large sets of data Binary search Fast, even for large sets of data More complex than linear search
2008 Pearson Education, Inc. All rights reserved.
6
19.1 Introduction (Cont.)
Sorting data
Place data in order
Typically ascending or descending Based on one or more sort keys
Algorithms
Insertion sort Selection sort Merge sort More efficient, but more complex
2008 Pearson Education, Inc. All rights reserved.
7
19.1 Introduction (Cont.)
Big O notation
Estimates worst-case runtime for an algorithm
How hard an algorithm must work to solve a problem
2008 Pearson Education, Inc. All rights reserved.
8
Chapter
Algorithm
Location
Searching Algorithms 7 20 Linear search Binary search Recursive linear search Recursive binary search 20 22 Binary tree search Linear search of a linked list binary_search standard library function Section 7.7 Section 19.2.2 Exercise 19.8 Exercise 19.9 Section 20.7 Exercise 20.21 Section 22.5.6
Fig. 19.1 | Searching and sorting algorithms in this text. (Part 1 of 2)
2008 Pearson Education, Inc. All rights reserved.
9
Chapter
Algorithm
Location
Sorting Algorithms 7 8 19 Insertion sort Selection sort Recursive merge sort Bubble sort Bucket sort Recursive quicksort 20 22 Binary tree sort sort standard library function Heap sort Section 7.8 Section 8.6 Section 19.3.3 Exercises 19.5 and 19.6 Exercise 19.7 Exercise 19.10 Section 20.7 Section 22.5.6 Section 22.5.12
Fig. 19.1 | Searching and sorting algorithms in this text. (Part 2 of 2)
2008 Pearson Education, Inc. All rights reserved.
10
19.2 Searching Algorithms
Searching algorithms
Find element that matches a given search key Major difference between search algorithms
Amount of effort they require to complete search Particularly dependent on number of data elements Can be described with Big O notation
2008 Pearson Education, Inc. All rights reserved.
11
19.2.1 Efficiency of Linear Search
Big O notation
Measures runtime growth of an algorithm relative to number of items processed
Highlights dominant terms Ignores terms that become unimportant as n grows Ignores constant factors
2008 Pearson Education, Inc. All rights reserved.
12
19.2.1 Efficiency of Linear Search (Cont.)
Big O notation (Cont.)
Constant runtime
Number of operations performed by algorithm is constant Does not grow as number of items increases Represented in Big O notation as O(1) Pronounced "on the order of 1" or "order 1" Example Test if the first element of an n-element vector is equal to the second element Always takes one comparison, no matter how large the vector
2008 Pearson Education, Inc. All rights reserved.
13
19.2.1 Efficiency of Linear Search (Cont.)
Big O notation (Cont.)
Linear runtime
Number of operations performed by algorithm grows linearly with number of items Represented in Big O notation as O(n) Pronounced "on the order of n" or "order n" Example Test if the first element of an n-vector is equal to any other element Takes n - 1 comparisons n term dominates, -1 is ignored
2008 Pearson Education, Inc. All rights reserved.
14
19.2.1 Efficiency of Linear Search (Cont.)
Big O notation (Cont.)
Quadratic runtime
Number of operations performed by algorithm grows as the square of the number of items Represented in Big O notation as O(n2) Pronounced "on the order of n2" or "order n2" Example Test if any element of an n-vector is equal to any other element Takes n2/2 n/2 comparisons n2 term dominates, constant 1/2 is ignored, -n/2 is ignored
2008 Pearson Education, Inc. All rights reserved.
15
19.2.1 Efficiency of Linear Search (Cont.)
Efficiency of linear search
Linear search runs in O(n) time
Worst case: every element must be checked If size of the vector doubles, number of comparisons also doubles
2008 Pearson Education, Inc. All rights reserved.
16
Performance Tip 19.1
Sometimes the simplest algorithms perform poorly. Their virtue is that they are easy to program, test and debug. Sometimes more complex algorithms are required to realize maximum performance.
2008 Pearson Education, Inc. All rights reserved.
17
19.2.2 Binary Search
Binary search algorithm
Requires that the vector first be sorted
Can be performed by Standard Library function sor t Takes two random-access iterators Sorts elements in ascending order
First iteration (assuming sorted ascending order)
Test the middle element in the vector If it matches search key, the algorithm ends If it is greater than search key, continue with only the first half of the vector If it is less than search key, continue with only the second half of the vector
2008 Pearson Education, Inc. All rights reserved.
18
19.2.2 Binary Search (Cont.)
Binary search algorithm (Cont.)
Subsequent iterations
Test the middle element in the remaining subvector If it matches search key, the algorithm ends If not, eliminate half of the subvector and continue
Terminates when
Element matching search key is found Current subvector is reduced to zero size Can conclude that search key is not in vector
2008 Pearson Education, Inc. All rights reserved.
1 2 3 4 5 6 7 8 9
// Fig 19.2: BinarySearch.h // Class that contains a vector of random integers and a function // that uses binary search to find an integer. #include <vector> using std::vector;
Outline
19
BinarySearch.h
class BinarySearch { public:
(1 of 1)
10 BinarySearch( int ); // constructor initializes vector 11 int binarySearch( int ) const; // perform a binary search on vector 12 void displayElements() const; // display vector elements 13 private: 14 int size; // vector size 15 vector< int > data; // vector of ints 16 void displaySubElements( int, int ) const; // display range of values 17 }; // end class BinarySearch
2008 Pearson Education, Inc. All rights reserved.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
// Fig 19.3: BinarySearch.cpp // BinarySearch class memberfunction definition. #include <iostream> using std::cout; using std::endl; #include <cstdlib> // prototypes for functions srand and rand using std::rand; using std::srand; #include <ctime> // prototype for function time using std::time; #include <algorithm> // prototype for sort function #include "BinarySearch.h" // class BinarySearch definition // constructor initializes vector with random ints and sorts the vector BinarySearch::BinarySearch( int vectorSize ) { size = ( vectorSize > 0 ? vectorSize : 10 ); // validate vectorSize srand( time( 0 ) ); // seed using current time // fill vector with random ints in range 1099 for ( int i = 0; i < size; i++ ) data.push_back( 10 + rand() % 90 ); // 1099 std::sort( data.begin(), data.end() ); // sort the data } // end BinarySearch constructor
Outline
20
BinarySearch.cp p
(1 of 3)
Initialize the vector with random ints from 10-99
Sort the elements in vector data in ascending order 2008 Pearson Education,
Inc. All rights reserved.
29 30 // perform a binary search on the data 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 int BinarySearch::binarySearch( int searchElement ) const { int low = 0; // low end of the search area int high = size 1; // high end of the search area int middle = ( low + high + 1 ) / 2; // middle element int location = 1; // return value; 1 if not found do // loop to search for element { // print remaining elements of vector to be searched displaySubElements( low, high ); // output spaces for alignment for ( int i = 0; I < middle; i++ ) cout << " ";
Outline
21
Calculate the low end index, high end index and middle index of the BinarySearch.cp portion of the vector being searched p (2 of 3) Initialize the location of the found element to -1, indicating that the search key has not (yet) been found
Test if the middle element is cout << " * " << endl; // indicate current middle equal to searchElement
// if the element is found at the middle if ( searchElement == data[ middle ] )
51 location = middle; // location is the current middle 52 else if ( searchElement < data[ middle ] ) // middle is too high 53 54 55 56 57 58 high = middle 1; // eliminate the higher half else // middle element is too low Eliminate low = middle + 1; // eliminate the lower half middle = ( low + high + 1 ) / 2; // recalculate the middle } while ( ( low <= high ) && ( location == 1 ) );
half of the remaining values in the vector
2008 Pearson Education, located Inc. All rights reserved.
Loop until the subvector is of zero size or the search key is
59 60 return location; // return location of search key 61 } // end function binarySearch 62 63 // display values in vector 64 void BinarySearch::displayElements() const 65 { 66 displaySubElements( 0, size 1 ); 67 } // end function displayElements 68 69 // display certain values in vector 70 void BinarySearch::displaySubElements( int low, int high ) const 71 { 72 for ( int i = 0; i < low; i++ ) // output spaces for alignment 73 cout << " "; 74 75 for ( int i = low; i <= high; i++ ) // output elements left in vector 76 cout << data[ i ] << " "; 77 78 cout << endl; 79 } // end function displaySubElements
Outline
22
BinarySearch.cp p
(3 of 3)
2008 Pearson Education, Inc. All rights reserved.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
// Fig 19.4: Fig19_04.cpp // BinarySearch test program. #include <iostream> using std::cin; using std::cout; using std::endl; #include "BinarySearch.h" // class BinarySearch definition int main() { int searchInt; // search key int position; // location of search key in vector // create vector and output it BinarySearch searchVector ( 15 ); searchVector.displayElements(); // get input from user cout << "\nPlease enter an integer value (1 to quit): "; cin >> searchInt; // read an int from user cout << endl; // repeatedly input an integer; 1 terminates the program while ( searchInt != 1 ) { // use binary search to try to find integer position = searchVector.binarySearch( searchInt );
Outline
Fig20_04.cpp
23
(1 of 3)
Perform a binary search on the data
2008 Pearson Education, Inc. All rights reserved.
30 // return value of 1 indicates integer was not found 31 if ( position == 1 ) 32 cout << "The integer " << searchInt << " was not found.\n"; 33 else 34 cout << "The integer " << searchInt 35 << " was found in position " << position << ".\n"; 36 37 // get input from user 38 cout << "\n\nPlease enter an integer value (1 to quit): "; 39 cin >> searchInt; // read an int from user 40 cout << endl; 41 } // end while 42 43 return 0; 44 } // end main 26 31 33 38 47 49 49 67 73 74 82 89 90 91 95 Please enter an integer value (1 to quit): 38 26 31 33 38 47 49 49 67 73 74 82 89 90 91 95 * 26 31 33 38 47 49 49 * 26 31 33 38 47 49 49 67 73 74 82 89 90 91 95
Outline
Fig19_04.cpp
24
(2 of 3)
(Continued at top of next slide... )
2008 Pearson Education, Inc. All rights reserved.
(...continued from bottom of previous slide ) Please enter an integer value (1 to quit): 38 26 31 33 38 47 49 49 67 73 74 82 89 90 91 95 * 26 31 33 38 47 49 49 * The integer 38 was found in position 3. Please enter an integer value (1 to quit): 91 26 31 33 38 47 49 49 67 73 74 82 89 90 91 95 * 73 74 82 89 90 91 95 * 90 91 95 * The integer 91 was found in position 13. Please enter an integer value (1 to quit): 25 26 31 33 38 47 49 49 67 73 74 82 89 90 91 95 * 26 31 33 38 47 49 49 * 26 31 33 * 26 * The integer 25 was not found. Please enter an integer value (1 to quit): 1
Outline
Fig19_04.cpp
25
(3 of 3)
2008 Pearson Education, Inc. All rights reserved.
26
19.2.2 Binary Search (Cont.)
Efficiency of binary search
Logarithmic runtime
Number of operations performed by algorithm grows logarithmically as number of items increases Represented in Big O notation as O(log n) Pronounced "on the order of log n" or "order log n" Example Binary searching a sorted vector of 1023 elements takes at most 10 comparisons ( 10 = 2 log ( 1023 + 1 ) ) Repeatedly dividing 1023 by 2 and rounding down results in 0 after 10 iterations
2008 Pearson Education, Inc. All rights reserved.
27
19.3 Sorting Algorithms
Sorting algorithms
Placing data into some particular order
Such as ascending or descending
One of the most important computing applications End result, a sorted vector, will be the same no matter which algorithm is used
Choice of algorithm affects only runtime and memory use
2008 Pearson Education, Inc. All rights reserved.
28
19.3.1 Efficiency of Selection Sort
Selection sort
At ith iteration
Swaps the ith smallest element with element i
After ith iteration
Smallest i elements are sorted in increasing order in first i positions
Requires a total of (n2 n)/2 comparisons
Iterates n - 1 times In ith iteration, locating ith smallest element requires n i comparisons Has Big O of O(n2)
2008 Pearson Education, Inc. All rights reserved.
29
19.3.2 Efficiency of Insertion Sort
Insertion sort
At ith iteration
Insert (i + 1)th element into correct position with respect to first i elements
After ith iteration
First i elements are sorted
Requires a worst-case of n2 inner-loop iterations
Outer loop iterates n - 1 times Inner loop requires n 1iterations in worst case For determining Big O, nested statements mean multiply the number of iterations Has Big O of O(n2)
2008 Pearson Education, Inc. All rights reserved.
19.3.3 Merge Sort (A Recursive Implementation)
Merge sort
Sorts vector by
Splitting it into two equal-size subvectors If vector size is odd, one subvector will be one element larger than the other Sorting each subvector Merging them into one larger, sorted vector Repeatedly compare smallest elements in the two subvectors The smaller element is removed and placed into the larger, combined vector
30
2008 Pearson Education, Inc. All rights reserved.
19.3.3 Merge Sort (A Recursive Implementation) (Cont.)
Merge sort (Cont.)
Our recursive implementation
Base case A vector with one element is already sorted Recursion step Split the vector (of 2 elements) into two equal halves If vector size is odd, one subvector will be one element larger than the other Recursively sort each subvector Merge them into one larger, sorted vector
31
2008 Pearson Education, Inc. All rights reserved.
19.3.3 Merge Sort (A Recursive Implementation) (Cont.)
Merge sort (Cont.)
Sample merging step
Smaller, sorted vectors
32
A: 4 1 0 34 56 77 B: 5 30 5 1 52 93
Compare smallest element in A to smallest element in B
4 (A) is less than 5 (B)
4 becomes first element in merged vector
5 (B) is less than 1 (A) 0
5 becomes second element in merged vector
1 (A) is less than 30 (B) 0
1 becomes third element in merged vector 0 2008 Pearson Education, Inc.
All rights reserved.
1 2 3 4 5 6 7 8 9
// Fig 19.5: MergeSort.h // Class that creates a vector filled with random integers. // Provides a function to sort the vector with merge sort. #include <vector> using std::vector;
Outline
MergeSort.h
33
// MergeSort class definition class MergeSort {
(1 of 1)
10 public: 11 MergeSort( int ); // constructor initializes vector 12 void sort(); // sort vector using merge sort 13 void displayElements() const; // display vector elements 14 private: 15 int size; // vector size 16 vector< int > data; // vector of ints 17 void sortSubVector( int, int ); // sort subvector 18 void merge( int, int, int, int ); // merge two sorted vectors 19 void displaySubVector( int, int ) const; // display subvector 20 }; // end class SelectionSort
2008 Pearson Education, Inc. All rights reserved.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
// Fig 19.6: MergeSort.cpp // Class MergeSort memberfunction definition. #include <iostream> using std::cout; using std::endl; #include <vector> using std::vector; #include <cstdlib> // prototypes for functions srand and rand using std::rand; using std::srand; #include <ctime> // prototype for function time using std::time; #include "MergeSort.h" // class MergeSort definition // constructor fill vector with random integers MergeSort::MergeSort( int vectorSize ) { size = ( vectorSize > 0 ? vectorSize : 10 ); // validate vectorSize srand( time( 0 ) ); // seed random number generator using current time // fill vector with random ints in range 1099 for ( int i = 0; i < size; i++ ) data.push_back( 10 + rand() % 90 ); } // end MergeSort constructor
Outline
MergeSort.cpp
34
(1 of 5)
2008 Pearson Education, Inc. All rights reserved.
29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57
// split vector, sort subvectors and merge subvectors into sorted vector void MergeSort::sort() { sortSubVector( 0, size 1 ); // recursively sort entire vector } // end function sort
Outline
35
// recursive function to sort subvectors void MergeSort::sortSubVector( int low, int high ) { // test base case; size of vector equals 1 Test the base case if ( ( high low ) >= 1 ) // if not base case { int middle1 = ( low + high ) / 2; // calculate middle of vector int middle2 = middle1 + 1; // calculate next element over // output split step cout << "split: "; Split the vector in two displaySubVector( low, high ); cout << endl << " "; displaySubVector( low, middle1 ); cout << endl << " "; displaySubVector( middle2, high ); Recursively call function cout << endl << endl; sortSubVector on the two subvectors // split vector in half; sort each half (recursive calls) sortSubVector( low, middle1 ); // first half of vector sortSubVector( middle2, high ); // second half of vector
MergeSort.cpp Call function sortSubVector with 0 and size5) 1 as the (2 of beginning and ending indices
2008 Pearson Education, Inc. All rights reserved.
58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87
// merge two sorted vectors after split calls return merge( low, middle1, middle2, high ); } // end if Combine } // end function sortSubVector
the two sorted vectors into one larger, sorted vector
Outline
MergeSort.cpp
36
// merge two sorted subvectors into one sorted subvector void MergeSort::merge( int left, int middle1, int middle2, int right ) { int leftIndex = left; // index into left subvector int rightIndex = middle2; // index into right subvector int combinedIndex = left; // index into temporary working vector vector< int > combined( size ); // working vector // output two subvectors before merging cout << "merge: "; displaySubVector( left, middle1 ); cout << endl << " "; displaySubVector( middle2, right ); cout << endl; // merge vectors until reaching end of either while ( leftIndex <= middle1 && rightIndex <= right ) { // place smaller of two current elements into result // and move to next space in vector if ( data[ leftIndex ] <= data[ rightIndex ] ) combined[ combinedIndex++ ] = data[ leftIndex++ ]; else combined[ combinedIndex++ ] = data[ rightIndex++ ]; } // end while
(3 of 5)
Loop until the end of either subvector is reached
Test which element at the beginning of the vectors is smaller Place the smaller element in the combined vector
2008 Pearson Education, Inc. All rights reserved.
88 37 89 if ( leftIndex == middle2 ) // if at end of left vector 90 { 91 while ( rightIndex <= right ) // copy in rest of right vector Fill the combined vector with 92 combined[ combinedIndex++ ] = data[ rightIndex++ ];...