CS 310 Unit 1 Introduction to Algorithms

CS 310 Unit 1 Introduction to Algorithms - CS 310 Unit 1...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: CS 310 Unit 1 Introduction to Algorithms Furman Haddix Ph.D. Assistant Professor Minnesota State University, Mankato Spring 2008 Unit 1 Introduction to Algorithms Objectives • • • • • • Analysis of Algorithms, the Course What is an algorithm? Analysis of Algorithms, the Subject Why Algorithms? Analysis and Design of Algorithms Simple Sort Examples – Bubble Sort – Insertion Sort – Merge Sort – Recurrence Tree • Notation • Text, Chapters 1 and 2 Analysis of Algorithms, the Course • Evaluation – Homework – 30% – Exams – 70% • General Concept – Industrial-strength algorithms for industrial-strength applications – Implementing an algorithm correctly – Implementing the correct algorithm – Algorithm assumptions – Application stability • Instructor: Administrivia – Name • Furman Haddix – Office • WH 0227 – Phone • 389-1966 – e-mail • furman.haddix@mnsu.edu – Office Hours • WH 0227 – 10-11 MTH; 1-2 H; 2-3 MTH • WH 0283 – 1-2 MT • Text – Introduction to Algorithms, Cormen, Leiserson, Rivest, Stein What Is an Algorithm? • An algorithm is – A type of tool which should be in every computer scientist’s toolbox – A precise definition of a computational procedure – Started with a good problem definition • Algorithm Example: – Input – Process – Output Analysis of Algorithms, the Subject • Many algorithms and data structures already introduced in 110, 111, 210 • This course will cover these and others in several dimensions – Computability – The Big Oh – Real World Application – Don’t use bubble sort at work – Appropriateness of Assumptions – How to modify algorithm for a Real World Scenario – Using algorithms within a large scale application – Developing your own algorithms – Testing algorithms – Correctness of algorithms – Correctness of algorithm implementation – Appropriateness of algorithm for an application Why Study Algorithms? • Match problem solutions with resources available • Argument that current general purpose processors are fast enough with enough memory resources, that algorithm efficiency and resource utilization doesn’t matter • Doesn’t hold for special purpose processors • Doesn’t hold for large problems, e.g., sorting a million tuple database. • The increase in resource not consumed by Windows has been used to solve Algorithm for Solving a Programming Assignment 1. 2. 3. 4. 5. Read the problem Understand the problem Choose a design methodology Develop a design Validate the design against the problem statement 6. Choose an implementation methodology 7. Write a bit of code 8. Test the code 9. Repeat last two steps until each code unit is complete 10.Integrate the code units 11.Test the integration Concerns Addressed by Algorithms • • • • performance performance performance scalability – problem size – resource size • modularity • correctness • locality • cost of implementation – programmer time • • • • • • • maintainability functionality robustness user-friendliness simplicity extensibility reliability Analysis and Design of Algorithms The objectives of our study of algorithms • Analyze algorithms in terms of the concerns of the previous slide • Determine suitability of algorithms for implementation in specific contexts • Communicate regarding algorithmic implementations – Algorithmic mathematics • Implement algorithms to solve Expression of Algorithms • • • • • • • Narratives Inputs Outputs Initialization Steps, usually iterative or recursive Termination Pseudocode – Include declaration and description of use • • • • Mathematical formulae Model of computation Less detail than in a program May address issues not addressed in a program • • Example 1: Bubble Sort Don’t use at home without adult supervision! – Compare successive elements in a data structure. If true comparison, swap, otherwise compare next. – Repeat until all sorted. Example Sort Array in Increasing Order Bubble up Step [0] [1] [2] [3] [4] approach Initial 20 19 18 16 13 1 20 19 18 13 16 2 20 19 13 18 16 3 20 13 19 18 16 4 13 20 19 18 16 5 13 20 19 16 18 6 13 20 16 19 18 7 13 16 20 19 18 8 13 16 20 18 19 9 13 16 18 20 19 10 13 16 18 19 20 Notation -- Θ(n) Mathematical Description: • f(n) = Θ(g(n)) if there exist positive constants c1, c2, and n0, such that 0 ≤ c1 g(n) ≤ f(n) ≤ c2 g(n) for all n ≥ n0. Computer Science: • Drop low-order terms; ignore leading constants. • Example: 3n3 + 90n2–5n+ 6046 = Θ(n3) • Still required that – 0 ≤c1 n3 ≤ 3n3 + 90n2–5n+ 6046 ≤ c2 n3 for all n ≥ n0 – 0 ≤ 3n3 ≤ 3n3 + 90n2–5n+ 6046 ≤ 18n3 for all n ≥ 10 – Note that c1, c2, and n0, are not independent Graph of 3n3 + 90n2–5n+ 6046 vs. Θ(n3) Limits 70000 60000 50000 40000 3nnn 30000 T(n) 18nnn 20000 10000 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 • In this example, a larger c1 is not possible, because T(n) asymptotically approaches c1n3, as n increases. • Conversely, a smaller c2 is possible if a larger n0 is selected. Graph of 3n3 + 90n2–5n+ 6046 vs. Θ(n3) Limits 35000000 30000000 25000000 3nnn 20000000 T(n) 15000000 4nnn 18nnn 10000000 5000000 0 10 20 30 40 50 60 70 80 90 100 110 120 • Here, c2 can be 4, provided that n is greater than or equal to n0 = 91. Bubble Sort Pseudocode void bubble(int A, int n) { // array A has n elements, // indexed from 0 to n-1 for (i = 1; i < n; i++) // if n-1 elements sorted, the nth is as well for (j = n-1; j>= i; j--) // first pass guarantees element 0 sorted if(A[j] < A[j-1]) swap(A, j, j-1); // swap pseudo code not provided } • • • • • • • • Bubble Sort Θ swap = Θ(1) () inner loop is executed between 1 and n-1 times outer loop is executed n-1 times inner loop is executed from 1 to n-1 times, average times of ((n-1)+1)/2 bubble(n) operation count = T(n) T(n) = (n-1)((n-1+1)/2) Θ(1)+ Θ(1) T(n) = (n(n-1)/2+1)Θ(1) = ((n2-n)/2 +1)Θ(1) bubble(n) = Θ(n2) – c0 = .4 – c1 = ½ Graph of Bubble Sort T(n) vs. Θ(n) Limits 35 30 25 20 .4nn 15 T(n) .5nn 10 5 0 1 2 3 4 5 6 7 8 Insertion Sort • Another sort used primarily for instruction • Partition data set into two lists – sorted and unsorted. • Move elements from unsorted to sorted until unsorted is empty • Heuristic – Initially sorted list has one element, unsorted, n-1 – Take first element in unsorted list and insert into proper place in sorted list – Repeat until unsorted list is empty • Nice feature is that it executes in Θ(n) time if list sorted Insertion Sort Example Step [0] [1] [2] [3] [4] next Initial 20 19 18 16 13 1 20 19 18 16 13 19 2 19 20 18 16 13 19 3 19 20 18 16 13 18 4 19 20 20 16 13 18 5 18 19 20 16 13 18 6 18 19 20 16 13 16 7 18 19 20 20 13 16 8 18 19 19 20 13 16 9 16 18 19 20 13 16 10 16 18 19 20 13 13 11 16 18 19 20 20 13 12 16 18 19 19 20 13 13 16 18 18 19 20 13 14 13 16 19 19 20 13 Initial area of 1 is sorted by default. Since target At end (20) is of iteration, greater Since 19 next is is than next, greater moved to it is moved than indicated back. next, it is moved place, [0] in up in the this case. array. Then, the value in next is copied to [0]. Insertion Sort Pseudocode int A[n]; // indexed from 0 to n-1 int next; for (int i = 1; i < n; i++) { next = A[i]; j = i – 1; while (j >= 0 && A[j] > next) { A[j+1] = A[j]; j--; } A[j+1] = next; Insertion Sort Θ() • Outer loop has n-1 iterations. • Inner loop has from 1 to n-1 iterations • loop overhead, outer loop operations = Θ(1) • average number of time inner loop is executed per outer loop iterations is ((n-1)+1)/2 • insertion(n) operation count = T(n) • T(n) = (n-1)((n-1+1)/2) Θ(1)+ Θ(1) • insertion(n) = Θ(n2) Graph of Insertion Sort T(n) vs. Θ (n) Limits 35 30 25 20 .4nn 15 T(n) .5nn 10 5 0 1 2 3 4 5 6 7 8 Merging Two Sorted Lists Initial Step Step Step Step Step Step Step A B C A 1 C A 2 C A 3 C A 4 C A 5 C A 6 C A 7 C B B B B B B B 9 8 7 7 6 6 6 5 5 5 5 9 8 8 6 7 9 7 9 8 3 3 3 3 5 3 6 3 6 7 2 6 8 2 9 8 2 8 2 2 2 1 2 5 2 1 5 3 1 5 7 1 6 7 1 9 7 1 9 8 1 1 9 3 Merge Pseudocode int A[n]; // indexed from 0 to n-1 int nl, nh; // range of elements in a to be merged void merge(int A, int B, int nl, int nh) { int mid = (nl + nh + 1)/2; int cx = nl, dx = mid +1, bx = nl; for(bx = nl; bx <= nh; bx++) if(cx <= mid && (A[cx] < A[dx] || dx > nh)) { B[bx] = A[cx]; // write from lower end cx++; // if smaller or upper end empty } // end if else { B[bx] = A[dx]; dx++; } // end else for(bx = nl; bx <= nh; bx++) A(bx)= B(bx); // copy back to original array } // end merge Merge Θ() • Comparisons vary from the number of elements in the shorter list to n-1. If the two lists represent a subdivision of a longer division into two (nearly) equal parts, the lower limit would be , n/2 . cMINn ≤ TC (n) ≤ cMAXn. Both the minimum and maximum support TC (n) = Θ(n) • 2n moves, as every value is moved from A to B and back. Thus, TM(n) = cMn = Θ(n). Notation – x, x • Ceiling function – The smallest integer greater than {expression}. – expression – Example: 4.78 = 5 • Floor function – The largest integer less than {expression}. – expression – Example: 4.78 = 4 • Terminology – Most of the real numbers can be placed in rooms – The room contains all the real numbers between the floor and the ceiling. – However, for an integer x: x = x = x • For a non-integer real numbers y : y < x < x – The ceiling of one room equals the floor of the Floor, Ceiling, Logarithms Floor and Ceiling Function Implementations • C++ provides floor() and ceil() for floats, doubles, and long doubles in the <cmath> library. Requires –lm flag for compiler. • ints can be “casted” to floats. Logarithmic Notation • logb means logarithm base b, e.g., log10 is a decimal logarithm • log means logarithm any base we don’t care what base • lg means logarithm base 2 same as log2 • ln means natural logarithm same as loge Merge Sort • The idea of merge sort is to subdivide a list into two lists successively until all of the lists are of size 1. • Since the lists of size 1 are already sorted they can be successively merged back into one sorted list. • An alternative is to subdivide the lists into short lists, say 8, sort the short lists by a simple sort, and then Merge Sort Example Step [0] [1] [2] [3] [4] [5] Comment Initial 10 19 18 16 13 11 Subdivide into two lists 1 10 19 18 16 13 11 Recursively subdivide 2 10 19 18 16 13 11 Continue until all are size 1 3 10 19 18 16 13 11 Merge back 4 10 19 18 13 16 11 Merge back again 5 10 18 19 11 13 16 Merge back again 6 10 11 13 16 18 19 Sorted into Array B Merge Sort Pseudocode int A[n]; // indexed from 0 to n-1 int n; // number of elements in A void mergeSort(int A, int B, int nl, int nh) { int mid = (nl + nh + 1)/2; if(nl != mid) mergeSort(A, B, nl, mid); if(mid + 1 != nh) mergeSort(A, B, mid + 1, nh); merge(A, B, nl, nh); } Merge Sort Θ() • Since we are using recursive function calls, T(n) is measured by a recurrence relation. • The function T(n) for mergeSort() can be divided into four parts: – Calculation and computation: c – First recursive call: T( n/2 ) – Second recursive call: T( n/2 ) – Merge call: Θ(n) • Combining and simplifying, we have T(n) = 2T(n/2) + Θ(n) Solving a Recurrence Relation • To be soluble, a recurrence relation must have a base case. • Thus, Θ(1), if n=1 T(n) = T(n) = 2T(n/2) + Θ(n), if n>1 • We will study several methods for solving recurrence relations; one of the simpler methods is illustrated on the next slide. • Note that the base case is usually omitted from the statement of a recurrence relation. Using a Recursion Tree to Solve a Recurrence T Θ(n) (n) Θ(n) T(n) = 2T(n/2) + Θ(n) We start with T(n) as the root T(n/2 Θ T(n/2 Θ Θ(n) • Then we substitute our (n/2) ) (n/2) ) recurrence equation for each T term until we reach the base case. T(n/4 Θ T(n/4 Θ T(n/4 Θ T(n/4 This actually is a map of Θ Θ(n) (n/4) ) (n/4) ) (n/4) ) (n/4) our recursive calls. ) ht = ln n … … Summing by • • Level of Tree Θ(1) … n leaves … Θ(1) if n>1 Θ(n) Total = Θ(n ln n) Comparing Algorithms • Number of operations – – – – – – – Reads Writes Memory Operations Disk Operations Maximum Average Expected • Utilization of Memory • Delivery of first results • Time between results • Parallelizability • Implementation Issues Unit 1 Introduction to Algorithms Objectives • • • • • • Analysis of Algorithms, the Course What is an algorithm? Analysis of Algorithms, the Subject Why Algorithms? Analysis and Design of Algorithms Simple Sort Examples – Bubble Sort – Insertion Sort – Merge Sort – Recurrence Tree • Notation • Text, Chapters 1 and 2 ...
View Full Document

Ask a homework question - tutors are online