computational Sorting
Algorithm:
well-defined procedure that transforms input into output steps for the computer to follow to solve a problem
Text Chapters 1, 2
instance
Sorting Problem:
Input: A sequence of n numbers < a1 , a2 , L , an > Output: A permutation (reordering) < a ' , a ' , L , a ' > of 1 2 n the input sequence such that:
a '1
a '2
L
a 'n
Insertion Sort Animation
Finding a place for item with value 5 in position 1: Swap item in position 0 with item in position 1.
Insertion Sort Animation
Positions 0 through 1 are now in non-decreasing order.
http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html
http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html
Insertion Sort Animation
Finding a place for item with value 1 in position 2: Swap item in position 1 with item in position 2.
Insertion Sort Animation
Finding a place for item with value 1: Swap item in position 0 with item in position 1. Positions 0 through 2 are now in non-decreasing order.
http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html
http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html
Insertion Sort Animation
Finding a place for item with value 3 in position 3: Swap item in position 2 with item in position 3.
Insertion Sort Animation
Finding a place for item with value 3: Swap item in position 1 with item in position 2.
http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html
http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html
Insertion Sort Animation
Positions 0 through 3 are now in non-decreasing order.
Insertion Sort Animation
Finding a place for item with value 2 in position 4: Swap item in position 3 with item in position 4.
http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html
http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html
Insertion Sort Animation
Finding a place for item with value 2: Swap item in position 2 with item in position 3.
Insertion Sort Animation
Finding a place for item with value 2: Swap item in position 1 with item in position 2.
http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html
http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html
Insertion Sort Animation
Positions 0 through 4 are now in non-decreasing order.
Insertion Sort Animation
Finding a place for item with value 6 in position 5: Swap item in position 4 with item in position 5.
http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html
http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html
Insertion Sort Animation
Positions 0 through 5 are now in non-decreasing order.
Insertion Sort Animation
Finding a place for item with value 4 in position 6: Swap item in position 5 with item in position 6.
http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html
http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html
Insertion Sort Animation
Finding a place for item with value 4: Swap item in position 4 with item in position 5.
Insertion Sort Animation
Positions 0 through 6 are now in non-decreasing order.
http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html
http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html
Insertion Sort Animation
Finding a place for item with value 7 in position 7: Swap item in position 6 with item in position 7.
Insertion Sort Animation
Positions 0 through 7 are now in non-decreasing order.
http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html
http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html
Insertion Sort Animation
Positions 0 through 7 are now in non-decreasing order.
Insertion Sort Animation
Positions 0 through 7 are now in non-decreasing order.
http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html
http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html
Asymptotic Analysis
Asymptotic Notation
O(g(n)) is a set of functions, so we often say f(n) is in O(g(n)).
Asymptotic Notation (cont.)
1
Function Order of Growth
lg(lg(n)) lg(n) lg(lg(n)) lg(n)
n
n lg(n) lg(n)
n lg2(n)
n2
n5
2n
know how to order functions asymptotically (behavior as n becomes large)
O( ) upper bound ( ) lower bound ( ) upper & lower bound
know how to use asymptotic complexity notation to describe time or space complexity
ANALYSIS
OF
ALGORITHMS
Average Case vs. Worst Case Running Time of an Algorithm
An algorithm may run faster on certain data sets than on others, Finding the average case can be very difficult, so typically algorithms are measured by the worst-case time complexity. In certain application domains (e.g., air traffic control, surgery) knowing the worst-case time complexity is of crucial importance.
Mathematical Review Running Time Pseudo-Code Analysis of Algorithms Asymptotic Notation Asymptotic Analysis
Quick
Measuring the Running Time
How should we measure the running time of an algorithm? Experimental Study Write a program that implements the algorithm Run the program with data sets of varying size and composition. Use a method like System.currentTimeMillis() to get an accurate measure of the actual running time. The resulting data set should look something like:
Beyond Experimental Studies Experimental studies have several limitations:
- It is necessary to implement and test the algorithm in order to determine its running time. - Experiments can be done only on a limited set of inputs, and may not be indicative of the running time on other inputs not included in the experiment. - In order to compare two algorithms, the same hardware and software environments should be used. We will now develop a general methodology for analyzing the running time of algorithms that - Uses a high-level description of the algorithm instead of testing one of its implementations. - Takes into account all possible inputs. - Allows one to evaluate the efficiency of any algorithm in a way that is independent from the hardware and software environment.
Pseudo-Code
Pseudo-code is a description of an algorithm. It is more structured than usual prose but less formal than a programming language. Example: finding the maximum element of an array. Algorithm arrayMax(A, n): Input: An array A storing n integers. Output: The maximum element in A. A[0] currentMax for i 1 to n -1 do if currentMax < A[i] then currentMax A[i] return currentMax Pseudo-code is our preferred notation for describing algorithms. However, pseudo-code hides program design issues.
What is Pseudo-Code?
A mixture of natural language and high-level programming concepts that describes the main ideas behind a generic implementation of a data structure or algorithm. - Expressions: use standard mathematical symbols to describe numeric and boolean expressions - use for assignment - - use = for the equality relationship - Method Declarations: - Algorithm name(param1, param2)
What is Pseudo-Code?
A Quick Math Review (cont.)
- Programming Constructs: - decision structures: if ... then ... [else ... ] - while-loops: while ... do - repeat-loops: repeat ... until ... - for-loop: for ... do - array indexing: A[i] - Methods: - calls: object method(args) - returns: return value
Analysis of Algorithms
Analysis of Algorithms
Primitive Operations: Low-level computations that are largely independent from the programming language and can be identified in pseudocode, e.g: - calling a method and returning from a method - performing an arithmetic operation (e.g. addition) - comparing two numbers, etc. By inspecting the pseudo-code, we can count the number of primitive operations executed by an algorithm.
Example: Algorithm arrayMax(A, n): Input: An array A storing n integers. Output: The maximum element in A. A[0] currentMax for i 1 to n -1 do if currentMax < A[i] then currentMax A[i]
Asymptotic Analysis of Running Time
Simple Justification Techniques
By Example - Find an example - Find a counter example The "Contra" Attack - Find a contradiction in the negative statement - Contrapositive Induction and Loop-Invariants - Induction - 1) Prove the base case - 2) Prove that any case n implies the next case (n + 1) is also true - Loop invariants - Prove initial claim S 0 - Show that S i-1 implies S i will be true after iteration i
Use the Big-Oh notation to express the number of primitive operations executed as a function of the input size. For example, we say that the arrayMax algorithm runs in O(n) time. Comparing the asymptotic running time - an algorithm that runs in O(n) time is better than one that runs in O(n 2) time - similarly, O(log n) is better than O( n) - hierarchy of functions: log n << n << n 2 << n 3 << 2 n
Caution! - Beware of very large constant factors. An algorithm running in time 1,000,000 n is still O(n) but might be less efficient on your data set than one running in time 2n 2 , which is O(n 2 )
Advanced Topics: Other Justification Techniques
Proof by Excessive Waving of Hands Proof by Incomprehensible Diagram Proof by Very Large Bribes See me after class Proof by Violent Metaphor Don't argue with anyone who always assumes a sequence consists of hand grenades The Emperor's New Clothes Method "This proof is so obvious only an idiot wouldn't understand it."
A Quick Math Review (cont.)
Types of Algorithmic Input
Best-Case Input: of all possible algorithm inputs of size n, it generates the "best" result
for Time Complexity: "best" is smallest running time Best-Case Input Produces Best-Case Running Time
& so it provides a lower bound on the algorithm's asymptotic running time -- (subject to any implementation assumptions) for Space Complexity: "best" is smallest storage
Average-Case Input Worst-Case Input
these are defined similarly
Best-Case Time < Average-Case Time < Worst-Case Time
Bounding Algorithmic Time
(using cases) Using "case" we can discuss lower and/or upper bounds on: case"
best-case running time or average-case running time or worst-case running time
Bounding Algorithmic Time
(tightening bounds)
for example... 1 lglg(n) ...