Unformatted text preview: CS 310 Unit 1
Introduction to Algorithms
Furman Haddix Ph.D.
Assistant Professor
Minnesota State University,
Mankato
Spring 2008 Unit 1 Introduction to Algorithms
Objectives
•
•
•
•
•
• Analysis of Algorithms, the Course
What is an algorithm?
Analysis of Algorithms, the Subject
Why Algorithms?
Analysis and Design of Algorithms
Simple Sort Examples
– Bubble Sort
– Insertion Sort
– Merge Sort
– Recurrence Tree
• Notation
• Text, Chapters 1 and 2 Analysis of Algorithms, the
Course • Evaluation – Homework – 30%
– Exams – 70% • General Concept
– Industrialstrength algorithms for
industrialstrength applications
– Implementing an algorithm correctly
– Implementing the correct algorithm
– Algorithm assumptions
– Application stability • Instructor:
Administrivia
– Name
• Furman Haddix
– Office
• WH 0227
– Phone
• 3891966
– email
• [email protected]
– Office Hours
• WH 0227
– 1011 MTH; 12 H; 23 MTH
• WH 0283
– 12 MT
• Text
– Introduction to Algorithms, Cormen, Leiserson,
Rivest, Stein What Is an Algorithm?
• An algorithm is
– A type of tool which should be in every
computer scientist’s toolbox
– A precise definition of a computational
procedure
– Started with a good problem definition • Algorithm Example:
– Input
– Process
– Output Analysis of Algorithms, the
Subject
• Many algorithms and data structures
already introduced in 110, 111, 210
• This course will cover these and others in
several dimensions – Computability – The Big Oh
– Real World Application – Don’t use bubble sort
at work
– Appropriateness of Assumptions – How to
modify algorithm for a Real World Scenario
– Using algorithms within a large scale
application
– Developing your own algorithms
– Testing algorithms
– Correctness of algorithms
– Correctness of algorithm implementation
– Appropriateness of algorithm for an application Why Study Algorithms?
• Match problem solutions with resources
available
• Argument that current general purpose
processors are fast enough with enough
memory resources, that algorithm
efficiency and resource utilization
doesn’t matter
• Doesn’t hold for special purpose
processors
• Doesn’t hold for large problems, e.g.,
sorting a million tuple database.
• The increase in resource not consumed
by Windows has been used to solve Algorithm for Solving a Programming
Assignment 1.
2.
3.
4.
5. Read the problem
Understand the problem
Choose a design methodology
Develop a design
Validate the design against the problem
statement
6. Choose an implementation methodology
7. Write a bit of code
8. Test the code
9. Repeat last two steps until each code unit
is complete
10.Integrate the code units
11.Test the integration Concerns Addressed by
Algorithms
•
•
•
• performance
performance
performance
scalability
– problem size
– resource size • modularity
• correctness
• locality • cost of
implementation
– programmer time •
•
•
•
•
•
• maintainability
functionality
robustness
userfriendliness
simplicity
extensibility
reliability Analysis and Design of
Algorithms
The objectives of our study of
algorithms
• Analyze algorithms in terms of the
concerns of the previous slide
• Determine suitability of algorithms
for implementation in specific
contexts
• Communicate regarding algorithmic
implementations
– Algorithmic mathematics • Implement algorithms to solve Expression of Algorithms
•
•
•
•
•
•
• Narratives
Inputs
Outputs
Initialization
Steps, usually iterative or recursive
Termination
Pseudocode – Include declaration and description of use •
•
•
• Mathematical formulae
Model of computation
Less detail than in a program
May address issues not addressed in a
program • • Example 1: Bubble Sort Don’t use at home without adult supervision!
– Compare successive elements in a data structure. If true
comparison, swap, otherwise compare next.
– Repeat until all sorted.
Example Sort Array in Increasing Order Bubble up
Step
[0]
[1]
[2]
[3]
[4]
approach
Initial 20 19 18 16 13 1 20 19 18 13 16 2 20 19 13 18 16 3 20 13 19 18 16 4 13 20 19 18 16 5 13 20 19 16 18 6 13 20 16 19 18 7 13 16 20 19 18 8 13 16 20 18 19 9 13 16 18 20 19 10 13 16 18 19 20 Notation  Θ(n)
Mathematical Description:
• f(n) = Θ(g(n)) if there exist positive constants c1, c2, and
n0, such that 0 ≤ c1 g(n) ≤ f(n) ≤ c2 g(n) for all n ≥ n0.
Computer Science:
• Drop loworder terms; ignore leading constants.
• Example: 3n3 + 90n2–5n+ 6046 = Θ(n3)
• Still required that
– 0 ≤c1 n3 ≤ 3n3 + 90n2–5n+ 6046 ≤ c2 n3 for all n ≥ n0
– 0 ≤ 3n3 ≤ 3n3 + 90n2–5n+ 6046 ≤ 18n3 for all n ≥ 10 – Note that c1, c2, and n0, are not independent Graph of 3n3 + 90n2–5n+ 6046 vs. Θ(n3)
Limits
70000
60000
50000
40000 3nnn 30000 T(n)
18nnn 20000
10000
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 • In this example, a larger c1 is not possible, because T(n) asymptotically
approaches c1n3, as n increases.
• Conversely, a smaller c2 is possible if a larger n0 is selected. Graph of 3n3 + 90n2–5n+ 6046 vs. Θ(n3)
Limits
35000000
30000000
25000000
3nnn 20000000 T(n) 15000000 4nnn
18nnn 10000000
5000000
0
10 20 30 40 50 60 70 80 90 100 110 120
• Here, c2 can be 4, provided that n is greater than or equal to n0 = 91. Bubble Sort Pseudocode
void bubble(int A, int n) {
// array A has n elements,
//
indexed from 0 to n1
for (i = 1; i < n; i++)
// if n1 elements sorted, the nth is as well
for (j = n1; j>= i; j)
// first pass guarantees element 0 sorted
if(A[j] < A[j1])
swap(A, j, j1);
// swap pseudo code not provided
} •
•
•
•
•
•
•
• Bubble Sort Θ
swap = Θ(1)
()
inner loop is executed between 1 and
n1 times
outer loop is executed n1 times
inner loop is executed from 1 to n1
times, average times of ((n1)+1)/2
bubble(n) operation count = T(n)
T(n) = (n1)((n1+1)/2) Θ(1)+ Θ(1)
T(n) = (n(n1)/2+1)Θ(1) = ((n2n)/2
+1)Θ(1)
bubble(n) = Θ(n2)
– c0 = .4
– c1 = ½ Graph of Bubble Sort T(n) vs. Θ(n)
Limits
35
30
25
20 .4nn 15 T(n)
.5nn 10
5
0
1 2 3 4 5 6 7 8 Insertion Sort
• Another sort used primarily for instruction
• Partition data set into two lists – sorted and
unsorted.
• Move elements from unsorted to sorted until
unsorted is empty
• Heuristic
– Initially sorted list has one element, unsorted,
n1
– Take first element in unsorted list and insert
into proper place in sorted list
– Repeat until unsorted list is empty
• Nice feature is that it executes in Θ(n) time if list
sorted Insertion Sort Example
Step [0] [1] [2] [3] [4] next Initial 20 19 18 16 13 1 20 19 18 16 13 19 2 19 20 18 16 13 19 3 19 20 18 16 13 18 4 19 20 20 16 13 18 5 18 19 20 16 13 18 6 18 19 20 16 13 16 7 18 19 20 20 13 16 8 18 19 19 20 13 16 9 16 18 19 20 13 16 10 16 18 19 20 13 13 11 16 18 19 20 20 13 12 16 18 19 19 20 13 13 16 18 18 19 20 13 14 13 16 19 19 20 13 Initial area
of 1 is
sorted by
default.
Since target
At end
(20) is of
iteration,
greater
Since 19
next is is
than next,
greater
moved to
it is moved
than
indicated
back. next,
it is moved
place, [0] in
up in the
this case.
array. Then,
the value in
next is
copied to
[0]. Insertion Sort Pseudocode
int A[n]; // indexed from 0 to n1
int next;
for (int i = 1; i < n; i++) {
next = A[i];
j = i – 1;
while (j >= 0 && A[j] > next) {
A[j+1] = A[j];
j;
}
A[j+1] = next; Insertion Sort Θ()
• Outer loop has n1 iterations.
• Inner loop has from 1 to n1
iterations
• loop overhead, outer loop operations
= Θ(1)
• average number of time inner loop is
executed per outer loop iterations is
((n1)+1)/2
• insertion(n) operation count = T(n)
• T(n) = (n1)((n1+1)/2) Θ(1)+ Θ(1)
• insertion(n) = Θ(n2) Graph of Insertion Sort T(n) vs. Θ
(n) Limits
35
30
25
20 .4nn 15 T(n)
.5nn 10
5
0
1 2 3 4 5 6 7 8 Merging Two Sorted Lists
Initial Step Step Step Step Step Step Step
A B C A 1 C A 2 C A 3 C A 4 C A 5 C A 6 C A 7 C
B
B
B
B
B
B
B
9
8
7 7 6 6 6 5 5 5 5 9 8 8 6 7 9 7 9 8 3 3 3 3 5 3 6 3 6 7 2 6 8 2 9 8 2 8 2 2 2 1 2 5 2 1 5 3 1 5 7 1 6 7 1 9 7 1 9 8 1 1 9 3 Merge Pseudocode
int A[n]; // indexed from 0 to n1
int nl, nh; // range of elements in a to be merged
void merge(int A, int B, int nl, int nh) {
int mid = (nl + nh + 1)/2;
int cx = nl, dx = mid +1, bx = nl;
for(bx = nl; bx <= nh; bx++)
if(cx <= mid && (A[cx] < A[dx]  dx > nh)) {
B[bx] = A[cx]; // write from lower end
cx++;
// if smaller or upper end empty
} // end if
else {
B[bx] = A[dx];
dx++;
} // end else
for(bx = nl; bx <= nh; bx++)
A(bx)= B(bx); // copy back to original array
} // end merge Merge Θ()
• Comparisons vary from the number
of elements in the shorter list to n1.
If the two lists represent a
subdivision of a longer division into
two (nearly) equal parts, the lower
limit would be , n/2 . cMINn ≤ TC (n) ≤
cMAXn. Both the minimum and
maximum support TC (n) = Θ(n)
• 2n moves, as every value is moved
from A to B and back. Thus, TM(n)
= cMn = Θ(n). Notation – x, x • Ceiling function
– The smallest integer greater than {expression}.
– expression
– Example: 4.78 = 5
• Floor function
– The largest integer less than {expression}.
– expression
– Example: 4.78 = 4
• Terminology
– Most of the real numbers can be placed in
rooms
– The room contains all the real numbers between
the floor and the ceiling.
– However, for an integer x: x = x = x
• For a noninteger real numbers y : y < x < x
– The ceiling of one room equals the floor of the Floor, Ceiling, Logarithms Floor and Ceiling
Function
Implementations
• C++ provides floor()
and ceil() for floats,
doubles, and long
doubles in the
<cmath> library.
Requires –lm flag for
compiler.
• ints can be “casted” to
floats. Logarithmic Notation
• logb means logarithm
base b, e.g., log10 is a
decimal logarithm
• log means logarithm
any base we don’t
care what base
• lg means logarithm
base 2 same as log2
• ln means natural
logarithm same as
loge Merge Sort
• The idea of merge sort is to
subdivide a list into two lists
successively until all of the lists are
of size 1.
• Since the lists of size 1 are already
sorted they can be successively
merged back into one sorted list.
• An alternative is to subdivide the lists
into short lists, say 8, sort the short
lists by a simple sort, and then Merge Sort Example
Step [0] [1] [2] [3] [4] [5] Comment Initial 10 19 18 16 13 11 Subdivide into two lists 1 10 19 18 16 13 11 Recursively subdivide 2 10 19 18 16 13 11 Continue until all are size 1 3 10 19 18 16 13 11 Merge back 4 10 19 18 13 16 11 Merge back again 5 10 18 19 11 13 16 Merge back again 6 10 11 13 16 18 19 Sorted into Array B Merge Sort Pseudocode
int A[n]; // indexed from 0 to n1
int n;
// number of elements in A
void mergeSort(int A, int B, int
nl, int nh) {
int mid = (nl + nh + 1)/2;
if(nl != mid)
mergeSort(A, B, nl, mid);
if(mid + 1 != nh)
mergeSort(A, B, mid + 1, nh);
merge(A, B, nl, nh);
} Merge Sort Θ()
• Since we are using recursive function
calls, T(n) is measured by a
recurrence relation.
• The function T(n) for mergeSort() can
be divided into four parts:
– Calculation and computation: c
– First recursive call: T( n/2 )
– Second recursive call: T( n/2 )
– Merge call: Θ(n) • Combining and simplifying, we have
T(n) = 2T(n/2) + Θ(n) Solving a Recurrence Relation
• To be soluble, a recurrence relation must
have a base case.
• Thus,
Θ(1), if n=1
T(n) =
T(n) = 2T(n/2) + Θ(n), if n>1
• We will study several methods for solving
recurrence relations; one of the simpler
methods is illustrated on the next slide.
• Note that the base case is usually omitted
from the statement of a recurrence
relation. Using a Recursion Tree to Solve a
Recurrence
T
Θ(n)
(n) Θ(n)
T(n) = 2T(n/2) + Θ(n)
We start with T(n) as
the root
T(n/2
Θ
T(n/2
Θ
Θ(n)
• Then we substitute our
(n/2)
)
(n/2)
)
recurrence equation for
each T term until we
reach the base case.
T(n/4
Θ
T(n/4
Θ
T(n/4
Θ
T(n/4 This actually is a map of
Θ
Θ(n)
(n/4)
)
(n/4)
)
(n/4)
)
(n/4) our recursive calls.
)
ht = ln
n
…
…
Summing by
•
• Level of Tree Θ(1) … n
leaves … Θ(1)
if n>1 Θ(n)
Total = Θ(n
ln n) Comparing Algorithms
• Number of
operations
–
–
–
–
–
–
– Reads
Writes
Memory Operations
Disk Operations
Maximum
Average
Expected • Utilization of
Memory
• Delivery of first
results
• Time between
results
• Parallelizability
• Implementation
Issues Unit 1 Introduction to Algorithms
Objectives
•
•
•
•
•
• Analysis of Algorithms, the Course
What is an algorithm?
Analysis of Algorithms, the Subject
Why Algorithms?
Analysis and Design of Algorithms
Simple Sort Examples
– Bubble Sort
– Insertion Sort
– Merge Sort
– Recurrence Tree
• Notation
• Text, Chapters 1 and 2 ...
View
Full Document
 Spring '08
 FurmanHaddix
 Algorithms, Sort, Insertion Sort, Big O notation, Furman Haddix

Click to edit the document details