Chapter 5
Fundamental Techniques
Acknowledgement
In addition to the textbook slides and my
slides, I used some from Dr. Ying Lu of
University of Nebraska at Lincoln, especially
on dynamic programming solution of the 0/1
Knapsack Problem.
2
Well look at 3
Comments
This algorithm only finds the max possible
value that can be carried in the knapsack
i.e., the value in B[n,W]
To know the items that make this maximum
value, an addition to this algorithm is
necessary
1
How to Find Actual Knapsack Items
All
Dictionary ADT
The dictionary ADT models a
searchable collection of keyelement items
The main operations of a
dictionary are searching, inserting,
and deleting items
Multiple items with the same key
are allowed
Applications:
address book
credit card aut
1
2
3
4
1
0
2
120
0
n/a 0
n/a n/a
n/a n/a
3
Trace of M - 4
4
1: A is 30x1
2: B is 1x40
3: C is 40x10
4: D is 10x25
d=1 (loop to 3)
0
n/a
i=2 (loop to 4-1=3), j = i+d = 3
M[2,3] =
0
for d := 1 to n-1 do / diagonals
for i := 1 to n-d do / rows with an entr
When using a greedy algorithm, if we want to
guarantee an optimal solution we must prove
that our method of choosing the next item
works.
There are times, as we will see later, when we
are willing to settle for a good approximation to
an optimal solutio
Vector-based Heap Implementation (2.4.3)
We can represent a heap with n keys
by means of a vector of length n 1
For the node at rank i
the left child is at rank 2i
the right child is at rank 2i 1
Links between nodes are not
explicitly stored
The leaves
Still another method The Guessand-Test Method
In the guess-and-test method, we guess a closed form solution
for a recurrence relation and try to prove it is true by induction:
b
if n 1
T (n)
2T (n / 2) bn log n if n 2
Note: Changed n < 2 in text to n =1
Sequence ADT
The Sequence ADT is the
union of the Vector and List
ADTs
Elements accessed by
Rank, or
Position
Generic methods:
size(), isEmpty()
Vector-based methods:
elemAtRank(r), replaceAtRank(r,
o), insertAtRank(r, o),
removeAtRank(r)
List-ba
Example Trace-9
i\w
0
1
2
3
4
0
0
0
0
0
0
1
0
0
0
2
0
3
3
0
3
4
0
3
5
0
3
if wi <= w / item i can be part of the solution
if bi + B[i-1,w-wi] > B[i-1,w]
B[i,w] = bi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] / wi > w
i=2
bi=4
Items:
1: (
Postorder Traversal
In a postorder traversal, a node
is visited after its descendants
Application: compute space
used by files in a directory and
its subdirectories
9
Algorithm postOrder(v)
for each child w of v
postOrder (w)
visit(v)
cs16/
3
8
7
homework
Lists and Sequences
Singly Linked List
A singly linked list is a
concrete data structure
consisting of a sequence of
nodes
Each node stores
element
link to the next node
next
node
elem
A
B
C
D
2
Stack with a Singly Linked List
We can implement a stac
Heaps and Priority Queues
2
5
9
6
7
What is a heap (2.4.3)
A heap is a binary tree storing The last node of a heap is
keys at its internal nodes and
the rightmost internal
satisfying the following
node of depth h 1
properties:
Heap-Order: for every inte
Queue Operations (cont.)
Algorithm enqueue(o)
Operation enqueue throws
if size() = N 1 then
an exception if the array is
throw
full
FullQueueException
This exception is
else
implementationQ[r] o
dependent
r (r + 1) mod N
Q
0 1 2
f
r
Q
0 1 2
r
f
1
Queue
Knapsack problem (Review)
Given some items, pack the knapsack to get the maximum
total value. Each item has some weight and some
value/benefit. The total weight that we can carry is no
more than some fixed number W.
So we must consider weights of items as
The 0/1 Knapsack Algorithm - Textbook
Recall definition of B[k,w]:
B[k 1, w]
if wk w
B[k , w]
else
maxcfw_ B[k 1, w], B[k 1, w wk ] bk
Since B[k,w] is defined in terms
of B[k-1,*], we can reuse the
same array.
We didn't do this so you can see
the al
Slight change from one in text on pg 280. This uses a 2dimensional array that illustrates what is going on better.
for w = 0 to W
Algorithm 01Knapsack(S,W)
B[0,w] = 0
Input: Set S of n items such that each item i
for i = 1 to n
has benefit bi and weight w
Master Method, Example 7
The form:
c
T ( n )
aT ( n / b) f ( n )
if n d
if n d
The Master Theorem:
log a
1. if f ( n) is O( n
b
), then T ( n) is ( n logb a )
2. if f (n) is ( n logb a log k n), then T ( n) is ( n logb a log k 1 n)
3. if f (n) is ( n
Data Structure for Binary Trees
A node is represented by
an object storing
Element
Parent node
Left child node
Right child node
B
Node objects implement
the Position ADT
B
A
A
D
C
D
E
C
E
1
Priority Queues
Sell 100 IBM
Sell 300 IBM
Bu
y
Bu
y
500 IBM
400 I
Not Optimal, But a Good Approximation
Sometimes the greedy method can be used even when
the greedy-choice property doesn't hold.
That will often lead to a pretty good approximation to the
optimal solution.
An Example: A county is in its early stages of
Worst-case Running Time
The worst case for quick-sort occurs when the pivot is the unique
minimum or maximum element
One of L and G has size n 1 and the other has size 0
The running time is proportional to the sum
n (n 1) 2
Thus, the worst-case running t
Radix-Sort for Binary
Numbers
Consider a sequence of n b-bit
integers
x xb x1x0
We represent each element as
a b-tuple of integers in the
range [0, 1] and apply radixsort with N 2
This application of the radixsort algorithm runs in O(bn)
time
For exam
Quick-Sort
Quick-sort is a randomized
sorting algorithm based on
the divide-and-conquer
paradigm:
x
Divide: pick a random element
x (called pivot) and partition S
into
L elements less than x
E elements equal x
G elements greater than x
Recur: sort L
An Improved Integer Multiplication Algorithm
The recursion on the last slide requires 3
products of n/2 bits each plus O(n) additional
work.
So, T(n) = 3T(n/2) + n, which implies T(n) is
(nlog23), by the Master Theorem.
Thus, T(n) is (n1.585).
That's
Sets
Sets
1
Set Operations
We represent a set by the
sorted sequence of its
elements
By specializing the auxiliary
methods, the generic merge
algorithm can be used to
perform basic set operations:
aIsLess(a, S)
S.insertFirst(a)
bIsLess(b, S)
S.insertLas
The Basic Idea
The technique solves problems with overlapping
subproblems.
Typically the subproblems arise through a recursive
solution to a problem.
Rather than solve the subproblems repeatedly, we
solve the smaller subproblem and save the results in
Other Examples for Using Dynamic
Programming:
(O. Slotterbeck, J. W. Baker and R. Aron)"An Algorithm for
Computing the Tsirelson's Space Norm", published as
Appendix B (44 pages) in Tsirelson's Space by P. Casazza and T.
Shura, Lecture Notes in Mathemati
Note this property holds:
Subproblem optimality: The optimal solution can be defined
in terms of optimal subproblems
There has to be a final multiplication (root of the
expression tree) for the optimal solution.
Say, the final multiply is at index i: