Unformatted text preview: UMass L UM Lowell Computer Science 91.503 ll C S i 91 503 Analysis of Algorithms y f g
Prof. Karen Daniels
Spring, Spring 2011 Lecture 2
Tuesday, 2/1/11 y, Design Patterns for Optimization Problems Greedy Algorithms Algorithmic Paradigm Context
Divide & Conquer View problem as collection of subproblems "Recursive" nature Independent subproblems Number of subproblems Preprocessing Characteristic running time Ch t i ti i ti Primarily for optimization problems Optimal substructure: optimal solution to problem contains within it optimal solutions to subproblems Greedy choice property: locally optimal produces globally optimal Heuristic version useful for bounding optimal value
Subproblem solution order Dynamic Programming Greedy Algorithm overlapping typically sequential q dependence depends on partitioning factors typically l t i ll log function of n typically small depends on number d d b and difficulty of subproblems typically sort often dominated ft d i t d by nlogn sort Solve subproblem(s), then make choice Make choice, then solve subproblem(s) Greedy Algorithms What is a Greedy Algorithm? Solves a opt So ves an optimization problem at o p ob e Optimal Substructure: optimal solution contains in it optimal solutions to subproblems Greedy Strategy: At each decision point do what looks best "locally" point, locally Choice does not depend on evaluating potential future choices or presolving overlapping subproblems Topdown algorithmic structure Top With each step, reduce problem to a smaller problem Greedy Choice Property: "locally best" = globally best A Greedy Strategy Approach
source: 91.503 textbook Cormen, et al. Cormen, 1. 2. 3. 4. 5. 6. Determine the optimal substructure of the problem. Develop a recursive solution. l i l i Prove that, at any stage of the recursion, one of the optimal choices is the greedy choice. choice Show that all but one of the subproblems caused by making the greedy choice are empty. y g g y py Develop a recursive greedy algorithm. Convert it to an iterative algorithm. g
With experience, it is also possible to directly devise a (possibly iterative) greedy strategy. Examples of Greedy Algorithms Activity Selection Minimum Spanning Tree Dijkstra Sh Dijk Shortest P h Path Huffman Codes Fractional Knapsack Activity Selection Activity Selection Optimization Problem Problem Instance: Set S = {a1, a2,..., an} of n activities {a Each activity ai has: start time: si finish time: fi si f i Activities i, j are compatible iff nonoverlapping: non[ si f i ) and [ s j f j ) and si f j or s j f i ( ) Objective: select a maximumsized set of mutually compatible activities maximumsource: 91.503 textbook Cormen, et al. Cormen, Activity Selection
Activity Time Duration
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Activity Number 1 2 3 4 5 6 7 8 Algorithmic Progression "Brute"BruteForce" Check all possible solutions Exponential number of subproblems! i l b f b bl Quadratic number of subproblems Linear number of subproblems p Dynamic Programming Greedy Algorithm Activity Selection: Dynamic Programming Formulation
Sij = {ak S : f i sk < f k s j }
Solution to Sij including ak produces 2 subproblems: 1) Sik (start after ai finishes; finish before ak starts) 2) Skj (start after ak finishes; finish before aj starts) c[i j] size of maximumsize subset of i,j]=size i,j]=size maximumj] mutually compatible activities in Sij. 0 if Sij = 0 / c[i, j ] = max ak Sij {c[i, k ] + c[k , j ] + 1} if Sij 0 / We can do better with a greedy strategy...
source: 91.404 textbook Cormen, et al. Activity Selection: Recursive Greedy Algorithm
simplified notation for activities that start after ak finishes: S k = {ai S : si f k }
k = index of current subproblem n = total number of activities Only 1 subproblem to solve at each "level".
s, f = arrays of start finish times start, times, sorted by increasing finish time (compatible with ak ) HighHighlevel call: RECACTIVITYSELECTOR(s,f,0,n) RECACTIVITYSELECTOR(s ,0,n 0 Returns an optimal solution for S Running time?
source: 91.503 textbook Cormen, et al. (3rd edition) Cormen, source: web site accompanying 91.503 textbook Cormen, et al. Activity Selection: Iterative Greedy
source: 91.503 textbook Cormen, et al. Algorithm: g Presort activities in S by nondecreasing finish time and renumber. non Running time?
source: 91.503 textbook Cormen, et al. (3rd edition) Cormen, Streamlined Greedy Strategy Approach
1. 2. 3. View optimization problem as one in which making choice leaves one subproblem to solve. Prove there always exists an optimal solution that makes the greedy choice. Show that greedy choice + optimal solution to subproblem optimal solution to problem.
source: 91.503 textbook Cormen, et al. Greedy Choice Property: "locally best" = globally best Minimum Spanning Tree Minimum Spanning Tree
Where are the greedy choices made?
Time: Time: O(ElgE) given fast FINDFINDSET, UNION Invariant: Minimum weight i forest t spanning f Produces minimum weight tree of edges that includes every vertex. vertex. Becomes single B i l tree at end A 3 1 G 5 2 4 6 E 6 8 2 7 F 4 B Time: Time: O(ElgV) = O(ElgE) slightly faster with fast priority queue 1 Invariant: Minimum weight tree g D C for Undirected, Connected, Weighted Graph G=(V,E) G (V E)
Spans all vertices at end
source: 91.503 textbook Cormen et al. (Ch 23) Dijkstra Shortest Path Single Source Shortest Paths: Dijkstra s Algorithm Dijkstra's
for (nonnegative) weighted, directed graph G=(V,E) A 3 1 G 5 6 D 2 4 6 E 8 2 7 F 4 B 1 C Running time? It depends on time? priority queue implementation.
We will revisit this algorithm when we study shortest paths.
source: 91.503 textbook Cormen et al. (Ch 24) Huffman Codes Huffman Code Motivation Prefix Code: No code is a prefix of any other. Code: source: web site accompanying 91.503 textbook Cormen, et al. Huffman Code Tree Example
In general, tree need not be so unbalanced. Goal: Goal: Minimize number of bits (cost) required to encode a file.
B(T ) = (c. freq )(dT (c) )
cC = length of codeword for character c = depth of c's leaf in tree = # edges from root to node source: 91.503 textbook Cormen, et al. Code Tree Comparison
How many leaves? How many internal nodes? FixedFixedlength code: not optimal Huffman prefix code: optimal source: web site accompanying 91.503 textbook Cormen, et al. Example of Huffman Code Steps source: web site accompanying 91.503 textbook Cormen, et al. Huffman Code Greedy Algorithm
= = = = z.left = = z.right i ht z.freq x.freq + y.freq = = = = = =
Running Time?
source: web site accompanying 91.503 textbook Cormen, et al. Huffman Correctness: A Key Idea y
Lemma 16.2: Let C be an alphabet in which each character c in C has 16.2: frequency c.freq. Let x and y be 2 characters in C with lowest frequencies. c.freq. Then there exists an optimal prefix code for C in which codewords x and y have the same length and differ only in the last bit.
optimal x a y b (T represents arbitrary optimal p p y p prefix code.) ) ( (Claim: T'' also has optimal cost.) p ) source: web site accompanying 91.503 textbook Cormen, et al. Cormen, Fractional Knapsack Knapsack
Each item has value and weight. g Goal: Goal: maximize total value of items chosen, subject to weight limit. fractional: can take part of an item 01: take all or none of an item
wt wt wt wt capacity 50 30 20 item3 it 3 $120 "knapsack" "k k" 10 item1 item2 it 1 it 2 Value: $60 $100 source: web site accompanying 91.503 textbook Cormen, et al. Greedy Heuristic If optimization problem does not have "greedy choice property", greedy approach g y p p y ,g y pp may still be useful as a heuristic in bounding the optimal solution g p Example: minimization problem
Upper B U Bound (heuristic) d (h i ti ) Optimal (unknown value) Lower Bound Solution Values ...
View
Full Document
 Spring '11
 Staff
 Algorithms, Dynamic Programming, Greedy algorithm, ET AL, textbook Cormen, Cormen

Click to edit the document details