This preview shows page 1. Sign up to view the full content.
Unformatted text preview: UMass Lowell Computer Science 91.503 Analysis of Algorithms Analysis of Algorithms
Prof. Karen Daniels
Spring, 2011 Spring 2011 Lecture 9
Tuesday, 4/19/11 y Linear Programming 1 Overview Motivation & Basics Standard & Slack Forms Formulating Problems as Linear Programs Simplex Algorithm Simplex Algorithm Example HighLevel Algorithm Correctness Roadmap Key Concepts Initial Basic Feasible Solution Duality More examples... 2 Motivation & Basics Motivation & Basics 3 Motivation: A Political Problem Motivation: A Political Problem
Goal: Win election by winning majority of votes in each region.
100,000 voters 200,000 voters 50,000 voters Thousands of voters who could be won with $1,000 of ads related to a policy Subgoal: Subgoal: Win majority of votes in each region while g g minimizing advertising cost.
source: 91.503 textbook Cormen et al.
4 Motivation: A Political Problem (continued) This search space is 4dimensional.
urban b suburban rural Thousands of Th d f voters representing majority. source: 91.503 textbook Cormen et al. 5 General Linear Programs General Linear Programs
real numbers Linear function
= realvalued variables Linear constraints Linear inequalities source: 91.503 textbook Cormen et al. 6 Overview of Linear Programming Overview of Linear Programming
Different from voter example: this example has 2dimensional search space p p p
Objective function Convex feasible region: intersection of i t ti f halfplanes Objective value 7 source: 91.503 textbook Cormen et al. Overview of Linear Programming Overview of Linear Programming
More examples (on board) of 2dimensional search space: Infeasible example in which constraints conflict Unbounded example Standard & Slack Forms Standard & Slack Forms 9 Standard Form Standard Form objective function constraints source: 91.503 textbook Cormen et al. 10 Standard Form (compact) Standard Form (compact)
ndimensional vectors mxn matrix mdimensional vector Can specify linear program in standard form by (A,b,c).
11 source: 91.503 textbook Cormen et al. Converting to Standard Form Converting to Standard Form source: 91.503 textbook Cormen et al. 12 Converting to Standard Form (continued) Transforming minimization to maximization minimization maximization
Negate coefficients 13 source: 91.503 textbook Cormen et al. Converting to Standard Form (continued)
Giving each variable a nonnegativity constraint non 3(x2' x2") ( ) x2 has no nonnegativity constraint If xj has no nonnegativity constraint, non negativity replace each occurrence of xj with xj' xj". New nonnegativity constraints 14 source: 91.503 textbook Cormen et al. Converting to Standard Form (continued) Transforming equality constraints to inequality constraints 15 source: 91.503 textbook Cormen et al. Converting to Standard Form (continued) Changing sense of an inequality constraint (Rename variables for notational consistency.) Rationale:
16 source: 91.503 textbook Cormen et al. Converting Linear Programs into Slack Form
for algorithmic ease, transform all constraints except nonnonnegativity ones into equalities
slack variable for inequality constraint: define slack instead of s nonbasic variables b i i bl basic variables 17 source: 91.503 textbook Cormen et al. Converting Linear Programs into Slack Form (continued) objective function happens to be empty in this case 18 source: 91.503 textbook Cormen et al. Converting Linear Programs into Slack Form (continued) into Slack Form (continued)
set of indices of basic variables set of indices of nonbasic variables Compact Form: (N, B, A, b, c, v)
Slack Form Example Compact Form negative of slack form coefficients Basic B i
basic variables NonBasic 19 source: 91.503 textbook Cormen et al. Formulating Problems l i bl as Linear Programs 20 Shortest Paths Shortest Paths
SingleSingle i h t t th i i i "distance" f Si l pair shortest path: minimize "di t " from source s t sink t. to i k t. Use BellmanFord idea: translate relaxation into a constraint. BellmanRecall: ( s, v) ( s, u ) + w(u , v) where is shortest path weight Can we replace maximize with minimize here? Why or why not? source: 91.503 textbook Cormen et al., al., 2nd edition 21 Maximum Flow Maximum Flow maximize f sv  f vs vV vV subject to
Capacity constraints C it t i t Flow conservation f uv c(u , v) for f each u , v V h for each u V  {s, t} f
vV vu = f
vV uv f uv 0 for each u, v V source: 91.503 textbook Cormen et al., al., 3rd edition 22 Minimum Cost Flow
minimize
subject to
Capacity constraints Flow conservation Flow target d is prespecified.
( u ,v )E a(u, v) f
vV uv uv f  f f  f
vV vu
vV sv f uv c(u , v)
=0
=d for each u , v V
for each u V  {s, t} f uv 0 vV vs for each u , v V
cost = 4 cost = 4 cost = 7 cost = 7
cost = 3 t Flow target d is prespecified here as d=4. cost = 3 cost = 3 cost = 10 cost = 10 Total weighted g Total weighted flow cost = 27 flow cost = 27 23 source: 91.503 textbook Cormen et al., al., 3rd edition Multicommodity Flow
k is number of commodities di is target flow value for commodity i minimize 0
subject to
Capacity constraints p y si is source for commodity i ti is sink for commodity i f
i =1 k iuv c(u, v) for each u , v V Flow conservation f
vV ivu  f iuv = 0 for each i = 1,2 ,...k and each u V  {si , ti }
vV Flow target di is prespecified for each i. f
vV isi v  f ivsi = d i
vV for each i = 1,2,...k f iuv 0 for each u , v V and for each i = 1,2,...k "The only known polynomialtime algorithm for this problem expresses it as a linear program and then solves it with a polynomialtime linearprogramming algorithm."
source: 91.503 textbook Cormen et al., al., 3rd edition
24 Simplex Method Simplex Method 25 Solving a Linear Program Solving a Linear Program Simplex algorithm Geometric interpretation Visit some vertices on the boundary of the simplex Visit some vertices on the boundary of the simplex representing the convex feasible region Transforms set of inequalities using process similar to Gaussian elimination Runtime not polynomial in worstcase p y often very fast in practice 1972 exponential time result sparse matrix techniques provide speedup Ellipsoid method Iterative approach creating sequence of ellipsoids: Iterative approach creating sequence of ellipsoids: holds current point as center of ellipsoid that holds a feasible point, if one exists. Cut off part of ellipsoid violated by some constraint. InteriorPoint methods Runtime polynomial slow in practice Runtime polynomial for large inputs, performance can be competitive with simplex method Moves through interior of feasible region source: 91.503 textbook Cormen et al. 26 Simplex Algorithm: Example Basic Solution Basic Solution Standard Form Slack Form Basic Solution: ( x1 , x2 K x6 ) = (0,0,0,30,24,36) Basic Solution: set each nonbasic variable to 0.
27 source: 91.503 textbook Cormen et al. Simplex Algorithm: Example Reformulating the LP Model Reformulating the LP Model
Main Idea: In each iteration, reformulate the LP model so basic Idea: solution has larger objective value (since we're maximizing) we re maximizing).
Select a nonbasic variable whose objective coefficient is positive (since we're maximizing): x1 Increase its value as much as possible. Identify tightest constraint on increase. For basic variable x6 of that constraint, swap role with x1. Rewrite other equations that use x1 , resulting in x6 on RHS. entering variable PIVOT new objective value 28 leaving variable source: 91.503 textbook Cormen et al. Simplex Algorithm: Example Reformulating the LP Model Reformulating the LP Model
Next Iteration: select x3 as entering variable variable. entering variable new objective value ( x1 , x2 K x6 ) = (33 / 4,0,3 / 2,69 / 4,0,0) PIVOT leaving variable New Basic Solution: ( x1 , x2 K x6 ) = (33 / 4,0,3 / 2,69 / 4,0,0)
29 source: 91.503 textbook Cormen et al. Simplex Algorithm: Example Reformulating the LP Model Reformulating the LP Model
Next Iteration: select x2 as entering variable variable. Stop since all coefficients are negative. p g entering variable new objective value ( x1 , x2 K x6 ) = (8,4,0,18,0,0) PIVOT leaving variable New Basic Solution: ( x1 , x2 K x6 ) = (8,4,0,18,0,0)
30 source: 91.503 textbook Cormen et al. Simplex Algorithm: Pivoting p g g
leaving variable entering variable Rewrite the equation that has xl on LHS to have xe on LHS Update remaining equations by substituting RHS of new equation for each occurrence of xe. Do the same for objective function. Update sets of p nonbasic, basic variables. source: 91.503 textbook Cormen et al. 31 Simplex Algorithm: Pseudocode Simplex Algorithm: Pseudocode
initial basic solution to be defined later (detects infeasibility) detects unboundedness optimal solution source: 91.503 textbook Cormen et al. 32 Correctness: Roadmap (Key Pieces) ( i )
Lemma 29.3: Algebraic lemma 29.3: Lemma 29.4: Slack form uniqueness 29.4: Lemma 29.1: Pivot results 29.1: Lemma 29 6: L 29.6: 29.6
TieTiebreaking Lemma 29 8: 29.8: 29.8 8:
Weak LP duality Lemma 29.5: 29.5:
Iteration bound, cycling n + m # variables m # basic variables Corollary 29.9: 29.9:
Conditions for which feasible solutions for primal, primal, (max) and dual (min) programs are optimal Lemma 29.11: Laux 29.11: Lemma 29.12: 29.12:
Infeasibility detection Lemma 29.2: 29.2:
Basic solution feasible > if SIMPLEX finds solution it is feasible; if reports unbounded, then model is unbounded Lemma 29.7: Basic solution 29.7:
feasible > SIMPLEX either reports unbounded or finds feasible solution in n + m m iterations it ti Theorem 29.10: 29.10:
LP d lit duality: SIMPLEX primal result is optimal & dual is optimal * * analogous to max flow/ Theorem 29.13: Fundamental Theorem of Linear Programming 29.13: g g min cut idea For LP model in standard form, either: 1. exists optimal solution with finite objective function value & SIMPLEX returns one, or 2. infeasible & SIMPLEX returns INFEASIBLE, or 3. unbounded & SIMPLEX returns UNBOUNDED (Lemmas 29.2 & 29.7) 33 Proof of Correctness Key Concepts Proof of Correctness Key Concepts
Initial Basic Feasible Solution 34 Finding an Initial Solution Finding an Initial Solution
An LP model whose initial basic solution is not feasible Board work for picture, slack form and initial basic solution. p ,
35 source: 91.503 textbook Cormen et al. Finding an Initial Solution
(continued)
Auxiliary LP model Laux: max x0 Laux in standard but not slack form x0 0  x0 0
L feasible > 0 optimal for Laux 0
L feasible < 0 optimal for Laux <36 source: 91.503 textbook Cormen et al. Finding an Initial Solution
(continued)
most negative see p. 888 for details (also example in next slides) source: 91.503 textbook Cormen et al. al. 3rd edition 37 Finding an Initial Solution
(continued) Original LP model Laux Laux in slack form 38 source: 91.503 textbook Cormen et al. Laux Basic (infeasible) Solution: ( x0 , x1 , x2 , x3 , x4 ) = (0,0,0,2,4) Finding an Initial Solution
(continued)
Entering entering g Leaving (most restrictive) PIVOT Entering most leaving g negative Leaving (most negative, restrictive) (restrictive) PIVOT Laux Basic (feasible) Solution: ( x0 , x1 , x2 , x3 , x4 ) = (4,0,0,6,0)
Why is this feasible for Laux? Why can we stop here? Feasible Basic Solution: (0, 4/5, 14/5, 0) 39 source: 91.503 textbook Cormen et al. Finding an Initial Solution
(continued) See textbook for proof (p. 890). 40 source: 91.503 textbook Cormen et al. Proof of Correctness Key Concepts
Duality 41 Linear Programming Duality Linear Programming Duality
max becomes min x variables go away RHS coefficients swap places with objective function coefficients y variables appear sense changes transpose of A matrix source: 91.503 textbook Cormen et al. 42 Duality Example Duality Example source: 91.503 textbook Cormen et al. 43 Weak Linear Programming Duality Weak Linear Programming Duality
Any feasible solution to p y primal LP has value no g greater than that of any feasible solution to the dual LP.
standard form
29.18 29.8329.85 29.84 source: 91.503 textbook Cormen et al. 44 Weak Linear Programming Duality (continued) maximized minimized source: 91.503 textbook Cormen et al. 45 Finding a Dual Solution Finding a Dual Solution
Finding a dual solution whose value is equal to that of an g q optimal primal solution... Suppose th l t slack f S the last l k form of the primal i f th i l is:
z = v'+
jN c' j x j x i = b'i  jN a''ij x j f i B. for Then, to produce an optimal dual solution, we set:
 c ' n + i yj = 0 if (n + i ) N , otherwise. (29.91) source: 91.503 textbook Cormen et al. al. 3rd edition 46 Optimality
Theorem 29.10 (LinearProgramming Duality). Th 29 10 (Li P i D li ) x = ( x1 , x2 , K , xn ) Suppose that SIMPLEX returns values for the primal linear program (A, b, c). Let N and B for the primal linear program (A b c) Let N and B denote the nonbasic and basic variables for the final , slack form, let c' denote the coefficient in the final y = ( y1 , y2 ,K, ym ) slack form, and let be defined by (29.91). Then is an optimal solution the primal x y program, is an optimal solution to the dual n m program and c j x j = bi y i ,
j =1 1 i 1 i=1 Proof: see text (p. 882885). source: 91.503 textbook Cormen et al. 47 Correctness: Roadmap (Key Pieces) ( i )
Lemma 29.3: Algebraic lemma 29.3: Lemma 29.4: Slack form uniqueness 29.4: Lemma 29.1: Pivot results 29.1: Lemma 29 6: L 29.6: 29.6
TieTiebreaking Lemma 29 8: 29.8: 29.8 8:
Weak LP duality Lemma 29.5: 29.5:
Iteration bound, cycling n + m # variables m # basic variables Corollary 29.9: 29.9:
Conditions for which feasible solutions for primal, primal, (max) and dual (min) programs are optimal Lemma 29.11: Laux 29.11: Lemma 29.12: 29.12:
Infeasibility detection Lemma 29.2: 29.2:
Basic solution feasible > if SIMPLEX finds solution it is feasible; if reports unbounded, then model is unbounded Lemma 29.7: Basic solution 29.7:
feasible > SIMPLEX either reports unbounded or finds feasible solution in n + m m iterations it ti Theorem 29.10: 29.10:
LP d lit duality: SIMPLEX primal result is optimal & dual is optimal * analogous to max flow/ Theorem 29.13: Fundamental Theorem of Linear Programming 29.13: g g min cut idea For LP model in standard form, either: 1. exists optimal solution with finite objective function value & SIMPLEX returns one, or 2. infeasible & SIMPLEX returns INFEASIBLE, or 3. unbounded & SIMPLEX returns UNBOUNDED (Lemmas 29.2 & 29.7) 48 Beyond Chapter 29... Beyond Chapter 29 49 Some Variations on Linear Programming Some Variations on Linear Programming Integer Programming: Linear constraints and objective Mixed Integer Programming: Linear constraints and objective Quadratic Programming: Linear constraints Convex Programming Quadratic objective function Quadratic objective function Realvalued variables Some variable(s) are restricted to integer values Some variable(s) are restricted to integer values. Some variables are not restricted to integers. Variables are restricted to integer values Variables are restricted to integer values. Feasibility is NPcomplete. Generalization of Linear Programming Generalization of Linear Programming Minimize a convex function, subject to convex constraints Interior point methods can be applied if the functions satisfy additional properties. Reference: "A Tutorial on Convex Optimization" by Hindi f " l "b d Increasingly being used.
50 ...
View
Full
Document
This note was uploaded on 02/13/2012 for the course CS 91.503 taught by Professor Staff during the Spring '11 term at UMass Lowell.
 Spring '11
 Staff
 Algorithms

Click to edit the document details