Unformatted text preview: Duality Theory Duality Consider the following Linear Program Min  4 x1  x2  5 x3  3 x4 subject to :  x1 + x2 + x3  3 x4 1 (1) 5 x1x2 3 x38 x4 55 (2) x12 x2 3 x3 + 5 x4 3 (3) x1,x2 ,x3 ,x4 0 We wish to give a quick lower bound on the optimal objective function value Trivially, negative infinity (likely to get you fired!) Lower Bound Consider
 4 x1  x2  5 x3  3 x4 
5 (2) * 3
25 5 40 275 x1  x 2  5 x3  x4  3 3 3 3 Min  4 x1  x2  5 x3  3 x4 subject to :  x1 + x2 + x3  3 x4 1 5 x1x2 3 x38 x4 55 x12 x2 3 x3 + 5 x4 3 x1,x2 ,x3 ,x4 0 (1) (2) (3) How about In general, 1* (2) + 1* (3)  4 x1  x2  5 x3  3 x4  4 x1  3x 2  6 x3  3 x 4 58
w1 * (1) + w2 * (2) + w3 * (3)  ( w1 + 5w2  w3 ) x1  (  w1 + w2 + 2 w3 ) x 2  (  w1 + 3w2 + 3w3 ) x3  ( 3w1 + 8w2  5w3 ) x 4  w1  55w2  3w3 (4)  ( w1 + 5w2  w3 ) 4  (  w1 + w2 + 2w3 ) 1  (  w1 + 3w2 + 3w3 ) 5  ( 3w1 + 8w2  5w3 ) 3 w1 , w2 , w3 0 Dual Problem If conditions are met then z *  w1  55w2  3w3 Since we want a good lower bound, we would like  w1  55w2  3w3 to be as large as possible, while maintaining the constraints required for it to be a lower bound. max  w1  55w2  3w3
subject to :  ( w1 + 5w2  w3 ) 4  (  w1 + w2 + 2w3 ) 1  (  w1 + 3w2 + 3w3 ) 5  ( 3w1 + 8w2  5w3 ) 3 w1 , w2 , w3 0 Another LP! Primal and Dual
Primal
min cT x s.t. Ax x b xx 0 Dual
max bT w s.t. AT w x c wx 0 Primal Constraint Primal Variable Dual Variable Dual Constraint Amx n ; xn 1 ; wm 1 Let be feasible solutions of the T w b x w Ax cT x problem T Weak Duality Duality Theorem 1: Any feasible solution to the
dual value of the primal primal dual upper provides lower a bound on the optimal objective function problem Duality Theorem 2: primal If the dual problem has an unbounded objective then the dual has no feasible solution. primal Duality Duality Theorem 3: If x* and w* are feasible solutions to the primal and dual problems respectively and cTx*=bTw* then x* and w* are optimal solutions for the primal and dual respectively. Gross Slackness Conditions Duality Theorem 4: If x* and w* are feasible solutions to the Primal and Dual respectively such that cTx*=bTw* then, w*T(Ax*b)=0 (cTw*TA)x*=0 Proof:
T DualTSlacks cTx* = w*TAx* = w*Tb
Primal Slacks w*T (Ax*b)=0 (c w* A)x*=0 Complementary Slackness Conditions Duality Theorem 5 If x* and w* are feasible solutions to the Primal and Dual, respectively, such that cTx*=bTw*, then
wi* ( Ai. x * bi ) = 0 i (c  w* T A.j x* = 0 j j j ) wi* ( Ai. x *  bi ) = 0 Proof: From DT 4 : w*T(Ax*b) = 0 m
wi* ( Ai. x *  bi ) = 0 i =1 w 0
* i (A x
i. *  bi 0 i ) wi* ( Ai. x * bi ) 0 i wi* ( Ai. x * bi ) = 0 i Strong Duality Theorem Duality Theorem 6: If the
dual primal
T 1 primal dual has an optimal solution, then so does the Since Ax b AxIs = b c B* ( B *) A. j  c j 0 j N 14 2 43 Proof: Let x* be an optimal solution with optimal basis B*0 x0 x,s w*T
A.T w * c j j A w* c
T and their objective functions are equal j j (1) N B 0 =0 For surplus variables the cost is 0 And column is  e i or ( e ) i T w* 0 wi* 0 i AT w* x c x w* is dual feasible w* 4 0 Duality Theorem Duality Theorem 7 Let x* and w* be any feasible solution to the primal and dual problems in the canonical form. Then they are respectively optimal if (c j  A. j w ) x j = 0 j
T * * w ( Ai x  bi ) = 0
* i * KarushKuhnTucker Conditions Duality Theorem 8 x* is an optimal solution to the LP {min cTx s.t. Ax b, x 0} if there exists an mvector w* such that : 1. Ax* b, x* 0 2. ATw* c ; w* 0 3. w*T(Ax*b) = 0 (cTATw* )x* = 0 Primal Feasible Dual Feasible Gross Slackness Conditions Example
min  x1  3 x2 x1  2 x2 s.t.  x1  x2 x1 , x2 Is (0,0) optimal? Is (4/3, 8/3) optimal? 4 4 0 Example
min  x1  3 x2 x1  2 x2 s.t.  x1  x2 x1 , x2 4 4 0 Consider solution 1: (0,0) 1. Satisfied 2. No constraint binding set w1=w2=0 3. ATw = (0,0)T > c Consider solution 2: (4/3, 8/3) 1. Satisfied 3. (cATw*) = 0 c = ATw* w1 = 2 ; w* = 0 3 2. Dual Feasibility satisfied Primal/Dual Relationship max Constraint Constraint = Constraint x0 x0 x unrestricted min u0 u0 u unrestricted Constraint Constraint = Constraint Primal/Dual Relationship Dual rules:
Primal Maximize Constraint i = Variable xj xj 0 xj 0 xj unrestricted Dual Minimize Variable ui ui 0 ui 0 ui unrestricted Constraint j = Dual of Dual Theorem: Dual of dual is primal. Duality Gap Four possibilities Primal optimal, dual optimal (no gap) Primal unbounded, dual infeasible (no gap) Primal infeasible, dual unbounded (no gap) Primal infeasible, dual infeasible (infinite gap) Geometric Implication of Duality Farkas' Lemma
System 1: Ay x 0 cT y > 0 System 2: AT w = c w 0 Physical Interpretation Economic Interpretation of Duality Table Production Example Table Production Example
Table 1 10 Wood 1 Wood 2 2 1 Table 2 14 1 3 10 10 Resource LP Model of Table Production Example Decision variables Table Production Example 1 2 Profit
1 Table 1 x 10 2 1 Table 2 x 14 1 3 Resource Wood 1 u
2 10 10 Wood 2 u Table Production Example: Dual Decision variables An Economic Interpretation
max ( P) s.t.
n c x
j =1 j n n
j min
j =1 bi wi aij x j bi
j =1 i = 1, , m ( D) s.t. n i =1 aij wi c j = 1, , n j xj 0 j = 1, , n wi 0 = 1, , m i x j number of units of product j to be produced aij amount of resource i required per unit of product j bi amount of resource i available c j profit from selling one unit of product j wi " rent" to be charge for resource i c j loss from not producing unit of product j a w
ij i i rent available from not producing 1 unit of product j wi  Fair rent or shadow price for resource i Production Example Production Example
Activity 1 Activity 2
1 2 Demand cost u 10 u 10 1 3 10 14 Product 1 2 Product 2 1 Production Example: Primal Decision variables Production Example Production Example
Activity 1 cost Product 1
1 Activity 2 10 1 Demand 10 2 10 x Product 2
2 1 3 14 x Production Example: Dual Decision variables Another Economic Interpretation
max ( P) s.t.
n c x
j =1 j n n
j min
j =1 bi wi aij x j bi
j =1 i = 1, , m ( D) s.t. n i =1 aij wi c j = 1, , n j xj 0 j = 1, , n wi 0 = 1, , m i x j price to be charged per unit of product j c j the needed quantity of product j aij amount of product j produced by one unit of activity i bi per unit cost of activity i a x
j =1 ij n j revenue cannot exceed the per unit cost of activity i a w
ij i wi level of activity i
i amount of product j produced x j  Fair price or shadow price for product j Shadow price No Degeneracy
z = cT B 1b  B = bT w* 
j N (z j  c j )x j
j N (z j  c j )x j z = bi
* wi*bi
i bi = wi* Holds as long as the basis remains unchanged * w 0 implies that as bi increases z increases Eventually some other basis will become optimal This will happen when some basic variable reaches its bound Implies Degeneracy more than one basis is optimal at this point More than one dual solution What is the shadow price? Shadow Price under Degeneracy
min cT x s.t. Ax x b xx 0
max bT w s.t. AT w x c wx 0 Assume that D is feasible P is bounded for all values of b Consider values of bi such that P remains feasible Note the feasible region of D does not change Since D is feasible and P is feasible the optimal solution is bounded dual optimal solution is at an extreme point of the dual polyhedron w j , j = 1, , E be extreme points z * (bi ) = max { bT w j j = 1, , E} Shadow Prices Since we are only changing bi
z * (bi ) = max { bT w j j = 1, , E} = max bk wkj + wij bi j = 1, , E k x i = max { w0j + wij bi j = 1, , E} z * (bi ) w4 w3  z* = min wij w j is an optimal dual vertex at bi bi
w
1 { } } w2 + z* = max wij w j is an optimal dual vertex at bi bi  z* bi + z* bi { bi Find Shadow Price in the Primal Tableau Lagrangian Duality minimize cTx subject to Ax b x2 P Define the dual objective g(p) by g(p)=minx2 P cTx+pT(bAx) Define the dual problem maximize g(p) subject to p 0 Theorem: Both weak duality and strong
duality hold. Applications of Duality Asset Pricing I Given n different assets being traded There are m possible states of natures Payoff matrix: R=[rij], where rij is the payoff of asset j at state i x: the portfolio vector w: wealth vector: w=Rx p: the asset price vector Cost of acquiring portfolio x is pTx What the prices should be? Asset Pricing II: Absence of Arbitrage Absence of Arbitrage: Rx 0 p Tx 0 Theorem The absence of arbitrage condition holds if and only if there exists q 0, called the state price vector, such that p=RTq Robust Optimization I Consider a linear program minimize (c+Cu)Tx subject to Ax=b x 0, where u z and Cu e is a state of nature and beyond decision maker's control. Robust Solution: minimize maxu z, Cu e (c+Cu)Tx subject to Ax=b x 0, Robust Optimization II Nature's (primal) problem maximizeu cTx+xTCu subject to Cu e u 0 Dual of Nature's problem minimizey cTx+eTy subject to C Ty C Tx y 0 Robust Optimization III Decision maker's robust solution minimizex,y cTx+eTy subject to C Ty C Tx Ax=b x, y 0 Linear Production Game Cooperative Game: Example Table Production Example
Table 1 Table 2 10 Wood 1 Wood 2 2 1 14 1 3 10 10 Resource LP Model of Table Production Example Decision variables Table Production Example Now assume wood i is controlled by agent i, i=1,2 Do the agents put the wood together to make the production? If they do, how should they share the profit? l1+l2=V({1,2}) l1 V({1}) l2 V({2}) Table Production Example If we know the fair price of wood 1 and wood 2, we can quantify the contribution of each agent and figure out how much each agent should get Table Production Example 1 2 Profit
1 Table 1 x 10 2 1 Table 2 x 14 1 3 Resource Wood 1 u
2 10 10 Wood 2 u Table Production Example: Dual Minimize 10u1 + 10u2 value of wood s.t. 2u1 + u2 10
u1 + 3u2 14 u1 0, u2 0 value of Table #1
value of Table #2 Optimal solution u* = (3.2, 3.6) with optimal value $68 We can easily check that the profit allocation (32,36) is fair for both agents Cooperative Game: Core Cooperative game (N,V) N={1,2,...,n}: grand coalition For each coalition S N, a characteristic cost/profit function V(S) An Allocation (l1, l2, , ln): i2 N li = V(N) Core of the game: i2 N li = V(N) i2 S li V(S) 8 S N (reverse the direction for profit game) Questions Relate to Core Core is empty or not? How to find an allocation in the core? Check whether an allocation is in the core. Production Game (Owen `75) N={1,2,...,n}: grand coalition Player i has a resource vector bi V(S) = max c'x s.t. Ax i2 Sbi x 0 Production Game (Owen `75) V(N) = max c'x s.t. Ax i2 Nbi x 0 How much should be allocated to player i? Player i owns resource bi If the (marginal) price of the resource is y, then player i should get (bi)Ty How to get y? Is this allocation in the core? Production Game: Duality Approach Let y* be any optimal solution for the dual V(N)= min ( i2 Nb ) y s.t. ATy c y 0 li=(bi)Ty*
i T Define Then the allocation (l1,l2,...,ln) is in the core i2 N li = ( i2 Nb ) y = V(N) i2 S li = ( i2 Sb ) y V(S)
i T * i T * Production Game (Owen `75) i2 S li = ( i2 Sb ) y V(S) y* is a feasible solution for the following dual min ( i2 Sb ) y s.t. ATy c y 0 V(S) = max c'x s.t. Ax i2 Sbi x 0
i T i T * Primal Newsvendor Game Dual Simplex Method Optimality Conditions Alternative Methods for LP Primal Simplex Method: at each iteration, B1b 0 (primal feasible) target for cBTB1A cT (dual feasible) Dual Simplex Method at each iteration, cBTB1A cT (dual feasible) target for B1b 0 (primal feasible) PrimalDual Method At each iteration, maintain complementary slackness conditions target for primal feasibility and dual feasibility An Example Min 2x1+3x2+4x3 s.t. x1+2x2+ x3 3 2x1  x2+3x3 4 x1, x2, x3 0 z z
4 1 2 3 4 5 1 0 0 x 2 1 2 x 3 2 1 x 4 1 3 x 0 1 0 x 0 0 1 rhs 0 3 4 x
5 x z z
4 1 2 3 4 5 1 0 0 x 0 0 1 x 4 x 1 x 0 x 1 rhs 4 x
1 5/2 1/2 1 1/2 3/2 0 1/2 1 1/2 2 x z z
2 1 2 3 4 5 1 0 0 x 0 0 1 x 0 1 0 x x x 9/5 8/5 1/5 28/5 1/5 2/5 1/2 2/5 7/5 1/5 2/5 11/5 rhs x
1 x Dual Simplex Method Initial Dual Feasible Solution Initial Dual Feasible Solution One pivot will give a dual feasible solution ckzk=minj {cjzj} xk enters and xn+1 leaves the basis Solving the problem with the artificial constraint leads to three cases: 1. The dual problem is unbounded 2. Optimal primal and dual solutions are obtained with x*n+1>0 3. Optimal primal and dual solutions are obtained with x*n+1=0 (cn+1zn+1 =cBTB1e1) (a) cn+1zn+1>0 (unbounded) (b) cn+1zn+1 =0 (optimal) One Simple Example Dual Simplex Method
Try Java Pivot Tool http://www.princeton.edu/~rvdb/LPbook/index.html Sensitivity Analysis Sensitivity Analysis Changes in the Cost Vector An Example Changes in the RightHandSide An Example Changes in a nonbasic column of A Changes in a basic column of A Aj! Aj' (xj is a basic variable): complicated What's the impact of this change? Handle in two steps Add a new variable xj' with column Aj' Eliminate the old variable xj from the problem An Example Add a new variable Suppose that we add a new variable xn+1 with cost cn+1 and consumption column An+1 If cn+1zn+1 0, x*n+1=0 and the current solution is optimal If cn+1zn+1<0, xn+1 enters the basis An Example Add a new constraint Add Am+1x bm+1 xB+ B1NxN =B1b ((Am+1)N(Am+1)NB1N)xN+xn+1=bm+1(Am+1)BB1b Add Am+1x = bm+1 Use artificial variable as a basic variable in this row An Example Example: Continued Parametric Analysis Parametric Analysis: cost vector An Example Example: Continued Example: Continued Example: Continued Example: Continued Parametric Analysis: RightHandSide An Example Example: Continued Example: Continued Example: Continued ...
View
Full Document
 Fall '09
 XinChen
 Linear Programming, Optimization, WI, Dual problem, dual feasibility, Table Production Example

Click to edit the document details