Download
# Lecture LP Duality In Lecture we saw the Maxow Mincut Theorem which stated that the maximum ow from a source to a sink through a graph is always equal to the minimum capacity which needs to be remo PDF document - DocSlides

tawny-fly | 2014-12-14 | General

### Presentations text content in Lecture LP Duality In Lecture we saw the Maxow Mincut Theorem which stated that the maximum ow from a source to a sink through a graph is always equal to the minimum capacity which needs to be remo

Show

Page 1

Lecture 5 LP Duality In Lecture #3 we saw the Max-ﬂow Min-cut Theorem which stated that the maximum ﬂow from a source to a sink through a graph is always equal to the minimum capacity which needs to be removed from the edges of the graph to disconnect the source and the sink. This theorem gave us a method to prove that a given ﬂow is optimal; simply exhibit a cut with the same value. This theorem for ﬂows and cuts in a graph is a speciﬁc instance of the LP Duality Theorem which relates the optimal values of LP problems. Just like the Max-ﬂow Min-cut Theorem, the LP Duality Theorem can also be used to prove that a solution to an LP problem is optimal. 5.1 Primals and Duals Consider the following LP = max(2 + 3 s.t. 4 + 8 12 + 2 ,x In an attempt to solve we can produce upper bounds on its optimal value. Since 2 + 3 + 8 12, we know OPT( 12. Since 2 + 3 (4 + 8 6, we know OPT( 6. Since 2 + 3 ((4 + 8 ) + (2 )) 5, we know OPT( 5. In each of these cases we take a positive linear combination of the constraints, looking for better and better bounds on the maximum possible value of 2 + 3 . We can formalize Lecturer: Anupam Gupta. Scribe: Timothy Wilson.

Page 2

LECTURE5.LPDUALITY this, letting ,y ,y be the coeﬃcients of our linear combination. Then we must have + 2 + 3 + 2 ,y ,y and we seek min(12 + 3 + 4 This too is an LP! We refer to this LP as the dual and the original LP as the primal. The actual choice of which problem is the primal and which is the dual is not important since the dual of the dual is equal to the primal. We designed the dual to serve as a method of constructing an upperbound on the optimal value of the primal, so if is a feasible solution for the dual and is a feasible solution for the primal, then 2 + 3 12 + 3 + 4 . If we can ﬁnd two feasible solutions that make these equal, then we know we have found the optimal values of these LP. In this case the feasible solutions ,x and 16 ,y = 0 ,y give the same value 4.75, which therefore must be the optimal value. 5.1.1 Generalization In general, the primal LP = max( Ax b,x ,x corresponds to the dual LP, = min( c,y ,y where is an matrix. When there are equality constraints or variables that may be negative, the primal LP = max( s.t. for for 0 for for corresponds to the dual LP = min( s.t. 0 for for for for

Page 3

LECTURE5.LPDUALITY 5.2 The Duality Theorem The Duality Theorem will show that the optimal values of the primal and dual will be equal (if they are ﬁnite). First we will prove our earlier assertion that the optimal solution of a dual program gives a bound on the optimal value of the primal program. Theorem 5.1 (The Weak Duality Theorem) Let = max( Ax b,x ,x and let be its dual LP, min( c,y ,y . If is a feasible solution for and is a feasible solution for , then Proof. ) (Since feasible for and 0) = ( Ax (Since is feasible for and 0) From this we can conclude that if is unbounded (OPT( ) = ), then is infeasible. Similarly, if is unbounded (OPT( ) = ), then is infeasible. Therefore we have the following table of possibilities for the feasibility of and Unbounded Infeasible Feasible Unbounded no yes no Infeasible yes ??? ??? Feasible no ??? ??? The Duality Theorem allows us to ﬁll in the remaining four places in this table. Theorem 5.2 (Duality Theorem for LPs) If and are a primal-dual pair of LPs, then one of these four cases occurs: 1. Both are infeasible. 2. is unbounded and is infeasible. 3. is unbounded and is infeasible. 4. Both are feasible and there exist optimal solutions x,y to and such that We have already seen cases 2 and 3 as simple consequences of the Weak Duality Theorem. The ﬁrst case can easily be seen to occur: a simple example takes to be a matrix, to be strictly negative, and to be strictly positive). Therefore the only remaining case of interest is case 4. Geometric Proof. Let be the program max( Ax b,x ) and be dual program min( c,y 0). Suppose is an optimal feasible solution for . Let for be all the constraints tight at . We claim that the objective function vector is contained in the cone , generated by the vectors

Page 4

LECTURE5.LPDUALITY a x <= b a x <= b 2 2 Figure 5.1: The objective vector lies in the cone spanned by the constraint vectors Suppose for contradiction that does not lie in this cone. Then there must exist a separating hyperplane between and : i.e., there exists a vector such that for all , but d> 0. Now consider the point d for some tiny > 0. Note the following: For small enough , the point satisifes the constraints Az . Consider for 6 : since this constraint was not tight for , we won’t violate it if is small enough. And for with we have a a since > 0 and 0. The objective function value increases since c d>c This contradicts the fact that was optimal. Therefore the vector lies within the cone made of the normals to the constraints, so is a positive linear combination of these normals. Choose for so that , and set = 0 for 6 We know 0. Therefore is a solution to the dual with , so by The Weak Duality Theorem, OPT( ) = OPT( ). A somewhat more rigorous proof not relying on our geometric intuition that there should be a separating hyperplane between a cone and a vector not spanned by the cone relies on a lemma by Farkas that often comes in several forms. The forms we shall use are as follows Theorem 5.3 (Farkas’ Lemma (1894) - Form 1) Given and , exactly one of the following statements is true.

Page 5

LECTURE5.LPDUALITY 1. such that Ax 2. such that and b< Theorem 5.4 (Farkas’ Lemma - Form 2) Given and , exactly one of the following statements is true. 1. such that Ax 2. ,y = 0 , and b> Proofs. Left to the reader (homework 2). Duality Theorem using Farkas’ Lemma. Let be the program min( Ax b,x and be dual program max( c,y 0). Suppose the the dual is feasible and its maximum value is . Let Ax b,c . If has a feasible solution, then must also have a feasible solution with value at most . The LP is also equivalent to Ax b, Suppose for contradiction is infeasible. Then by Farkas’ Lemma (Form 2) there exists 0 such that = 0 and ( This implies λc = 0 and λδ> 0. If = 0, then = 0 and b > 0. Choose 0 such that and Then for > 0, y ) = 0 y 0 (Since 0) y ) = b > so y is a feasile solution of with value greater than , a contradiction. Otherwise we can scale and to make = 1 (since y, 0), so and b> . This means is a solution to with value greater than , a contradiction. Therefore is feasible, so is feasible with value at most . By The Weak Duality Theorem, OPT( ) = = OPT( ). In the next couple of lectures, we will continue to explore duality, and its applications.

This theorem gave us a method to prove that a given 64258ow is optimal simply exhibit a cut with the same value This theorem for 64258ows and cuts in a graph is a speci64257c instance of the LP Duality Theorem which relates the optimal values of LP ID: 23918

- Views :
**206**

**Direct Link:**- Link:https://www.docslides.com/tawny-fly/lecture-lp-duality-in-lecture
**Embed code:**

Download this pdf

DownloadNote - The PPT/PDF document "Lecture LP Duality In Lecture we saw t..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

Page 1

Lecture 5 LP Duality In Lecture #3 we saw the Max-ﬂow Min-cut Theorem which stated that the maximum ﬂow from a source to a sink through a graph is always equal to the minimum capacity which needs to be removed from the edges of the graph to disconnect the source and the sink. This theorem gave us a method to prove that a given ﬂow is optimal; simply exhibit a cut with the same value. This theorem for ﬂows and cuts in a graph is a speciﬁc instance of the LP Duality Theorem which relates the optimal values of LP problems. Just like the Max-ﬂow Min-cut Theorem, the LP Duality Theorem can also be used to prove that a solution to an LP problem is optimal. 5.1 Primals and Duals Consider the following LP = max(2 + 3 s.t. 4 + 8 12 + 2 ,x In an attempt to solve we can produce upper bounds on its optimal value. Since 2 + 3 + 8 12, we know OPT( 12. Since 2 + 3 (4 + 8 6, we know OPT( 6. Since 2 + 3 ((4 + 8 ) + (2 )) 5, we know OPT( 5. In each of these cases we take a positive linear combination of the constraints, looking for better and better bounds on the maximum possible value of 2 + 3 . We can formalize Lecturer: Anupam Gupta. Scribe: Timothy Wilson.

Page 2

LECTURE5.LPDUALITY this, letting ,y ,y be the coeﬃcients of our linear combination. Then we must have + 2 + 3 + 2 ,y ,y and we seek min(12 + 3 + 4 This too is an LP! We refer to this LP as the dual and the original LP as the primal. The actual choice of which problem is the primal and which is the dual is not important since the dual of the dual is equal to the primal. We designed the dual to serve as a method of constructing an upperbound on the optimal value of the primal, so if is a feasible solution for the dual and is a feasible solution for the primal, then 2 + 3 12 + 3 + 4 . If we can ﬁnd two feasible solutions that make these equal, then we know we have found the optimal values of these LP. In this case the feasible solutions ,x and 16 ,y = 0 ,y give the same value 4.75, which therefore must be the optimal value. 5.1.1 Generalization In general, the primal LP = max( Ax b,x ,x corresponds to the dual LP, = min( c,y ,y where is an matrix. When there are equality constraints or variables that may be negative, the primal LP = max( s.t. for for 0 for for corresponds to the dual LP = min( s.t. 0 for for for for

Page 3

LECTURE5.LPDUALITY 5.2 The Duality Theorem The Duality Theorem will show that the optimal values of the primal and dual will be equal (if they are ﬁnite). First we will prove our earlier assertion that the optimal solution of a dual program gives a bound on the optimal value of the primal program. Theorem 5.1 (The Weak Duality Theorem) Let = max( Ax b,x ,x and let be its dual LP, min( c,y ,y . If is a feasible solution for and is a feasible solution for , then Proof. ) (Since feasible for and 0) = ( Ax (Since is feasible for and 0) From this we can conclude that if is unbounded (OPT( ) = ), then is infeasible. Similarly, if is unbounded (OPT( ) = ), then is infeasible. Therefore we have the following table of possibilities for the feasibility of and Unbounded Infeasible Feasible Unbounded no yes no Infeasible yes ??? ??? Feasible no ??? ??? The Duality Theorem allows us to ﬁll in the remaining four places in this table. Theorem 5.2 (Duality Theorem for LPs) If and are a primal-dual pair of LPs, then one of these four cases occurs: 1. Both are infeasible. 2. is unbounded and is infeasible. 3. is unbounded and is infeasible. 4. Both are feasible and there exist optimal solutions x,y to and such that We have already seen cases 2 and 3 as simple consequences of the Weak Duality Theorem. The ﬁrst case can easily be seen to occur: a simple example takes to be a matrix, to be strictly negative, and to be strictly positive). Therefore the only remaining case of interest is case 4. Geometric Proof. Let be the program max( Ax b,x ) and be dual program min( c,y 0). Suppose is an optimal feasible solution for . Let for be all the constraints tight at . We claim that the objective function vector is contained in the cone , generated by the vectors

Page 4

LECTURE5.LPDUALITY a x <= b a x <= b 2 2 Figure 5.1: The objective vector lies in the cone spanned by the constraint vectors Suppose for contradiction that does not lie in this cone. Then there must exist a separating hyperplane between and : i.e., there exists a vector such that for all , but d> 0. Now consider the point d for some tiny > 0. Note the following: For small enough , the point satisifes the constraints Az . Consider for 6 : since this constraint was not tight for , we won’t violate it if is small enough. And for with we have a a since > 0 and 0. The objective function value increases since c d>c This contradicts the fact that was optimal. Therefore the vector lies within the cone made of the normals to the constraints, so is a positive linear combination of these normals. Choose for so that , and set = 0 for 6 We know 0. Therefore is a solution to the dual with , so by The Weak Duality Theorem, OPT( ) = OPT( ). A somewhat more rigorous proof not relying on our geometric intuition that there should be a separating hyperplane between a cone and a vector not spanned by the cone relies on a lemma by Farkas that often comes in several forms. The forms we shall use are as follows Theorem 5.3 (Farkas’ Lemma (1894) - Form 1) Given and , exactly one of the following statements is true.

Page 5

LECTURE5.LPDUALITY 1. such that Ax 2. such that and b< Theorem 5.4 (Farkas’ Lemma - Form 2) Given and , exactly one of the following statements is true. 1. such that Ax 2. ,y = 0 , and b> Proofs. Left to the reader (homework 2). Duality Theorem using Farkas’ Lemma. Let be the program min( Ax b,x and be dual program max( c,y 0). Suppose the the dual is feasible and its maximum value is . Let Ax b,c . If has a feasible solution, then must also have a feasible solution with value at most . The LP is also equivalent to Ax b, Suppose for contradiction is infeasible. Then by Farkas’ Lemma (Form 2) there exists 0 such that = 0 and ( This implies λc = 0 and λδ> 0. If = 0, then = 0 and b > 0. Choose 0 such that and Then for > 0, y ) = 0 y 0 (Since 0) y ) = b > so y is a feasile solution of with value greater than , a contradiction. Otherwise we can scale and to make = 1 (since y, 0), so and b> . This means is a solution to with value greater than , a contradiction. Therefore is feasible, so is feasible with value at most . By The Weak Duality Theorem, OPT( ) = = OPT( ). In the next couple of lectures, we will continue to explore duality, and its applications.

Today's Top Docs

Related Slides