Lecture 2 Convex Set CK Cheng Dept of Computer Science and Engineering University of California San Diego Convex Optimization Problem 2 is a convex function For Subject to ID: 1028950
Download Presentation The PPT/PDF document "1 CSE203B Convex Optimization" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
1. 1CSE203B Convex OptimizationLecture 2 Convex SetCK ChengDept. of Computer Science and EngineeringUniversity of California, San Diego
2. Convex Optimization Problem:2 is a convex functionFor , Subject to is a convex set
3. Convex Optimization Problem:3A. Convex Function Definition: B. Convex Set Definition: We have
4. Chapter 2 Convex SetSet Convexity and SpecificationConvexityImplicit vs. Explicit EnumerationConvex Set Terms and DefinitionsSeparating HyperplanesDual Cones4
5. Set Convexity and Specification: Convexity5A set is convex if we have Examples:
6. Set Convexity and Specification: Convexity6A set is convex if we have Remark: Most used sets in the classScalar set: Vector set: Matrix set: Set S is convex if every two points in S has the connected straight segment in the set.For convex sets and : is also convex
7. 1. Set Convexity and Specification: Set Specification via Implicit or Explicit EnumerationImplicit ExpressionExplicit Enumeration 7 Implicit Expression:ConstraintsMin Subject to Explicit Expression: EnumerationMin
8. 1. Implicit vs Explicit Enumeration of Convex SetImplicit Expression8 Example: {𝑥|𝐴𝑥≤𝑏} , , Remark: Simultaneous linear constraints imply AND, Intersection of the constraints
9. is a convex setProof: 9Given two vectors , For ( The inequality implies By definition, set is convex.Remark: Simultaneous linear constraints imply AND, Intersection of the constraintsLinear constraints constitute a convex set. 1. Implicit vs Explicit Enumeration of Convex Set
10. 1. Specification of Convex Set: Implicit ExpressionExample:10
11. 1. Implicit vs Explicit Enumeration of Convex SetExample:11
12. 1. Specification of Set: Explicit ExpressionExplicit Enumeration12 Example:
13. 1. Specification of Set: Explicit ExpressionImplicit and Explicit Conversion13Example:
14. 1. Implicit vs Explicit Enumeration of Convex Set14Remark: Implicit Expression: Constraints of the problem formulationExplicit Enumeration: Formulation of the objective functionThe interchange may not be trivial. Every vector in matrix is a solution of n equations in constraint
15. 1. Implicit vs Explicit Enumeration of Convex SetExplicit Enumeration15 is convex if
16. 1. Implicit vs Explicit Enumeration of Convex SetStatement: . Proof: 16Given ,Let us set Therefore, by definition . We then have
17. 2. Convex Set Terms and Definitions17Definitions: Affine Set, Cone, and Convex Hull,function and two conditions i. ii. iii.
18. Definitions: Hyperplane and Half Spaces18 = 2. Sets and Definitions
19. Ex : 3 variables Ex : 4 variables (1) degrees of freedom : . (2) all vectors ( are orthogonal to direction , i.e. Proof:Exercise: Conceptually (visually) construct hyperplane. 2. Sets and Definitions: Hyperplanes19
20. Hyperplane : as an Equal potential of cost function is the sensitivity or cost of vector 2. Sets and Definitions: Hyperplanes20
21. Hyperplane : as a linearized constraint Remark : Hyperplane is one key building block of convex optimization. (theory, algorithms, applications for machine learning, deep learning, …)Each hyperplane separates the space into two half spaces.If hyperplanes can separate the space into disjoint regions. 2. Sets and Definitions21
22. Ⅴ. Polyhedra (plural) : Poly (many) Hedron (face) 2. Sets and Definitions22
23. ⅤI. Matrix Space : Positive Semidefinite Cone Symmetric Matrix Ex: 2. Sets and Definitions23
24. Proof : Let We have 2. Sets and Definitions24
25. (Classification, Optimization, Duality)Theorem : Given two convex sets in Actually i.e. For 3. Separating Hyperplane 25
26. Proof : should be true By contradiction, suppose that Then we can show that is close to for Because We have 3. Separating Hyperplane 26
27. Given set and a point on the boundary of is called supporting hyperplane of C if .Supporting Hyperplane Theorem: For any nonempty convex set and a point on the boundary of There exists a support hyperplane to at .Proof: A separating hyperplane that separates interior and {} is a supporting hyperplane. 3. Supporting Hyperplane 27
28. Given Cone (i.e. )Ex: 1. 2. 3. 4. 4. Dual Cones28
29. Show that Proof : Statement L=>R By contradiction, suppose that We can find Setting t=1, then we have R=>L Given Thus, 294. Dual Cones
30. Examples 304. Dual Cones
31. The polyhedral cone |0} has its dual cone Proof : By definition Thus Let , we have if Ex: i.e. i.e. 314. Dual Cones
32. Remark: (1) cone can be translated to (2) If Then , i.e. is a supporting hyperplane of cone (3) Suppose that the current feasible search region is at corner and is a local feasible region of a convex set If , i.e. Then is an optimal solution 324. Dual Cones