Nicholas Ruozzi University of Texas at Dallas Where Were Going Multivariable calculus tells us where to look for global optima but our goal is to design algorithms that can actually find one ID: 1028949
Download Presentation The PPT/PDF document "Convex Sets and Functions" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
1. Convex Sets and FunctionsNicholas RuozziUniversity of Texas at Dallas
2. Where We’re GoingMultivariable calculus tells us where to look for global optima, but our goal is to design algorithms that can actually find one!This is inherently different than combinatorial optimization, the true solution may be given by a real number, e.g., , which we might not even be able to represent in closed form!Our notions of what it means to solve these problems needs to be differentWhat makes an optimization problem easy? What makes an optimization problem hard? 2
3. Convex SetsA set of points is convex if for every pair of points , any point on the line segment between and is also in That is, if , then for all 3Convex SetNot a Convex Set
4. Convex SetsA set of points is convex if for every pair of points , any point on the line segment between and is also in That is, if , then for all 4Convex SetNot a Convex Set
5. Examples of Convex SetsLine Segments: for some Lines, planes, hyperplanes, etc. also define convex sets 5
6. Examples of Convex SetsHalf spaces: for some and 6
7. Examples of Convex SetsBalls of Radius : for some This is called the Euclidean norm or Euclidean distance because is equal to the length of the line segment between the points and 7
8. Examples of Convex SetsBalls of Radius : for some 8
9. Convex CombinationsWe say that is a convex combination of the points if for some choice of such that Let be the set of all points that can be obtained as a convex combination of the In the special case , is just a line segment is a convex set called the convex hull of the where and 9
10. Convex CombinationsWe say that is a convex combination of the points if for some choice of such that Let be the set of all points that can be obtained as a convex combination of the is a convex set called the convex hull of the 10
11. Convex FunctionsA function is convex if is a convex set and for all and is called concave if is convex
12. Convex FunctionsA function is convex if is a convex set and for all and
13. Convex FunctionsA twice continuously differentiable function with is convex if is a convex set and for all This is likely the definition that you saw in high school Calculus
14. Convex FunctionsWhich of the following functions are convex? 14
15. Convex FunctionsWhich of the following functions are convex? 15
16. Operations Preserving ConvexityNonnegative weighted sums of convex functions are convex, i.e., if and are convex functions and then is a convex function. 16
17. Operations Preserving ConvexityComposition with an affine function, i.e., if is a convex function and , then given by is a convex function. 17
18. Operations Preserving ConvexityPointwise maximum of convex functions are convex, i.e., if and are convex functions then is a convex function. 18
19. Operations Preserving ConvexityPointwise supremum, i.e., if is convex in for each fixed then is a convex function. 19
20. Operations Preserving ConvexityMinimization of a convex function, i.e., if is jointly convex in and for some convex set , then is a convex function. 20
21. Other Characterizations of ConvexityA differentiable function is convex on a convex set if and only iffor all Proof: see Boyd 3.3.1 21
22. Other Characterizations of ConvexityA differentiable function is convex on a convex set if and only iffor all 22First order Taylor expansion: this is the best linear approximator of at the point
23. Other Characterizations of ConvexityA differentiable function is convex on a convex set if and only iffor all 23
24. Other Characterizations of ConvexityA differentiable function is convex on a convex set if and only iffor all 24Image: Lane Vosbury, Seminole State College
25. Other Characterizations of ConvexityA twice continuously differentiable function is convex on a convex set if and only if its Hessian matrix is positive semidefinite at all is positive semidefinite if all of its eigenvalues are nonnegativeEquivalently, for all vectors 25Note: we will have a more formal refresher of eigenvalues/eigenvectors later in the course