J McCalley 1 Realtime Electricity markets and tools Dayahead SCUC and SCED SCED Minimize f x s ubject to h x c g x lt b BOTH LOOK LIKE THIS SCUC x contains discrete amp continuous variables ID: 749591
Download Presentation The PPT/PDF document "EE 458 Introduction to Optimization" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
EE 458Introduction to Optimization
J. McCalley
1Slide2
Real-time
Electricity markets and tools
Day-ahead
SCUC and SCED
SCED
Minimize f(
x
)
s
ubject to
h
(x)=cg(x)< b
BOTH LOOK LIKE THIS
SCUC: x contains discrete & continuous variables.
SCED:x contains only continuous variables.
2Slide3
Optimization Terminology
Minimize f(
x
)
s
ubject toh(x)=cg(x)< bf(x): Objective function x: Decision variablesh(x)=c: Equality constraintg(x)<
b: Inequality constraint
An optimization problem or a mathematical program or a mathematical programming problem.x*: solution3Slide4
Classification of Optimization Problems
http://www.neos-guide.org/NEOS/index.php/Optimization_Tree
Continuous Optimization
Unconstrained Optimization
Bound Constrained Optimization Derivative-Free Optimization Global Optimization Linear Programming Network Flow Problems Nondifferentiable Optimization Nonlinear Programming Optimization of Dynamic Systems Quadratic Constrained Quadratic Programming Quadratic Programming Second Order Cone Programming Semidefinite Programming Semiinfinite Programming Discrete and Integer Optimization
Combinatorial Optimization Traveling Salesman Problem
Integer Programming Mixed Integer Linear Programming Mixed Integer Nonlinear Programming Optimization Under Uncertainty Robust Optimization Stochastic Programming Simulation/Noisy Optimization Stochastic Algorithms Complementarity Constraints and Variational Inequalities Complementarity Constraints Game Theory Linear Complementarity Problems Mathematical Programs with Complementarity Constraints Nonlinear Complementarity Problems Systems of Equations Data Fitting/Robust Estimation Nonlinear Equations
Nonlinear Least Squares Systems of Inequalities Multiobjective Optimization 4Slide5
Convex functions
Definition #1
:
A
function f(x) is convex in an interval if its second derivative is positive on that interval.Example: f(x)=x2 is convex since f’(x)=2x, f’’(x)=2>05Slide6
Convex functions
The second derivative test is sufficient but not necessary.
www.ebyte.it/library/docs/math09/AConvexInequality.html
Definition #2
: A
function f(x) is convex if a line drawn between any two points on the function remains on or above the function in the interval between the two points.6Slide7
Convex functions
Definition #2
: A
function f(x) is convex if a line drawn between any two points on the function remains on or above the function in the interval between the two
points.
Is a linear function convex?Answer is “yes” since a line drawn between any two points on the function remains on the function.7Slide8
Convex Sets
Definition #3
:
A set C is convex if a line segment between any two points in C lies in C.
Ex
: Which of the below are convex sets?The set on the left is convex. The set on the right is not.8Slide9
Convex Sets
Definition #3
:
A set C is convex if a line segment between any two points in C lies in C.
S. Boyd and L.
Vandenberghe, “Convex optimization,” Cambridge University Press, 2004.9Slide10
Global vs. local optima
Example
: Solve the
following:
Minimize f(x)=x
2Solution: f’(x)=2x=0x*=0.This solution is a local optimum. It is also the global optimum.Example: Solve the following:Minimize f(x)=x3-17x2+80x-100Solution: f’(x)=3x2-34x+80=0Solving the above results in x=3.33 and x=8. Issue#1: Which is the best solution?Issue#2: Is the best solution the global solution?10Slide11
Global vs. local optima
Example
: Solve the
following:
Minimize f(x)=x
2Solution: f’(x)=2x=0x*=0.This solution is a local optimum. It is also the global optimum.Example: Solve the following:Minimize f(x)=x3-17x2+80x-100Solution: f’(x)=3x2-34x+80=0Solving the above results in x=3.33 and x=8. Issue#1: Which is the best solution?Issue#2: Is the best solution the global solution?11Slide12
Global vs. local optima
Example
: Solve the
following:
Minimize f(x)=x
3-17x2+80x-100Solution: f’(x)=3x2-34x+80=0. Solving results in x=3.33, x=8. Issue#1: Which is the best solution?Issue#2: Is the best solution the global solution?x=8
No! It is unbounded.12Slide13
Convexity & global vs. local optima
When
minimizing a function, if we want to be sure that we can get a global solution via differentiation, we need to impose some requirements on our objective function.
We will also need to impose some requirements on the set of possible values that the solution x* may take. Min f(x)subject toh(x)=cg(x)< b
Definition
: If f(x) is a convex function, and if S is a convex set, then the above problem is a convex programming problem. Definition: If f(x) is not a convex function, or if S is
not a convex set, then the above problem is a non-convex programming problem. 13Slide14
Convex vs. nonconvex
programming problems
The desirable quality of a convex programming problem is that any
locally optimal solution
is also a
globally optimal solution. If we have a method of finding a locally optimal solution, that method also finds for us the globally optimum solution.14The undesirable quality of a non-convex programming problem is that any method which finds a locally optimal solution does not necessarily find the globally optimum solution.MATHEMATICAL PROGRAMMINGConvex
Non-convex
We address convex programming problems for the remainder of these notes, and beyond.We will later address a special form of non-convex programming problems.Slide15
A convex programming problem
15
Two variables with one equality-constraint
Multi-variable with one equality-constraint.
Multi-variable with multiple equality-constraints.
We focus on this one, but conclusions we derive will also apply to the other two. The benefit of focusing on this one is that we can visualize it.Slide16
Contour maps
16
Definition
: A
contour map is a 2-dimensional plane, i.e., a coordinate system in 2
variables,
say, x1, x2, that illustrates curves (contours) of constant functional value f(x1, x2). Example
: Draw the contour map for .
[X,Y] = meshgrid
(-2.0:.2:2.0,-2.0:.2:2.0);Z = X.^2+Y.^2;[c,h]=contour(X,Y,Z);clabel(
c,h);grid;xlabel('x1');ylabel('x2');Slide17
Contour maps and 3-D illustrations
17
Example
: Draw the 3-D surface for
.
[X,Y] =
meshgrid
(-2.0:.2:2.0,-2.0:.2:2.0);
Z = X.^2+Y.^2;surfc(X,Y,Z)xlabel('x1')ylabel('x2')zlabel('f(x1,x2)')
Height is f(x)
Contours
Each
contour of fixed value
f
is the projection onto the
x1-x2
plane of a horizontal slice made of the 3-D figure at a value
f
above the
x1-x2
plane.Slide18
Solving a convex program: graphical analysis
18
Example
: Solve this
convex program:
.
Superimpose this relation on top of the contour plot for f(x
1
,x
2
).
1. f(x
1
,x
2
)
must be minimized, and so we would like the solution to be as close to the origin as possible;
2. The
solution must be on the thick line in the right-hand corner of the plot, since this line represents the equality constraint
.Slide19
Solving a convex program: graphical analysis
19
.
Solution:
Any
contour
f<3
does not intersect the equality constraint;
Any
contour
f>3
intersects the equality constraint at two points
.
The contour f=3 and the equality constraint just touch each other at the point
x
*.
“Just touch”:
The
two curves are
tangent
to one another at the solution point
. Slide20
Solving a convex program: graphical analysis
20
.
The
two curves are
tangent
to one another at the solution point
.
The normal (gradient) vectors of the two curves, at the solution (tangent) point, are parallel.
“Parallel” means that the two vectors have the same direction. We do not know that they have the same magnitude. To account for this, we equate with a “multiplier”
λ
:
This means the following two vectors are parallel:Slide21
Solving a convex program: graphical analysis
21
.
Moving everything to the left:
Alternately:
Performing the gradient operation (taking derivatives with respect to x
1
and x
2
) :
In this problem, we already know the solution, but what if we did not?
Then could we use the above equations to find the solution?Slide22
Solving a convex program: analytical analysis
22
In this problem, we already know the solution, but what if we did not?
Then could we use the above equations to find the solution?
NO! Because we only have 2 equations, yet 3 unknowns: x
1
, x
2
,
λ
.
So we need another equation. Where do we get that equation?
Recall our equality constraint:
h(x
1
, x
2
)-c=0
.
This must be satisfied! Therefore:
Three equations, three unknowns, we can solve.Slide23
Solving a convex program: analytical analysis
23
Observation: The three equations are simply partial derivatives of the function
This is obviously true for the first two equations
,
but it is not so obviously true for the last one. But to see it, observeSlide24
Formal approach to solving our problem
24
Define the
Lagrangian
function:
In a convex programming problem, the “first-order conditions” for finding the solution is given by
OR
Or more compactly
where we have used
x
=(x
1
, x
2
)Slide25
Formal approach to solving our problem
25
Define the
Lagrangian
function:
In a convex programming problem, the “first-order conditions” for finding the solution is given by
OR
Or more compactly
where we have used
x
=(x
1
, x
2
)