We can choose the amount of stirfry ounce and boba fluid ounces Healthy Squad Goals 2000 Calories 2500 Sugar 100 g Calcium 700 mg Food Cost Calories Sugar ID: 813359
Download The PPT/PDF document "Warm-up: What to eat? We are trying heal..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Warm-up: What to eat?
We are trying healthy by finding the optimal amount of food to purchase.We can choose the amount of stir-fry (ounce) and boba (fluid ounces).
Healthy Squad Goals2000 Calories 2500Sugar 100 gCalcium 700 mg
FoodCostCaloriesSugarCalciumStir-fry (per oz)1100320Boba (per fl oz)0.550470
What is the cheapest way to stay “healthy” with this menu?
How much
stir-fry
(ounce) and
boba
(fluid ounces) should we buy?
Slide2Announcements
Assignments:P2Due Thu 10/3, 10 pmP3Will be released later todayDue Thu 10/17, 10 pm
HW5 (written)Released Tue 10/1Due Tue 10/8, 10 pmSat 10/5, 10 pm
Slide3AI: Representation and Problem Solving
Optimization & Linear ProgrammingInstructors: Fei Fang & Pat Virtue
Slide credits: CMU AI, http://ai.berkeley.edu
Slide4Learning Objectives
Formulate a problem as a Linear Program (LP)Convert a LP into a required form, e.g., inequality form
Plot the graphical representation of a linear optimization problem with two variables and find the optimal solutionUnderstand the relationship between optimal solution of an LP and the intersections of constraintsDescribe and implement a LP solver based on vertex enumerationDescribe the high-level idea of Simplex algorithmNext Lecture
Slide5Recap
What have we learned so far?What do they have in common?
This lecture: Continuous spaceGeneral formulation
Slide6Recap
What have we learned so far?Search: Depth/Breadth-first search, A* search, local searchConstraint Satisfaction Problem: 8-queen, graph coloringLogic and Planning: Propositional Logic, SAT, First-order LogicWhat do they have in common?
Variables/Symbols, Finite optionsThis lecture: Move to continuous space (with connections to the discrete space)Provide a general formulation that can be used to represent many of the previously seen problems
Slide7Focus of Today: (Linear) Optimization Problem
s.t.
Problem Description
Graphical RepresentationLinear Program Formulation
Notation Alert!
Linear Programming
Slide8Diet Problem: What to eat?
We are trying healthy by finding the optimal amount of food to purchase.We can choose the amount of stir-fry (ounce) and boba (fluid ounces).
Healthy Squad Goals2000 Calories 2500Sugar 100 gCalcium 700 mg
FoodCostCaloriesSugarCalciumStir-fry (per oz)1100320Boba (per fl oz)0.550470
What is the cheapest way to stay “healthy” with this menu?
How much
stir-fry
(ounce) and
boba
(fluid ounces) should we buy?
Slide9Problem Formulation
Can we formulate it as a Constraint Satisfaction Problem?Variable:Domain:Constraint:
Healthy Squad Goals2000 Calories 2500Sugar 100 gCalcium 700 mg
FoodCostCaloriesSugarCalciumStir-fry (per oz)1100320Boba (per fl oz)0.550470
What is the cheapest way to stay “healthy” with this menu?
How much
stir-fry
(ounce) and
boba
(fluid ounces) should we buy?
What are the issues with this CSP formulation?
Slide10Problem Formulation
Can we formulate it as a Constraint Satisfaction Problem?Variable:
(ounces for stir-fry), (ounces for boba)Domain: Constraint: Implicit:
Healthy Squad Goals2000
Calories 2500Sugar 100 gCalcium 700 mg Food
Cost
Calories
Sugar
Calcium
Stir-fry
(per oz)
1
100
3
20
Boba
(per
fl
oz)
0.5
50
4
70
What is the cheapest way to stay “healthy” with this menu?
How much
stir-fry
(ounce) and
boba
(fluid ounces) should we buy?
,
What are the issues with this CSP formulation?
Slide11Problem Formulation
Optimization problem: Finding the best solution from all feasible solutions
s.t.
Notation Alert!
Healthy Squad Goals
2000
Calories
2500
Sugar
100 g
Calcium
700 mg
Food
Cost
Calories
Sugar
Calcium
Stir-fry
(per oz)
1
100
3
20
Boba
(per
fl
oz)
0.5
50
4
70
Objective
Variable
Domain
Constraint
From CSP to Optimization Problem
Slide12Problem Formulation
Optimization problem: Finding the best solution from all feasible solutions
s.t.
Notation Alert!
Healthy Squad Goals
2000
Calories
2500
Sugar
100 g
Calcium
700 mg
Food
Cost
Calories
Sugar
Calcium
Stir-fry
(per oz)
1
100
3
20
Boba
(per
fl
oz)
0.5
50
4
70
Objective
(Optimization) Variable
Domain
Constraint
Optimization Objective
Can be represented as constraints
From CSP to Optimization Problem
Slide13Problem Formulation
Formulate Diet Problem as an optimization problem
s.t.
Healthy Squad Goals
2000 Calories 2500Sugar 100 gCalcium 700 mg
Food
Cost
Calories
Sugar
Calcium
Stir-fry
(per oz)
1
100
3
20
Boba
(per
fl
oz)
0.5
50
4
70
Variable:
Objective:
Constraints:
Slide14Problem Formulation
Formulate Diet Problem as an optimization problem
s.t.
Healthy Squad Goals
2000 Calories 2500Sugar 100 gCalcium 700 mg
Food
Cost
Calories
Sugar
Calcium
Stir-fry
(per oz)
1
100
3
20
Boba
(per
fl
oz)
0.5
50
4
70
Variable:
.
: ounces for stir-fry,
: ounces for
boba
Objective:
Constraints:
𝑖𝑛 𝑟𝑒𝑞𝑢𝑖𝑟𝑒𝑑 𝑟𝑎𝑛𝑔𝑒
Can be ignored in this problem. Why?
Slide15Problem Formulation
Formulate Diet Problem as an optimization problem
Healthy Squad Goals2000 Calories 2500Sugar 100 gCalcium 700 mg
s.t. 𝑖𝑛 𝑟𝑒𝑞𝑢𝑖𝑟𝑒𝑑 𝑟𝑎𝑛𝑔𝑒
.
: ounces for stir-fry,
: ounces for
boba
What is the expression of
Food
Cost
Calories
Sugar
Calcium
Stir-fry
(per oz)
1
100
3
20
Boba
(per
fl
oz)
0.5
50
4
70
Slide16Problem Formulation
Formulate Diet Problem as an optimization problem
Healthy Squad Goals2000 Calories 2500Sugar 100 gCalcium 700 mg
s.t. 𝑖𝑛 𝑟𝑒𝑞𝑢𝑖𝑟𝑒𝑑 𝑟𝑎𝑛𝑔𝑒
Food
Cost
Calories
Sugar
Calcium
Stir-fry
(per oz)
1
100
3
20
Boba
(per
fl
oz)
0.5
50
4
70
.
: ounces for stir-fry,
: ounces for
boba
What is the expression of
Slide17Problem Formulation
Formulate Diet Problem as an optimization problem
Healthy Squad Goals2000 Calories 2500Sugar 100 gCalcium 700 mg
s.t.
Food
Cost
Calories
Sugar
Calcium
Stir-fry
(per oz)
1
100
3
20
Boba
(per
fl
oz)
0.5
50
4
70
Slide18Piazza Poll 1
Is
, a feasible solution of the following optimization problem?
Healthy Squad Goals2000
Calories 2500Sugar 100 gCalcium 700 mg
s.t.
Food
Cost
Calories
Sugar
Calcium
Stir-fry
(per oz)
1
100
3
20
Boba
(per
fl
oz)
0.5
50
4
70
Slide19Linear Programming
Linear program in Inequality Form
s.t. Linear Programming: Technique for the optimization of a linear objective function subject to linear equality and linear inequality constraints
s.t.
Example Linear program
Slide20Recap of Linear Algebra
,
What is
? ? What does mean? (Note: It is fine if you directly write
)
Slide21Recap of Linear Algebra
,
What is
? ?
What does
mean?
(Note: It is fine if you directly write
)
Slide22Recap of Linear Algebra
,
What is ?
What is
?
Slide23Recap of Linear Algebra
,
What is ?
What is
?
Slide24Problem Formulation
s.t.
s.t.
Convert it into the following inequality form, what should
,
, and
be?
Slide25Problem Formulation
s.t.
s.t.
with
and
Convert it into the following inequality form, what should
,
, and
be?
Slide26Problem Formulation
s.t.
s.t.
Convert it into the following inequality form, what should
,
, and
be?
Slide27Problem Formulation
s.t.
s.t.
Convert it into the following inequality form, what should
,
, and
be?
Slide28Piazza Poll 2
What has to increase to add more nutrition constraints?
s.t.
Select all that apply
length length height width length
Healthy Squad Goals
2000
Calories
2500
Sugar
100 g
Calcium
700 mg
…(More Constraints)
Food
Cost
Calories
Sugar
Calcium
Stir-fry
(per oz)
1
100
3
20
Boba
(per
fl
oz)
0.5
50
4
70
Slide29Piazza Poll 2
What has to increase to add more nutrition constraints?
s.t.
Healthy Squad Goals
2000
Calories
2500
Sugar
100 g
Calcium
700 mg
Sugar
g
Food
Cost
Calories
Sugar
Calcium
Stir-fry
(per oz)
1
100
3
20
Boba
(per
fl
oz)
0.5
50
4
70
Slide30Piazza Poll 2
What has to increase to add more nutrition constraints?
s.t.
Healthy Squad Goals
2000
Calories
2500
Sugar
100 g
Calcium
700 mg
Sugar
g
Food
Cost
Calories
Sugar
Calcium
Stir-fry
(per oz)
1
100
3
20
Boba
(per
fl
oz)
0.5
50
4
70
Slide31Question
What has to increase to add more menu items?
s.t.
Select all that apply
length length height width length
Healthy Squad Goals
2000
Calories
2500
Sugar
100 g
Calcium
700 mg
Food
Cost
Calories
Sugar
Calcium
Stir-fry
(per oz)
1
100
3
20
Boba
(per
fl
oz)
0.5
50
4
70
New Item
…
…
…
…
Slide32Question
What has to increase to add more menu items?
s.t.
Healthy Squad Goals
2000 Calories 2500Sugar 100 gCalcium 700 mg
Food
Cost
Calories
Sugar
Calcium
Stir-fry
(per oz)
1
100
3
20
Boba
(per
fl
oz)
0.5
50
4
70
Beef (per
oz
)
2
80
1
30
Slide33Question
What has to increase to add more menu items?
s.t.
Healthy Squad Goals
2000 Calories 2500Sugar 100 gCalcium 700 mg
Food
Cost
Calories
Sugar
Calcium
Stir-fry
(per oz)
1
100
3
20
Boba
(per
fl
oz)
0.5
50
4
70
Beef (per
oz
)
2
80
1
30
Slide34Question
If
, which of the following also equals ? s.t.
Select all that applylength
length
length
Slide35Linear Programming
Different forms
s.t.
Inequality form
Important to pay attention to form!
s.t.
General form
s.t.
A
Standard form
Can switch between formulations!
Note: Different books have different definitions. We will refer to the forms in this slide (also consistent with B&V book).
Slide36Focus of Today: (Linear) Optimization Problem
s.t.
Problem Description
Graphical RepresentationLinear Program FormulationLinear Programming
Slide37Shape of Linear Equality
Geometry / Algebra QuestionWhat is the equality present this line?
Slide38Shape of Linear Equality
Geometry / Algebra QuestionWhat is the equality present this line?
To make sure the line intersects with the axes at (3,0) and (0,2)
Slide39Shape of Linear Inequality
Geometry / Algebra QuestionWhat shape does this inequality represent?
Slide40Shape of Linear Inequality
Geometry / Algebra QuestionWhat shape does this inequality represent?
Half Plane
Line
Slide41Shape of Linear Inequality
Geometry / Algebra QuestionWhat shape does these inequalities jointly represent?
Slide42Shape of Linear Inequality
Geometry / Algebra QuestionWhat shape does these inequalities jointly represent?
Intersection of half planes
Could be polyhedron
Could be empty
Slide43Piazza Poll 3
What is the relationship between the half plane:
and the vector:
Feasible
Infeasible
A
B
C
D
Slide44Lowering Cost
Given the cost vector
and initial point ,Which unit vector step will cause to have the lowest cost
?
A
B
C
D
Slide45Lowering Cost
Given the cost vector
and initial point ,Which unit vector step will cause to have the lowest cost
?
when
and
have opposite directions
Slide46Cost Contours
Given the cost vector
where will = 0 ?
Slide47Cost Contours
Given the cost vector
where will = 0 ?
Slide48Cost Contours
Given the cost vector
where will = 0 ? = 1 ? = 2 ? = -1 ?
= -2 ?
Slide49Cost Contours
Given the cost vector
where will = 0 ? = 1 ? = 2 ? = -1 ?
= -2 ?
= 0
= 1
= 2
= -1
Slide50Optimizing Cost
Given the cost vector
, which point in the red polyhedron below can minimize ?
= 0
= 1
= 2
= -1
Slide51Optimizing Cost
Given the cost vector
, which point in the red polyhedron below can minimize ?
= 0
= 1
= 2
= -1
Optimal point
Slide52LP Graphical Representation
s.t.
Inequality form
Feasible Region
Objective Function
Slide53Piazza Poll 4
s.t.
True or False: An minimizing LP with exactly one constraint, will always have a minimum objective of .
Slide54Piazza Poll 4
s.t.
True or False: An minimizing LP with exactly one constraint, will always have a minimum objective value of .
False:
or
Slide55Focus of Today: (Linear) Optimization Problem
s.t.
Problem Description
Graphical RepresentationLinear Program FormulationLinear Programming
Slide56Warm-up: What to eat?
We are trying healthy by finding the optimal amount of food to purchase.We can choose the amount of stir-fry (ounce) and boba (fluid ounces).
Healthy Squad Goals2000 Calories 2500Sugar 100 gCalcium 700 mg
FoodCostCaloriesSugarCalciumStir-fry (per oz)1100320Boba (per fl oz)0.550470
What is the cheapest way to stay “healthy” with this menu?
How much
stir-fry
(ounce) and
boba
(fluid ounces) should we buy?
Slide57Healthy Squad Goals
2000
Calories 2500Sugar 100 gCalcium 700 mg
s.t.
What is the feasible region?
What is the optimal solution?
Slide58Healthy Squad Goals
2000
Calories 2500Sugar 100 gCalcium 700 mg
s.t.
What is the feasible region?
What is the optimal solution?
Optimal point
Slide59Healthy Squad Goals
Calories
2500Sugar 100 gCalcium 700 mg
s.t.
What is the feasible region?
What is the optimal solution?
Slide60Healthy Squad Goals
Calories
2500Sugar 100 gCalcium 700 mg
s.t.
What is the feasible region?
What is the optimal solution?
Optimal point
Slide61Healthy Squad Goals
Calories
Sugar 100 g
s.t.
What is the feasible region?
What is the optimal solution?
Slide62Healthy Squad Goals
Calories
Sugar 100 g
s.t.
What is the feasible region?
What is the optimal solution?
Problem unbounded!
Slide63Healthy Squad Goals
Calories
Sugar 100 g
s.t.
What is the feasible region?
What is the optimal solution?
Slide64Healthy Squad Goals
Calories
Sugar 100 g
s.t.
What is the feasible region?
What is the optimal solution?
Optimal point
Slide65Healthy Squad Goals
2000
Calories 2500Sugar 100 gCalcium 700 mg
s.t.
What is the feasible region?
What is the optimal solution?
Slide66Healthy Squad Goals
2000
Calories 2500Sugar 100 gCalcium 700 mg
s.t.
What is the feasible region?
What is the optimal solution?
Infeasible!
Slide67Solving an LP
If LP is feasible and bounded, at least one solution is at feasible intersections of constraint boundaries!
AlgorithmsVertex enumeration: Find all vertices of feasible region (feasible intersections), check objective value
Slide68Solving an LP
But, how do we find the intersections?
s.t.
Calorie min
Calorie max
Sugar
Calcium
A point is the intersection of two lines
Pick two lines: two rows in
matrix time
equals the corresponding rows in
Intersection
Not any pair of rows can lead to an intersection
Slide69Solving an LP
AlgorithmsVertex enumeration: Find all vertices of feasible region (feasible intersections), check objective value
Simplex: Start with an arbitrary vertex. Iteratively move to a best neighboring vertex until no better neighbor foundSimplex is most similar to which Local Search algorithm?Hill Climbing!If LP is feasible and bounded, at least one solution is at feasible intersections of constraint boundaries!
Slide70Solving an LP
AlgorithmsVertex enumeration: Find all vertices of feasible region (feasible intersections), check objective value
Simplex: Start with an arbitrary vertex. Iteratively move to a best neighboring vertex until no better neighbor found
If LP is feasible and bounded, at least one solution is at feasible intersections of constraint boundaries!
Slide71Piazza Poll 5
Simplex Algorithm is most similar to which search algorithm?
IntersectionA: Depth First SearchB: Random WalkC: Hill ClimbingD: Beam SearchSimplex: Start with an arbitrary vertex. Iteratively move to a best neighboring vertex until no better neighbor found
Slide72Solving an LP
But, how do we find a neighboring intersection?
s.t.
Calorie min
Calorie max
Sugar
Calcium
Intersection determined by: two rows in
matrix time
equals the corresponding rows in
Intersection
A neighboring intersection only have one different row
Neighboring intersection
Neighboring intersection
Slide73Focus of Today: (Linear) Optimization Problem
s.t.
Problem Description
Graphical RepresentationLinear Program FormulationLinear Programming
Slide74“Marty
, you’re not thinking fourth-dimensionally”
Slide75Shapes in higher dimensions
How do these linear shapes extend to 3-D, N-D?
2-D 3-D N-D
Slide76What are intersections in higher dimensions?
How do these linear shapes extend to 3-D, N-D?
s.t.
Calorie min
Calorie max
Sugar
Calcium
Slide77How do we find intersections in higher dimensions?
s.t.
Calorie min
Calorie max
Sugar
Calcium
Still looking at subsets of rows in the
matrix
Rank(A[1:2,:])=?
Rank(A[(1,3),:])=?
Slide78Summary
Linear optimization problem handles continuous space but is closely connected to discrete space (only need to consider vertices)For a problem with two variables, graphical representation is helpfulLP is a special class of optimization problems
Optimization ProblemsConvex ProblemsLinear ProgrammingGradient DescentSimplex
Slide79Gradient Descent/Ascent
Find
that minimize / maximizes ?Gradient descent / ascentUse gradient to find best directionUse the magnitude of the gradient to determine how big a step you move Value space of variablesConvex Problems: Any local optima is global optima. Therefore, for convex problems, gradient descent/ascent leads to globally optimal solution.