Global Optimization General issues in global optimization

Global Optimization

General issues in global optimizationClassification of algorithmsThe DIRECT algorithmDivided rectanglesExploration and Exploitation as bi-objective optimizationApplication to High Speed Civil Transport

Global optimization issues

Optimization problem is NP-hardNo-free-lunch theorem (Wolpert and Macready)No single algorithm can do well on all problemsIf an algorithm is improved for one problem, it will suffer for others.

Great opportunity for engineers to use problem knowledge to tailor algorithms.

Big headache for journals because they get many worthless new algorithms.

Global Optimization

Global optimization algorithms by Thomas Weise

Classification of global optimization algorithms

The most popular algorithms imitate natural processes, including genetic algorithms, particle swarm optimization, ant colony optimization, and simulated annealing.They rely on randomness for exploration, so every time you run them you may get a different result.DIRECT is an example of a systematic deterministic exploration of the design space.

Adaptive sampling algorithms based on surrogates, such as EGO, are gaining popularity.

Problems global optimization

What does it mean that global optimization is NP hard?What is the no-free-lunch principle, and how does it affect engineering optimization.When should we use local optimizers to solve global optimization problems and when we should not?Answers in the notes page.

Lipschitzian

Optimization

DIRECT was inspired by

Lipschitzian

optimization.

Optimizer

divides space into boxes and samples the vertices of each

A

box is further divided based on

estimated maximum

rate of change of the function,

K (

Lipschitz

constant)

DIRECT

The

function value at the middle of each box and it’s largest dimension are used to determine potentially optimal boxes

Each potentially optimal box is divided

Lipschitzian

optimization for all possible

Lipschitz

constants

DIRECT Box Division

.

Exploration vs. Exploitation

DIRECT uses convex hull of box sizes to balance exploitation vs. explorationWith enough function evaluations every region in design space will be exploredThis is clearly not feasible for high dimensional spaces

Cox’s paper compares DIRECT to repeated local optimization with random start

Problems DIRECT

What is the Lipschtiz constant? What value would be appropriate for the function x2+x in the interval (-2,2)? Solution

Invent function values at every point where the function is evaluated in Slide 8 that are consistent with the diagram

.

Solution

What are the meanings of the term exploration and exploitation in the context of global optimization

?

Solution

Optimization of a High Speed Civil Transport

26 design variables defining the shape of the wing and fuselage and tail and the flight trajectoryLong range (NY Tokyo type flight) with 250 passengers for minimum takeoff gross weightConstraints on takeoff and landing maneuvers, engine out conditions.CFD analysis plus structural optimization needed for each design.

Results

Results

Results

Results