/
Menkes van den Briel Menkes van den Briel

Menkes van den Briel - PowerPoint Presentation

pasty-toler
pasty-toler . @pasty-toler
Follow
438 views
Uploaded On 2016-02-19

Menkes van den Briel - PPT Presentation

Member of Research Staff NICTA and ANU menkesnictacomau Combining Linear Programming Based Decomposition Techniques with Constraint Programming CPbased column generation Application Reference ID: 224427

decomposition variables benders programming variables decomposition programming benders column generation cost min problem master integer constraint based capacity demand

Share:

Link:

Embed:

Download Presentation from below link

Download Presentation The PPT/PDF document "Menkes van den Briel" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Slide1

Menkes van den Briel

Member of Research Staff

NICTA and ANU

menkes@nicta.com.au

Combining Linear Programming Based Decomposition Techniques with Constraint ProgrammingSlide2

CP-based column generation

ApplicationReference

Urban transit crew managementT.H.

Yunes., A.V. Moura, C.C. de Souza. Solving very large crew scheduling problems to optimality. Proceedings of ACM symposium on Applied Computing, pages 446-451, 2000.

T.H. Yunes., A.V.

Moura

, C.C. de Souza. Hybrid column generation approaches for urban transit crew management problems. Transportation Science 39(2):273-288, 2005.Travelling tournamentK. Easton, G.L.

Nemhauser, and M.A. Trick. Solving the travelling tournament problem: A combined integer programming and constraint programming approach. Proceedings of Practice and Theory of Automated Timetabling, volume 2740 of Lecture Notes in Computer Science, pages 100-112. Springer, 2002.Two-dimensional bin packing

D. Pisinger, M. Sigurd. Using decomposition techniques and constraint programming for solving the two-dimensional bin-packing problem. Journal on Computing 19(1):36-51, 2007.

Graph coloring

S.

Gualandi

. Enhancing constraint programming-based column generation for integer programs. PhD thesis,

Politechnico

di Milano, 2008.

Constrained cutting stock

T. Fahle, M. Sellmann. Cost based filtering for the constrained knapsack problem. Annals of Operations Research 115(1):73-93, 2002.

Employee timetabling

S. Demassey, G. Pesant, L.M. Rousseau. A cost-regular based hybrid column generation approach. Constraints 11(4):315-333, 2006.

Wireless mesh networks

A. Capone, G.

Carello

, I.

Filippini

, S.

Gualandi

, F.

Malucelli

. Solving a resource allocation problem in wireless mess networks: A comparison between a CP-based and a classical column generation. Networks 55(3):221-233, 2010.

Multi-machine scheduling

R.

Sadykov

, L.A. Wolsey. Integer programming and constraint programming in solving a

multimachine

assignment scheduling problem with deadlines and release dates. Journal on Computing 18(2):209-217, 2006.Slide3

CP-based column generation

ApplicationReference

Airline crew assignment

U. Junker, S.E. Karisch, N. Kohl, B. Vaaben, T.

Fahle, M. Sellmann. A framework for constraint programming based column generation. Proceedings of Principles and Practice of Constraint Programming, volume 1713 of Lecture Notes in Computer Science, pages 261-274, 1999.

T.

Fahle, U. Junker, S.E. Karisch, N. Kohl, M. Sellmann

, B. Vaaben. Constraint programming based column generation for crew assignment. Journal of Hueristics 8(1):59-81, 2002.

M. Sellmann, K. Zervoudakis

, P.

Stamatopoulos

, T.

Fahle

. Crew assignment via constraint programming: integrating column generation and heuristic tree search. Annals of Operations Research 115(1):207-225, 2002.

Vehicle routing with time windows

L.M. Rousseau. Stabilization issues for constraint programming based column generation. Proceedings of Integration of AI and OR techniques in CP for Combinatorial Optimization, volume 3011 of Lecture notes in Computer Science, pages 402-408. Springer, 2004.

L.M. Rousseau, M.

Gendreau

, G.

Pesant

, F.

Focacci

. Solving VRPTWs with constraint programming based column generation. Annals of Operations Research 130(1):199-216, 2004.Slide4

CP-based Benders decomposition

ApplicationReference

Parallel machine scheduling

V. Jain, I.E. Grossmann. Algorithms for hybrid MILP/CP models for a class of optimization problems. INFORMS Journal on Computing 13(4):258-276, 2001.Polypropylene batch scheduling

C. Timpe. Solving planning and scheduling problems with combined integer and constraint programming. OR Spectrum 24(4):431-448, 2002.

Call

center schedulingT. Benoist, E.

Gaudin, B. Rottembourg. Constraint programming contribution to Benders decomposition: A case study. Principles and Practice of Constraint Programming, volume 2470 of Lecture Notes in Computer Science, pages 603-617. Springer, 2002.

Multi-machine schedulingJ.N. Hooker. A hybrid method for planning and scheduling. Principles and Practice of Constraint Programming, volume 3258 of Lecture Notes in Computer Science, pages 305-316. Springer, 2004.

J.N. Hooker. Planning and scheduling to minimize tardiness. Principles and Practice of Constraint Programming, volume 3709 of Lecture Notes in Computer Science, pages 314-327. Springer, 2005.Slide5

CP versus IP

CPIP

VariablesFinite domain

Continuous, Binary, IntegerConstraintsSymbolic:

alldifferentcumulativeLinear,

algebraic:

(+, –, *, =, ≤, ≥)Inference

Constraint propagationLP relaxation

GlobalOptimalLocal

FeasibleSlide6

CP versus IP“MILP is very efficient when the relaxation is tight and models have a structure that can be effectively exploited”“CP works better for highly constrained discrete optimization problems where expressiveness of MILP is a major limitation”“From the work that has been performed, it is not clear whether a general integration strategy will always perform better than either CP or an MILP approach by itself. This is especially true for the cases where one of these methods is a very good tool to solve the problem at hand. However, it is usually possible to enhance the performance of one approach by borrowing some ideas from the other”

Source: Jain and Grossmann, 2001Slide7

OutlineBackgroundIntroduction

Dantzig Wolfe decompositionBenders decompositionConclusionsSlide8

What is your background?Have

implemented Benders and/or Dantzig Wolfe decompositionHave

heard about Benders and/or Dantzig Wolfe decompositionHave seen Bender and/or Dances with WolvesSlide9

Things to take awayA better understanding of how to combine

linear programming based decomposition techniques with constraint programmingA better understanding of column generation, Dantzig Wolfe decomposition and Benders decompositionA whole lot of Python code with example implementationsSlide10

Helpful installations Python 2.6.x or 2.7.x

“Python is a programming language that lets you work more quickly and integrate your systems more effectively”http://www.python.org/getit/Gurobi (Python interface)

“The state-of-the-art solver for linear programming (LP), quadratic and quadratically constrained programming (QP and QCP), and mixed-integer programming (MILP, MIQP, and MIQCP)”http://www.gurobi.com/products/gurobi-optimizer/try-for-yourself

NetworkX“NetworkX is a Python language software package for the creation, manipulation, and study of the structure, dynamics, and functions of complex networks”http://networkx.lanl.gov/download.htmlSlide11

AbbreviationsArtificial Intelligence (AI)Constraint Programming (CP)

Constraint Satisfaction Problem (CSP)Integer Programming (IP)Linear Programming (LP)Mixed Integer Programming (MIP)Mixed Integer Linear Programming (MILP)

Mathematical Programming (MP)Operations Research (OR)Slide12

OutlineBackgroundIntroduction

Dantzig Wolfe decompositionBenders decompositionConclusionsSlide13

What is decomposition?“Decomposition in 

computer science, also known as factoring, refers to the process by which a complex problem or system is broken down into parts that are easier to conceive, understand, program, and maintain”Source: http://en.wikipedia.org/wiki/Decomposition_(computer_science)

Decomposition in linear programming is a technique for solving linear programming problems where the constraints (or variables) of the problem can be divided into two groups, one group of “easy” constraints and another of “hard” constraintsSlide14

“easy” versus “hard” constraintsReferring to the constraints as “easy” and “hard” may be a bit misleading

The “hard” constraints need not be very difficult in themselves, but they can complicate the linear program making the overall problem more difficult to solveWhen the “hard” constraints are removed from the problem, then more efficient techniques could be applied to solve the resulting linear programSlide15

ExampleShortest path problem

(P) Min (i

,j)A

cijxij

s.t.

1 for i = s Source

j:(i

,j)A xij

– 

j

:(

j

,

i

)

A

x

ji

= 0 for

i

N

– {

s

,

t

}

Flow

-1 for

i

=

t

Sink

x

ij

 {0, 1}

Resource constrained shortest path problem

(NP-complete)Min (i,j)A cijxijs.t. 1 for i = s Source j:(i,j)A xij – j:(j,i)A xji = 0 for iN – {s, t} Flow -1 for i = t Sink (i,j)A dijxij ≤ C Capacity xij  {0, 1}

G

= (

N

,

A

)

, source

s

, sink

tSlide16

ExampleAssignment problem

(P)Max 

i=1,…, m

, j=1,…,n

cijx

ij

s.t. j

=1,…,n xij

= 1 for 1 ≤ i ≤ m Job

i

=1

,…,m

x

ij

= 1 for 1 ≤

j

n

Machine

x

ij

 {0, 1}

Generalized assignment problem

(NP-complete)

Max

i

=1,…,

m

,

j

=1

,…,n

c

ij

x

ij

s.t. j=1,…,n xij = 1 for 1 ≤ i ≤ m Job i=1,…,m dijxij ≤ Cj for 1 ≤ j ≤ n Capacity xij  {0, 1}m jobs, n machinesSlide17

ExampleConsider developing a strategic corporate plan for several production facilities. Each facility has its own capacity and production constraints, but decisions are linked together at the corporate level by budgetary considerations

Common constraints

Facility 1

Facility 2Facility n

Independent

constraintsSlide18

“easy” versus “hard” variablesReferring to the variables as “easy” and “hard” may be a bit misleading

The “hard” variables need not be very difficult in themselves, but they can complicate the linear program making the overall problem more difficult to solveWhen the “hard” variables are removed from the problem, then more efficient techniques could be applied to solve the resulting linear programSlide19

ExampleCapacitated facility location problem

(NP-complete)Min 

i=1,…,n,j=1,…,m

cijxij

+ j=1,…,m

fjyj

s.t. i=1,…,m

xij ≥ 1 for j = 1,…,

n

Demand

j

=1,…,

n

d

i

x

ij

C

i

y

i

for

i

= 1,…,

m

Roll

x

ij

y

i

for

i

= 1,…,

m j = 1,…, n Flow impl. xij ≥ 0 yi  {0, 1} m facilities, n customersSlide20

ExampleConsider solving a multi period scheduling problem. Each period has its own set of variables but is linked together through resource consumption variables

Independent variables

Common variables

Period 1

Period 2

Period nSlide21

OutlineBackgroundIntroduction

Dantzig Wolfe decompositionBenders decompositionConclusionsSlide22

PrimalMin cx

s.t. Ax ≥

b [y]

x ≥ 0Dual

Max yTb

s.t.

yTA ≤

c [x] y

≥ 0BackgroundSlide23

PrimalMin cx

s.t. Ax ≥

b [y]

x ≥ 0Dual

Max bTy

s.t.

ATy ≤

cT [x]

y ≥ 0Background

cx

c

A

x

Ax

b

b

T

y

b

T

A

T

c

T

A

T

y

ySlide24

Travelling salesmanG = (N

, A), cost c

ij

0

1

2

3

4

5

6

7

8

9

x

y

0

20

19

1

1

1

2

17

15

3

14

6

4

12

12

5

12

3

6

9

8

7

15

20

8

19

11

975Slide25

Travelling salesmanG = (N

, A), cost c

ij

xy020

19111217

15

3146412

125123698

7152081911975

0

1

2

3

4

5

6

7

8

9

Cost 60.78Slide26

Travelling salesmanVariables x

ij is 1 if arc (

i, j) is on the shortest tour, 0 otherwise

Formulation

Min (i,

j

)A cij

xijs.t. 

i:(i,j)

A

x

ij

= 1 for

j

N

Inflow

j

:(

i

,

j

)

A

x

ij

= 1 for

i

N

Outflow

i

,jS:(i,j)A xij ≤ |S| – 1 for S  N Subtour xij  {0, 1}Slide27

Travelling salesmanVariables x

ij is 1 if arc (

i, j) is on the shortest tour, 0 otherwise

Formulation

Min (i,

j

)A cij

xijs.t. 

i:(i,j)A

x

ij

= 1 for

j

N

Inflow

j

:(

i

,

j

)

A

x

ij

= 1 for

i

N

Outflow

x

ij

 {0, 1}Slide28

Example codeSlide29

Travelling salesmanG = (N

, A), cost c

ij

xy

020

19

11

121715

31464

12

12

5

12

3

6

9

8

7

15

20

8

19

11

9

7

5

1

2

3

4

5

6

7

9

0

8

Subtour

0, 2, 7Slide30

Travelling salesmanG = (N

, A), cost c

ij

xy

020

19

11

121715

31464

12

12

5

12

3

6

9

8

7

15

20

8

19

11

9

7

5

1

2

3

4

5

6

7

9

0

8

Subtour

0, 8, 1, 9Slide31

Travelling salesmanG = (N

, A), cost c

ij

xy

020

19

11

121715

31464

12

12

5

12

3

6

9

8

7

15

20

8

19

11

9

7

5

1

2

3

4

5

6

7

9

0

8

Subtour

0, 8, 2, 7Slide32

Travelling salesmanG = (N

, A), cost c

ij

xy

020

19

11

121715

31464

12

12

5

12

3

6

9

8

7

15

20

8

19

11

9

7

5

0

1

2

3

4

5

6

7

8

9

Cost 79.98Slide33

Travelling salesmanG = (N

, A), cost c

ij

xy

020

19

11

121715

31464

12

12

5

12

3

6

9

8

7

15

20

8

19

11

9

7

5

0

1

2

3

4

5

6

7

8

9

Cost 60.78Slide34

LPs with many constraintsThe number of constraints that are tight (or active

) is at most equal to the number of variables, so even with many constraints (possibly exponential many) only a small subset will be tight in the optimal solution

Active

Non-active

ASlide35

A

Ax

b

Row generation in the primal…

cx

c

xSlide36

y

b

T

A

T

… is column generation in the dual

b

T

y

c

T

A

T

ySlide37

…and vice versa

x

c

A

c

x

b

A

x

A

T

A

T

y

c

T

b

T

y

b

T

y

Column generation

in the primal

Row generation

in the dual

=Slide38

Resource constrained shortest pathG = (N

, A), source s, sink

t, for each (i, j

)  A, cost cij

, resource demand dij, and resource capacity C

1

2

3

4

5

6

1,10

10,3

1,7

2,2

1,2

10,1

1,1

12,3

2,3

5,7

i

j

c

ij

,

d

ij

Capacity = 14

Source:

Desrosiers

and

Lübbecke

, 2005Slide39

Resource constrained shortest pathG = (N

, A), source s, sink

t, for each (i, j

)  A, cost cij

, resource demand dij, and resource capacity C

1

6

1,10

10,3

1,7

2,2

1,2

10,1

1,1

12,3

2,3

5,7

i

j

c

ij

,

d

ij

Cost 13

Demand 13

Capacity = 14

2

3

4

5Slide40

Resource constrained shortest pathVariables x

ij is 1 if arc

(i, j) is on the shortest path,

0 otherwiseFormulation

Min (i

,

j)A c

ijxijs.t.

1 for i = s

Source

j

:(

i

,

j

)

A

x

ij

j

:(

j

,

i

)

A

x

ji

= 0 for

i

N

– {

s, t} Flow -1 for i = t Sink (i,j)A dijxij ≤ C Capacity xij  {0, 1}Slide41

Example codeSlide42

Resource constrained shortest pathVariables 

k is 1 if path

k is the shortest path, 0 otherwise

FormulationMin 

kK c

k

ks.t.

kK

k = 1 Convex

k

K

d

k

k

C

Capacity

k

≥ 0Slide43

Arc variablesPath variables

Arc versus path

1

2

3

4

5

6

1

2

3

4

5

6

1

2

3

4

5

6

1

2

3

4

5

6Slide44

Example codeSlide45

Revised Simplex methodMin cx

s.t. Ax ≥

b x ≥ 0

Min z = cx

s.t. Ax = b

x ≥ 0Let x

be a basic feasible solution, such that x = (xB,

xN) where xB is the vector of basic variables and x

N

is the vector of

non-basic variables

Add slack variablesSlide46

Revised Simplex methodMin z =

cxs.t. Ax

= b x

≥ 0Min z =

cBxB + c

N

xNs.t.

BxB + ANxN

= b xB, x

N

≥ 0

Min

z

=

c

B

x

B

+

c

N

x

N

s.t.

x

B

=

B

-1

b

B

-1

A

N

x

N

x

B

, xN ≥ 0Rearrangex = (xB, xN), c = (cB, cN), A = (B, AN)Slide47

Revised Simplex methodMin z =

cBxB +

cNxN

s.t. xB =

B-1b – B-1

A

NxN

xB, xN ≥ 0Min

z = cBB-1b

+

(

c

N

c

B

B

-1

A

N

)

x

N

s.t.

x

B

=

B

-1

b

B

-1

A

N

x

N

x

B, xN ≥ 0SubstituteSlide48

Revised Simplex methodMin z =

cBB-1b

+ (cN

– cBB-1

AN)xN

s.t.

xB =

B-1b – B-1A

NxN xB,

x

N

0

At the end of each iteration we have

Current value of non-basic variables

x

N

= 0

Current objective function value

z

=

c

B

B

-1

b

Current value of basic variables

x

B

=

B

-1

b

Objective coefficients of basic variables

0

Objective coefficients

of

non-basic

variables

(cN – cBB-1AN) are the so-called reduced costsWith a minimization objective we want non-basic variables with negative reduced costsSlide49

Revised Simplex methodSimplex algorithmSelect new basic variable (

xN to enter the basis)Select new non-basic variable (xB to exit the basis)

Update data structuresSlide50

Revised Simplex methodSimplex algorithmx

S = b (slack variables equal rhs)

x\S

= 0 (non-slack variables equal 0)while min

j{(cj

– cBB

-1Aj)} < 0Select new basic variable

j : (cj – c

B

B

-1

A

j

)

<

0

Select new non-basic variable

j’

by increasing

x

j

as much as possible

Update data structures by swapping columns between matrix

B

and matrix

A

NSlide51

ExampleMin z = –

x1 – 2x

2s.t. – 2x

1 + x2 ≥ 2

– x1 + 2

x

2 ≥ 7 x

1 ≥ 7 x1,

x2 ≥ 0Min z = –

x

1

– 2

x

2

s.t.

– 2

x

1

+

x

2

+

x

3

= 2

x

1

+ 2

x

2

+

x

4

= 7

x

1

+

x

5

= 7

x1 , x2, x3 , x4 , x5 ≥ 0Add slack variablesSlide52

Simplex methodRevised Simplex methodExample

bsc

x1

x2

x3

x4

x5

rhs

-z

1

-2

0

0

0

0

x3

-2

1

1

0

0

2

x4

-1

2

0

1

0

7

x5

1

0

0

0

1

3

bsc

x1

x2

x3

x4

x5

rhs

-z

-5

0

2

0

0

4

x2

-2

1

1

0

0

2

x4

3

0

-2

1

0

3

x5

1

0

0

0

1

3

bsc

x3

x4

x5

rhs

-z

0

0

0

0

x3

1

0

0

2

x4

0

1

0

7

x5

0

0

1

3

bsc

x3

x4

x5

rhs

-z

2

0

0

4

x3

1

0

0

2

x4

-2

1

0

3

x5

0

0

1

3

x2

-2

1

2

0

x1

-5

-2

3

1Slide53

Simplex methodRevised Simplex methodExample

bsc

x1

x2

x3

x4

x5

rhs

-z

0

0

-3/4

5/3

0

9

x2

0

1

-1/3

2/3

0

4

x1

1

0

-2/3

1/3

0

1

x5

0

0

2/3

-1/3

1

2

bsc

x1

x2

x3

x4

x5

rhs

-z

0

0

0

1

2

13

x2

0

1

0

1/2

1/2

5

x1

1

0

0

0

1

3

x3

0

0

1

-1/2

3/2

3

bsc

x3

x4

x5

rhs

-z

2

0

0

9

x2

-1/3

2/3

0

4

x1

-2/3

1/3

0

1

x5

2/3

-1/3

1

2

bsc

x3

x4

x5

rhs

-z

0

0

0

13

x2

0

1/2

1/2

5

x1

0

0

1

3

x3

1

-1/2

3/2

3

x3

-3/4

-1/3

-2/3

2/3Slide54

Column generationSimplex algorithmx

S = b (slack variables equal rhs)

x\S

= 0 (non-slack variables equal 0)while min

j{(cj

– cBB

-1Aj)} < 0Select new basic variable

j : (cj – c

B

B

-1

A

j

)

<

0

Select new non-basic variable

j’

by increasing

x

j

as much as possible

Update data structures by swapping columns between matrix

B

and matrix

A

N

In column generation, rather than checking the reduced cost for each variable, a

subproblem

is solved to find a variable with negative reduced costSlide55

LPs with many variablesThe number of basic (non-zero) variables is at most equal to the number of constraints, so even with many variables (possibly exponential many) only a small subset will be in the optimal solution

A

x

B

x

NSlide56

(cN

– cBB

-1AN) < 0

(cN

– yTA

N

) < 0

Column generation

SubstituteSlide57

(cN

– yT

AN) < 0

PrimalMin cxs.t.

Ax ≥ b

x ≥ 0

Dual

Max yTbs.t.

y

T

A

c

y

≥ 0

Column generation

x

c

A

c

x

b

A

x

A

T

A

T

y

c

T

b

T

y

b

T

y

Column with negative reduced cost

Row with violated rhsSlide58

Resource constrained shortest pathVariables 

k is 1 if path k

is the shortest path, 0 otherwiseFormulation

Min k

K ck

ks.t.

kK 

k = 1 Convex 

k

K

d

k

k

C

Capacity

k

 {0, 1}Slide59

PrimalMin 

kK

ckk

s.t. k

K 

k

= 1 []

kK

dkk ≤

C

[

]

k

≥ 0

Dual

Max

+

C

s.t

.

+

d

k

c

k

[

k

]

= free

 ≤ 0Resource constrained shortest pathNeed to find a path for which ck –  – dk < 0Implicitly search all paths by optimizing Min (i,j)A (cij – dij)s.t. Source, Flow, SinkSlide60

Resource constrained shortest pathG = (N

, A), source s, sink

t, for each (i, j

)  A, cost cij

, resource demand dij, and resource capacity C

1

2

3

4

5

6

1

10

1

2

1

10

1

12

2

5

i

j

(

c

ij

d

ij

)

Capacity = 14Slide61

Resource constrained shortest pathMaster

Min kK

ck

ks.t.

kK

k = 1 Convex

kK

dkk ≤ C

Capacity

k

≥ 0

Subproblem

Min

(

i

,

j

)

A

(

c

ij

d

ij

)

x

ij

s.t.

1 for

i

=

s Source j:(i,j)A xij – j:(j,i)A xji = 0 for iN – {s, t} Flow -1 for i = t SinkAdd variable to master if (i,j)A (cij – dij)xij –  < 0Slide62

Example codeSlide63

Roll width W, m orders of d

i rolls of length li, i

= 1,…, m Cutting stock

12

31

36

45

11 x

4 x

4 x

2 x

l

i

d

i

100Slide64

Roll width W, m orders of d

i rolls of length li, i

= 1,…, m Cutting stock

12

12

31

45

12

12

36

12

12

12

31

45

31

31

36

36

12

12

12

12

36

12

31

36

45

11 x

4 x

4 x

2 x

l

i

d

i

Rolls 5

100

100

98

96

96Slide65

Cutting stockVariables x

ik is the number of times order i is cut from roll

k yk is

1 if roll k is used, 0 otherwise

FormulationMin 

k=1,…,

K yk

s.t. k=1,…,K

xik ≥ di

for

i

= 1,…,

n

Demand

i

=1

,…,

n

l

i

x

ik

Wy

k

for

k

= 1,…,

K

Roll

x

ik

≥ 0 and integer yk  {0, 1}Slide66

Example codeSlide67

Cutting stockVariables 

k is the number of times cutting pattern k is used

FormulationMin

kK

k

s.t.

kK

aikk

≥ di for i = 1,…,

m

Demand

k

0 and integerSlide68

Cutting stockCutting pattern variables

12

12

36

36

k

a

ik

[2, 0, 2, 0]

12

31

36

45

11 x

4 x

4 x

2 x

12

12

31

45

k

a

ik

[2, 1, 0,

1

]Slide69

PrimalMin 

kK

k s.t.

kK

aik

k

≥ di [

i] 

k ≥ 0Dual

Max

i

=1,…,

n

d

i

i

s.t

.

i

=1

,…,

n

a

ik

i

≤ 1 [

k

]

i

≥ 0

Cutting stockNeed to find a cutting pattern for which 1 – i=1,…,n aiki < 0Implicitly search all cutting patterns by optimizing Max i=1,…,n aii s.t. i=1,…,n liai ≤ Wai ≥ 0 and integerSlide70

m items with value i and weight

li, i = 1,…, m, maximum allowed weight

WCutting stock

$0.50,

45lbs

$0.50, 36lbs

$0.33, 31lbs

$0.125, 12lbs

12

31

36

45

l

i

i

0.125

0.33

0.50

0

.50

100lbs

12

12

36

36Slide71

Cutting stockMaster

Min kK

k

s.t. k

K aik

k ≥ di

for i = 1,…, m Demand

k ≥ 0

Subproblem

Max

i

=1,…,

m

a

i

i

s.t.

i

=1,…,

m

l

i

a

i

W

a

i

≥ 0 and integer

Add

variable

to master

if 1 – aii < 0Slide72

Example codeSlide73

Generalized assignmentn jobs, m machines, cost

cij, demand dij, capacity

Ci

1

2

3

4

5

1

2

j

i

36

34

C

j

c

ij

,

d

ij

Job

1

2

1

17, 8

23, 15

2

21, 15

16, 7

3

22,

14

21,

23

4

18, 23

16,

22

5

24, 8

17, 11Slide74

Generalized assignmentn jobs, m machines, cost

cij, demand dij, capacity

Ci

Cost 9530

29

1

2

3

4

5

1

2

j

i

36

34

C

j

c

ij

,

d

ij

Job

1

2

1

17, 8

23, 15

2

21, 15

16, 7

3

22,

14

21,

23

4

18, 23

16,

22

5

24, 8

17, 11Slide75

Generalized assignmentVariables

xij is 1 if job

j is assigned to machine i, 0 otherwise

FormulationMax 

i=1,…,m

,

j=1,…,n c

ijxij

s.t. i=1,…,m

x

ij

= 1

for

1 ≤

j

n

Job

j

=1

,…,n

d

ij

x

ij

C

i

for

1 ≤

i

m

Capacity

xij  {0, 1}Slide76

Example codeSlide77

Generalized assignmentVariables

ik is 1 if machine

i has job assignment k, 0 otherwise

FormulationMax 

i=1,…,m,

k

=1,…,Ki c

ikiks.t.

i=1,…,m,k

=1,…,

Ki

a

ijk

ik

= 1

for

1 ≤

j

n

Job

k

=1

,…,

Ki

ik

= 1

for

1 ≤

i

m Convexity ik  {0, 1}Slide78

Generalized assignmentJob assignment variables

ik

aijk [1, 0, 1, 0, 1]

ik

a

ijk

[0, 1, 0, 1, 0]

1

2

3

4

5

1

2

3

4

5

1

2

1

2Slide79

Generalized assignmentFormulation

Max i=1,…,

m,k=1

,…,Ki cik

iks.t.

i=1,…,

m,k=1,…,Ki a

ijkik = 1 for 1 ≤

j

n

Job

k

=1

,…,

Ki

ik

= 1

for

1 ≤

i

m

Convexity

ik

 {0, 1}

Common constraints

Machine 1

Machine 2

Machine nIndependentconstraintsSlide80

PrimalMax

i=1,…,m

,k=1,…,Ki

cikik

s.t. i

=1

,…,m,k=1,…,

Ki aijk

ik = 1 k=1,…,Ki

ik

= 1

ik

0

Dual

Min

j

=1,…,

n

j

+

i

=1,…,

m

i

s.t

.

j

=1,…,

n

aijkj + i ≥ cik j = free i = freeGeneralized assignmentNeed to find a cutting pattern for which j=1,…,n (cik – aijkj ) – i > 0 for i = 1,…,mImplicitly search all cutting patterns by optimizing Max j=1,…,n (cij – aijj ) s.t. j=1,…,n d

ija

ij ≤

C

i

a

ij

≥ 0 and integerSlide81

$

55.00,

8lbsn items with value 

j and weight dij, j

= 1,…, n, maximum allowed weight W

Generalized assignment

$

52.00

, 23lbs

$

51.00

, 14lbs

$

55.00

, 15lbs

$44.00, 8lbs

36lbs

Job

1

1

44, 8

2

55, 15

3

51,

14

4

52, 23

5

55, 8

Job

2

1

40, 15

2

37, 7

3

43,

23

4

34,

22

541, 11

36

1

2

3

4

5

1

2

34

1

2

3

4

5

1

2Slide82

Generalized assignmentMaster

Max i=1,…,m

,k=1,…,Ki

cik

iks.t. 

i

=1,…,m,k=1,…,

Ki aijk

ik = 1 for 1 ≤ j ≤ n Job

k

=1,…,

Ki

ik

= 1 for 1 ≤

i

m

Convexity

ik

 {0, 1}

Subproblem

(for each machine

i

)

Max 

j

=1,…,

n

(

c

ij

a

ij

j

)

s.t. j=1,…,n dijaij ≤ Ci aij ≥ 0 and integerAdd variable to master if j=1,…,n (cij – aijj ) – i > 0Slide83

Example codeSlide84

History of column generation

1961

: A linear programming approach to the

cutting-stock

problemP.C. Gilmore and R.E. Gomory

1963

: A linear programming approach to the

cutting-stock

problem–Part IIP.C. Gilmore and R.E. Gomory

1960

:

Decomposition

principle for linear programs

G.B.

Dantzig

and P.

Wolfe

“Credit is due to Ford and Fulkerson for their proposal for solving

multicommodity

network problems as it served to inspire the present development.”

1958

: A suggested computation for maximal

multicommodity

network flows

L.R. Ford and D.R. Fulkerson

1969

: A

column generation

algorithm for a ship scheduling problem

L.E.

AppelgrenSlide85

Solving integer programs by column generation

2000

: On Dantzig-Wolfe decomposition in integer programming and ways to perform

branching

in a branch-and-price algorithmF. Vanderbeck

2005

: A primer in column generation

J. Desrosiers and M.E. Lubbecke

1998

:

Branch-and-price

: column generation for solving huge integer programs

C. Barnhart, E.L. Johnson, G.L. Nemhauser, M.W.P. Savelsbergh and P.H. Vance

1984

: Routing with time windows by column generation

Y. Dumas, F. Soumis and M. Desrochers

2011

:

Branching

in branch-and-price: a generic scheme

F. VanderbeckSlide86

CP-based column generation

2000

: Solving very large crew scheduling problems to optimality

T.H. Yunes, A.V. Moura and C.C. de Souza

1999

: A framework for

constraint programming based column generation

U. Junker, S.E. Karisch, N. Kohl, B. Vaaben, T. Fahle and M. SellmannSlide87

CP-based column generation

ApplicationReference

CP used to solve subproblemCP used within Branch-and-Price

Urban transit crew management

T.H. Yunes., A.V. Moura, C.C. de Souza. 2000.

Y

Y

T.H. Yunes., A.V. Moura, C.C. de Souza. 2005.

YYTravelling tournamentK. Easton, G.L.

Nemhauser

, and M.A. Trick. 2002.

Y

Y

Two-dimensional bin packing

D.

Pisinger

, M.

Sigurd

. 2007.

Y

Y

Graph coloring

S.

Gualandi

. 2008.

Y

Y

Constrained cutting stock

T.

Fahle

, M.

Sellmann

. 2002.

Y

N

Employee timetabling

S.

Demassey

, G.

Pesant

, L.M. Rousseau. 2006.

Y

YWireless mesh networksA. Capone, G. Carello, I. Filippini, S. Gualandi, F. Malucelli. 2010.YNMulti-machine schedulingR. Sadykov, L.A. Wolsey. 2006.YNSource: Gualandi and Malucelli, 2009Slide88

CP-based column generation

ApplicationReference

CP used to solve subproblemCP used within Branch-and-Price

Airline crew assignment

U. Junker, S.E. Karisch, N. Kohl, B. Vaaben

, T. Fahle

, M. Sellmann. 1999.Y

NT. Fahle

, U. Junker, S.E. Karisch, N. Kohl, M. Sellmann, B. Vaaben. 2002.

Y

N

M.

Sellmann

, K.

Zervoudakis

, P.

Stamatopoulos

, T.

Fahle

. 2002.

Y

N

Vehicle routing with time windows

L.M. Rousseau. 2004.

Y

N

L.M. Rousseau, M.

Gendreau

, G.

Pesant

, F.

Focacci

. 2004.

Y

Y

Source:

Gualandi

and

Malucelli

, 2009Slide89

CP-based column generationTypical implementation

Master

Subproblem

Linear

programming

Constraint

programming

Dual information

New

columnsSlide90

OutlineBackgroundIntroduction

Dantzig Wolfe decompositionBenders decompositionConclusionsSlide91

Two-stage optimization

Stage

1

Stage 2

Solution valuesSlide92

Benders decomposition

Stage

1

Stage 2

Solution

values

Benders

cutsSlide93

Benders decomposition“Learn from ones mistakes”

Distinguish primary variables from secondary variablesSearch over primary variables (master problem)For each trial value of primary variables, solve problem over secondary variables (subproblem)

If solution is suboptimal/infeasible, find out why and design a constraint that rules out not only this solution but a large class of solutions that are suboptimal/infeasible for the same reason (Benders cut)Add Benders cut to the master problem and resolve

Master

Subproblem

Solution

values

Benders

cutsSlide94

Capacitated facility locationm facilities, n

customers, cost cij, demand dj, capacity

Ci, fixed cost fi

1

2

3

4

5

1

3

2

i

j

C

i

,

f

i

c

ij

d

j

10, 3

10, 4

10, 4

6

7

4

8

5

Cust

1

2

3

1

2

4

5

2

3

3

4

3

4

1

2

4

5

2

1

5

7

6

3Slide95

Capacitated facility locationm facilities, n

customers, cost cij, demand dj, capacity

Ci, fixed cost fi

1

2

3

4

5

1

3

2

i

j

C

i

,

f

i

c

ij

d

j

10, 3

10, 4

10, 4

6

7

4

8

5

Cust

1

2

3

1

2

4

5

2

3

3

4

3

4

1

2

4

5

2

1

5

7

6

3

Cost 21.29Slide96

Capacitated facility locationVariables

xij fraction of demand supplied by facility i

to cusomter j y

i is 1 if facility i is open, 0

otherwiseFormulation

Min

i=1,…,n,j=1,…,

m cijx

ij + j=1,…,m f

j

y

j

s.t.

i

=1,…,

m

x

ij

1

for

j

= 1,…,

n

Demand

j

=1

,…,

n

d

i

x

ij

≤ Ciyi for i = 1,…, m Roll xij ≤ yi for i = 1,…, m j = 1,…, n Flow xij ≥ 0 yi  {0, 1}Slide97

Example codeSlide98

Master

Subproblem

Solution

values

Benders

cuts

Min

cx

+

dy

s.t.

Ax

b

Px

+

Qy

r

x

0 and integer

y ≥

0

Min

cx

+

s.t.

Ax

b

x ≥ 0 and integer  ≥ 0Min dys.t. Qy ≥ r – Px

y ≥

0Benders decomposition

What if the

subproblem

is infeasible?Slide99

Benders decompositionPrimal, dual possibilities

OptimalUnbounded

InfeasibleOptimalYesNo

NoUnboundedNoNo

YesInfeasibleNoYes

Yes

Dual

PrimalSlide100

Master

Subproblem

Solution

values

Benders

cuts

Min

cx

+

dy

s.t.

Ax

b

Px

+

Qy

r

x

0 and integer

y ≥

0

Min

cx

+

s.t.

Ax

b

optimality cuts

feasibility cuts

x ≥ 0 and integer  ≥ 0Min dys.t. Qy ≥ r – P

x y

≥ 0

Benders decompositionSlide101

Min dy

s.t. Qy ≥ r –

Px [u]

y ≥ 0OptimalInfeasible

Max uT(

r

– Px)

s.t. uTQ ≤

d [y] u

0

Optimality cut

u

k

T

(

r

Px

)

Infeasibility cut

v

k

T

(

r

Px

) ≤ 0

Benders decompositionSlide102

Master

Subproblem

Solution

values

Benders

cuts

Min

cx

+

dy

s.t.

Ax

b

Px

+

Qy

r

x

0 and integer

y ≥

0

Min

cx

+

s.t.

Ax

b

≥ ukT(r – Px) vkT(r – Px) ≤ 0 x ≥ 0 and integer  ≥ 0

Max

dy

s.t.

Qy

r

P

x

y

0

Benders decompositionSlide103

Benders decomposition

Solve master problem

Is optimal?

START

Solve sub problem

Terminate?

END

Add optimality

cut

Add feasibility

cut

yes

no

yes

noSlide104

Capacitated facility locationVariables

xij fraction of demand supplied by facility i

to cusomter j y

i is 1 if facility i is open, 0

otherwiseFormulation

Min

i=1,…,n,j=1,…,

m cijx

ij + j=1,…,m f

j

y

j

s.t.

i

=1,…,

m

x

ij

1

for

j

= 1,…,

n

Demand

j

=1

,…,

n

d

i

x

ij

≤ Ciyi for i = 1,…, m Roll xij ≤ yi for i = 1,…, m j = 1,…, n Flow xij ≥ 0 yi  {0, 1}Slide105

Capacitated facility locationMaster

Min j

=1,…,m fj

yj + 

s.t. optimality cuts

feasibility cuts

yi

 {0, 1} 

≥ 0SubproblemMin i=1,…,

n,j

=1,…,

m

c

ij

x

ij

s.t

.

i

=1,…,

m

x

ij

1 for

j

= 1,…,

n

Demand

j

=1,…,

n

d

i

xij ≤ Ciyi for i = 1,…, m Roll xij ≤ yi for i = 1,…, m j = 1,…, n Flow xij ≥ 0Slide106

Capacitated facility locationSubproblem

primalMin i=1,…,

n,j=1,…,m

cijxij

s.t. 

i

=1,…,m xij

≥ 1 [j

] j=1,…,n

d

i

x

ij

C

i

y

i

[

i

]

x

ij

y

i

[

ij

]

x

ij

0

Subproblem dualMax j=1,…,m j + i=1,…,n Ciyii + i=1,…,n,j=1,…,m yiijs.t. j + dii + ij ≥ 1 [xij] j ≥ 0 i ≤ 0 ij ≤ 0Slide107

Capacitated facility locationMaster

Min j

=1,…,m fj

yj + 

s.t.  ≥

j=1,…,m 

j + i=1,…,n

Ciiyi

+ 

i

=1,…,

n,j

=1,…,

m

ij

y

i

j

=1,…,

m

j

+ 

i

=1,…,

n

C

i

i

y

i

+ 

i

=1,…,n,j=1,…,m ij yi ≤ 0 yi  {0, 1}  ≥ 0Slide108

Example codeSlide109

Benders decomposition for stochastic

prog.

Master

Scenario 2

Scenario

1

Scenario 3Slide110

Capacitated facility locationm facilities, n

customers, cost cij, demand dj, capacity

Ci, fixed cost fi

1

2

3

4

5

1

3

2

i

j

C

i

,

f

i

c

ij

d

j

10, 3

10, 4

10, 4

6

7

4

8

5

Cust

1

2

3

1

2

4

5

2

3

3

4

3

4

1

2

4

5

2

1

5

7

6

3

5

6

3

7

4

4

5

2

6

3Slide111

Example codeSlide112

CP-based Benders decompositionTypical implementation(?)

Master

Subproblem

Constraint

programming

Linear

programming

Solution values

Benders

cutsSlide113

CP-based Benders decompositionRecent developments

Master

Subproblem

Integer

programming

Constraint

programming

Solution values

Benders

cutsSlide114

CP-based Benders decomposition

ApplicationReferenceMaster problem

SubproblemParallel machine scheduling

V. Jain, I.E. Grossmann. 2001.MILPCP

Polypropylene batch schedulingC. Timpe. 2002.MILPCP

Call

center schedulingT. Benoist, E. Gaudin, B. Rottembourg. 2002.

CPLPMulti-machine schedulingJ.N. Hooker. 2004.

MILPCPJ.N. Hooker. 2005.MILPCP

Source: Hooker, 2006Slide115

Nested Benders decompositionNested Benders decompositionWhen the

subproblem is decomposed into a master and subproblem

Master

Sub

Master

Sub

Master

Sub

Master

Sub

Forward pass

Solve master problems

Backward pass

Solve

subproblems

and add

Benders cutsSlide116

OutlineIntroduction

BackgroundDantzig Wolfe decompositionBenders decompositionConclusionsSlide117

Why use decomposition?Many real-world systems contain loosely connected components, and as a result, the corresponding mathematical models present a certain structure that can be exploitedIt may be your only choice when solving the model without decomposition is impossible, because it is too large (memory error or timeout)Slide118

When is decomposition likely most effective?When you have either

complicating constraints or complicating variables

Dantzig

Wolfe decomposition

Benders

decompositionSlide119

Further reading

Column GenerationGuy Desaulniers, Jacques Desrosiers, Marius M. SolomonDecomposition Techniques in Mathematical ProgrammingAntonio J. Conejo

, Enrique Castillo, Roberto Minguez and Raquel Garcia-BertrandLinear Programming and Network FlowsMokhtar S. Bazaraa, John J. Jarvis, Hanif D. SheraliSlide120

Related Contents


Next Show more