Member of Research Staff NICTA and ANU menkesnictacomau Combining Linear Programming Based Decomposition Techniques with Constraint Programming CPbased column generation Application Reference ID: 224427
Download Presentation The PPT/PDF document "Menkes van den Briel" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Menkes van den Briel
Member of Research Staff
NICTA and ANU
menkes@nicta.com.au
Combining Linear Programming Based Decomposition Techniques with Constraint ProgrammingSlide2
CP-based column generation
ApplicationReference
Urban transit crew managementT.H.
Yunes., A.V. Moura, C.C. de Souza. Solving very large crew scheduling problems to optimality. Proceedings of ACM symposium on Applied Computing, pages 446-451, 2000.
T.H. Yunes., A.V.
Moura
, C.C. de Souza. Hybrid column generation approaches for urban transit crew management problems. Transportation Science 39(2):273-288, 2005.Travelling tournamentK. Easton, G.L.
Nemhauser, and M.A. Trick. Solving the travelling tournament problem: A combined integer programming and constraint programming approach. Proceedings of Practice and Theory of Automated Timetabling, volume 2740 of Lecture Notes in Computer Science, pages 100-112. Springer, 2002.Two-dimensional bin packing
D. Pisinger, M. Sigurd. Using decomposition techniques and constraint programming for solving the two-dimensional bin-packing problem. Journal on Computing 19(1):36-51, 2007.
Graph coloring
S.
Gualandi
. Enhancing constraint programming-based column generation for integer programs. PhD thesis,
Politechnico
di Milano, 2008.
Constrained cutting stock
T. Fahle, M. Sellmann. Cost based filtering for the constrained knapsack problem. Annals of Operations Research 115(1):73-93, 2002.
Employee timetabling
S. Demassey, G. Pesant, L.M. Rousseau. A cost-regular based hybrid column generation approach. Constraints 11(4):315-333, 2006.
Wireless mesh networks
A. Capone, G.
Carello
, I.
Filippini
, S.
Gualandi
, F.
Malucelli
. Solving a resource allocation problem in wireless mess networks: A comparison between a CP-based and a classical column generation. Networks 55(3):221-233, 2010.
Multi-machine scheduling
R.
Sadykov
, L.A. Wolsey. Integer programming and constraint programming in solving a
multimachine
assignment scheduling problem with deadlines and release dates. Journal on Computing 18(2):209-217, 2006.Slide3
CP-based column generation
ApplicationReference
Airline crew assignment
U. Junker, S.E. Karisch, N. Kohl, B. Vaaben, T.
Fahle, M. Sellmann. A framework for constraint programming based column generation. Proceedings of Principles and Practice of Constraint Programming, volume 1713 of Lecture Notes in Computer Science, pages 261-274, 1999.
T.
Fahle, U. Junker, S.E. Karisch, N. Kohl, M. Sellmann
, B. Vaaben. Constraint programming based column generation for crew assignment. Journal of Hueristics 8(1):59-81, 2002.
M. Sellmann, K. Zervoudakis
, P.
Stamatopoulos
, T.
Fahle
. Crew assignment via constraint programming: integrating column generation and heuristic tree search. Annals of Operations Research 115(1):207-225, 2002.
Vehicle routing with time windows
L.M. Rousseau. Stabilization issues for constraint programming based column generation. Proceedings of Integration of AI and OR techniques in CP for Combinatorial Optimization, volume 3011 of Lecture notes in Computer Science, pages 402-408. Springer, 2004.
L.M. Rousseau, M.
Gendreau
, G.
Pesant
, F.
Focacci
. Solving VRPTWs with constraint programming based column generation. Annals of Operations Research 130(1):199-216, 2004.Slide4
CP-based Benders decomposition
ApplicationReference
Parallel machine scheduling
V. Jain, I.E. Grossmann. Algorithms for hybrid MILP/CP models for a class of optimization problems. INFORMS Journal on Computing 13(4):258-276, 2001.Polypropylene batch scheduling
C. Timpe. Solving planning and scheduling problems with combined integer and constraint programming. OR Spectrum 24(4):431-448, 2002.
Call
center schedulingT. Benoist, E.
Gaudin, B. Rottembourg. Constraint programming contribution to Benders decomposition: A case study. Principles and Practice of Constraint Programming, volume 2470 of Lecture Notes in Computer Science, pages 603-617. Springer, 2002.
Multi-machine schedulingJ.N. Hooker. A hybrid method for planning and scheduling. Principles and Practice of Constraint Programming, volume 3258 of Lecture Notes in Computer Science, pages 305-316. Springer, 2004.
J.N. Hooker. Planning and scheduling to minimize tardiness. Principles and Practice of Constraint Programming, volume 3709 of Lecture Notes in Computer Science, pages 314-327. Springer, 2005.Slide5
CP versus IP
CPIP
VariablesFinite domain
Continuous, Binary, IntegerConstraintsSymbolic:
alldifferentcumulativeLinear,
algebraic:
(+, –, *, =, ≤, ≥)Inference
Constraint propagationLP relaxation
GlobalOptimalLocal
FeasibleSlide6
CP versus IP“MILP is very efficient when the relaxation is tight and models have a structure that can be effectively exploited”“CP works better for highly constrained discrete optimization problems where expressiveness of MILP is a major limitation”“From the work that has been performed, it is not clear whether a general integration strategy will always perform better than either CP or an MILP approach by itself. This is especially true for the cases where one of these methods is a very good tool to solve the problem at hand. However, it is usually possible to enhance the performance of one approach by borrowing some ideas from the other”
Source: Jain and Grossmann, 2001Slide7
OutlineBackgroundIntroduction
Dantzig Wolfe decompositionBenders decompositionConclusionsSlide8
What is your background?Have
implemented Benders and/or Dantzig Wolfe decompositionHave
heard about Benders and/or Dantzig Wolfe decompositionHave seen Bender and/or Dances with WolvesSlide9
Things to take awayA better understanding of how to combine
linear programming based decomposition techniques with constraint programmingA better understanding of column generation, Dantzig Wolfe decomposition and Benders decompositionA whole lot of Python code with example implementationsSlide10
Helpful installations Python 2.6.x or 2.7.x
“Python is a programming language that lets you work more quickly and integrate your systems more effectively”http://www.python.org/getit/Gurobi (Python interface)
“The state-of-the-art solver for linear programming (LP), quadratic and quadratically constrained programming (QP and QCP), and mixed-integer programming (MILP, MIQP, and MIQCP)”http://www.gurobi.com/products/gurobi-optimizer/try-for-yourself
NetworkX“NetworkX is a Python language software package for the creation, manipulation, and study of the structure, dynamics, and functions of complex networks”http://networkx.lanl.gov/download.htmlSlide11
AbbreviationsArtificial Intelligence (AI)Constraint Programming (CP)
Constraint Satisfaction Problem (CSP)Integer Programming (IP)Linear Programming (LP)Mixed Integer Programming (MIP)Mixed Integer Linear Programming (MILP)
Mathematical Programming (MP)Operations Research (OR)Slide12
OutlineBackgroundIntroduction
Dantzig Wolfe decompositionBenders decompositionConclusionsSlide13
What is decomposition?“Decomposition in
computer science, also known as factoring, refers to the process by which a complex problem or system is broken down into parts that are easier to conceive, understand, program, and maintain”Source: http://en.wikipedia.org/wiki/Decomposition_(computer_science)
Decomposition in linear programming is a technique for solving linear programming problems where the constraints (or variables) of the problem can be divided into two groups, one group of “easy” constraints and another of “hard” constraintsSlide14
“easy” versus “hard” constraintsReferring to the constraints as “easy” and “hard” may be a bit misleading
The “hard” constraints need not be very difficult in themselves, but they can complicate the linear program making the overall problem more difficult to solveWhen the “hard” constraints are removed from the problem, then more efficient techniques could be applied to solve the resulting linear programSlide15
ExampleShortest path problem
(P) Min (i
,j)A
cijxij
s.t.
1 for i = s Source
j:(i
,j)A xij
–
j
:(
j
,
i
)
A
x
ji
= 0 for
i
N
– {
s
,
t
}
Flow
-1 for
i
=
t
Sink
x
ij
{0, 1}
Resource constrained shortest path problem
(NP-complete)Min (i,j)A cijxijs.t. 1 for i = s Source j:(i,j)A xij – j:(j,i)A xji = 0 for iN – {s, t} Flow -1 for i = t Sink (i,j)A dijxij ≤ C Capacity xij {0, 1}
G
= (
N
,
A
)
, source
s
, sink
tSlide16
ExampleAssignment problem
(P)Max
i=1,…, m
, j=1,…,n
cijx
ij
s.t. j
=1,…,n xij
= 1 for 1 ≤ i ≤ m Job
i
=1
,…,m
x
ij
= 1 for 1 ≤
j
≤
n
Machine
x
ij
{0, 1}
Generalized assignment problem
(NP-complete)
Max
i
=1,…,
m
,
j
=1
,…,n
c
ij
x
ij
s.t. j=1,…,n xij = 1 for 1 ≤ i ≤ m Job i=1,…,m dijxij ≤ Cj for 1 ≤ j ≤ n Capacity xij {0, 1}m jobs, n machinesSlide17
ExampleConsider developing a strategic corporate plan for several production facilities. Each facility has its own capacity and production constraints, but decisions are linked together at the corporate level by budgetary considerations
Common constraints
Facility 1
Facility 2Facility n
Independent
constraintsSlide18
“easy” versus “hard” variablesReferring to the variables as “easy” and “hard” may be a bit misleading
The “hard” variables need not be very difficult in themselves, but they can complicate the linear program making the overall problem more difficult to solveWhen the “hard” variables are removed from the problem, then more efficient techniques could be applied to solve the resulting linear programSlide19
ExampleCapacitated facility location problem
(NP-complete)Min
i=1,…,n,j=1,…,m
cijxij
+ j=1,…,m
fjyj
s.t. i=1,…,m
xij ≥ 1 for j = 1,…,
n
Demand
j
=1,…,
n
d
i
x
ij
≤
C
i
y
i
for
i
= 1,…,
m
Roll
x
ij
≤
y
i
for
i
= 1,…,
m j = 1,…, n Flow impl. xij ≥ 0 yi {0, 1} m facilities, n customersSlide20
ExampleConsider solving a multi period scheduling problem. Each period has its own set of variables but is linked together through resource consumption variables
Independent variables
Common variables
Period 1
Period 2
Period nSlide21
OutlineBackgroundIntroduction
Dantzig Wolfe decompositionBenders decompositionConclusionsSlide22
PrimalMin cx
s.t. Ax ≥
b [y]
x ≥ 0Dual
Max yTb
s.t.
yTA ≤
c [x] y
≥ 0BackgroundSlide23
PrimalMin cx
s.t. Ax ≥
b [y]
x ≥ 0Dual
Max bTy
s.t.
ATy ≤
cT [x]
y ≥ 0Background
cx
c
A
x
Ax
b
b
T
y
b
T
A
T
c
T
A
T
y
ySlide24
Travelling salesmanG = (N
, A), cost c
ij
0
1
2
3
4
5
6
7
8
9
x
y
0
20
19
1
1
1
2
17
15
3
14
6
4
12
12
5
12
3
6
9
8
7
15
20
8
19
11
975Slide25
Travelling salesmanG = (N
, A), cost c
ij
xy020
19111217
15
3146412
125123698
7152081911975
0
1
2
3
4
5
6
7
8
9
Cost 60.78Slide26
Travelling salesmanVariables x
ij is 1 if arc (
i, j) is on the shortest tour, 0 otherwise
Formulation
Min (i,
j
)A cij
xijs.t.
i:(i,j)
A
x
ij
= 1 for
j
N
Inflow
j
:(
i
,
j
)
A
x
ij
= 1 for
i
N
Outflow
i
,jS:(i,j)A xij ≤ |S| – 1 for S N Subtour xij {0, 1}Slide27
Travelling salesmanVariables x
ij is 1 if arc (
i, j) is on the shortest tour, 0 otherwise
Formulation
Min (i,
j
)A cij
xijs.t.
i:(i,j)A
x
ij
= 1 for
j
N
Inflow
j
:(
i
,
j
)
A
x
ij
= 1 for
i
N
Outflow
x
ij
{0, 1}Slide28
Example codeSlide29
Travelling salesmanG = (N
, A), cost c
ij
xy
020
19
11
121715
31464
12
12
5
12
3
6
9
8
7
15
20
8
19
11
9
7
5
1
2
3
4
5
6
7
9
0
8
Subtour
0, 2, 7Slide30
Travelling salesmanG = (N
, A), cost c
ij
xy
020
19
11
121715
31464
12
12
5
12
3
6
9
8
7
15
20
8
19
11
9
7
5
1
2
3
4
5
6
7
9
0
8
Subtour
0, 8, 1, 9Slide31
Travelling salesmanG = (N
, A), cost c
ij
xy
020
19
11
121715
31464
12
12
5
12
3
6
9
8
7
15
20
8
19
11
9
7
5
1
2
3
4
5
6
7
9
0
8
Subtour
0, 8, 2, 7Slide32
Travelling salesmanG = (N
, A), cost c
ij
xy
020
19
11
121715
31464
12
12
5
12
3
6
9
8
7
15
20
8
19
11
9
7
5
0
1
2
3
4
5
6
7
8
9
Cost 79.98Slide33
Travelling salesmanG = (N
, A), cost c
ij
xy
020
19
11
121715
31464
12
12
5
12
3
6
9
8
7
15
20
8
19
11
9
7
5
0
1
2
3
4
5
6
7
8
9
Cost 60.78Slide34
LPs with many constraintsThe number of constraints that are tight (or active
) is at most equal to the number of variables, so even with many constraints (possibly exponential many) only a small subset will be tight in the optimal solution
Active
Non-active
ASlide35
A
Ax
b
Row generation in the primal…
cx
c
xSlide36
y
b
T
A
T
… is column generation in the dual
b
T
y
c
T
A
T
ySlide37
…and vice versa
x
c
A
c
x
b
A
x
A
T
A
T
y
c
T
b
T
y
b
T
y
Column generation
in the primal
Row generation
in the dual
=Slide38
Resource constrained shortest pathG = (N
, A), source s, sink
t, for each (i, j
) A, cost cij
, resource demand dij, and resource capacity C
1
2
3
4
5
6
1,10
10,3
1,7
2,2
1,2
10,1
1,1
12,3
2,3
5,7
i
j
c
ij
,
d
ij
Capacity = 14
Source:
Desrosiers
and
Lübbecke
, 2005Slide39
Resource constrained shortest pathG = (N
, A), source s, sink
t, for each (i, j
) A, cost cij
, resource demand dij, and resource capacity C
1
6
1,10
10,3
1,7
2,2
1,2
10,1
1,1
12,3
2,3
5,7
i
j
c
ij
,
d
ij
Cost 13
Demand 13
Capacity = 14
2
3
4
5Slide40
Resource constrained shortest pathVariables x
ij is 1 if arc
(i, j) is on the shortest path,
0 otherwiseFormulation
Min (i
,
j)A c
ijxijs.t.
1 for i = s
Source
j
:(
i
,
j
)
A
x
ij
–
j
:(
j
,
i
)
A
x
ji
= 0 for
i
N
– {
s, t} Flow -1 for i = t Sink (i,j)A dijxij ≤ C Capacity xij {0, 1}Slide41
Example codeSlide42
Resource constrained shortest pathVariables
k is 1 if path
k is the shortest path, 0 otherwise
FormulationMin
kK c
k
ks.t.
kK
k = 1 Convex
k
K
d
k
k
≤
C
Capacity
k
≥ 0Slide43
Arc variablesPath variables
Arc versus path
1
2
3
4
5
6
1
2
3
4
5
6
1
2
3
4
5
6
1
2
3
4
5
6Slide44
Example codeSlide45
Revised Simplex methodMin cx
s.t. Ax ≥
b x ≥ 0
Min z = cx
s.t. Ax = b
x ≥ 0Let x
be a basic feasible solution, such that x = (xB,
xN) where xB is the vector of basic variables and x
N
is the vector of
non-basic variables
Add slack variablesSlide46
Revised Simplex methodMin z =
cxs.t. Ax
= b x
≥ 0Min z =
cBxB + c
N
xNs.t.
BxB + ANxN
= b xB, x
N
≥ 0
Min
z
=
c
B
x
B
+
c
N
x
N
s.t.
x
B
=
B
-1
b
–
B
-1
A
N
x
N
x
B
, xN ≥ 0Rearrangex = (xB, xN), c = (cB, cN), A = (B, AN)Slide47
Revised Simplex methodMin z =
cBxB +
cNxN
s.t. xB =
B-1b – B-1
A
NxN
xB, xN ≥ 0Min
z = cBB-1b
+
(
c
N
–
c
B
B
-1
A
N
)
x
N
s.t.
x
B
=
B
-1
b
–
B
-1
A
N
x
N
x
B, xN ≥ 0SubstituteSlide48
Revised Simplex methodMin z =
cBB-1b
+ (cN
– cBB-1
AN)xN
s.t.
xB =
B-1b – B-1A
NxN xB,
x
N
≥
0
At the end of each iteration we have
Current value of non-basic variables
x
N
= 0
Current objective function value
z
=
c
B
B
-1
b
Current value of basic variables
x
B
=
B
-1
b
Objective coefficients of basic variables
0
Objective coefficients
of
non-basic
variables
(cN – cBB-1AN) are the so-called reduced costsWith a minimization objective we want non-basic variables with negative reduced costsSlide49
Revised Simplex methodSimplex algorithmSelect new basic variable (
xN to enter the basis)Select new non-basic variable (xB to exit the basis)
Update data structuresSlide50
Revised Simplex methodSimplex algorithmx
S = b (slack variables equal rhs)
x\S
= 0 (non-slack variables equal 0)while min
j{(cj
– cBB
-1Aj)} < 0Select new basic variable
j : (cj – c
B
B
-1
A
j
)
<
0
Select new non-basic variable
j’
by increasing
x
j
as much as possible
Update data structures by swapping columns between matrix
B
and matrix
A
NSlide51
ExampleMin z = –
x1 – 2x
2s.t. – 2x
1 + x2 ≥ 2
– x1 + 2
x
2 ≥ 7 x
1 ≥ 7 x1,
x2 ≥ 0Min z = –
x
1
– 2
x
2
s.t.
– 2
x
1
+
x
2
+
x
3
= 2
–
x
1
+ 2
x
2
+
x
4
= 7
x
1
+
x
5
= 7
x1 , x2, x3 , x4 , x5 ≥ 0Add slack variablesSlide52
Simplex methodRevised Simplex methodExample
bsc
x1
x2
x3
x4
x5
rhs
-z
1
-2
0
0
0
0
x3
-2
1
1
0
0
2
x4
-1
2
0
1
0
7
x5
1
0
0
0
1
3
bsc
x1
x2
x3
x4
x5
rhs
-z
-5
0
2
0
0
4
x2
-2
1
1
0
0
2
x4
3
0
-2
1
0
3
x5
1
0
0
0
1
3
bsc
x3
x4
x5
rhs
-z
0
0
0
0
x3
1
0
0
2
x4
0
1
0
7
x5
0
0
1
3
bsc
x3
x4
x5
rhs
-z
2
0
0
4
x3
1
0
0
2
x4
-2
1
0
3
x5
0
0
1
3
x2
-2
1
2
0
x1
-5
-2
3
1Slide53
Simplex methodRevised Simplex methodExample
bsc
x1
x2
x3
x4
x5
rhs
-z
0
0
-3/4
5/3
0
9
x2
0
1
-1/3
2/3
0
4
x1
1
0
-2/3
1/3
0
1
x5
0
0
2/3
-1/3
1
2
bsc
x1
x2
x3
x4
x5
rhs
-z
0
0
0
1
2
13
x2
0
1
0
1/2
1/2
5
x1
1
0
0
0
1
3
x3
0
0
1
-1/2
3/2
3
bsc
x3
x4
x5
rhs
-z
2
0
0
9
x2
-1/3
2/3
0
4
x1
-2/3
1/3
0
1
x5
2/3
-1/3
1
2
bsc
x3
x4
x5
rhs
-z
0
0
0
13
x2
0
1/2
1/2
5
x1
0
0
1
3
x3
1
-1/2
3/2
3
x3
-3/4
-1/3
-2/3
2/3Slide54
Column generationSimplex algorithmx
S = b (slack variables equal rhs)
x\S
= 0 (non-slack variables equal 0)while min
j{(cj
– cBB
-1Aj)} < 0Select new basic variable
j : (cj – c
B
B
-1
A
j
)
<
0
Select new non-basic variable
j’
by increasing
x
j
as much as possible
Update data structures by swapping columns between matrix
B
and matrix
A
N
In column generation, rather than checking the reduced cost for each variable, a
subproblem
is solved to find a variable with negative reduced costSlide55
LPs with many variablesThe number of basic (non-zero) variables is at most equal to the number of constraints, so even with many variables (possibly exponential many) only a small subset will be in the optimal solution
A
x
B
x
NSlide56
(cN
– cBB
-1AN) < 0
(cN
– yTA
N
) < 0
Column generation
SubstituteSlide57
(cN
– yT
AN) < 0
PrimalMin cxs.t.
Ax ≥ b
x ≥ 0
Dual
Max yTbs.t.
y
T
A
≤
c
y
≥ 0
Column generation
x
c
A
c
x
b
A
x
A
T
A
T
y
c
T
b
T
y
b
T
y
Column with negative reduced cost
Row with violated rhsSlide58
Resource constrained shortest pathVariables
k is 1 if path k
is the shortest path, 0 otherwiseFormulation
Min k
K ck
ks.t.
kK
k = 1 Convex
k
K
d
k
k
≤
C
Capacity
k
{0, 1}Slide59
PrimalMin
kK
ckk
s.t. k
K
k
= 1 []
kK
dkk ≤
C
[
]
k
≥ 0
Dual
Max
+
C
s.t
.
+
d
k
≤
c
k
[
k
]
= free
≤ 0Resource constrained shortest pathNeed to find a path for which ck – – dk < 0Implicitly search all paths by optimizing Min (i,j)A (cij – dij)s.t. Source, Flow, SinkSlide60
Resource constrained shortest pathG = (N
, A), source s, sink
t, for each (i, j
) A, cost cij
, resource demand dij, and resource capacity C
1
2
3
4
5
6
1
10
1
2
1
10
1
12
2
5
i
j
(
c
ij
–
d
ij
)
Capacity = 14Slide61
Resource constrained shortest pathMaster
Min kK
ck
ks.t.
kK
k = 1 Convex
kK
dkk ≤ C
Capacity
k
≥ 0
Subproblem
Min
(
i
,
j
)
A
(
c
ij
–
d
ij
)
x
ij
s.t.
1 for
i
=
s Source j:(i,j)A xij – j:(j,i)A xji = 0 for iN – {s, t} Flow -1 for i = t SinkAdd variable to master if (i,j)A (cij – dij)xij – < 0Slide62
Example codeSlide63
Roll width W, m orders of d
i rolls of length li, i
= 1,…, m Cutting stock
12
31
36
45
11 x
4 x
4 x
2 x
l
i
d
i
100Slide64
Roll width W, m orders of d
i rolls of length li, i
= 1,…, m Cutting stock
12
12
31
45
12
12
36
12
12
12
31
45
31
31
36
36
12
12
12
12
36
12
31
36
45
11 x
4 x
4 x
2 x
l
i
d
i
Rolls 5
100
100
98
96
96Slide65
Cutting stockVariables x
ik is the number of times order i is cut from roll
k yk is
1 if roll k is used, 0 otherwise
FormulationMin
k=1,…,
K yk
s.t. k=1,…,K
xik ≥ di
for
i
= 1,…,
n
Demand
i
=1
,…,
n
l
i
x
ik
≤
Wy
k
for
k
= 1,…,
K
Roll
x
ik
≥ 0 and integer yk {0, 1}Slide66
Example codeSlide67
Cutting stockVariables
k is the number of times cutting pattern k is used
FormulationMin
kK
k
s.t.
kK
aikk
≥ di for i = 1,…,
m
Demand
k
≥
0 and integerSlide68
Cutting stockCutting pattern variables
12
12
36
36
k
a
ik
[2, 0, 2, 0]
12
31
36
45
11 x
4 x
4 x
2 x
12
12
31
45
k
a
ik
[2, 1, 0,
1
]Slide69
PrimalMin
kK
k s.t.
kK
aik
k
≥ di [
i]
k ≥ 0Dual
Max
i
=1,…,
n
d
i
i
s.t
.
i
=1
,…,
n
a
ik
i
≤ 1 [
k
]
i
≥ 0
Cutting stockNeed to find a cutting pattern for which 1 – i=1,…,n aiki < 0Implicitly search all cutting patterns by optimizing Max i=1,…,n aii s.t. i=1,…,n liai ≤ Wai ≥ 0 and integerSlide70
m items with value i and weight
li, i = 1,…, m, maximum allowed weight
WCutting stock
$0.50,
45lbs
$0.50, 36lbs
$0.33, 31lbs
$0.125, 12lbs
12
31
36
45
l
i
i
0.125
0.33
0.50
0
.50
100lbs
12
12
36
36Slide71
Cutting stockMaster
Min kK
k
s.t. k
K aik
k ≥ di
for i = 1,…, m Demand
k ≥ 0
Subproblem
Max
i
=1,…,
m
a
i
i
s.t.
i
=1,…,
m
l
i
a
i
≤
W
a
i
≥ 0 and integer
Add
variable
to master
if 1 – aii < 0Slide72
Example codeSlide73
Generalized assignmentn jobs, m machines, cost
cij, demand dij, capacity
Ci
1
2
3
4
5
1
2
j
i
36
34
C
j
c
ij
,
d
ij
Job
1
2
1
17, 8
23, 15
2
21, 15
16, 7
3
22,
14
21,
23
4
18, 23
16,
22
5
24, 8
17, 11Slide74
Generalized assignmentn jobs, m machines, cost
cij, demand dij, capacity
Ci
Cost 9530
29
1
2
3
4
5
1
2
j
i
36
34
C
j
c
ij
,
d
ij
Job
1
2
1
17, 8
23, 15
2
21, 15
16, 7
3
22,
14
21,
23
4
18, 23
16,
22
5
24, 8
17, 11Slide75
Generalized assignmentVariables
xij is 1 if job
j is assigned to machine i, 0 otherwise
FormulationMax
i=1,…,m
,
j=1,…,n c
ijxij
s.t. i=1,…,m
x
ij
= 1
for
1 ≤
j
≤
n
Job
j
=1
,…,n
d
ij
x
ij
≤
C
i
for
1 ≤
i
≤
m
Capacity
xij {0, 1}Slide76
Example codeSlide77
Generalized assignmentVariables
ik is 1 if machine
i has job assignment k, 0 otherwise
FormulationMax
i=1,…,m,
k
=1,…,Ki c
ikiks.t.
i=1,…,m,k
=1,…,
Ki
a
ijk
ik
= 1
for
1 ≤
j
≤
n
Job
k
=1
,…,
Ki
ik
= 1
for
1 ≤
i
≤
m Convexity ik {0, 1}Slide78
Generalized assignmentJob assignment variables
ik
aijk [1, 0, 1, 0, 1]
ik
a
ijk
[0, 1, 0, 1, 0]
1
2
3
4
5
1
2
3
4
5
1
2
1
2Slide79
Generalized assignmentFormulation
Max i=1,…,
m,k=1
,…,Ki cik
iks.t.
i=1,…,
m,k=1,…,Ki a
ijkik = 1 for 1 ≤
j
≤
n
Job
k
=1
,…,
Ki
ik
= 1
for
1 ≤
i
≤
m
Convexity
ik
{0, 1}
Common constraints
Machine 1
Machine 2
Machine nIndependentconstraintsSlide80
PrimalMax
i=1,…,m
,k=1,…,Ki
cikik
s.t. i
=1
,…,m,k=1,…,
Ki aijk
ik = 1 k=1,…,Ki
ik
= 1
ik
≥
0
Dual
Min
j
=1,…,
n
j
+
i
=1,…,
m
i
s.t
.
j
=1,…,
n
aijkj + i ≥ cik j = free i = freeGeneralized assignmentNeed to find a cutting pattern for which j=1,…,n (cik – aijkj ) – i > 0 for i = 1,…,mImplicitly search all cutting patterns by optimizing Max j=1,…,n (cij – aijj ) s.t. j=1,…,n d
ija
ij ≤
C
i
a
ij
≥ 0 and integerSlide81
$
55.00,
8lbsn items with value
j and weight dij, j
= 1,…, n, maximum allowed weight W
Generalized assignment
$
52.00
, 23lbs
$
51.00
, 14lbs
$
55.00
, 15lbs
$44.00, 8lbs
36lbs
Job
1
1
44, 8
2
55, 15
3
51,
14
4
52, 23
5
55, 8
Job
2
1
40, 15
2
37, 7
3
43,
23
4
34,
22
541, 11
36
1
2
3
4
5
1
2
34
1
2
3
4
5
1
2Slide82
Generalized assignmentMaster
Max i=1,…,m
,k=1,…,Ki
cik
iks.t.
i
=1,…,m,k=1,…,
Ki aijk
ik = 1 for 1 ≤ j ≤ n Job
k
=1,…,
Ki
ik
= 1 for 1 ≤
i
≤
m
Convexity
ik
{0, 1}
Subproblem
(for each machine
i
)
Max
j
=1,…,
n
(
c
ij
–
a
ij
j
)
s.t. j=1,…,n dijaij ≤ Ci aij ≥ 0 and integerAdd variable to master if j=1,…,n (cij – aijj ) – i > 0Slide83
Example codeSlide84
History of column generation
1961
: A linear programming approach to the
cutting-stock
problemP.C. Gilmore and R.E. Gomory
1963
: A linear programming approach to the
cutting-stock
problem–Part IIP.C. Gilmore and R.E. Gomory
1960
:
Decomposition
principle for linear programs
G.B.
Dantzig
and P.
Wolfe
“Credit is due to Ford and Fulkerson for their proposal for solving
multicommodity
network problems as it served to inspire the present development.”
1958
: A suggested computation for maximal
multicommodity
network flows
L.R. Ford and D.R. Fulkerson
1969
: A
column generation
algorithm for a ship scheduling problem
L.E.
AppelgrenSlide85
Solving integer programs by column generation
2000
: On Dantzig-Wolfe decomposition in integer programming and ways to perform
branching
in a branch-and-price algorithmF. Vanderbeck
2005
: A primer in column generation
J. Desrosiers and M.E. Lubbecke
1998
:
Branch-and-price
: column generation for solving huge integer programs
C. Barnhart, E.L. Johnson, G.L. Nemhauser, M.W.P. Savelsbergh and P.H. Vance
1984
: Routing with time windows by column generation
Y. Dumas, F. Soumis and M. Desrochers
2011
:
Branching
in branch-and-price: a generic scheme
F. VanderbeckSlide86
CP-based column generation
2000
: Solving very large crew scheduling problems to optimality
T.H. Yunes, A.V. Moura and C.C. de Souza
1999
: A framework for
constraint programming based column generation
U. Junker, S.E. Karisch, N. Kohl, B. Vaaben, T. Fahle and M. SellmannSlide87
CP-based column generation
ApplicationReference
CP used to solve subproblemCP used within Branch-and-Price
Urban transit crew management
T.H. Yunes., A.V. Moura, C.C. de Souza. 2000.
Y
Y
T.H. Yunes., A.V. Moura, C.C. de Souza. 2005.
YYTravelling tournamentK. Easton, G.L.
Nemhauser
, and M.A. Trick. 2002.
Y
Y
Two-dimensional bin packing
D.
Pisinger
, M.
Sigurd
. 2007.
Y
Y
Graph coloring
S.
Gualandi
. 2008.
Y
Y
Constrained cutting stock
T.
Fahle
, M.
Sellmann
. 2002.
Y
N
Employee timetabling
S.
Demassey
, G.
Pesant
, L.M. Rousseau. 2006.
Y
YWireless mesh networksA. Capone, G. Carello, I. Filippini, S. Gualandi, F. Malucelli. 2010.YNMulti-machine schedulingR. Sadykov, L.A. Wolsey. 2006.YNSource: Gualandi and Malucelli, 2009Slide88
CP-based column generation
ApplicationReference
CP used to solve subproblemCP used within Branch-and-Price
Airline crew assignment
U. Junker, S.E. Karisch, N. Kohl, B. Vaaben
, T. Fahle
, M. Sellmann. 1999.Y
NT. Fahle
, U. Junker, S.E. Karisch, N. Kohl, M. Sellmann, B. Vaaben. 2002.
Y
N
M.
Sellmann
, K.
Zervoudakis
, P.
Stamatopoulos
, T.
Fahle
. 2002.
Y
N
Vehicle routing with time windows
L.M. Rousseau. 2004.
Y
N
L.M. Rousseau, M.
Gendreau
, G.
Pesant
, F.
Focacci
. 2004.
Y
Y
Source:
Gualandi
and
Malucelli
, 2009Slide89
CP-based column generationTypical implementation
Master
Subproblem
Linear
programming
Constraint
programming
Dual information
New
columnsSlide90
OutlineBackgroundIntroduction
Dantzig Wolfe decompositionBenders decompositionConclusionsSlide91
Two-stage optimization
Stage
1
Stage 2
Solution valuesSlide92
Benders decomposition
Stage
1
Stage 2
Solution
values
Benders
cutsSlide93
Benders decomposition“Learn from ones mistakes”
Distinguish primary variables from secondary variablesSearch over primary variables (master problem)For each trial value of primary variables, solve problem over secondary variables (subproblem)
If solution is suboptimal/infeasible, find out why and design a constraint that rules out not only this solution but a large class of solutions that are suboptimal/infeasible for the same reason (Benders cut)Add Benders cut to the master problem and resolve
Master
Subproblem
Solution
values
Benders
cutsSlide94
Capacitated facility locationm facilities, n
customers, cost cij, demand dj, capacity
Ci, fixed cost fi
1
2
3
4
5
1
3
2
i
j
C
i
,
f
i
c
ij
d
j
10, 3
10, 4
10, 4
6
7
4
8
5
Cust
1
2
3
1
2
4
5
2
3
3
4
3
4
1
2
4
5
2
1
5
7
6
3Slide95
Capacitated facility locationm facilities, n
customers, cost cij, demand dj, capacity
Ci, fixed cost fi
1
2
3
4
5
1
3
2
i
j
C
i
,
f
i
c
ij
d
j
10, 3
10, 4
10, 4
6
7
4
8
5
Cust
1
2
3
1
2
4
5
2
3
3
4
3
4
1
2
4
5
2
1
5
7
6
3
Cost 21.29Slide96
Capacitated facility locationVariables
xij fraction of demand supplied by facility i
to cusomter j y
i is 1 if facility i is open, 0
otherwiseFormulation
Min
i=1,…,n,j=1,…,
m cijx
ij + j=1,…,m f
j
y
j
s.t.
i
=1,…,
m
x
ij
≥
1
for
j
= 1,…,
n
Demand
j
=1
,…,
n
d
i
x
ij
≤ Ciyi for i = 1,…, m Roll xij ≤ yi for i = 1,…, m j = 1,…, n Flow xij ≥ 0 yi {0, 1}Slide97
Example codeSlide98
Master
Subproblem
Solution
values
Benders
cuts
Min
cx
+
dy
s.t.
Ax
≥
b
Px
+
Qy
≥
r
x
≥
0 and integer
y ≥
0
Min
cx
+
s.t.
Ax
≥
b
x ≥ 0 and integer ≥ 0Min dys.t. Qy ≥ r – Px
y ≥
0Benders decomposition
What if the
subproblem
is infeasible?Slide99
Benders decompositionPrimal, dual possibilities
OptimalUnbounded
InfeasibleOptimalYesNo
NoUnboundedNoNo
YesInfeasibleNoYes
Yes
Dual
PrimalSlide100
Master
Subproblem
Solution
values
Benders
cuts
Min
cx
+
dy
s.t.
Ax
≥
b
Px
+
Qy
≥
r
x
≥
0 and integer
y ≥
0
Min
cx
+
s.t.
Ax
≥
b
optimality cuts
feasibility cuts
x ≥ 0 and integer ≥ 0Min dys.t. Qy ≥ r – P
x y
≥ 0
Benders decompositionSlide101
Min dy
s.t. Qy ≥ r –
Px [u]
y ≥ 0OptimalInfeasible
Max uT(
r
– Px)
s.t. uTQ ≤
d [y] u
≥
0
Optimality cut
≥
u
k
T
(
r
–
Px
)
Infeasibility cut
v
k
T
(
r
–
Px
) ≤ 0
Benders decompositionSlide102
Master
Subproblem
Solution
values
Benders
cuts
Min
cx
+
dy
s.t.
Ax
≥
b
Px
+
Qy
≥
r
x
≥
0 and integer
y ≥
0
Min
cx
+
s.t.
Ax
≥
b
≥ ukT(r – Px) vkT(r – Px) ≤ 0 x ≥ 0 and integer ≥ 0
Max
dy
s.t.
Qy
≤
r
–
P
x
y
≥
0
Benders decompositionSlide103
Benders decomposition
Solve master problem
Is optimal?
START
Solve sub problem
Terminate?
END
Add optimality
cut
Add feasibility
cut
yes
no
yes
noSlide104
Capacitated facility locationVariables
xij fraction of demand supplied by facility i
to cusomter j y
i is 1 if facility i is open, 0
otherwiseFormulation
Min
i=1,…,n,j=1,…,
m cijx
ij + j=1,…,m f
j
y
j
s.t.
i
=1,…,
m
x
ij
≥
1
for
j
= 1,…,
n
Demand
j
=1
,…,
n
d
i
x
ij
≤ Ciyi for i = 1,…, m Roll xij ≤ yi for i = 1,…, m j = 1,…, n Flow xij ≥ 0 yi {0, 1}Slide105
Capacitated facility locationMaster
Min j
=1,…,m fj
yj +
s.t. optimality cuts
feasibility cuts
yi
{0, 1}
≥ 0SubproblemMin i=1,…,
n,j
=1,…,
m
c
ij
x
ij
s.t
.
i
=1,…,
m
x
ij
≥
1 for
j
= 1,…,
n
Demand
j
=1,…,
n
d
i
xij ≤ Ciyi for i = 1,…, m Roll xij ≤ yi for i = 1,…, m j = 1,…, n Flow xij ≥ 0Slide106
Capacitated facility locationSubproblem
primalMin i=1,…,
n,j=1,…,m
cijxij
s.t.
i
=1,…,m xij
≥ 1 [j
] j=1,…,n
d
i
x
ij
≤
C
i
y
i
[
i
]
x
ij
≤
y
i
[
ij
]
x
ij
≥
0
Subproblem dualMax j=1,…,m j + i=1,…,n Ciyii + i=1,…,n,j=1,…,m yiijs.t. j + dii + ij ≥ 1 [xij] j ≥ 0 i ≤ 0 ij ≤ 0Slide107
Capacitated facility locationMaster
Min j
=1,…,m fj
yj +
s.t. ≥
j=1,…,m
j + i=1,…,n
Ciiyi
+
i
=1,…,
n,j
=1,…,
m
ij
y
i
j
=1,…,
m
j
+
i
=1,…,
n
C
i
i
y
i
+
i
=1,…,n,j=1,…,m ij yi ≤ 0 yi {0, 1} ≥ 0Slide108
Example codeSlide109
Benders decomposition for stochastic
prog.
Master
Scenario 2
Scenario
1
Scenario 3Slide110
Capacitated facility locationm facilities, n
customers, cost cij, demand dj, capacity
Ci, fixed cost fi
1
2
3
4
5
1
3
2
i
j
C
i
,
f
i
c
ij
d
j
10, 3
10, 4
10, 4
6
7
4
8
5
Cust
1
2
3
1
2
4
5
2
3
3
4
3
4
1
2
4
5
2
1
5
7
6
3
5
6
3
7
4
4
5
2
6
3Slide111
Example codeSlide112
CP-based Benders decompositionTypical implementation(?)
Master
Subproblem
Constraint
programming
Linear
programming
Solution values
Benders
cutsSlide113
CP-based Benders decompositionRecent developments
Master
Subproblem
Integer
programming
Constraint
programming
Solution values
Benders
cutsSlide114
CP-based Benders decomposition
ApplicationReferenceMaster problem
SubproblemParallel machine scheduling
V. Jain, I.E. Grossmann. 2001.MILPCP
Polypropylene batch schedulingC. Timpe. 2002.MILPCP
Call
center schedulingT. Benoist, E. Gaudin, B. Rottembourg. 2002.
CPLPMulti-machine schedulingJ.N. Hooker. 2004.
MILPCPJ.N. Hooker. 2005.MILPCP
Source: Hooker, 2006Slide115
Nested Benders decompositionNested Benders decompositionWhen the
subproblem is decomposed into a master and subproblem
Master
Sub
Master
Sub
Master
Sub
Master
Sub
Forward pass
Solve master problems
Backward pass
Solve
subproblems
and add
Benders cutsSlide116
OutlineIntroduction
BackgroundDantzig Wolfe decompositionBenders decompositionConclusionsSlide117
Why use decomposition?Many real-world systems contain loosely connected components, and as a result, the corresponding mathematical models present a certain structure that can be exploitedIt may be your only choice when solving the model without decomposition is impossible, because it is too large (memory error or timeout)Slide118
When is decomposition likely most effective?When you have either
complicating constraints or complicating variables
Dantzig
Wolfe decomposition
Benders
decompositionSlide119
Further reading
Column GenerationGuy Desaulniers, Jacques Desrosiers, Marius M. SolomonDecomposition Techniques in Mathematical ProgrammingAntonio J. Conejo
, Enrique Castillo, Roberto Minguez and Raquel Garcia-BertrandLinear Programming and Network FlowsMokhtar S. Bazaraa, John J. Jarvis, Hanif D. SheraliSlide120