Lecture 4 Part 2 M Pawan Kumar pawankumarecpfr Finite set Σ called alphabet of size 2 01 abcde xyz Set Σ of all finite length strings called words ID: 383579
Download Presentation The PPT/PDF document "Discrete Optimization" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Discrete OptimizationLecture 4 – Part 2
M. Pawan Kumar
pawan.kumar@ecp.frSlide2
Finite set Σ
called alphabet of size ≥ 2
{0,1}{a,b,c,d,e,…,x,y,z}Set Σ* of all finite length strings (called words)0, 1, 00, 01, 10, 11, 000,….discrete, optimizationSize of word size(w) = number of letterssize(00) = 2size(discrete) = 8
Decision ProblemSlide3
Problem Π
is a subset
of Σ*All words with the answer “yes”Informal problemGiven input word x Σ*, does x Π ?Polynomial-time solvable problem ΠThere exists a polynomial-time algorithm for the informal problem
Polynomial in size(x)
Decision ProblemSlide4
P
Polynomial-time solvable problem
ΠThere exists a polynomial-time algorithm that decides whether x Σ* belongs to Π or not.Polynomial in size(x)P = {all Π,
Π
is polynomial time solvable}
For example
Shortest path with +
ve
lengths
P
Shortest path with no –
ve
length
circuit PSlide5
NP
Π
NPThere exists a problem Π’ PThere exists a polynomial pThere exists an x, size(x) ≤ p(size(w)) such thatw Π if and only if
wx
Π
’
Polynomial-time checkable ‘certificate’
For example
Hamiltonian graph problem
NPSlide6
Reduction
Π
is reducible to ΛThere exists a polynomial time algorithmGiven w, returns xw Π if and only if x Λ
Λ
NP-complete
All
Π
NP can be
reduced to
Λ
“Hardest” problems in NPSlide7
Outline
Reduction
“SAT” is reducible to “3-SAT”“3-SAT” is reducible to “Partition”“Partition” is reducible to “Hamiltonian Path”NP-hard ProblemsNP-completeness of SATSlide8
Alphabet Σ
containing
variables x1,x2,…,xnAnd special symbols (, ), ∧,∨,~And not containing 0 and 1
A variable is a Boolean expression
Boolean ExpressionSlide9
Alphabet Σ
containing
variables x1,x2,…,xnAnd special symbols (, ), ∧,∨,~And not containing 0 and 1
If v and w are Boolean expressions then
(v
∧
w) is a Boolean expression
Boolean ExpressionSlide10
Alphabet Σ
containing
variables x1,x2,…,xnAnd special symbols (, ), ∧,∨,~And not containing 0 and 1
If v and w are Boolean expressions then
(v
∨
w) is a Boolean expression
Boolean ExpressionSlide11
Alphabet Σ
containing
variables x1,x2,…,xnAnd special symbols (, ), ∧,∨,~And not containing 0 and 1
If v and w are Boolean expressions then
~v is a Boolean expression
~w is a Boolean expression
Boolean ExpressionSlide12
Alphabet Σ
containing
variables x1,x2,…,xnAnd special symbols (, ), ∧,∨,~And not containing 0 and 1
For example, f(x
1,x
2
,…,
x
n
)
((x
2
∧
x
3
)
∨
~(x
3
∨ x5) ∧ x2) ∨ ~(x2 ∧ x5)~(~x2 ∨ ~x3) ∧ ~x1
Boolean ExpressionSlide13
f(x1
,x
2,…,xn) is satisfiable ifthere exists an assignment x1 = α1, …, xn = αnwhere αi {0,1}
such that f(x
1
,x
2
,…,
x
n
) = 1
The following identities hold
0
∧
0 = 0, 0
∧
1 = 0, 1
∧
0 = 0, 1
∧
1 = 1
0
∨0 = 0, 0∨1 = 1, 1∨0 = 1, 1∨1 = 1~0 = 1, ~1 = 0(0) = 0, (1) = 1SatisfiabilitySlide14
Given the alphabet Σ
Given the identities for 0 and 1
SAT is a subset of Σ* that is satisfiableInformal ProblemGiven a Boolean expression w, is w satisfiableSATSlide15
A special case of SAT
Given variables x
1,x2,…,xn ΣB1 consists of x1,~x1,…,xn
,~
x
n
B
2
consists of (w
1
∨
…
∨
w
k
),
w
i
B
1
, 1≤k≤3
B3 consists of w1∧w2∧…∧wm, wi B2e.g., (x1∨
~x
2
∨
x
3
) ∧ (x2∨~x3∨x4) ∧ (~x1∨~x2)3-SAT is the subset of B3 that is satisfiable
3-SATSlide16
x1
= x
2 ∨ x3(x1∨~x2) ∧ (x1
∨
~x
3
)
∧
(~x
1
∨
x
2
∨
x
3
)
x
1
= x
2
∧ x3(~x1∨x2) ∧ (~x1∨x3) ∧ (x1∨~x2∨~x3
)
x
1
= ~x
2
(x1∨x2) ∧ (~x1∨~x2)SAT is reducible to 3-SATSlide17
Outline
Reduction
“SAT” is reducible to “3-SAT”“3-SAT” is reducible to “Partition”“Partition” is reducible to “Hamiltonian Path”NP-hard ProblemsNP-completeness of SATSlide18
A special case of SAT
Given variables x
1,x2,…,xn ΣB1 consists of x1,~x1,…,xn
,~
x
n
B
2
consists of (w
1
∨
…
∨
w
k
),
w
i
B
1
, 1≤k≤3
B3 consists of w1∧w2∧…∧wm, wi B2e.g., (x1∨
~x
2
∨
x
3
) ∧ (x2∨~x3∨x4) ∧ (~x1∨~x2)3-SAT is the subset of B3 that is satisfiable
3-SATSlide19
A finite set XPartition of X is a collection of subsets
Mutually exclusive
Collectively exhaustive For example, X = {a,b,c,d,e,f}{{a,b},{c},{d,e,f}} is a partition{{a,b},{a,c},{d,e,f}} is not a partition{{
a,b
},{c},{
d,e
}} is not a partition
PartitionSlide20
A finite set XPartition of X is a collection of subsets
Mutually exclusive
Collectively exhaustive Problem: Given collection of subsets CDoes C contain a partition of X?Or not?PartitionSlide21
f = w1
∧ w2 ∧ … wmBipartite undirected graph with V = V1 U V2V1
are variables x
1
,x
2
,…,
x
n
V
2
are words w
1
,w
2
,…,
w
m
Edges E = E
1
U E2E1 = {(wi,xj)}, xj wiE2 = {(wi,xj)}, ~xj wi
3-SAT is reducible to PartitionSlide22
Collection C1
of sets {
wi} U EiEi is non-emptyEi is a subset of edge set incident with wiCollection C2 of sets {
x
j
} U
E
j
and {
x
j
} U
E’
j
E
j
is the set of all edges in E
1
incident with
x
j
E’j is the set of all edges in E2 incident with xjf is satisfiable iff C1 U C2 contains a partition3-SAT is reducible to PartitionSlide23
Outline
Reduction
“SAT” is reducible to “3-SAT” (review)“3-SAT” is reducible to “Partition”“Partition” is reducible to “Hamiltonian Path”NP-hard ProblemsNP-completeness of SATSlide24
A finite set XPartition of X is a collection of subsets
Mutually exclusive
Collectively exhaustive Problem: Given collection of subsets CDoes C contain a partition of X?Or not?PartitionSlide25
Digraph D = (V, A)Path P is Hamiltonian if
It traverses each vertex in V
All vertices in the path are distinctProblem: Given DDoes it contain a Hamiltonian Path?Or not?Hamiltonian PathConnection to Shortest Path?Slide26
Partition Problem: Set X, Collection CX = {1,2,…,k}
C = {C
1,C2,…,Cm}Let Ci = {j1,…,jt}
Partition is reducible to Hamiltonian
Introduce r
0
and s
0
. Connect
r
m
to s
0
.Slide27
Partition Problem: Set X, Collection CX = {1,2,…,k}
C = {C
1,C2,…,Cm}Let Ci = {j1,…,jt}
Partition is reducible to Hamiltonian
C has a partition
iff
G has Hamiltonian r
0
-s
k
pathSlide28
Partition Problem: Set X, Collection CX = {1,2,…,k}
C = {C
1,C2,…,Cm}Let Ci = {j1,…,jt}
Partition is reducible to Hamiltonian
Left as Exercise !!Slide29
Outline
Reduction
NP-hard ProblemsNP-completeness of SATSlide30
NP-hard
Need not be in NP
No polynomial-time checkable certificateNot even a decision problem (e.g. optimization)At least as hard as NP-complete problemsΛ NP-hardThere exists Π NP-completeΠ is reducible to
ΛSlide31
NP-hard
Image Courtesy of WikipediaSlide32
Outline
Reduction
NP-hard ProblemsNP-completeness of SATRandom Access Machine (review)Cook’s TheoremSlide33
Given an array f with elements = 0,1 stringsRAM executes a set of instructions
Each instruction can
Read entries from prescribed positionsPerform arithmetic operation on read entriesWrite answers to prescribed positionsRandom Access MachineSlide34
Given an array f with elements = 0,1 strings
Finite set of variables z
0, z1, … , zkInitially, zi = 0 and f contains inputInstructions numbered 0, 1, …, tVariable z0 stores the instruction to execute
Random Access MachineSlide35
Given an array f with elements = 0,1 strings
Finite set of variables z
0, z1, … , zkInitially, zi = 0 and f contains inputInstructions numbered 0, 1, …, tStop if z0 > t
Random Access MachineSlide36
Given an array f with elements = 0,1 strings
Finite set of variables z
0, z1, … , zkInitially, zi = 0 and f contains inputRead instructionzi := f(zj
)
Random Access MachineSlide37
Given an array f with elements = 0,1 strings
Finite set of variables z
0, z1, … , zkInitially, zi = 0 and f contains inputWrite instructionf(zi) := z
j
Random Access MachineSlide38
Given an array f with elements = 0,1 strings
Finite set of variables z
0, z1, … , zkInitially, zi = 0 and f contains inputAdd instructionzi := zj
+
z
k
Random Access MachineSlide39
Given an array f with elements = 0,1 strings
Finite set of variables z
0, z1, … , zkInitially, zi = 0 and f contains inputSubtract instructionzi := zj
-
z
k
Random Access MachineSlide40
Given an array f with elements = 0,1 strings
Finite set of variables z
0, z1, … , zkInitially, zi = 0 and f contains inputMultiply instructionzi := zj
z
k
Random Access MachineSlide41
Given an array f with elements = 0,1 strings
Finite set of variables z
0, z1, … , zkInitially, zi = 0 and f contains inputDivide instructionzi := zj
/
z
k
Random Access MachineSlide42
Given an array f with elements = 0,1 strings
Finite set of variables z
0, z1, … , zkInitially, zi = 0 and f contains inputIncrement instructionzi := zi
+ 1
Random Access MachineSlide43
Given an array f with elements = 0,1 strings
Finite set of variables z
0, z1, … , zkInitially, zi = 0 and f contains inputBinarize instructionzi := 1, if z
i
> 0z
i
:=
0, otherwise
Random Access MachineSlide44
Outline
Reduction
NP-hard ProblemsNP-completeness of SATRandom Access Machine (review)Cook’s TheoremSlide45
Key Observation
z
i = f(zj)Remember zi, zj and f(zj) are Boolean strings
z
i
= a
1
a
2
a
3
a
4
…a
n
,
a
{0,1}
n
f(
z
j) = b1b2b3b4…bn, b {0,1}
n
“Read” as Boolean expression
a
k
=
bk implies (ak∧bk) ∨ (~ak∧~bk) = 1ak = bk = 1 OR ak = bk = 0B(a,b) = 1Slide46
Key Observation
f(
zi) = zjRemember zi, zj and f(zi) are Boolean strings
f(
z
i
) = a
1
a
2
a
3
a
4
…a
n
,
a
{0,1}
n
z
j = b1b2b3b4…bn, b
{0,1}
n
“Write” as Boolean expression
a
k
=bk implies (ak∧bk) ∨ (~ak∧~bk) = 1ak = bk = 1 OR ak = bk = 0B(a,b) = 1Slide47
Key Observation
z
i = zj + zkRemember zi, zj and z
k
are Boolean strings
z
i
= a
1
a
2
a
3
a
4
…a
n
,
a
{0,1}
n
zj = b1b2b3b4…bn, b
{0,1}
n
z
k
= c1c2c3c4…cn, c {0,1}n“Add” as Boolean expressionB(a,b,c) = 1Boolean algebra !! Slide48
Key Observation
z
i = zj - zkRemember zi, zj and z
k
are Boolean strings
z
i
= a
1
a
2
a
3
a
4
…a
n
,
a
{0,1}
n
zj = b1b2b3b4…bn, b
{0,1}
n
z
k
= c1c2c3c4…cn, c {0,1}n“Subtract” as Boolean expressionB(a,b,c) = 1Boolean algebra !! Slide49
Key Observation
z
i = zj zkRemember zi, zj and z
k
are Boolean strings
z
i
= a
1
a
2
a
3
a
4
…a
n
,
a
{0,1}
n
zj = b1b2b3b4…bn, b
{0,1}
n
z
k
= c1c2c3c4…cn, c {0,1}n“Multiply” as Boolean expressionB(a,b,c) = 1Boolean algebra !! Slide50
Key Observation
z
i = zj / zkRemember zi, zj and z
k
are Boolean strings
z
i
= a
1
a
2
a
3
a
4
…a
n
,
a
{0,1}
n
zj = b1b2b3b4…bn, b
{0,1}
n
z
k
= c1c2c3c4…cn, c {0,1}n“Divide” as Boolean expressionB(a,b,c) = 1Boolean algebra !! Slide51
Key Observation
z
i = zi + 1Remember old/new zi are Boolean stringsOld zi
= a
1
a
2
a
3
a
4
…a
n
,
a
{0,1}
n
New
z
i
= b1b2b3b4…bn, b {0,1}n
“Increment” as Boolean expression
B(
a
,
b
) = 1Boolean algebra !! Slide52
Key Observation
z
i = 1 if zi > 0 and 0 otherwiseRemember old/new zi are Boolean stringsOld zi
= a
1
a
2
a
3
a
4
…a
n
,
a
{0,1}
n
New
z
i
= b1b2b3b4…bn, b {0,1}n
“
Binarize
” as Boolean expression
B(
a
,b) = 1Boolean algebra !! Slide53
Cook’s Theorem
Any problem in NP can be reduced to SAT
Consider a problem Π NPThere is a corresponding problem Π’ P
w
Π
There exists x, size(x) = poly(size(w))
wx
Π
’Slide54
Input for RAM of Π
’
Input f(1), f(2), …, f(q)Each f(i) {0,1}nLet w = f(1)f(2)…f(q’)Then x=f(q’+1)f(q’+2)…f(q)
Size of input = q*nSlide55
Output for RAM of Π
’
W.l.o.g., at termination output is written at f(0)E.g., f(0) = {1}n implies wx Π’, otherwise not.Slide56
Variables for RAM of Π
’
Variables z0,z1,…,zkk is polynomial in the size of inputSlide57
Machine Memory
The word m = z
0z1..zkf(0)f(1)…f(q)Size of m = (k+q+2)*n = poly(size(input))
Machine memory at iteration
i
= m
i
m
i
= z
i
0
z
i
1
..
z
i
k
f
i
(
0)fi(1)…fi(q)Slide58
History
The algorithm terminates after r iterations
Define history h = m0m1m2…mr
Size of h = (r+1)*size(m) = s = poly(size(input))Slide59
Correct and Incorrect History
Given h = {0,1}
s, it is “correct” if there is an input wx such that the RAM has history hGiven h = {0,1}s, it is “incorrect” if there is
no input
wx
such that the RAM has history
hSlide60
SAT
Boolean expression g(h)
g(h) = 1 if h is “correct”g(h) = 0 if h is “incorrect”Set f0(1)f0(2)f0
(q’) = w
Set
f
r
(0) = {1}
n
w
Π
if and only if g is
satisfiable