M Pawan Kumar Center for Visual Computing Ecole Centrale Paris Post Metric Labeling Random variables V v 1 v 2 v n Label set L l 1 l 2 l h Labelings ID: 785141
Download The PPT/PDF document "Rounding-based Moves for Metric Labeling" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Rounding-based Movesfor Metric Labeling
M. Pawan Kumar
Center for Visual Computing
Ecole Centrale Paris
Slide2Post
Metric Labeling
Random variables V = {v
1
, v2, …, vn}
Label set L = {l1, l2, …, lh}
Labelings quantatively distinguished by energy E(y)
Labeling y ∈ Ln
Unary potential of variable va ∈ V
∑
a
θ
a
(
y
a
)
Slide3Post
Metric Labeling
Random variables V = {v
1
, v
2, …, vn}Label set L = {l1, l
2, …, lh}Labelings
quantatively distinguished by energy E(y)
Labeling y ∈ Ln Pairwise potential of variables (
v
a
,v
b
)
∑
a
θ
a(ya)
+ ∑(a,b) wab d(ya,yb)
wab is non-negative
d(.,.) is a metric distance function
min
y
Slide4PostExisting Work
Move-Making Algorithms (Efficient)
Linear Programming Relaxation (Accurate)
Rounding-based MovesEquivalence
Complete Rounding and Complete MoveInterval Rounding and Interval MoveHierarchical Rounding and Hierarchical MoveOutline
Slide5Post
Expansion Algorithm
Sky
House
Tree
Ground
Initialize with Tree
Expand Ground
Expand House
Expand Sky
Variables take label
l
α
or
retain current label
Boykov
,
Veksler
and
Zabih
, ICCV 1999
Slide6Post
Move-Making Algorithms
Iteration t
Define S
t ⊆ Ln containing current labeling
yt
∑a θa(ya)
+ ∑(a,b)
wab d(ya,yb
)
argmin
y
s.t.
y
∈ S
t
Sometimes
it can even be solved exactly
Above problem is easier than original problem
yt+1 =
Start with an initial labeling
y
0
Slide7PostExisting Work
Move-Making Algorithms (Efficient)
Linear Programming Relaxation (Accurate)
Rounding-based Moves
EquivalenceComplete Rounding and Complete MoveInterval Rounding and Interval MoveHierarchical Rounding and Hierarchical MoveOutline
Slide8Post
Linear Programming Relaxation
Chekuri
,
Khanna, Naor and Zosin, SODA 2001
Binary indicator xa(i) ∈ {0,1}
If variable ‘a’ takes the label ‘i’ then xa(i) = 1
∑i xa(
i) = 1Each variable ‘a’ takes one label
Similarly, binary indicator
x
ab
(
i,k
) ∈ {0,1}
Slide9Post
Linear Programming Relaxation
Minimize a linear function over feasible
x
Indicators
x
a
(
i
),
x
ab
(
i,k
)
{0,1}
Relaxed
xa
(i), xab(i,k) [0,1] Rounding
Chekuri, Khanna, Naor and Zosin
, SODA 2001
Slide10PostExisting Work
Move-Making Algorithms (Efficient)
Linear Programming Relaxation (Accurate)
Rounding-based Moves
EquivalenceComplete Rounding and Complete MoveInterval Rounding and Interval MoveHierarchical Rounding and Hierarchical MoveOutline
Slide11Post
Move-Making Bound
y
*: Optimal Labeling
y
: Estimated LabelingΣa
θa(ya) + Σ(a,b) wabd
(ya,yb)
Σa θa(y*a) + Σ(
a,b
)
w
ab
d
(y*
a
,y
*b)≥
Slide12Post
Move-Making Bound
y
*: Optimal Labeling
y
: Estimated LabelingΣa
θa(ya) + Σ(a,b) wabd
(ya,yb)
Σa θa(y*a) + Σ(
a,b
)
w
ab
d
(y*
a
,y
*b)B≤
For all possible values of θa(i) and wab
Slide13Post
Rounding Approximation
x
*: LP Optimal Solution
x
: Rounded SolutionΣa
Σi θa(i)xa(i) + Σ
(a,b) Σ(i,k) wabd(i,k)
xab(i,k)
≥
Σ
a
Σ
i
θ
a
(i)x*a(i) + Σ(
a,b) Σ(i,k) wabd(i,k)x*ab(i,k)
Slide14Post
Rounding Approximation
x
*: LP Optimal Solution
x
: Rounded SolutionΣa
Σi θa(i)xa(i) + Σ
(a,b) Σ(i,k) wabd(i,k)
xab(i,k)
≤
Σ
a
Σ
i
θ
a
(i)x*a(i) + Σ(
a,b) Σ(i,k) wabd(i,k)x*ab(i,k)A
For all possible values of θa(i
) and wab
Slide15Post
Equivalence
For any known rounding with approximation A
there exists a move-making algorithm
such that the move-making bound B = A
We know how to design such an algorithm
Slide16PostExisting Work
Move-Making Algorithms (Efficient)
Linear Programming Relaxation (Accurate)
Rounding-based MovesEquivalence
Complete Rounding and Complete MoveInterval Rounding and Interval MoveHierarchical Rounding and Hierarchical MoveOutline
Slide17Post
Complete Rounding
Treat x*
a
(
i
) [0,1] as probability that ya = li
Cumulative probability
z
a
(
i
)
=
Σ
j≤i
x*
a(j)
0
z
a
(1)
z
a
(2)
z
a
(h) = 1
z
a
(k)
z
a
(
i
)
Generate a random number r
(0,1]
Assign the label next to r
r
Slide18Post
Complete Rounding - Example
0
z
a
(1)
z
a
(4)
z
a
(3)
z
a
(2)
0.25
0.5
0.75
1.0
0
z
b
(1)
z
b
(4)
z
b
(3)
z
b
(2)
0.7
0.8
0.9
1.0
0
z
c
(1)
z
c
(4)
z
c
(3)
z
c
(2)
0.1
0.2
0.3
1.0
r
r
r
Slide19Post
Equivalent Move
Complete Move !!
Slide20Post
Complete Move
Iteration t
Define S
t ⊆
Ln∑
a θa(ya)
+ ∑(a,b)
wab d(ya,yb)
argmin
y
s.t.
y
∈ S
t
y
t+1
=
Start with an initial labeling
y
0
Slide21Post
Complete Move
Iteration t
Define S
t =
Ln∑
a θa(ya)
+ ∑(a,b)
wab d(ya,yb)
argmin
y
s.t.
y
∈ S
t
How do we solve this problem?
Above problem is the same as the original problem
y
t+1
=
Start with an initial labeling
y
0
Slide22Post
Complete Move
Define S
t
= Ln
∑a θa
(ya)+ ∑
(a,b) wab d’(
ya,yb)
argmin
y
s.t.
y
∈ S
t
How do we solve this problem?
Above problem is the same as the original problem
y
t+1
=
Slide23Post
Complete Move
Define S
t
= Ln
∑a θa
(ya)+ ∑
(a,b) wab d’(
ya,yb)
argmin
y
s.t.
y
∈ S
t
Obtained by solving a small LP
Submodular
overestimation d’ of d
y
t+1 =
Slide24Post
Submodular
Overestimation
max
i,k d’(li,l
k)/d(li,lk)
mind’d’(li
,lk) ≥ d(li,lk
)s.t.
d’(l
i
,l
k+1
) +
d’(
l
i+1
,l
k) ≥ d(li,lk) +
d(li+1,lk+1)
Slide25Post
Submodular
Overestimation
b
mind’
d’(li,lk) ≥ d(li
,lk)s.t.
d’(li,lk+1) +
d’(li+1,lk) ≥ d(li
,l
k
) +
d(
l
i+1
,l
k+1
)
bd(li,l
k) ≥ d’(li,lk)Dual provides worst-case instance for complete rounding
Slide26PostExisting Work
Move-Making Algorithms (Efficient)
Linear Programming Relaxation (Accurate)
Rounding-based MovesEquivalence
Complete Rounding and Complete MoveInterval Rounding and Interval MoveHierarchical Rounding and Hierarchical MoveOutline
Slide27Post
Interval Rounding
Treat x*
a
(
i
) [0,1] as probability that ya = li
Cumulative probability
z
a
(
i
)
=
Σ
j≤i
x*
a(j)
0
z
a
(1)
z
a
(2)
z
a
(h) = 1
z
a
(k)
z
a
(
i
)
Choose an interval of length h’
Slide28Post
Interval Rounding
Treat x*
a
(
i
) [0,1] as probability that ya = li
Cumulative probability
z
a
(
i
)
=
Σ
j≤i
x*
a(j)
r
Generate a random number r
(0,1]
Assign the label next to r
if it is within the interval
z
a
(k)-
z
a
(
i
)
0
Choose an interval of length
h’
REPEAT
Slide29Post
Interval Rounding - Example
0
z
a
(1)
z
a
(4)
z
a
(3)
z
a
(2)
0.25
0.5
0.75
1.0
0
z
b
(1)
z
b
(4)
z
b
(3)
z
b
(2)
0.7
0.8
0.9
1.0
0
z
c
(1)
z
c
(4)
z
c
(3)
z
c
(2)
0.1
0.2
0.3
1.0
Slide30Post
Interval Rounding - Example
0
z
a
(1)
z
a
(2)
0.25
0.5
0
z
b
(1)
z
b
(2)
0.7
0.8
0
z
c
(1)
z
c
(2)
0.1
0.2
r
r
r
Slide31Post
Interval Rounding - Example
0
z
a
(1)
z
a
(4)
z
a
(3)
z
a
(2)
0.25
0.5
0.75
1.0
0
z
b
(1)
z
b
(4)
z
b
(3)
z
b
(2)
0.7
0.8
0.9
1.0
0
z
c
(1)
z
c
(4)
z
c
(3)
z
c
(2)
0.1
0.2
0.3
1.0
Slide32Post
Interval Rounding - Example
0
z
c
(1)
z
c
(4)
z
c
(3)
z
c
(2)
0.1
0.2
0.3
1.0
Slide33Post
Interval Rounding - Example
0
z
c
(3)
z
c
(2)
0.1
0.2
r
-
z
c
(1)
-
z
c
(1)
Slide34Post
Interval Rounding - Example
0
z
a
(1)
z
a
(4)
z
a
(3)
z
a
(2)
0.25
0.5
0.75
1.0
0
z
b
(1)
z
b
(4)
z
b
(3)
z
b
(2)
0.7
0.8
0.9
1.0
0
z
c
(1)
z
c
(4)
z
c
(3)
z
c
(2)
0.1
0.2
0.3
1.0
Slide35Post
Equivalent Move
Interval Move !!
Slide36Post
Interval Move
Iteration t
y
∈ St iff
ya = yta or ya
∈ interval of labels
∑a θa(ya
)+ ∑(a,b)
w
ab
d(
y
a
,y
b
)
argminy
s.t.y ∈ Styt+1 =
Start with an initial labeling
y
0
Choose an interval of labels of length h’
How do we solve this problem?
Slide37Post
Interval Move
Iteration t
y
∈ St iff
ya = yta or ya
∈ interval of labels
∑a θa(ya
)+ ∑(a,b)
w
ab
d’(
y
a
,y
b
)
argminy
s.t.y ∈ Styt+1 =
Start with an initial labeling
y
0
Choose an interval of labels of length h’
Submodular
overestimation d’ of d
Slide38PostExisting Work
Move-Making Algorithms (Efficient)
Linear Programming Relaxation (Accurate)
Rounding-based MovesEquivalence
Complete Rounding and Complete MoveInterval Rounding and Interval MoveHierarchical Rounding and Hierarchical MoveOutline
Slide39Post
Hierarchical Rounding
L
1
L
2l1
l2
l3l4
l5
l
6
l
7
l
8
l
9
L
3
Hierarchical clustering of labels (e.g. r-HST metrics)
Slide40Post
Hierarchical Rounding
L
1
L
2l1
l2
l3l
4l5
l
6
l
7
l
8
l
9
L
3
Assign variables to labels L
1
, L
2
or L
3
Move down the hierarchy until the leaf level
Slide41Post
Hierarchical Rounding
L
1
L
2l1
l2
l3l4
l5
l
6
l
7
l
8
l
9
L
3
Assign variables to labels l
1
, l
2
or l
3
Slide42Post
Hierarchical Rounding
L
1
L
2l1
l2
l3l
4l5
l
6
l
7
l
8
l
9
L
3
Assign variables to labels l
4
, l
5
or l
6
Slide43Post
Hierarchical Rounding
L
1
L
2l1
l2
l3l
4l5
l
6
l
7
l
8
l
9
L
3
Assign variables to labels l
7
, l
8
or l
9
Slide44Post
Equivalent Move
Hierarchical Move !!
Slide45Post
Hierarchical Move
L
1
L
2l1
l2l
3l4
l5
l
6
l
7
l
8
l
9
L
3
Hierarchical clustering of labels (e.g. r-HST metrics)
Slide46Post
Hierarchical Move
L
1
L
2l1
l2l
3l4
l5
l
6
l
7
l
8
l
9
L
3
Obtain labeling
y
1
restricted to labels {l
1
,l
2
,l
3
}
Slide47Post
Hierarchical Move
L
1
L
2l1
l2
l3l4
l5
l
6
l
7
l
8
l
9
L
3
Obtain labeling
y
2
restricted to labels {l
4
,l
5
,l
6
}
Slide48Post
Hierarchical Move
L
1
L
2l1
l2
l3l
4l5
l
6
l
7
l
8
l
9
L
3
Obtain labeling
y
3
restricted to labels {l
7
,l
8
,l
9
}
Slide49Post
Hierarchical Move
L
1
L
2
L3
V
a
V
b
y
1
(a)
y
2
(a)
y
3
(a)
Move up the hierarchy until we reach the root
y
1
(b)
y
2
(b)
y
3
(b)
Slide50Questions?
http://
mpawankumar.info
Slide51Post
Simple Example - Rounding
θ
a
(1)xa(1) + θ
a(2)xa(2) + θb(1)
xb(1) + θb(2)
xb(2)minx
≥0+ d(1,1)xab
(1,1)
+
d(1,2)
x
ab
(1,2)
+ d(2,1
)
x
ab(2,1) + d(2,2)xab(2,2)
xa(1) + xa(2) = 1s.t.
xb(1) + x
b(2) = 1
x
ab
(1,1)
+
x
ab
(1,2) =
x
a
(1)
x
ab
(2,1)
+
x
ab
(2,2) =
x
a
(2)x
ab(1,1) + xab(2,1) = xb(1)xab(1,2) + xab(2,2) = xb(2)
Slide52Post
Simple Example - Rounding
x*
a
(1) + x*
a
(2) = 1
x*
a
(1)
0
x*
b
(1) + x*
b
(2) = 1
x*
b
(1)
0
Generate a uniform random number r
(0,1]
Assign the label next to r
r
r
Probability that
V
a
is assigned label l
1
?
x*
a
(1)
Probability that
V
a
is assigned label l
2
?
x*
a
(2)
Slide53Post
Simple Example - Rounding
x*
a
(1) + x*
a
(2) = 1
x*
a
(1)
0
x*
b
(1) + x*
b
(2) = 1
x*
b
(1)
0
Generate a uniform random number r
(0,1]
Assign the label next to r
r
r
Probability that
V
a
and
V
b
are assigned l
1
and l
1
?
min{x*
a
(1), x*
b
(1)}
Slide54Post
Simple Example - Rounding
x*
a
(1) + x*
a
(2) = 1
x*
a
(1)
0
x*
b
(1) + x*
b
(2) = 1
x*
b
(1)
0
Generate a uniform random number r
(0,1]
Assign the label next to r
r
r
Probability that
V
a
and
V
b
are assigned l
1
and l
1
?
min{x*
ab
(1,1)+x*
ab
(1,2), x*
ab
(1,1) + x*
ab
(2,1)}
x*
ab
(1,1)
+
min{x*
ab
(1,2), x*
ab
(2,1)}
Slide55Post
Simple Example - Rounding
x*
a
(1) + x*
a
(2) = 1
x*
a
(1)
0
x*
b
(1) + x*
b
(2) = 1
x*
b
(1)
0
Generate a uniform random number r
(0,1]
Assign the label next to r
r
r
Probability that
V
a
and
V
b
are assigned l
1
and l
2
?
max{0,x*
a
(1)
-
x*
b
(1)}
Slide56Post
Simple Example - Rounding
x*
a
(1) + x*
a
(2) = 1
x*
a
(1)
0
x*
b
(1) + x*
b
(2) = 1
x*
b
(1)
0
Generate a uniform random number r
(0,1]
Assign the label next to r
r
r
Probability that
V
a
and
V
b
are assigned l
1
and l
2
?
x*
ab
(1,2)
-
min{x*
ab
(1,2), x*
ab
(2,1)}
max{0,x*
ab
(1,2)
-
x*
ab
(2,1)}
Slide57Post
Simple Example - Rounding
x*
a
(1) + x*
a
(2) = 1
x*
a
(1)
0
x*
b
(1) + x*
b
(2) = 1
x*
b
(1)
0
Generate a uniform random number r
(0,1]
Assign the label next to r
r
r
Probability that
V
a
and
V
b
are assigned l
2
and l
1
?
max{0,x*
b
(1)
-
x*
a
(1)}
Slide58Post
Simple Example - Rounding
x*
a
(1) + x*
a
(2) = 1
x*
a
(1)
0
x*
b
(1) + x*
b
(2) = 1
x*
b
(1)
0
Generate a uniform random number r
(0,1]
Assign the label next to r
r
r
Probability that
V
a
and
V
b
are assigned l
2
and l
1
?
x*
ab
(2,1)
-
min{x*
ab
(1,2), x*
ab
(2,1)}
max{0,x*
ab
(2,1)
-
x*
ab
(1,2)}
Slide59Post
Simple Example - Rounding
x*
a
(1) + x*
a
(2) = 1
x*
a
(1)
0
x*
b
(1) + x*
b
(2) = 1
x*
b
(1)
0
Generate a uniform random number r
(0,1]
Assign the label next to r
r
r
Probability that
V
a
and
V
b
are assigned l
2
and l
2
?
1-max{x*
a
(1)
,
x*
b
(1)}
Slide60Post
Simple Example - Rounding
x*
a
(1) + x*
a
(2) = 1
x*
a
(1)
0
x*
b
(1) + x*
b
(2) = 1
x*
b
(1)
0
Generate a uniform random number r
(0,1]
Assign the label next to r
r
r
Probability that
V
a
and
V
b
are assigned l
2
and l
2
?
min{x*
a
(2)
,
x*
b
(2)}
Slide61Post
Simple Example - Rounding
x*
a
(1) + x*
a
(2) = 1
x*
a
(1)
0
x*
b
(1) + x*
b
(2) = 1
x*
b
(1)
0
Generate a uniform random number r
(0,1]
Assign the label next to r
r
r
Probability that
V
a
and
V
b
are assigned l
2
and l
2
?
min{x*
ab
(2,2)+x*
ab
(1,2), x*
ab
(2,2) + x*
ab
(2,1)}
x*
ab
(2,2)
+
min{x*
ab
(1,2), x*
ab
(2,1)}
Slide62Post
Simple Example - Move
θ
a
(ya) + θb
(yb)miny
+ d(ya,yb)
ya ,
yb ∈ {1,2}If d is submodular
, solve using graph cuts
Otherwise
Slide63Post
Simple Example - Move
θ
a
(ya) + θb
(yb)miny
+ d’(ya,yb)
ya ,
yb ∈ {0,1}If d is submodular
, solve using graph cuts
Otherwise
use
submodular
overestimation d’
Estimate d’ by minimizing distortion
Slide64Post
Simple Example - Move
b
min
d'
d’(1,1) ≤ b d(1,1)s.t.d’(1,2) ≤ b d(1,2)
d’(2,1) ≤ b d(2,1)d’(2,2) ≤ b d(2,2)
d(1,1) ≤ d’(1,1)d(1,2) ≤ d’(1,2)
d(2,1) ≤ d’(2,1)d(2,2) ≤ d’(2,2)
d’(1,1) + d’(2,2) ≤ d’(2,1) + d’(2,2)
Dual LP provides worst-case rounding example
LP in the variables d’(
i,k
)
Slide65Post
Simple Example - Move
d(1,1)β(1,1)+d(
1,2
)β(1,2)+d(2,1)β(2,1
)+d(2,2)β(2,2)minα,
β,γ≥0s.t.
d(1,1)α(1,1)+d(1,2)α(1,2)+d(2,1)α(2,1
)+d(2,2)α(2,2) = 1β(1,1) = α(1,1) + γ
β(
1,2) = α(1,2) -
γ
β
(2,1) = α(2,1) -
γ
β
(2,2) = α(2,2) +
γ
Set
xab*(i,k) = α(i,k)Set
γ = min{xab*(1,2), x
ab*(2,1)}