Instructor Kris Hauser httpcsindianaeduhauserk 1 Constraint Propagation is the process of determining how the constraints and the possible values of one variable affect the possible values of other variables ID: 777248
Download The PPT/PDF document "CS B551: Elements of Artificial Intellig..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
CS B551: Elements of Artificial Intelligence
Instructor: Kris Hauserhttp://cs.indiana.edu/~hauserk
1
Slide2Constraint Propagation …
… is the process of determining how the constraints and the possible values of one variable affect the possible values of other variables
It is an important form of “least-commitment” reasoning
2
2
Slide3Forward Checking
3
Whenever a pair
(
X
v) is added to assignment A
do:
For each variable Y not in A do:
For every constraint C relating Y to
the variables in A do: Remove all values from Y’s domain that do not satisfy C
n
= number of variables
d
= size of initial domains
s
= maximum number of constraints involving a given variable (s
n-1)
Forward checking takes
O(
nsd
)
time
Slide4Forward Checking in Map Coloring
4
WA
NT
Q
NSW
V
SA
T
R
G
B
R
G
B
R
G
B
R
G
B
R
G
B
R
G
B
R
G
B
R
R
G
B
R
G
B
R
G
B
R
G
B
R
G
B
R
G
B
R
G
B
G
R
G
B
R
G
B
G
B
R
G
B
R
B
G
R
BBBRGB
Empty set: the current assignment
{(WA
R), (Q G), (V B)}does not lead to a solution
4
Slide5Forward Checking in Map Coloring
5
T
WA
NT
SA
Q
NSW
V
Contradiction that forward
checking did not detect
WA
NT
Q
NSW
V
SA
T
R
G
B
R
G
B
R
G
B
R
G
B
R
G
B
R
G
B
R
G
B
R
R
G
B
R
G
B
R
G
B
R
G
B
R
G
B
R
G
B
R
G
B
G
R
G
B
R
G
B
G
B
R
G
B
R
B
G
R
B
B
B
R
G
B
5
Slide6Forward Checking in Map Coloring
6
T
WA
NT
SA
Q
NSW
V
Contradiction that forward
checking did not detect
WA
NT
Q
NSW
V
SA
T
R
G
B
R
G
B
R
G
B
R
G
B
R
G
B
R
G
B
R
G
B
R
R
G
B
R
G
B
R
G
B
R
G
B
R
G
B
R
G
B
R
G
B
G
R
G
B
R
G
B
G
B
R
G
B
R
B
G
R
B
B
B
R
G
B
Detecting this contradiction requires a more
powerful
constraint propagation technique
6
Slide7Constraint Propagation
for Binary Constraints
REMOVE-VALUES(
X
,
Y
) removed false
For every value
v
in the domain of
Y doIf there is no value u in the domain of X such that the constraint on (X,Y) is satisfied then
Remove
v
from Y‘s domain
removed true
Return removed
7
Slide8Constraint Propagation
for Binary Constraints
8
AC3
Initialize queue
Q with all variables
(not yet instantiated)
While
Q
doX Remove(Q)For every (not yet instantiated)
variable Y
related to X
by a (binary) constraint doIf REMOVE-VALUES(
X,Y) then
If Y’s domain = then exit
Insert(Y,Q
)
Slide9Edge Labeling
9
We consider an image of a scene composed of polyhedral objects such that each vertex is the endpoint of exactly three edges
9
Slide10Edge Labeling
10
An “edge extractor” has accurately extracted all the visible edges in the image. The problem is to label each edge as convex (+), concave (-), or occluding (
) such that the complete labeling is physically possible
10
Slide1111
Convex
edges
Concave
edges
Occluding
edges
11
Slide1212
+
-
-
+
+
+
+
+
The arrow is
oriented such
that the object
is on the
right
of
the occluding
edge
12
Slide13One Possible Edge Labeling
13
+
+
+
+
+
+
+
+
+
+
-
-
13
Slide14Junction Types
14
Fork
L
T
Y
14
Slide15Junction Label Sets
15
+
+
-
-
-
-
-
+
+
+
+
+
+
+
-
-
-
-
-
+
(Waltz, 1975; Mackworth, 1977)
15
Slide16Edge Labeling as a CSP
A variable
is associated with each junction
The
domain
of a variable is the label set associated with the junction typeConstraints: The values assigned to two adjacent junctions must give the same label to the joining edge
1616
Slide17AC3 Applied to Edge Labeling
17
Q = (X
1
, X
2
, X
3
, ...)
X
1
X
5
X
3
X
8
X
12
X
2
X
4
17
Slide1818
+
-
+
-
+
-
-
+
+
X
1
X
5
Q = (
X
1
, ...)
18
AC3 Applied to Edge Labeling
Slide1919
+
-
+
-
+
-
-
+
+
X
1
X
5
Q = (
X
1
, ...)
19
Slide2020
+
-
+
-
+
-
-
+
+
X
5
Q = (X
5
, ...)
20
Slide2121
+
+
+
+
-
-
-
-
-
-
+
Q = (
X
5
, ...)
X
5
X
3
21
Slide2222
+
+
+
+
-
-
-
-
-
-
+
Q = (
X
5
, ...)
X
5
X
3
22
Slide2323
+
+
+
+
-
-
-
-
-
-
+
Q = (X
3
, ...)
X
3
23
Slide2424
+
+
+
+
+
-
-
-
+
+
+
+
Q = (
X
3
, ...)
X
3
X
8
+
24
Slide2525
+
+
+
+
+
-
-
-
+
+
+
+
Q = (
X
3
, ...)
X
3
X
8
+
25
Slide2626
+
+
+
+
+
-
-
-
+
+
+
+
Q = (X
8
, ...)
X
8
+
26
Slide2727
+
+
-
-
+
+
+
+
-
-
+
X
12
X
8
Q = (
X
8
, ...)
27
Slide28Complexity Analysis of AC3
n
= number of variables
d
= size of initial domains
s = maximum number of constraints involving a given variable (
s n-1)Each variables is inserted in Q up to d timesREMOVE-VALUES takes O(d
2
) time
AC3 takes O(
ndsd2) = O(nsd3) time
Usually more expensive than forward checking
28
AC3
Initialize queue
Q with all variables
(not yet instantiated)While
Q
doX
Remove(Q
)For every
(not yet instantiated) variable Y related to
X by a (binary) constraint do
If REMOVE-VALUES(X
,Y) then
If Y’s domain =
then exitInsert(
Y,Q)
REMOVE-VALUES(
X
,
Y
)
removed
false
For every value
v
in the domain of
Y
doIf there is no value
u in the domain of X
such that the constraint on (X,
Y) is satisfied then
Remove v from
Y‘s domain
removed true
Return removed
Slide29Is AC3 all that we need?
No !!AC3 can’t detect all contradictions among binary constraints
29
X
Z
Y
X
Y
X
Z
Y
Z
{1, 2}
{1, 2}
{1, 2}
29
Slide30Is AC3 all that we need?
No !!AC3 can’t detect all contradictions among binary constraints
30
X
Z
Y
X
Y
X
Z
Y
Z
{1, 2}
{1, 2}
{1, 2}
30
REMOVE-VALUES(
X
,
Y
)
removed
false
For every value
v
in the domain of
Y
do
If there is no value
u
in the domain of
X
such that the constraint on (
X
,
Y
) is satisfied then
Remove
v
from
Y
‘s domain
removed
true
Return
removed
Slide31Is AC3 all that we need?
No !!AC3 can’t detect all contradictions among binary constraints
31
X
Z
Y
X
Y
X
Z
Y
Z
{1, 2}
{1, 2}
{1, 2}
31
REMOVE-VALUES(
X
,
Y
)
removed
false
For every value
v
in the domain of
Y
do
If there is no value
u
in the domain of
X
such that the constraint on (
X
,
Y
) is satisfied then
Remove
v
from
Y
‘s domain
removed
true
Return
removed
REMOVE-VALUES(
X
,
Y,
Z
)
removed
false
For every value
w
in the domain of
Z
do
If there is no pair (
u
,
v
) of values in the domains of
X
and Y verifying the constraint on (X,Y) such that the constraints on (X,Z) and (Y,Z) are satisfied then Remove w from Z‘s domain removed true Return removed
Slide32Is AC3 all that we need?
No !!AC3 can’t detect all contradictions among binary constraints
Not all constraints are binary
32
X
Z
Y
X
Y
X
Z
Y
Z
{1, 2}
{1, 2}
{1, 2}
32
Slide33Tradeoff
Generalizing the constraint propagation algorithm increases its time complexity
Tradeoff between time spent in
backtracking search and time spent in
constraint propagation
A good tradeoff when all or most constraints are binary is often to combine backtracking with forward checking and/or AC3 (with REMOVE-VALUES for two variables)
33
33
Slide34Modified Backtracking Algorithm with AC3
34
CSP-BACKTRACKING(A,
var
-domains)
If assignment A is complete then return A
Run
AC3
and update
var
-domains accordinglyIf a variable has an empty domain then return failureX select a variable not in A
D select an ordering on the domain of X
For each value v in D do
Add (X
v) to A
var-domains forward checking
(var-domains, X, v, A)
If no variable has an empty domain then(i
) result CSP-BACKTRACKING(A, var-domains)
(ii) If result
failure then return resultRemove (X
v) from A
Return failure
Slide35A Complete Example:4-Queens Problem
35
1
3
2
4
3
2
4
1
X
1
{1,2,3,4}
X
3
{1,2,3,4}
X
4
{1,2,3,4}
X
2
{1,2,3,4}
1)
The modified backtracking algorithm starts by calling AC3, which removes no value
35
Slide364-Queens Problem
36
1
3
2
4
3
2
4
1
X
1
{
1
,2,3,4}
X
3
{1,2,3,4}
X
4
{1,2,3,4}
X
2
{1,2,3,4}
2)
The backtracking algorithm then selects a variable and a value for this variable. No heuristic helps in this selection. X
1
and the value 1 are arbitrarily selected
36
Slide374-Queens Problem
37
1
3
2
4
3
2
4
1
X
1
{
1
,2,3,4}
X
3
{
1
,2,
3
,4}
X
4
{
1
,2,3,
4
}
X
2
{
1
,
2
,3,4}
3)
The algorithm performs forward checking, which eliminates 2 values in each other variable’s domain
Slide384-Queens Problem
38
1
3
2
4
3
2
4
1
X
1
{
1
,2,3,4}
X
3
{
1
,2,
3
,4}
X
4
{
1
,2,3,
4
}
X
2
{
1
,
2
,3,4}
4)
The algorithm calls AC3
38
Slide394-Queens Problem
39
1
3
2
4
3
2
4
1
X
1
{
1
,2,3,4}
X
3
{
1
,2,
3
,4}
X
4
{
1
,2,3,
4
}
X
2
{
1
,
2
,
3
,4}
4)
The algorithm calls AC3,
which eliminates 3 from the domain of X
2
X
2
= 3 is
incompatible
with any of the
remaining values
of X
3
39
REMOVE-VALUES(
X
,
Y
)
removed
false
For every value
v
in the domain of
Y
do
If there is no value
u
in the domain of
X
such that the constraint on (
X,Y) is satisfied then Remove v from Y‘s domain removed true Return removed
Slide404-Queens Problem
40
1
3
2
4
3
2
4
1
X
1
{
1
,2,3,4}
X
3
{
1
,
2
,
3
,4}
X
4
{
1
,2,3,
4
}
X
2
{
1
,
2
,
3
,4}
4)
The algorithm calls AC3, which eliminates 3 from the domain of X
2
,
and 2 from the domain of X
3
40
Slide414-Queens Problem
41
1
3
2
4
3
2
4
1
X
1
{
1
,2,3,4}
X
3
{
1
,
2
,
3
,
4
}
X
4
{
1
,2,3,
4
}
X
2
{
1
,
2
,
3
,4}
The algorithm calls AC3, which eliminates 3 from the domain of X
2
, and 2 from the domain of X
3
,
and 4 from the domain of X
3
41
Slide424-Queens Problem
42
1
3
2
4
3
2
4
1
X
1
{
1
,2,3,4}
X
3
{
1
,
2
,
3
,
4
}
X
4
{
1
,2,3,
4
}
X
2
{
1
,
2
,
3
,4}
The domain of X
3
is
empty
backtracking
42
Slide434-Queens Problem
43
1
3
2
4
3
2
4
1
X
1
{
1
,
2
,3,4}
X
3
{1,2,3,4}
X
4
{1,2,3,4}
X
2
{1,2,3,4}
The algorithm removes 1 from X
1
’s domain and assign 2 to X
1
43
Slide444-Queens Problem
44
1
3
2
4
3
2
4
1
X
1
{
1
,
2
,3,4}
X
3
{1,
2
,3,
4
}
X
4
{1,
2
,3,4}
X
2
{
1,2,3
,4}
The algorithm performs forward checking
44
Slide454-Queens Problem
45
1
3
2
4
3
2
4
1
X
1
{
1
,
2
,3,4}
X
3
{1,
2
,3,
4
}
X
4
{1,
2
,3,4}
X
2
{
1,2,3
,4}
The algorithm calls AC3
45
Slide464-Queens Problem
46
1
3
2
4
3
2
4
1
X
1
{
1
,
2
,3,4}
X
3
{1,
2
,
3
,
4
}
X
4
{
1
,
2
,3,
4
}
X
2
{
1,2,3
,4}
The algorithm calls AC3, which reduces the domains of X
3
and X
4
to a single value
46
Slide47Exploiting the Structure of CSP
If the constraint graph contains several components, then solve one independent CSP per component
47
T
WA
NT
SA
Q
NSW
V
47
Slide48Exploiting the Structure of CSP
If the constraint graph is a tree, then :
48
Order the variables from the root to the leaves
(X
1
, X
2
, …,
X
n)For j = n, n-1, …, 2 callREMOVE-VALUES(Xj, Xi
) where
Xi
is the parent of Xj
Assign any valid value to X1
For j = 2, …, n do Assign any value to
Xj consistent with the value assigned to its parent X
i
X
Y
Z
U
V
W
(
X, Y, Z, U, V, W)
48
Slide49Exploiting the Structure of CSP
Whenever a variable is assigned a value by the backtracking algorithm, propagate this value and remove the variable from the constraint graph
49
WA
NT
SA
Q
NSW
V
49
Slide50Exploiting the Structure of CSP
Whenever a variable is assigned a value by the backtracking algorithm, propagate this value and remove the variable from the constraint graph
50
WA
NT
Q
NSW
V
If the graph becomes
a tree, then proceed
as shown in previous
slide
50
Slide5151
8
5
8
9
3
2
2
9
5
6
9
2
7
7
4
Representation
Variables
X
ij
for i, j in {1,..,9}
Domains
{1,…,9}
Constraints
:
X
ij
X
ik
, for j
k
X
ij
X
kj
, for i
k
X
ij
X
mn
, for (i,j), (m,n) in same cell
Slide5252
8
5
8
9
3
2
2
9
5
6
9
2
7
7
4
Representation
Variables
X
ij
for i, j in {1,..,9}
Domains
{1,…,9}
Constraints
:
X
ij
X
ik
, for j
k
X
ij
X
kj
, for i
k
X
ij
X
mn
, for (i,j), (m,n) in same cell
Slide5353
8
5
8
9
3
2
2
9
5
6
9
2
7
7
4
Representation
Variables
X
ij
for i, j in {1,..,9}
Domains
{1,…,9}
Constraints
:
X
ij
X
ik
, for j
k
X
ij
X
kj
, for i
k
X
ij
X
mn
, for (i,j), (m,n) in same cell
Can we detect this using constraint propagation?
Slide5454
8
5
8
9
3
2
2
9
5
6
9
2
7
7
4
Representation
Variables
X
ij
for i, j in {1,..,9}
Domains
{1,…,9}
Constraints
:
X
ij
X
ik
, for j
k
X
ij
X
kj
, for i
k
X
ij
X
mn
, for (i,j), (m,n) in same cell
Must detect 9-way interactions
Slide5555
8
5
8
9
3
2
2
9
5
6
9
2
7
7
4
Representation 2
Variables
R
i
j
in {1,…,9}
C
i
j
in {1,…,9}
X
i
j
in {(1,1),…,(3,3)}
Constraints
:
???
R
1
8
=3
R
2
2
=8
R
3
2
=6
R
3
7
=9
…
C
1
5
=4
C
1
2
=9
C
2
8
=4
C
3
8
=1
C
3
3
=8
C
3
9
=5
…
Slide5656
8
5
8
9
3
2
2
9
5
6
9
2
7
7
4
Representation 2
Variables
R
i
j
in {1,…,9}
C
i
j
in {1,…,9}
X
i
j
in {(1,1),…,(3,3)}
Constraints
:
???
R
1
8
=3
R
2
2
=8
R
3
2
=6
R
3
7
=9
…
C
1
5
=4
C
1
2
=9
C
2
8
=4
C
3
8
=1
C
3
3
=8
C
3
9
=5
…
X
1
8
= (1,3)
Slide5757
8
5
8
9
3
2
2
9
5
6
9
2
7
7
4
Representation 2
Variables
R
i
j
in {1,…,9}
C
i
j
in {1,…,9}
X
i
j
in {(1,1),…,(3,3)}
Constraints
:
???
R
1
8
=3
R
2
2
=8
R
3
2
=6
R
3
7
=9
…
C
1
5
=4
C
1
2
=9
C
2
8
=4
C
3
8
=1
C
3
3
=8
C
3
9
=5
…
X
1
8
= (1,3)
X
2
2
= (3,3)
Slide5858
8
5
8
9
3
2
2
9
5
6
9
2
7
7
4
Representation 2
Variables
R
i
j
in {1,…,9}
C
i
j
in {1,…,9}
X
i
j
in {(1,1),…,(3,3)}
Constraints
:
???
R
1
8
=3
R
2
2
=8
R
3
2
=6
R
3
7
=9
…
C
1
5
=4
C
1
2
=9
C
2
8
=4
C
3
8
=1
C
3
3
=8
C
3
9
=5
…
X
1
8
= (1,3)
X
2
2
= (3,3)
X
3
2
= (2,3)
X
3
7
= (3,3)
Slide5959
8
5
8
9
3
2
2
9
5
6
9
2
7
7
4
Representation 2
Variables
R
i
j
in {1,…,9}
C
i
j
in {1,…,9}
X
i
j
in {(1,1),…,(3,3)}
Constraints
:
R
i
j
=k
C
k
j
=I
Similar constraints between X’s and R’s, X’s and C’s
R
i
j
R
i
k
for j
k
C
i
j
C
i
k
for j
k
X
i
j
X
i
k
for j
k
R
1
8
=3
R
2
2
=8
R
3
2
=6
R
3
7
=9
…
C
1
5
=4
C
1
2
=9
C
2
8
=4
C
3
8
=1
C
3
3
=8
C
3
9
=5
…X18 = (1,3)X22 = (3,3)X32 = (2,3)X37 = (3,3)
Slide6060
8
5
8
9
3
2
2
9
5
6
9
2
7
7
4
Representation 2
Variables
R
i
j
in {1,…,9}
C
i
j
in {1,…,9}
X
i
j
in {(1,1),…,(3,3)}
Constraints
:
R
i
j
=k
C
k
j
=I
Similar constraints between X’s and R’s, X’s and C’s
R
i
j
R
i
k
for j
k
C
i
j
C
i
k
for j
k
X
i
j
X
i
k
for j
k
C
1
2
= 9
R
1
2
=
{1-9}
Slide6161
8
5
8
9
3
2
2
9
5
6
9
2
7
7
4
Representation 2
Variables
R
i
j
in {1,…,9}
C
i
j
in {1,…,9}
X
i
j
in {(1,1),…,(3,3)}
Constraints
:
R
i
j
=k
C
k
j
=i
Similar constraints between X’s and R’s, X’s and C’s
R
i
j
R
i
k
for j
k
C
i
j
C
i
k
for j
k
X
i
j
X
i
k
for j
k
C
2
2
= {1-9}
C
1
2
= 9
R
1
2
=
{2-9}
Slide6262
8
5
8
9
3
2
2
9
5
6
9
2
7
7
4
Representation 2
Variables
R
i
j
in {1,…,9}
C
i
j
in {1,…,9}
X
i
j
in {(1,1),…,(3,3)}
Constraints
:
R
i
j
=k
C
k
j
=I
Similar constraints between X’s and R’s, X’s and C’s
R
i
j
R
i
k
for j
k
C
i
j
C
i
k
for j
k
X
i
j
X
i
k
for j
k
C
1
2
= 9
R
1
2
=
{2-9}
X
2
2
= (3,3)
X
3
2
= (2,2)
Slide6363
8
5
8
9
3
2
2
9
5
6
9
2
7
7
4
Representation 2
Variables
R
i
j
in {1,…,9}
C
i
j
in {1,…,9}
X
i
j
in {(1,1),…,(3,3)}
Constraints
:
R
i
j
=k
C
k
j
=I
Similar constraints between X’s and R’s, X’s and C’s
R
i
j
R
i
k
for j
k
C
i
j
C
i
k
for j
k
X
i
j
X
i
k
for j
k
C
1
2
= 9
R
1
2
=2
X
2
2
= (3,3)
X
3
2
= (2,2)
Slide6464
2
8
5
8
9
3
2
2
9
5
6
9
2
7
7
4
Representation 2
Variables
R
i
j
in {1,…,9}
C
i
j
in {1,…,9}
X
i
j
in {(1,1),…,(3,3)}
Constraints
:
R
i
j
=k
C
k
j
=I
Similar constraints between X’s and R’s, X’s and C’s
R
i
j
R
i
k
for j
k
C
i
j
C
i
k
for j
k
X
i
j
X
i
k
for j
k
C
1
2
= 9
R
1
2
=2
X
2
2
= (3,3)
X
3
2
= (2,2)
Slide65Local Search for CSPs
Init: Make an arbitrary assignmentRepeat: Modify some variable to reduce # of violated constraints
65
Slide66Boolean Satisfiability Problems
Highly successful local search algorithms
WalkSAT
See R&N 7.3
66
p constraints of form
u
i
*
uj* uk*= 1 where ui* is either u
i or
ui
n variables u
i, …, un
Slide67Observations…
If a CSP has few constraints, local search solves it quicklyRandom starting assignment not too far from a solutionMillion-queens puzzles solved in < 1min, c. 1990
If a CSP has many constraints, local search solves it quickly
Constraints “guide” solver to a solution (if one exists)
67
Slide68Hard Sudoku’s
68
1 . . | . . . | . . 2
. 9 . | 4 . . | . 5 .
. . 6 | . . . | 7 . .
------+-------+------
. 5 . | 9 . 3 | . . .
. . . | . 7 . | . . .
. . . | 8 5 . | . 4 .
------+-------+------
7 . . | . . . | 6 . .. 3 . | . . 9 | . 8 .. . 2 | . . . | . . 1 Human solvers: Lot of logic (deep constraint propagation)
Computer solvers: Lot of backtracking
Slide69Hard & Easy 3-SAT Problems
Let R = # of constraints / # of variables
As n
, the fraction of hard problems reduces to 0
69
Slide70Recap
Constraint propagation, AC3Taking advantage of CSP structureLocal search for CSPs
70
Slide71Next Class
Intro to uncertaintyR&N 4.3-4, 13.1-2
71