Constraint Satisfaction Daniel Weld Slides adapted from Dan Klein Stuart Russell Andrew Moore amp Luke Zettlemoyer Recap Search Problem States configurations of the world Successor function ID: 783988
Download The PPT/PDF document "CSE 473: Artificial Intelligence" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
CSE 473: Artificial Intelligence
Constraint Satisfaction
Daniel Weld
Slides
adapted from Dan Klein, Stuart
Russell, Andrew Moore & Luke
Zettlemoyer
Recap:
Search Problem
States
configurations
of the
world
Successor function:
function
from states
to lists
of
triples
(state
, action,
cost)
Start state
Goal
test
Slide3General Tree Search Paradigm
3
function tree-search(root-node)
fringe
successors
(root-node)
while (
notempty
(fringe) ) {node remove-first(fringe) state state(node) if goal-test(state) return solution(node) fringe insert-all(successors(node),fringe) } return failureend tree-search
Fringe managed asStackQueuePriority Queue
Depth first
Breadth first
Best first, uniform cost, greedy, A*
Slide44
Space of Search
Strategies
Blind
Search
DFS, BFS, IDS
Informed
Search
Uniform cost, greedy, A*, IDA*Constraint SatisfactionBacktracking=DFS, FC, k-consistencyAdversary SearchSystematic / Stochastic (“Local”)
Fri
Fri
5
Constraint Satisfaction
Kind of
search
in which
States are
factored
into sets of variables
Search = assigning values to these variablesGoal test is encoded with constraints Gives structure to search space Exploration of one part informs othersBacktracking-style algorithms workBut other techniques add speedPropagationVariable orderingPreprocessing
Slide6Example: N-Queens
Formulation
as search
States
Operators
Start State
Goal Test
Slide7Constraint Satisfaction Problems
Standard search problems:
State is a “black box”: arbitrary data structure
Goal test: any function over states
Successor function can be anything
Simple example of a
formal representation language
Allows
more powerful
search algorithms
Constraint satisfaction problems (CSPs):
A special subset of search problems
State is defined by
variables
X
i
with values from a
domain
D
(often
D
depends on
i
)
Goal test is a
set of constraints
specifying allowable combinations of values for subsets of variables
Slide8Example: N-Queens
CSP Formulation
1:
Variables:
Domains:
Constraints
X
ij
+
X
ik
≤ 1
X
ij
+
X
kj
≤ 1
X
ij
+
X
i+k,j+
k
≤ 1
X
ij
+
X
i+k,j-
k
≤ 1
Slide9Example: N-Queens
CSP Formulation
1:
Variables:
Domains:
Constraints
Slide10Example: N-Queens
Formulation 1.5:
Variables:
Domains:
Constraints:
… there’s an even better way! What is it?
Slide11Example: N-Queens
Formulation 2:
Variables:
Domains:
Constraints:
Implicit:
Explicit:
-or-
Slide12Example: Map-Coloring
Variables:
Domain:
Constraints: adjacent regions must have different colors
Solutions are assignments satisfying all constraints, e.g.:
Constraint Graphs
Binary CSP: each constraint relates (at most) two variables
Binary constraint graph: nodes are variables, arcs show constraints
General-purpose CSP algorithms use the graph structure to speed up search. E.g., Tasmania is an independent subproblem!
Slide14Example: Sudoku
Variables:
Domains:
Constraints:
9-way alldiff for each row
9-way alldiff for each column
9-way alldiff for each region
Each (open) square
{1,2,…,9}
Slide15Example: Cryptarithmetic
Variables (circles):
Domains:
Constraints (boxes):
Slide16Example: The Waltz Algorithm
The Waltz algorithm is for interpreting line drawings of solid polyhedra
An early example of a computation posed as a CSP
Look at all intersections
Adjacent intersections impose constraints on each other
?
Slide17Waltz on Simple Scenes
Assume all objects:
Have no shadows or cracks
Three-faced vertices
“General position”: no junctions change with small movements of the eye.
Then each line on image is one of the following:
Boundary line (edge of an object) (
→
) with right hand of arrow denoting “solid” and left hand denoting “space”Interior convex edge (+)Interior concave edge (-)
Slide18Legal Junctions
Only certain junctions are physically possible
How can we formulate a CSP to label an image?
Variables: vertices
Domains: junction labels
Constraints: both ends of a line should have the same label
x
y
(x,y) in
,
, …
Slide19Varieties of CSPs
Discrete Variables
Finite domains
Size
d
means
O(
d
n) complete assignmentsE.g., Boolean CSPs, including Boolean satisfiability (NP-complete)Infinite domains (integers, strings, etc.)E.g., job scheduling, variables are start/end times for each jobLinear constraints solvable, nonlinear undecidableContinuous variablesE.g., start/end times for Hubble Telescope observationsLinear constraints solvable in polynomial time by LP methods
Slide20Varieties of Constraints
Varieties of Constraints
Unary constraints involve a single variable (equiv. to shrinking domains):
Binary constraints involve pairs of variables:
Higher-order constraints involve 3 or more variables:
e.g., cryptarithmetic column constraints
Preferences (soft constraints):
E.g., red is better than greenOften representable by a cost for each variable assignmentGives constrained optimization problems(We’ll ignore these until we get to Bayes’ nets)
Slide21Real-World CSPs
Assignment problems: e.g., who teaches what class
Timetabling problems: e.g., which class is offered when and where?
Hardware
configuration
Gate assignment in airports
Transportation scheduling
Factory scheduling
Fault diagnosis… lots more!Many real-world problems involve real-valued variables…
Slide22Standard Search Formulation
Standard search formulation of CSPs (incremental)
Start
with
straightforward approach
, then
improve
States are defined by the values assigned so far
Initial state: the empty assignment, {}Successor function: assign a value to an unassigned variableGoal test: the current assignment is complete and satisfies all constraints
Slide23Search Methods
What does BFS do?
What does DFS do?
Slide24DFS, and BFS would be much worse!
Slide25Backtracking Search
Idea 2: Only allow legal assignments at each point
I.e. consider only values which do not conflict previous assignments
Might have to do some computation to figure out whether a value is ok
“Incremental goal test”
Depth-first search for CSPs with these two improvements is called
backtracking search
(useless name, really)
Backtracking search is the basic uninformed algorithm for CSPsCan solve n-queens for n ≈ 25
Idea 1: Only consider a single variable at each pointVariable assignments are commutative, so fix orderingI.e., [WA = red then NT = green] same as [NT = green then WA = red]
Only need to consider assignments to a single variable at each step
How many leaves are there?
Slide26Backtracking Search
What are the choice points?
Slide27Backtracking Example
Slide28Backtracking
Slide29Are we done?
Slide30Improving Backtracking
General-purpose ideas give huge gains in speed
Ordering:
Which variable should be assigned next?
In what order should its values be tried?
Filtering: Can we detect inevitable failure early?
Structure: Can we exploit the problem structure?
Slide31Forward Checking
Idea: Keep track of remaining legal values for unassigned variables (using immediate constraints)
Idea: Terminate when any variable has no legal values
WA
SA
NT
Q
NSW
V
Slide32Forward Checking
Slide33Are We Done?
Slide34Constraint Propagation
Forward checking propagates information from assigned to adjacent unassigned variables, but doesn't detect more distant failures:
WA
SA
NT
Q
NSW
V
NT and SA cannot both be blue!
Why didn’t we detect this yet?
Constraint propagation
repeatedly enforces constraints (locally)
Slide35Arc Consistency
Simplest form of propagation makes each arc
consistent
X
→
Y is consistent iff for
every
value x there is
some allowed y
WA
SA
NT
Q
NSW
V
If X loses a value, neighbors of X need to be rechecked!
Arc consistency detects failure earlier than forward checking
What’s the downside of arc consistency?
Can be run as a preprocessor or after each assignment
Slide36Arc Consistency
Runtime: O(n
2
d
3
), can be reduced to O(n
2
d
2)… but detecting all possible future problems is NP-hard – why?[demo: arc consistency animation]
Slide37Constraint
Propagation
Slide38Are We Done?
Slide39Limitations of Arc Consistency
After running arc consistency:
Can have one solution left
Can have multiple solutions left
Can have no solutions left (and not know it)
What went wrong here?
Slide40K-Consistency*
Increasing degrees of consistency
1-Consistency (Node Consistency): Each single node’s domain has a value which meets that node’s unary constraints
2-Consistency (Arc Consistency): For each pair of nodes, any consistent assignment to one can be extended to the other
K-Consistency: For each k nodes, any consistent assignment to k-1 can be extended to the k
th
node.
Higher k more expensive to compute
(You need to know the k=2 algorithm)
Slide41Strong K-Consistency
Strong k-consistency: also k-1, k-2, … 1 consistent
Claim: strong n-consistency means we can solve without backtracking!
Why?
Choose any assignment to any variable
Choose a new variable
By 2-consistency, there is a choice consistent with the first
Choose a new variable
By 3-consistency, there is a choice consistent with the first 2…Lots of middle ground between arc consistency and n-consistency! (e.g. path consistency)
Slide42Ordering: Minimum Remaining Values
Minimum remaining values (MRV):
Choose the variable with the fewest legal values
Why min rather than max?
Also called “most constrained variable”
“Fail-fast” ordering
Slide43Ordering: Degree Heuristic
Tie-breaker among MRV variables
Degree heuristic:
Choose the variable participating in the most constraints on remaining variables
Why most rather than fewest constraints?
Slide44Ordering: Least Constraining Value
Given a choice of variable:
Choose the
least constraining value
The one that rules out the fewest values in the remaining variables
Note that it may take some computation to determine this!
Why least rather than most?
Combining these heuristics makes 1000 queens feasible
Slide45Propagation with Ordering
Slide46Problem Structure
Tasmania and mainland are independent subproblems
Identifiable as connected components of constraint graph
Suppose each subproblem has c variables out of n total
Worst-case solution cost is O((n/c)(d
c
)), linear in n
E.g., n = 80, d = 2, c =20
280 = 4 billion years at 10 million nodes/sec(4)(220) = 0.4 seconds at 10 million nodes/sec
Slide47Tree-Structured CSPs
Choose a variable as root, order
variables from root to leaves such
that every node's parent precedes
it in the ordering
For i = n : 2, apply RemoveInconsistent(Parent(X
i
),X
i)For i = 1 : n, assign Xi consistently with Parent(Xi)Runtime: O(n d2)
Slide48Tree-Structured CSPs
Theorem: if the constraint graph has no loops, the CSP can be solved in O(n d
2
) time!
Compare to general CSPs, where worst-case time is O(d
n
)
This property also applies to logical and probabilistic reasoning: an important example of the relation between syntactic restrictions and the complexity of reasoning.
Slide49Nearly Tree-Structured CSPs
Conditioning: instantiate a variable, prune its neighbors' domains
Cutset conditioning: instantiate (in all ways) a set of variables such that the remaining constraint graph is a tree
Cutset size c gives runtime O( (d
c
) (n-c) d
2
), very fast for small c
Slide50Iterative Algorithms for CSPs
Greedy and local methods typically work with “complete” states, i.e., all variables assigned
To apply to CSPs:
Allow states with unsatisfied constraints
Operators
reassign
variable values
Variable selection: randomly select any conflicted variable
Value selection by min-conflicts heuristic:Choose value that violates the fewest constraintsI.e., hill climb with h(n) = total number of violated constraints
Slide51Example: 4-Queens
States: 4 queens in 4 columns (4
4
= 256 states)
Operators: move queen in column
Goal test: no attacks
Evaluation: h(n) = number of attacks
Slide52Performance of Min-Conflicts
Given random initial state, can solve n-queens in almost constant time for arbitrary n with high probability (e.g., n = 10,000,000)
The same appears to be true for any randomly-generated CSP
except
in a narrow range of the ratio
Slide53Summary
CSPs are a special kind of search problem:
States defined by values of a fixed set of variables
Goal test defined by constraints on variable values
Backtracking = depth-first search with one legal variable assigned per node
Variable ordering and value selection heuristics help significantly
Forward checking prevents assignments that guarantee later failure
Constraint propagation (e.g., arc consistency) does additional work to constrain values and detect inconsistencies
The constraint graph representation allows analysis of problem structureTree-structured CSPs can be solved in linear timeIterative min-conflicts is usually effective in practice
Slide54© Daniel S. Weld
54Chinese Food as Search?
States?
Operators?
Start state?
Goal states?
Partially specified meals
Add, remove, change dishes
Null meal
Meal meeting certain conditions (rating?)
Slide55© Daniel S. Weld
55Factoring States
Rather than state = meal
Model state’s (independent) parts, e.g.
Suppose every meal for n people
Has n dishes plus soup
Soup =
Meal 1 =
Meal 2 = …Meal n = Or… physical state =X coordinate =Y coordinate =
Slide56© Daniel S. Weld
56Chinese Constraint
Network
Soup
Total Cost
< $30
Chicken
Dish
Vegetable
Rice
Seafood
Pork Dish
Appetizer
Must be
Hot&Sour
No
Peanuts
No
Peanuts
Not
Chow Mein
Not Both
Spicy
Slide57© Daniel S. Weld
57CSPs in the Real World
Scheduling space shuttle repair
Airport gate assignments
Transportation Planning
Supply-chain management
Computer configuration
Diagnosis
UI optimizationEtc...
Slide58© Daniel S. Weld
58Classroom Scheduling
Variables?
Domains (possible values for variables)?
Constraints?
Slide59© Daniel S. Weld
59CSP as a search problem?
What are states?
(nodes in graph)
What are the operators?
(arcs between nodes)
Initial state?
Goal test?
Q
Q
Q
Slide60© Daniel S. Weld
60Crosswords