CS 188: Artificial Intelligence - PowerPoint Presentation

Slide1

CS 188: Artificial Intelligence

Constraint Satisfaction Problems

Instructor: Anca Dragan

A few Slides about Integer CSPs have been added by Christoph F. Eick (UH)

University of California, Berkeley

[These slides adapted from Dan Klein and Pieter

Abbeel

]

Slide2

Constraint Satisfaction Problems

N variables

x

1

x

2

domain D

constraints

states

goal test

successor function

partial assignment

complete; satisfies constraints

assign an unassigned variable

Slide3

CSP Examples

Slide4

Example: Map Coloring

Variables:

Domains:Constraints: adjacent regions must have different colors

Solutions are assignments satisfying all constraints, e.g.:

Implicit:

Explicit:

Slide5

Constraint Graphs

Slide6

Constraint Graphs

Binary CSP: each constraint relates (at most) two variables

Binary constraint graph: nodes are variables, arcs show constraintsGeneral-purpose CSP algorithms use the graph structure to speed up search. E.g., Tasmania is an independent

subproblem!

Slide7

Example: N-Queens

Formulation 1:Variables:Domains:Constraints

Slide8

Example: N-Queens

Formulation 2:Variables:

Domains:Constraints:

Implicit:

Explicit:

Slide9

Screenshot of Demo N-Queens

Slide10

News Feb. 9, 2022

Group A’s Group Homework Credit Presentation will start 3:35p!

Today’s Background: A Scrimp Boat Near Palacios---Dr. Eick will visit Palacios in April

The two Problem Set1 tasks are individual tasks; UH academic honesty applies; we will check submitted reports and programs for similarity.

Today’s Lecture

CSP

Search 7

Games

…

Group A Presentation

Taking a look at Wikipedia Page discussing Backtracking (on Feb. 14)

Demo of Search Algorithms (on Feb. 14)

[

Slide11

Example: Cryptarithmetic

Variables:Domains:

Constraints:

X

1

More on that Problem

 2021 Group A’’s Online Task

Solution: 0=7, R=4, W=6, U=2, T=8, F=1; 867 + 867 = 1734

Slide12

Group A in 2021!

Slide13

Solution

Variables:

S, E, N, D, M, O, R, Y, X1, X2, X3, X4

X1-X4 corresponds to each of the columns. X1 being the column furthest to the right, then continuing on from thereDomains:

(S, E, N, D, M, O, R, Y) ∈ {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}(X1, X2, X3, X4) ∈ {0,1}

Constraints:

alldiff

(

S, E, N, D, M, O, R, Y)

D + E = Y + 10 * X1

N + R + X1= E + 10 * X2

E + O + X2 = N + 10 * X3

S + M + X3 = O + 10 * X4M = X4 = 1

Slide14

Solution cont.

M has to be 1 because it comes from a carry

Knowing that no two variables have the same value, also S and M are leading digits they can not be zero, so there always be a carry because M is not 0

Solution: M=1, S=9, O=0, E=5, N=6, R=8, D=7, Y=2; hence 9567+1085=10652 As mentioned above, M has to be 1 since it is a carry. So we can automatically determine that S = 9, and O = 0, because 9 + 1 = 10

Since we have determined O = 0, and each letter cannot have the same digit, E =/= N, so therefore a carry is required in this column, thus E can be assigned a 5 and N = 6Furthermore, on the next column, since N is 6 and E is 5, R can be determined to be a value of 8 (since 9 is already taken) so when added with N, it can create a carry for the latter column

Finally, we can assign any remaining value to D (7) and then add the value of E to determine Y, which is 2 with a carry of 1 for the latter column

Slide15

Example: Sudoku

Variables:

Each (open) square

Domains:

{1,2,…,9}

Constraints:

9-way

alldiff

for each row

9-way alldiff for each column

9-way alldiff for each region

(or can have a bunch of pairwise inequality constraints)

Slide16

Constraint Satisfaction Problems

Standard search problems:State is a “black box”: arbitrary data structure

Goal test can be any function over statesSuccessor function can also be anything

Constraint satisfaction problems (CSPs):A special subset of search problemsState is defined by

variables Xi

with values from a

domain

D

(sometimes

D

depends on

i)

Goal test is a set of constraints specifying allowable combinations of values for subsets of variables

Allows useful general-purpose algorithms with more power than standard search algorithms

Slide17

Integer Constraint Satisfaction Example Problem

Variables A, B, C, D which take values in {1,…,10}

Constraints:(C1) C**2+D**2<A**2(C2) B+C= A

(C3) C**2=A*B+1Solution: A=8, B=3, C=5, D=2

Slide18

Brute Force Search

Variable A, B, C, D which take values in {1,…,10}

Constraints:(C1) C**2+D**2<A**2(C2) B+C= A

(C3) C**2=A*B+1FOR A=1,…,A=10 FOR B=1,…,B=10

FOR C=1,…,C=10 FOR D=1,…,D=10 DO {

IF C1 and C2 and C3 THEN

WriteSolution

(A,B,C,D)}

Any ideas how to reduce the complexity of this search?

Slide19

Mathematical Analysis of the Constraints

Variable A, B, C, D which take values in {1,…,10}

Constraints:(C1) C**2+D**2<A**2  C<A and D<A

(C2) B+C= A B<A(C3) C**2=A*B+1

More efficient loop FOR A=1,…,A=10 FOR B=1,…,B=A-1

FOR C=1,…,C=A-1

FOR D=1,…,D=A-1 DO {

IF C1 and C2 and C3 THEN

WriteSolution

(A,B,C,D)}

Slide20

Variable Elimination

Variable A, B, C, D which take values in {1,…,10}

Constraints:(C1) C**2+D**2<A**2 (C2) B+C= A

(C3) C**2=A*B+1Idea: Remove Variable A by replacing it by B+C!New Constraints only using 3 variables

(C1’) C**2+D**2< (B+C)**2(C2’) C**2=(B+C)*B+1 Solve it finding valid assignments for (B, C, D) and then set A to B+C

Slide21

Varieties of CSPs and Constraints

Slide22

Varieties of CSPs

Discrete VariablesFinite domains

Size d means O(

dn) complete assignments

E.g., Boolean CSPs, including Boolean satisfiability (NP-complete)Infinite domains (integers, strings, etc.)

E.g., job scheduling, variables are start/end times for each job

Linear constraints solvable, nonlinear

undecidable

Continuous variables

E.g., start/end times for Hubble Telescope observations

Linear constraints solvable in polynomial time by LP methods (see cs170 for a bit of this theory)

Slide23

Varieties of Constraints

Varieties of ConstraintsUnary constraints involve a single variable (equivalent to reducing domains), e.g.:

Binary constraints involve pairs of variables, e.g.:Higher-order constraints involve 3 or more variables:

e.g., cryptarithmetic column constraints

Preferences (soft constraints):

E.g., red is better than green

Often

representable

by a cost for each variable assignment

Gives constrained optimization problems

(We’ll ignore these until we get to

Bayes’ nets)

Slide24

Real-World CSPs

Assignment problems: e.g., who teaches what classTimetabling problems: e.g., which class is offered when and where?

Hardware configurationTransportation schedulingFactory scheduling

Circuit layoutFault diagnosis… lots more!

Many real-world problems involve real-valued variables…

Slide25

Solving CSPs

Slide26

Standard Search Formulation

Standard search formulation of CSPs

States defined by the values assigned so far (partial assignments)Initial state: the empty assignment, {}Successor function: assign a value to an unassigned variable

Goal test: the current assignment is complete and satisfies all constraintsWe’ll start with the straightforward, naïve approach, then improve it

Slide27

Backtracking Search

Slide28

Backtracking Search

Backtracking search is the basic uninformed algorithm for solving CSPs

Idea 1: One variable at a time

Variable assignments are commutative, so fix ordering -> better branching factor! I.e., [WA = red then NT = green] same as [NT = green then WA = red]

Only need to consider assignments to a single variable at each stepIdea 2: Check constraints as you go

I.e. consider only values which do not conflict previous assignments

Might have to do some computation to check the constraints

“Incremental goal test”

Depth-first search with these two improvements

is called

backtracking search

(not the best name)

Can solve n-queens for n

 25

Slide29

Backtracking Example

[Demo: coloring -- backtracking]

Slide30

Backtracking Search

Backtracking = DFS + variable-ordering + fail-on-violationWhat are the choice points?

Slide31

Improving Backtracking

General-purpose ideas give huge gains in speed

Ordering:

Which variable should be assigned next?In what order should its values be tried?

Filtering: Can we detect inevitable failure early?

Slide32

Ideas Already Discussed So far for Task 3 PS1

Intelligent Variable Ordering---can be dynamic e.g. (A=1, B=40, C=39) but then (A=2, D=12, B=12)

Intelligent Variable Assignment Ordering not necessarily A=1, A=2, A=3 but maybe A=5, A=4,A=7 Create “<“ from constraints and use those to shorten loopsUse variable elimination for constraints <variable>=<expression not containing variable> e.g. A=B+C but not A=A**2*B-A***3 to reduce number of loops

Do forward checking for aborting hopeless search paths quickly

Slide33

Other Ideas for Task 3 PS1

Take advantage of the hierarchical nature of the CSP problems in Task3: if S=(A=32,B=1, C=1, D=12, E=50, F=12, G=15,H=49,…) is a solution for Task C then S=(A=32,B=1, C=1, D=12, E=50, F=12) is for problem A!

Idea: Find all solutions S1,…,S

r for Problem AFOR i=1,..,r DO { Use variable assignments of S

i for A,B,C,D, E, F Solve CSP Problem B for the remaining variable G, H, I, and J}Comment: Stop as soon as you find a solution for problem B!

The name of the game is everything that reduces the value of the

nva

counter for the search of the 3 CSPs is good!

I am quite certain, there are many other ideas you can use to reduce search complexity!

Will have more discussions about Task3 next week!

Slide34

Hill Climbing Appraoches Task 3 PS1

This approach searches complete solutions; e.g. S=(A=32,B=1, C=1, D=12, E=50, F=12) is for problem A!

The objective function could be the number of constraints violated, but there are other choices… The neighborhood could be the set of solutions that can be obtained by changing the value of one variablethat can be obtained by changing the value of two variables

Now classical hill climbing, randomized hill climbing, and simulated annealing could be used for Task3.Due to initialization sensitivity: use randomized hill climbing with restart; e.g. run the hill climbing approach 1000 times with a different starting solution. The outlined approach is kind of brute force and does not use any domain specific knowledge! There are ways to incorporate domain specific knowledge into the hill climbing approaches, but this will be quite challenging!

There will be a lot of premature convergence---e.g. hill climbing terminates with a solution that still violates two constraints---when using this approach…On the positive side: Hill climbing approaches are very fast and capable to search a lot of solutions!

Slide35

Ordering

Resume here

Slide36

On Variable and Variable Assignment Ordering

Variable A, B, C, D which take values in {1,…,10}

Constraints:(C1) C**2+D**2<A**2(C2) B+C= A

(C3) C**2=A*B+1FOR A=1,…,A=10 FOR B=1,…,B=10

FOR C=1,…,C=10 FOR D=1,…,D=10 DO {

IF C1 and C2 and C3 THEN

WriteSolution

(A,B,C,D)}

Claim:

The order in which we process the variables (here : A,B,C,D) and the order in which assign values to the variables (here: 1,…, 10) makes a big difference concerning how quickly we find a solution to a CSP!

Slide37

News and Preview Lecture Feb. 14, 2021

Today’s Background: Amboseli National Park, Kenya with Kilimanjaro (Tanzania) in the background but the elephant is a citizen of Kenya!Undergraduate research

Undergraduate Research - Course Credit - University of Houston (uh.edu) in Dr. Eick’s Lab:

http://www2.cs.uh.edu/~ceick/ai/UH-DAIS-Research-2022.pptx Group A can you submit your slides via e-mail to Dr. Eick and also post them in the respective channel.

Today’s Program:

Group B GHC Presentation

Brief Discussion: Report for Task1

Finish Discussion CSP

Games

Discussion of Backtracking Wiki page

Demo of Search Algorithms

Group C GHC Presentation

Slide38

Ordering: Minimum Remaining Values

Variable Ordering: Minimum remaining values (MRV):

Choose the variable with the fewest legal left values in its domain

Why min rather than max?

Also called “most constrained variable”“Fail-fast” ordering

Slide39

Ordering: Least Constraining Value

Value Ordering: Least Constraining Value

Given a choice of variable, choose the

least constraining valueI.e., the one that rules out the fewest values in the remaining variablesNote that it may take some computation to determine this! (E.g., rerunning filtering)

Why least rather than most?

Combining these ordering ideas makes

1000 queens feasible

Slide40

Filtering

Keep track of domains for unassigned variables and cross off bad options

Slide41

Filtering: Forward Checking

Filtering: Keep track of domains for unassigned variables and cross off bad optionsForward checking: Cross off values that violate a constraint when added to the existing assignment

In general, this approach interleaves inference with search

[Demo: coloring -- forward checking]

Slide42

Filtering: Constraint Propagation

Forward checking propagates information from assigned to unassigned variables, but doesn't provide early detection for all failures:

NT and SA cannot both be blue!

Why didn’t we detect this yet?

Constraint propagation: reason from constraint to constraint

WA

SA

NT

Q

NSW

V

Reminder of the slides will only very briefly discussed in 2022!

Slide43

Consistency of A Single Arc

An arc X  Y is

consistent iff for every x in the tail there is some

y in the head which could be assigned without violating a constraint

Forward checking?

Enforcing consistency of arcs pointing to each new assignment

Delete from the tail!

WA

SA

NT

Q

NSW

V

Slide44

Arc Consistency of an Entire CSP

A simple form of propagation makes sure

all

arcs are consistent:

Important: If X loses a value, neighbors of X need to be rechecked!

Arc consistency detects failure earlier than forward checking

Can be run as a preprocessor or after each assignment

What’s the downside of enforcing arc consistency?

Remember: Delete from the tail!

WA

SA

NT

Q

NSW

V

Slide45

Enforcing Arc Consistency in a CSP

Runtime: O(n2d

3), can be reduced to O(n2d2)

… but detecting all possible future problems is NP-hard – why?

Slide46

Limitations of Arc Consistency

After enforcing arc consistency:Can have one solution leftCan have multiple solutions left

Can have no solutions left (and not know it)Arc consistency still runs inside a backtracking search!

[Demo: coloring -- arc consistency]

[Demo: coloring -- forward checking]

Slide47

K-Consistency

Increasing degrees of consistency

1-Consistency (Node Consistency): Each single node’s domain has a value which meets that node’s unary constraints2-Consistency (Arc Consistency): For each pair of nodes, any consistent assignment to one can be extended to the other

K-Consistency: For each k nodes, any consistent assignment to k-1 can be extended to the kth

node.Higher k more expensive to compute

(You need to know the k=2 case: arc consistency)

Slide48

Strong K-Consistency

Strong k-consistency: also k-1, k-2, … 1 consistent

Claim: strong n-consistency means we can solve without backtracking!Why?Choose any assignment to any variable

Choose a new variableBy 2-consistency, there is a choice consistent with the firstChoose a new variable

By 3-consistency, there is a choice consistent with the first 2…

Lots of middle ground between arc consistency and n-consistency! (e.g. k=3, called path consistency)