Transcript modelling

Constraint Programming:
modelling
Toby Walsh
NICTA and UNSW
Golomb rulers
• Mark ticks on a ruler
 Distance between any two ticks (not just
neighbouring ticks) is distinct
• Applications in radio-astronomy,
cystallography, …
 http://www.csplib.org/prob/prob006
Golomb rulers
• Simple solution
 Exponentially long ruler
 Ticks at 0,1,3,7,15,31,63,…
• Goal is to find minimal length rulers
 turn optimization problem into sequence of satisfaction
problems
Is there a ruler of length m?
Is there a ruler of length m-1?
….
Optimal Golomb rulers
• Known for up to 23 ticks
• Distributed internet project to find large rulers
0,1
0,1,3
0,1,4,6
0,1,4,9,11
0,1,4,10,12,17
0,1,4,10,18,23,25
Solutions grow as approximately O(n^2)
Modelling the Golomb ruler
• Variable, Xi for each tick
• Value is position on ruler
• Naïve model with quaternary constraints
 For all i>j,k>l>j
|Xi-Xj| \= |Xk-Xl|
Problems with naïve model
• Large number of quaternary constraints
 O(n^4) constraints
• Looseness of quaternary constraints
 Many values satisfy |Xi-Xj| \= |Xk-Xl|
 Limited pruning
A better non-binary model
• Introduce auxiliary variables for inter-tick
distances
 Dij = |Xi-Xj|
 O(n^2) ternary constraints
• Post single large non-binary constraint
 alldifferent([D11,D12,…]).
 Tighter constraints and denser constraint graph
Other modeling issues
• Symmetry
 A ruler can always be reversed!
 Break this symmetry by adding constraint:
D12 < Dn-1,n
 Also break symmetry on Xi
X1 < X2 < … Xn
 Such tricks important in many problems
Other modelling issues
• Additional (implied) constraints
 Don’t change set of solutions
 But may reduce search significantly
E.g. D12 < D13, D23 < D24, …
E.g. D1k at least sum of first k integers
• Pure declarative specifications are not
enough!
Solving issues
• Labeling strategies often very important
 Smallest domain often good idea
 Focuses on “hardest” part of problem
• Best strategy for Golomb ruler is instantiate
variables in strict order
 Heuristics like fail-first (smallest domain) not
effective on this problem!
Experimental results
Runtime/sec Naïve model Alldifferent
model
8-Find
2.0
0.1
8-Prove
12.0
10.2
9-Find
31.7
1.6
9-Prove
168
9.7
10-Find
657
24.3
10-Prove
> 10^5
68.3
Something to try at home?
• Circular (or modular)
Golomb rulers
 Inter-tick distance
variables more central,
removing rotational
symmetry?
• 2-d Golomb rulers
All examples of
“graceful” graphs
Summary
• Modelling decisions:
 Auxiliary variables
 Implied constraints
 Symmetry breaking constraints
• More to constraints than just declarative
problem specifications!
Case study 2:
all interval series
All interval series
• Prob007 at www.csplib.org
• Comes from musical composition
 Traced back to Alban Berg
 Extensively used by Ernst Krenek
Op.170 “Quaestio temporis”
All interval series
• Take the 12 standard pitch classes
 c, c#, d, ..
 Represent them by numbers 0, .., 11
• Find a sequence so each occurs once
 Each difference occurs once
All interval series
• Can generalize to any n (not just 12)
Find Sn, a permutation of [0,n)
such that |Sn+1-Sn| are all distinct
• Finding one solution is easy
All interval series
• Can generalize to any n (not just 12)
Find Sn, a permutation of [0,n) such that |Sn+1-Sn| are
all distinct
• Finding one solution is easy
[n,1,n-1,2,n-2,.., floor(n/2)+2,floor(n/2)-1,floor(n/2)+1,floor(n/2)]
Giving the differences [n-1,n-2,..,2,1]
Challenge is to find all solutions!
Basic methodology
• Devise basic CSP model
 What are the variables? What are the
constraints?
•
•
•
•
Introduce auxiliary variables if needed
Consider dual or combined models
Break symmetry
Introduce implied constraints
Basic CSP model
• What are the variables?
Basic CSP model
• What are the variables?
Si = j if the ith note is j
• What are the constraints?
Basic CSP model
• What are the variables?
Si = j if the ith note is j
• What are the constraints?
Si in [0,n)
All-different([S1,S2,… Sn])
Forall i<i’ |Si+1 - Si| =/ |Si’+1 - Si’|
Will this model be any good? If so, why?
If not, why not?
Basic methodology
• Devise basic CSP model
 What are the variables? What are the
constraints?
•
•
•
•
Introduce auxiliary variables if needed
Consider dual or combined models
Break symmetry
Introduce implied constraints
Improving basic CSP model
• Is it worth introducing any auxiliary
variables?
 Are there any loose or messy constraints we
could better (more compactly?) express via
some auxiliary variables?
Improving basic CSP model
• Is it worth introducing any auxiliary
variables?
 Yes, variables for the pairwise differences
Di = |Si+1 - Si|
• Now post single large all-different constraint
Di in [1,n-1]
All-different([D1,D2,…Dn-1])
Basic methodology
• Devise basic CSP model
 What are the variables? What are the
constraints?
•
•
•
•
Introduce auxiliary variables if needed
Consider dual or combined models
Break symmetry
Introduce implied constraints
Break symmetry
• Does the problem have any symmetry?
Break symmetry
• Does the problem have any symmetry?
 Yes, we can reverse any sequence
S1, S2, … Sn is an all-inverse series
Sn, …, S2, S1 is also
• How do we eliminate this symmetry?
Break symmetry
• Does the problem have any symmetry?
 Yes, we can reverse any sequence
S1, S2, …, Sn is an all-inverse series
Sn, …, S2, S1 is also
• How do we eliminate this symmetry?
• As with Golomb ruler!
D1 < Dn-1
Break symmetry
• Does the problem have any other
symmetry?
Break symmetry
• Does the problem have any other
symmetry?
 Yes, we can invert the numbers in any sequence
0, n-1, 1, n-2, …
n-1, 0, n-2, 1, …
map x onto n-1-x
• How do we eliminate this symmetry?
Break symmetry
• Does the problem have any other
symmetry?
 Yes, we can invert the numbers in any sequence
0, n-1, 1, n-2, …
n-1, 0, n-2, 1, …
map x onto n-1-x
• How do we eliminate this symmetry?
S1 < S2
Basic methodology
• Devise basic CSP model
 What are the variables? What are the
constraints?
•
•
•
•
Introduce auxiliary variables if needed
Consider dual or combined models
Break symmetry
Introduce implied constraints
Implied constraints
• Are there useful implied constraints to add?
Implied constraints
• Are there useful implied constraints to add?
 Hmm, unlike Golomb ruler, we only have
neighbouring differences
 So, no need to consider transitive closure
Implied constraints
• Are there useful implied constraints to add?
 Hmm, unlike Golomb ruler, we are not
optimizing
 So, no need to improve propagation for
optimization variable
Performance
• Basic model is poor
• Refined model able to compute all solutions
up to n=14 or so
 GAC on all-different constraints very beneficial
 As is enforcing GAC on Di = |Si+1-Si|
This becomes too expensive for large n
So use just bounds consistency (BC) for larger n
Case study 3: orthogonal
Latin squares
Modelling decisions
• Many different ways to model even simple
problems
• Combining models can be effective
 Channel between models
• Need additional constraints
 Symmetry breaking
 Implied (but logically) redundant
Latin square
• Each colour appears
once on each row
• Each colour appears
once on each column
• Used in experimental
design
 Six people
 Six one-week drug
trials
Orthogonal Latin squares
• Find a pair of Latin
squares
 Every cell has a
different pair of
elements
• Generalized form:
 Find a set of m Latin
squares
 Each possible pair is
orthogonal
Orthogonal Latin squares
1234
2143
3412
4321
11
23
34
42
22
14
43
31
1234
3412
4321
2143
33
41
12
24
44
32
21
13
• Two 4 by 4 Latin
squares
• No pair is repeated
History of (orthogonal) Latin
squares
• Introduced by Euler in 1783
 Also called Graeco-Latin or Euler squares
• No orthogonal Latin square of order 2
 There are only 2 (non)-isomorphic Latin
squares of order 2 and they are not orthogonal
History of (orthogonal) Latin
squares
• Euler conjectured in 1783 that there are no orthogonal
Latin squares of order 4n+2
 Constructions exist for 4n and for 2n+1
 Took till 1900 to show conjecture for n=1
 Took till 1960 to show false for all n>1
• 6 by 6 problem also known as the 36 officer problem
“… Can a delegation of six regiments, each of which sends a colonel,
a lieutenant-colonel, a major, a captain, a lieutenant, and a sublieutenant be arranged in a regular 6 by 6 array such that no row or
column duplicates a rank or a regiment?”
More background
• Lam’s problem
 Existence of finite projective plane of order 10
 Equivalent to set of 9 mutually orthogonal Latin
squares of order 10
 In 1989, this was shown not to be possible after 2000
hours on a Cray (and some major maths)
• Orthogonal Latin squares also used in
experimental design
A simple 0/1 model
• Suitable for integer programming
 Xijkl = 1 if pair (i,j) is in row k column l, 0 otherwise
 Avoiding advice never to use more than 3 subscripts!
• Constraints
 Each row contains one number in each square
Sum_jl Xijkl = 1
Sum_il Xijkl = 1
 Each col contains one number in each square
Sum_jk Xijkl = 1
Sum_ik Xijkl = 1
A simple 0/1 model
• Additional constraints
 Every pair of numbers occurs exactly once
Sum_kl Xijkl = 1
 Every cell contains exactly one pair of numbers
Sum_ij Xijkl = 1
Is there any symmetry?
Symmetry removal
• Important for solving CSPs
 Especially for proofs of optimality?
• Orthogonal Latin square has lots of
symmetry
 Permute the rows
 Permute the cols
 Permute the numbers 1 to n in each square
• How can we eliminate such symmetry?
Symmetry removal
• Fix first row
11 22 33 …
• Fix first column
11
23
32
..
• Eliminates all symmetry?
What about a CSP model?
• Exploit large finite domains possible in
CSPs
 Reduce number of variables
 O(n^4) -> ?
• Exploit non-binary constraints
 Problem states that squares contain pairs that
are all different
 All-different is a non-binary constraint our
solvers can reason with efficiently
CSP model
• 2 sets of variables
 Skl = i if the 1st element in row k col l is i
 Tkl = j if the 2nd element in row k col l is j
• How do we specify all pairs are different?
 All distinct (k,l), (k’,l’)
if Skl = i and Tkl = j then Sk’l’=/ i or Tk’l’ =/ j
O(n^4) loose constraints, little constraint propagation!
What can we do?
CSP model
• Introduce auxiliary variables
 Fewer constraints, O(n^2)
 Tightens constraint graph => more propagation
 Pkl = i*n + j if row k col l contains the pair i,j
• Constraints
 2n all-different constraints on Skl, and on Tkl
 All-different constraint on Pkl
 Channelling constraint to link Pkl to Skl and
Tkl
CSP model v O/1 model
• CSP model
 3n^2 variables
 Domains of size n, n
and n^2+n
 O(n^2) constraints
 Large and tight nonbinary constraints
• 0/1 model




n^4 variables
Domains of size 2
O(n^4) constraints
Loose but linear
constraints
• Use IP solver!
Solving choices for CSP
model
• Variables to assign
 Skl and Tkl, or Pkl?
• Variable and value ordering
• How to treat all-different constraint
 GAC using Regin’s algorithm O(n^4)
 AC using the binary decomposition
Good choices for the CSP
model
• Experience and small instances suggest:
 Assign the Skl and Tkl variables
 Choose variable to assign with Fail First
(smallest domain) heuristic
• Break ties by alternating between Skl and Tkl
 Use GAC on all-different constraints for Skl
and Tkl
 Use AC on binary decomposition of large alldifferent constraint on Pkl
Performance
n
4
0-1 model
CSP model
Fails t/sec AC
Fails t/sec
4
0.11 2
0.18
CSP model
GAC
Fails t/sec
2
0.38
5
1950
4.05 295
6
?
?
7*
20083 59.8 91687 51.1 57495 66.1
1.39 190
1.55
640235 657 442059 773
Case study 4:
Langford’s problem
Langford’s problem
• Prob024 @
www.csplib.org
• Find a sequence of 8
numbers
 Each number [1,4]
occurs twice
 Two occurrences of i
are i numbers apart
• Unique solution
 41312432
Langford’s problem
• L(k,n) problem
 To find a sequence of k*n
numbers [1,n]
 Each of the k successive
occrrences of i are i apart
 We just saw L(2,4)
• Due to the mathematician
Dudley Langford
 Watched his son build a
tower which solved L(2,3)
Langford’s problem
• L(2,3) and L(2,4) have unique solutions
• L(2,4n) and L(2,4n-1) have solutions
 L(2,4n-2) and L(2,4n-3) do not
 Computing all solutions of L(2,19) took 2.5 years!
• L(3,n)
 No solutions: 0<n<8, 10<n<17, 20, ..
 Solutions: 9,10,17,18,19, ..
A014552
Sequence: 0,0,1,1,0,0,26,150,0,0,17792,108144,0,0,39809640,326721800,
0,0,256814891280,2636337861200
Basic model
• What are the variables?
Basic model
• What are the variables?
Variable for each occurrence of a number
X11 is 1st occurrence of 1
X21 is 1st occurrence of 2
..
X12 is 2nd occurrence of 1
X22 is 2nd occurrence of 2
..
• Value is position in the sequence
Basic model
• What are the constraints?
 Xij in [1,n*k]
 Xij+1 = i+Xij
 Alldifferent([X11,..Xn1,X12,..Xn2,..,X1k,..Xnk
])
Recipe
• Create a basic model
 Decide on the variables
• Introduce auxiliary variables
 For messy/loose constraints
• Consider dual, combined or
0/1 models
• Break symmetry
• Add implied constraints
• Customize solver
 Variable, value ordering
Break symmetry
• Does the problem have any symmetry?
Break symmetry
• Does the problem have any symmetry?
 Of course, we can invert any sequence!
Break symmetry
• How do we break this symmetry?
Break symmetry
• How do we break this symmetry?
 Many possible ways
 For example, for L(3,9)
• Either X92 < 14 (2nd occurrence of 9 is in 1st half)
• Or X92=14 and X82<14 (2nd occurrence of 8 is in
1st half)
Recipe
• Create a basic model
 Decide on the variables
• Introduce auxiliary variables
 For messy/loose constraints
• Consider dual, combined or
0/1 models
• Break symmetry
• Add implied constraints
• Customize solver
 Variable, value ordering
What about dual model?
• Can we take a dual view?
What about dual model?
• Can we take a dual view?
• Of course we can, it’s a permutation!
Dual model
• What are the variables?
 Variable for each position i
• What are the values?
Dual model
• What are the variables?
 Variable for each position i
• What are the values?
 If use the number at that position, we cannot
use an all-different constraint
 Each number occurs not once but k times
Dual model
• What are the variables?
 Variable for each position i
• What are the values?
 Solution 1: use values from [1,n*k] with the
value i*n+j standing for the ith occurrence of j
 Now want to find a permutation of these
numbers subject to the distance constraint
Dual model
• What are the variables?
 Variable for each position i
• What are the values?
 Solution 2: use as values the numbers [1,n]
 Each number occurs exactly k times
 Fortunately, there is a generalization of all-different
called the global cardinality constraint (gcc) for this
Global cardinality constraint
• Gcc([X1,..Xn],l,u) enforces values used by
Xi to occur between l and u times
 All-different([X1,..Xn]) = Gcc([X1,..Xn],1,1)
• Regin’s algorithm enforces GAC on Gcc in
O(n^2.d)
 Regin’s papers are tough to follow but this
seems to beat his algorithm for all-different!?
Dual model
• What are the constraints?
 Gcc([D1,…Dk*n],k,k)
 Distance constraints?
Dual model
• What are the constraints?
 Gcc([D1,…Dk*n],k,k)
 Distance constraints:
• Di=j then Di+j+1=j
Combined model
• Primal and dual variables
• Channelling to link them
 What do the channelling constraints look like?
Combined model
• Primal and dual variables
• Channelling to link them
 Xij=k implies Dk=i
Solving choices?
• Which variables to assign?
 Xij or Di
Solving choices?
• Which variables to assign?
 Xij or Di, doesn’t seem to matter
• Which variable ordering heuristic?
 Fail First or Lex?
Solving choices?
• Which variables to assign?
 Xij or Di, doesn’t seem to matter
• Which variable ordering heuristic?
 Fail First very marginally better than Lex
Recipe
• Create a basic model
 Decide on the variables
• Introduce auxiliary variables
 For messy/loose constraints
• Consider dual, combined or 0/1
models
• Break symmetry
• Add implied constraints
• Customize solver
 Variable, value ordering