/
Knowledge Compilation: Representations and Lower Bounds Paul Beame Knowledge Compilation: Representations and Lower Bounds Paul Beame

Knowledge Compilation: Representations and Lower Bounds Paul Beame - PowerPoint Presentation

calandra-battersby
calandra-battersby . @calandra-battersby
Follow
345 views
Uploaded On 2019-11-04

Knowledge Compilation: Representations and Lower Bounds Paul Beame - PPT Presentation

Knowledge Compilation Representations and Lower Bounds Paul Beame University of Washington with Jerry Li Vincent Liew Sudeepa Roy Dan Suciu Representing Boolean Functions Circuits Boolean formulas treelike circuits CNFs DNFs ID: 763251

dnnfs decision node decomposable decision dnnfs decomposable node read dnnf bps size fbdds boolean diagrams model bounds variables nodes

Share:

Link:

Embed:

Download Presentation from below link

Download Presentation The PPT/PDF document "Knowledge Compilation: Representations a..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Knowledge Compilation: Representations and Lower Bounds Paul Beame University of Washington with Jerry Li, Vincent Liew , Sudeepa Roy, Dan Suciu

Representing Boolean Functions Circuits Boolean formulas (tree-like circuits), CNFs, DNFs De Morgan basis ( ⋀,⋁,¬ ) Can be negation normalized (all negations at inputs) Decomposable ⋀ gates (DNNF) Decision Diagrams Decision Trees Branching Programs (Decision DAGs) OBDDs (oblivious read-once BPs) FBDDs (read-once BPs) k-OBDDs, read-k BPs, etc. Hybrid and non-syntactic representations d-DNNFs (DNNFs with deterministic ⋁ gates) Decision-DNNFs (FBDDs with decomposable ⋀ nodes) Sentential Decision Diagrams (SDDs) Decomposable Logic Decision Diagrams (DLDDs)

Decomposable ⋀ [ Darwiche 2001] ∧ No variables in common between subcircuits

Representing Boolean Functions Circuits Boolean formulas (tree-like circuits), CNFs, DNFs De Morgan basis ( ⋀,⋁,¬ ) Can be negation normalized (all negations at inputs)Decomposable ⋀ gates (DNNF)Decision DiagramsDecision TreesBranching Programs (Decision DAGs)OBDDs (oblivious read-once BPs) FBDDs (read-once BPs)k-OBDDs, read-k BPs, etc.Hybrid and non-syntactic representationsd-DNNFs (DNNFs with deterministic ⋁ gates)Decision-DNNFs (FBDDs with decomposable ⋀ nodes)Sentential Decision Diagrams (SDDs) Decomposable Logic Decision Diagrams (DLDDs)

Decision Trees 0 1 1 0 1 0 0 1 1 0 0 1 0 1 1 0                                                                                           Decision Tree for  

Decision DAGs (Branching Programs/BDDs) 1 0                             OBDD for                

0 1                                                         Branching programs FBDD : Diagram is read-once i.e., every path contains distinct variables not allowed for FBDD

Representing Boolean Functions Circuits Boolean formulas (tree-like circuits), CNFs, DNFs De Morgan basis ( ⋀,⋁,¬ ) Can be negation normalized (all negations at inputs)Decomposable ⋀ gates (DNNF)Decision DiagramsDecision Trees Branching Programs (Decision DAGs)OBDDs (oblivious read-once BPs)FBDDs (read-once BPs) k-OBDDs, read-k BPs, etc.Hybrid and non-syntactic representationsd-DNNFs (DNNFs with deterministic ⋁ gates)Decision-DNNFs (FBDDs with decomposable ⋀ nodes)Sentential Decision Diagrams (SDDs)Decomposable Logic Decision Diagrams (DLDDs)

d-DNNF Decomposable ⋀ ∧ No variables in common between subcircuits Deterministic ⋁ ∨ Subcircuits cannot both evaluate to true FBDD Decision Node ⇒ Both       ∨ ∧ ∧     decision-DNNF

Decomposable Logic Decision Diagrams (DLDDs) Generalization of decision-DNNF s: not just FBDDs + decomposable ⋀ nodesAlso ¬ nodes, decomposable binary ⋁, XOR, etcsub-DAGs for each node are labelled by disjoint sets of variables 10

Model Counting Model Counting Problem: Given: a Boolean function Compute: = #Models (satisfying assignments) of with weights: Probability Computation Problem Given: , and independent C ompute: Problems are #P hard but can be computed in practice for classes of problems on 100-10,000 vars.   11

d-DNNF to Sum-Product Network [ Darwiche 2001] d-DNNF Sum-Product Network ∨ ∧ ∧ ⊕ ⊗ ⊗

Networks of Sums and Products Directed acyclic graphs E ach input node is labeled or for some variable Each internal node is labeled by ⊕ or ⊗ . For a given set of probabilities for variables, each node is assigned a value given by:Node labeled has value Node labeled has value ⊕ node sums the values of its children ⊗ node multiplies the values of its children Easily computed in time linear in size of the network  

Compilation Desiderata for Compiled form Makes model counting simple Concise Easy to find Syntactic, if possible Canonical, if possibleIncrementally constructible, if possible 14

Representing Boolean Functions Circuits Boolean formulas (tree-like circuits), CNFs, DNFs De Morgan basis ( ⋀,⋁,¬ ) Can be negation normalized (all negations at inputs) Decomposable ⋀ gates (DNNF)Decision DiagramsDecision TreesBranching Programs (Decision DAGs)OBDDs (oblivious read-once BPs)FBDDs (read-once BPs)k-OBDDs, read-k BPs, etc.Hybrid and non-syntactic representations d-DNNFs (DNNFs with deterministic ⋁ gates)Decision-DNNFs (FBDDs with decomposable ⋀ nodes)Sentential Decision Diagrams (SDDs)Decomposable Logic Decision Diagrams (DLDDs) All these yield sum-product networks for weighted model counting of same size as the representation Follows from result for d-DNNFs since they are the most general

Representing Boolean Functions Circuits Boolean formulas (tree-like circuits), CNFs, DNFs De Morgan basis ( ⋀,⋁,¬ ) Can be negation normalized (all negations at inputs) Decomposable ⋀ gates (DNNF) Decision Diagrams Decision Trees Branching Programs (Decision DAGs) OBDDs (oblivious read-once BPs) FBDDs (read-once BPs) k-OBDDs, read-k BPs, etc. Hybrid and non-syntactic representations d-DNNFs (DNNFs with deterministic ⋁ gates) Decision-DNNFs (FBDDs with decomposable ⋀ nodes) Sentential Decision Diagrams (SDDs) Decomposable Logic Decision Diagrams (DLDDs) Syntactic: Consistency of representation can be verified efficiently

Circuits Boolean formulas (tree-like circuits), CNFs, DNFs De Morgan basis ( ⋀,⋁,¬ ) Can be negation normalized (all negations at inputs) Decomposable ⋀ gates (DNNF)Decision DiagramsDecision Trees Branching Programs (Decision DAGs)OBDDs (oblivious read-once BPs)FBDDs (read-once BPs)k-OBDDs, read-k BPs, etc. Hybrid and non-syntactic representationsd-DNNFs (DNNFs with deterministic ⋁ gates)Decision-DNNFs (FBDDs with decomposable ⋀ nodes)Sentential Decision Diagrams (SDDs)Decomposable Logic Decision Diagrams (DLDDs) Representing Boolean FunctionsCanonical: Unique given small side information

Outline Basic Knowledge Representations Connections to exact model counters SAT-solvers vs compiled forms Lower bounds for decision-DNNFs/DLDDs Applications and “lifted vs. grounded” inference Conversion theoremSDDs and lower bounds via communication complexityDNNF lower boundsA stronger general representation

CDP Relsat Cachet SharpSAT c2d Dsharp …19Exact Model Counters for CNF Search-based/DPLL-based (explore the assignment-space and count the satisfying ones) Knowledge Compilation-based (compile F into a “computation-friendly” form) [Survey by Gomes et. al. ’09] Both techniques explicitly or implicitly use DPLL-based algorithms produce FBDD or Decision-DNNF compiled forms [Huang-Darwiche’05, ’07] [ Birnbaum et. al. ’99] [ Bayardo Jr. et. al. ’97, ’00] [Sang et. al. ’05] [ Thurley ’06] [ Darwiche ’04] [ Muise et. al. ’12]

Compilation Desiderata for Compiled form Makes model counting simple Concise E asy to find Syntactic, if possibleCanonical, if possible Compiled size Search time Even if construction of compiled form is only implicitCan be exponential gap in terms of # of variables e.g. an UNSAT formula has constant compiled size  20and Search

Model Counters use Extensions to DPLL Caching Subformulas Cachet, SharpSAT , c2d, DsharpComponent Analysis Relsat, c2d, Cachet , SharpSAT , DsharpConflict Directed Clause Learning Cachet, SharpSAT, c2d, Dsharp 21Traces of DPLL + caching + (clause learning)  FBDD DPLL + caching + components + (clause learning)  Decision-DNNF

DPLL Algorithms for Model Counting 22 x z 0 y 1 u 0 1 1 0 w 1 0 0 1 1 0 u 1 1 1 0 w 1 0 0 1 1 0 1 0 1 0 0 1 1 1 F: ( xy )  (x uw )  (x u w z)  u  w  z  u  w w  u  w ½ ¾ ¾ y(  u  w ) 3 / 8 7 / 8 5 / 8 w ½ Assume uniform distribution for simplicity // basic DPLL: Function Pr ( F ) : if F = false then return 0 if F = true then return 1 select a variable x , return ½ Pr (FX=0 ) + ½ Pr( FX=1)

DPLL Algorithms for Model Counting 23 x z 0 y 1 u 0 1 1 0 w 1 0 0 1 1 0 u 1 1 1 0 w 1 0 0 1 1 0 1 0 1 0 0 1 1 1 F: ( xy )  (x uw )  (x u w z)  u  w  z  u  w w  u  w ½ ¾ ¾ y(  u  w ) 3 / 8 7 / 8 5 / 8 w ½ The trace is a Decision-Tree for F Decision-Node

Caching 24 x z 0 y 1 u 0 1 1 0 w 1 0 0 1 1 0 u 1 1 1 0 w 1 0 0 1 1 0 F: ( xy )  (x uw )  (x u w z)  u  w  z  u  w w  u  w y(  u  w ) w // DPLL with caching: Cache F and Pr ( F ) ; look it up before computing // basic DPLL: Function Pr ( F ) : if F = false then return 0 if F = true then return 1 select a variable x , return ½ Pr (FX=0 ) + ½ Pr( FX=1)

Caching & FBDDs 25 x z 0 y 1 0 1 0 u 1 1 1 0 w 1 0 0 1 1 0 F: ( xy )  (x uw )  (x u w z)  u  w  z  u  w w y(  u  w ) The trace is an FBDD for F

Component Analysis 26 x z 0 y 1 0 1 0 u 1 1 1 0 w 1 0 0 1 1 0 F: ( xy )  (x uw )  (x u w z)  u  w  z  u  w w y  (  u  w ) // DPLL with component analysis (and caching): if F = G  H where G and H have disjoint sets of variables Pr ( F ) = Pr ( G ) × Pr ( H) // basic DPLL: Function Pr( F): if F = false then return 0 if F = true then return 1 select a variable x, return ½ Pr( FX=0) + ½ Pr (FX=1 )

Components & decision-DNNF 27  x z 1 u 1 1 1 0 w 1 0 0 1 1 0  u  w  z w y  (  u  w ) 0 y 1 0 F: ( xy )  (x uw )  (x u w z) The trace is a decision-DNNF [Huang- Darwiche ’05, ’07] FBDD + Decomposable ⋀ nodes Decision Node y 0 1 AND Node  u  w

Decomposable Logic Decision Diagrams (DLDDs) Generalization of decision-DNNF s: not just decomposable ⋀ nodesAlso ¬ nodes, decomposable binary ⋁, XOR, etcsub-DAGs for each node are labelled by disjoint sets of variables Advantage over decision-DNNFs: Set of functions with concise representations is closed under negationConvenient since model counters are usually designed to work with CNF but formulas often naturally expressed in DNF. 28

Outline Basic Knowledge Representations Connections to exact model counters SAT-solvers vs compiled forms Lower bounds for decision-DNNFs/DLDDs Applications and “lifted vs. grounded” inference Conversion theoremSDDs and lower bounds via communication complexityDNNF lower boundsA stronger general representation

Conversion Theorem [B-Li-Roy- Suciu ICDT 2014, ACM TODS 2017] Any Boolean function represented by a DLDD (or decision-DNNF) 𝓓 with nodes can be represented by an FBDD 𝓕 with at most nodesMoreover, from 𝓓 we can construct 𝓕 in time linear in its size.[Razgon 2016] Conversion is asymptotically optimal for decision-DNNFs, even when nondeterministic FBDDs are allowed   30

Implications Many previous exponential lower bounds for FBDD s ⇒ exponential lower bounds for DLDDs/decision-DNNFsA lower bound for FBDDs implies a lower bound for DLDDs/decision-DNNFs Even applies to simple explicit 2-DNF formulas [Bollig-Wegener 2000] [Wegener 2002]But what about functions we really care about?  31

Applications of exact model counters Finite model theory: First order formulas over finite domains Bayesian inference Statistical relational models Combinations of logic and probability Probabilistic databasesRestrictions of statistical relational models32

A class of database examples Dichotomy Theorem [ Dalvi , Suciu 12 ] Model counting a Boolean combination of is either #P-hard, e.g. , or Polynomial time computable using “lifted inference” (inclusion-exclusion), e.g. and there is a simple condition to tell which case holds   33      

Example: is hard for FBDDs   Intuition: First term of easy if read by rows Second term of easy if read by columns Proving hardness is more complicated [B-Li-Roy- Suciu 14] uses “admissible paths” in any FBDD  

Lower Bounds Theorem [B-Li-Roy- Suciu ] : Any Boolean function of that depends on all of them requires FBDD size Corollary : Any decision-DNNF/DLDD for a Boolean combination of that depends on all them requires size Corollary : SAT-solver based exact model counting requires time e ven on probabilistic DB instances that are solvable via time algorithms using “lifted inference”.   35

“Lifted” vs “Grounded” Inference “Grounded” inference Works with propositional groundings of the first-order expressions given by the model “Lifted” inference Works with the first-order formulas and does higher level calculations (e.g. inclusion-exclusion) Folklore sentiment: Lifted inference is strictly stronger than grounded inferenceThese examples give a clear proof36

37 Proof of Conversion Theorem Decision-DNNF FBDD Efficient construction Size   Size  

38 Convert decomposable ⋀ -nodes to paths between decision-nodes while representing the same function   Decision-DNNF  FBDD

First attempt 39  G H 0 1 0 1 G H 0 0 1 Decision-DNNF FBDD G and H do not share variables, so every variable is still tested at most once on any path 1 FBDD

But, sub-DAGs may be shared 40  G H 0 1 0 1 Decision-DNNF   Conflict! g ’ h G H 0 0 1 H G 0 1 0 g ’ h

41  G H 0 1 0 1   g ’ h Obvious Solution: Replicate Nodes G H No conflict  can apply original idea But, may need recursive replication Can have exponential blowup!

Main Idea: Replicate Smaller Sub-DAG 42  Edges coming from other nodes in the decision-DNNF Smaller sub-DAG Larger sub-DAG Each ⋀ -node creates a private copy of its smaller sub-DAG

Light and Heavy Edges 43  Smaller sub-DAG Larger sub-DAG Light Edge Heavy Edge Each ⋀ -node creates a private copy of its smaller sub-DAG Recursively, each node is replicated every time it is in a smaller sub-DAG #Copies of = #sequences of light edges leading to  

Quasipolynomial Conversion 44    = Max # light edges on any path since #Copies of each node   # Nodes in FBDD ≤  

Outline Basic Knowledge Representations Connections to exact model counters SAT-solvers vs compiled forms Lower bounds for decision-DNNFs/DLDDs Applications and “lifted vs. grounded” inference Conversion theoremSDDs and lower bounds via communication complexityDNNF lower boundsA stronger general representation

Sentential Decision Diagrams (SDDs)[ Darwiche 2011] Special case of d-DNNFs, generalize OBDDs Preserves many nice OBDD properties Determined by a vtree: an ordered binary tree with distinct variables on the leavesyields hierarchical partitioning of the variablesSDD: an AND-OR dag based on the vtreeAND: splits the variables by the partition in the vtreeOR: left children of its AND s are disjoint (partition)                     OR OR OR OR OR AND AND   1 2 3

f Input a   Input b   010 11 0   Communication complexity is: # required message bits between Alice and Bob to compute   Alice Bob Communication Complexity

Communication protocols from vtrees 1/3-2/3 node   What are Alice and Bob’s inputs given vtree ?   Bob’s Variables Alice’s Variables        

Communication protocols from SDDs On input to , Alice sends Bob the name of an SDD node corresponding to 1/3-2/3 node that is consistent with and she guesses is consistent with Bob’s input to .   Bob’s Variables Alice’s Variables         Bob’s outputs 1 if node is consistent with and output on value is 1.  

Communication Complexity and SDDs Lemma: Among all SDD nodes consistent with , there is precisely one node consistent with Corollary: Any SDD for of size yields a bit unambiguous nondeterministic protocol for under some 1/3-2/3 partition of its variables.Theorem [B-Liew UAI 2015]: Any SDD for of size yields a bit deterministic protocol for under some 1/3-2/3 partition of its variables Proof : Use result of [ Yannakakis 1991] showing that unambiguous protocols of complexity can be simulated by deterministic ones of complexity .  

Applications: SDD lower bounds [B- Liew 2015] Fact : Every under every 1/3-2/3 partition of its variables contains a set disjointness problem of size requires communication complexity Yields a lower bound for SDDs for all UCQ queries except those with small OBDD representations. The following DB query has an size FBDD but requires a size SDD   Results also apply to all “structured d-DNNF” via [ Pipatsrisawat-Darwiche 2010]

Representing Boolean Functions Circuits Boolean formulas (tree-like circuits), CNFs, DNFs De Morgan basis ( ⋀,⋁,¬ ) Can be negation normalized (all negations at inputs) Decomposable ⋀ gates (DNNF) Decision DiagramsDecision Trees Branching Programs (Decision DAGs)OBDDs (oblivious read-once BPs)FBDDs (read-once BPs)k-OBDDs, read-k BPs, etc.Hybrid and non-syntactic representationsd-DNNFs (DNNFs with deterministic ⋁ gates)Decision-DNNFs (FBDDs with decomposable ⋀ nodes) Sentential Decision Diagrams (SDDs) Decomposable Logic Decision Diagrams (DLDDs) Require size size for simple DB queries  

Representing Boolean Functions Circuits Boolean formulas (tree-like circuits), CNFs, DNFs De Morgan basis ( ⋀,⋁,¬ )Can be negation normalized (all negations at inputs) Decomposable ⋀ gates (DNNF)Decision DiagramsDecision TreesBranching Programs (Decision DAGs)OBDDs (oblivious read-once BPs) FBDDs (read-once BPs)k-OBDDs, read-k BPs, etc.Hybrid and non-syntactic representationsd-DNNFs (DNNFs with deterministic ⋁ gates)Decision-DNNFs (FBDDs with decomposable ⋀ nodes)Sentential Decision Diagrams (SDDs)Decomposable Logic Decision Diagrams (DLDDs)

Lower Bounds for DNNFs Generalized version of Conversion Theorem: Size DNNF yields size nondeterministic FBDD. [ Razgon 2015, B-Liew 2015]Optimal even for decision-DNNF [Razgon 2015]Corollary: PERM on given by PERM ()=1 iff is a permutation matrix requires DNNF size [ Bova,Capelli,Mengel,Slivovsky 2016] Wide range of separations and purely exponential lower bounds on DNNFs and d-DNNFs Separations use general communication complexity methods of [ Pipatsrisawat-Darwiche 2010]  

Outline Basic Knowledge Representations Connections to exact model counters SAT-solvers vs compiled forms Lower bounds for decision-DNNFs/DLDDs Applications and “lifted vs. grounded” inference Conversion theoremSDDs and lower bounds via communication complexityDNNF lower bounds A stronger general representation

Sum-Product-Complement (SPC) Networks Directed acyclic graphs Each input node is labeled or for some variable Each internal node is labeled by ⊕ or ⊗ or unary ⊖ . For a given set of probabilities for variables, each node is assigned a value given by:Node labeled has value Node labeled has value ⊕ node sums the values of its children ⊗ node multiplies the values of its children ⊖ node computes 1 – (the value of its child) Easily computed in time linear in size of the network  

The ultimate model-counting representation? An SPC network corresponds to a Boolean representation as a D e Morgan circuit ( ⋀,⋁,¬ ) with:Decomposable ⋀ gatesDeterministic ⋁ gatesNo requirement to normalize negationsWhat to call them? DDDM or d-DDM or D3M circuits? No non-trivial lower bounds are known for such representations![B, Liew 2015 unpublished] These D3M representations have quasipolynomial size simulations by a special case of read-once parity branching programs.

Nondeterministic Read-Once BPs/FBDDs                   Value is 1 iff there exists a consistent path to the 1-sink node ≡ Parity Read-Once BPs Value is 1 iff there are an odd # of consistent paths to the 1-sink node         ⊕

Directions and Open Problems 59 Other separations and lower bounds shown by [ Bova,Capelli,Mengel,Slivovsky 2016] See tomorrow’s talk by Simone BovaFind algorithms, on par with those that build decision-DNNFs and SDDs, to build the more general D3M representationsProve lower bounds on the size of D3 M representations

Thank You 60