Lasserre - PowerPoint Presentation

Slide1

Lasserre Hierarchy, Higher Eigenvalues, and Graph Partitioning

Venkatesan GuruswamiCarnegie Mellon University

--- Mysore Park Workshop, August 10, 2012 ---

Joint work with

Ali

Kemal

SinopSlide2

Talk OutlineIntroduction to problems we studyLaplacian eigenvalues and our resultsLasserre hierarchyCase study: Minimum bisectionConcluding remarksSlide3

Graph partitioning problems Minimum bisection: Given edge-weighted graph G=(V,E,W), find partition of vertices into two equal parts cutting as few (in weight) edges as possible:More generally: Minimum -section

Find subset S  V of size  to minimize cut size G(S) = weight of edges leaving S = |E(S,Sc)|Related problems:Small set expansion: weight vertices by degree, find

S  V minimizing G

(S)

with

Vol

(S)= sum of degrees in S = 

Sparsest cut (find best ratio cut over all sizes)

1

2

3

4

1

2

3

4

Cost=2Slide4

Many Practical ApplicationsBuilding block for divide-and-conquer on graphsVLSI layoutPacket routing in distributed networks Clustering and image segmentationRoboticsScientific computingSlide5

Approximation AlgorithmsUnfortunately such cut problems are NP-hard. Find an α-factor approximation instead.If minimum cost = OPT, algorithm always finds a solution with value

≤ α OPT.Rounding Algorithm: Solve a convex relaxation and round the “fractional” solution to “integral” solution.

OPT

Algorithm

α

OPT

0

Relaxation

Integrality Gap

Slide6

A notorious problem: Unique GamesGraph G=(V,E)Number of labels k For each edge e=(u,v), A permutationGoal: Label vertices with k colors to minimize number of unsatisfied edges Slide7

Unique Games: ExampleSuppose k=3.

1

2

3

1

2

3

1

2

3

1

2

3

Label Extended Graph

Constraint Graph

A cloud of k=3 vertices

per vertex of GSlide8

Example

A labeling:

1

2

3

1

2

3

1

2

3

1

2

3

Label Extended Graph

Constraint GraphSlide9

Unique Games: ExampleUnsatisfied constraints:(in red’s neighborhood)

1

2

3

1

2

3

1

2

3

1

2

3

Label Extended Graph

Constraint GraphSlide10

Unique Games: ExampleUnsatisfied constraints:(in red’s neighborhood)

1

2

3

1

2

3

1

2

3

1

2

3

Label Extended Graph

Constraint GraphSlide11

Unique Games = Special sparse cut

1

2

3

1

2

3

1

2

3

1

2

3

Label Extended Graph

Constraint Graph

Find subset S of fraction 1/k vertices

in label extended graph,

containing

one vertex in each cloud,

minimizing

(S)Slide12

Unique Games conjecture [Khot’02] > 0 

k = k() s.t. it is NP-hard to tell if an instance of Unique Games with k labels has OPT 

 or

OPT



1- 

.

Let OPT = fraction of unsatisfied edges in optimal labeling

i.e., even if  a “special cut” with expansion  ,

it is hard to find a “special cut” with expansion  1- Slide13

Our workApproximation algorithms for these problems using semidefinite programs from the Lasserre hierarchy

Min-BisectionSmall Set ExpansionSparsest CutMin-Uncut / Max-CutIndependent SetAs well as k-partitioning variants:Min-k-SectionUnique GamesSlide14

Motivations: Algorithmic PerspectiveThese are fundamental, well-studied, practically relevant, optimization problemsYet huge gap:Approximability: or Hardness: not even factor 1.1 known to be NP-hard

Natural goal: Close (or reduce) the gapSDPs one of the principal algorithmic toolsExtend algorithmic techniques to more powerful SDPs.Slide15

Motivations: Complexity Perspective[Khot’02] Unique Games ConjectureUGC has many implications: tight results for all constraint satisfaction and ordering problems, vertex cover,… [

Raghavendra, Steurer’10] Small Set Expansion Conj. (SSEC):O(1)-approximation for Small Set Expansion is hard.Implies the UGC [RS’10], plus (1) hardness for bisection, sparsest cut, minimum linear arrangement, etc. [R,S,Tulsiani’12]Despite these bold conjectures, following is not ruled out:Can 5-rounds of Lasserre Hierarchy SDP relaxation

refute the UGC?

Investigate this possibility…

Identify candidate hard instances (if there are any!)

No consensus opinion

on its validitySlide16

Talk OutlineIntroduction to problems we studyLaplacian eigenvalues and our resultsLasserre hierarchy

Case study: Minimum bisectionConcluding remarksSlide17

Laplacian and Graph Spectrum

12

3

4

rows and cols

indexed by V

λ

2

: Measures expansion of the graph through

Cheeger’s

inequality

.

λ

r

: Related to small set expansion

[

Arora

, Barak, Steurer’10]

,

[Louis,Raghavendra,Tetali,Vempala’11], [

Gharan

, Lee,Trevisan’11]

.

0 = λ1 ≤

λ2 ≤ … ≤ λ

n ≤2 and λ1 + λ

2 + … + λn = n,Slide18

Our Results IBy rounding r/O(1) round Lasserre hierarchy SDPs, in time we obtain approximation

factorNote we get approximation scheme0 = λ1 ≤ λ2 ≤ … ≤ λn ≤2 and λ1 + λ2

+ … + λn = n,

Minimum Bisection*

Small Set Expansion*

Uniform Sparsest Cut

Their k-way generalizations*

* Satisfies constraints within factor of 1

 o(1)

For r

=n,

λ

r

>1,

λn-r <1

Minimum Uncut (min. version of Max Cut)

More generally, our methods apply to quadratic integer programming problems with positive

semidefinite

objective functionsSlide19

Our Results II: Unique GamesFor Unique Games, a direct bound will involve spectrum of label extended graph, whereas we want to bound using spectrum of original graph.We give a simple embedding and work directly on the original graph.We obtain factor in time

Concurrent to our work, [Barak-Steurer-Raghavendra’11] obtained factor in time (using weaker Sherali-Adams SDP).

Combining with

[Arora-Barak-Steurer’10]

“higher order

Cheeger

”

thm.

 UG with

compl. 1- O(1

) is easy for n

levels of Lasserre HierarchySlide20

InterpretationOur results show that these graph partitioning problems are easy on many graphsAlso hints at why showing even weak hardness results has been elusivePoints to the power of

Lasserre hierarchyCould be a serious threat to small set expansion conjecture, or even UGC.Recent work [Barak,Brandao,Harrow,Kelner,Steurer,Zhou’12] shows O(1) rounds enough to solve known gap instancesSlide21

Our Results IIIIndependent set: O(1)

approximation in nO(r) time when r’th

largest

eigenvalue



n-r

 1 + O(1/

d

max)



O(d

max/t) approximation if n-r

 1 + 1/t

Normalized

Laplacian

eigenvalues

0 = λ1

≤ λ2 ≤ … ≤ λ

n ≤ 2 and λ1 + λ

2 + … + λn = n,

[Arora-Ge’11] Given 3-colorable graph, find independent set of size

n/12 in nO

(r) time if



n-r < 17/16. Slide22

Talk OutlineIntroduction to problems we studyLaplacian eigenvalues

and our resultsLasserre hierarchyCase study: Minimum bisectionConcluding remarksSlide23

Basic Lasserre Hierarchy RelaxationQuadratic IP formulation for a k-labeling problem: For each S of size ≤ r, and each possible labeling f : S  {0,1,…,k-1}

Boolean variable representing: with all implied pairwise consistency constraints.(SDP Relaxation) Replace with

r

= number of

rounds/levels

of

Lasserre

hierarchy;

Resulting SDP can be solved in

n

O(r) timeSlide24

ConsistencyLasserre Relaxation for Minimum Bisection

Relaxation for consistent labeling of all subsets of size r:

Marginalization

Distribution

Partition

SIze

Cut cost

Given d-regular graph G, find subset U of size

 minimizing 

G

(U)

Intended integral value of

x

u

(1) : 1 if u

 U, and 0 otherwise.

MinimizeSlide25

Intuition Behind Lasserre RelaxationFor each S, the vectors xS(f) give a local distribution on labelings f : S

 {0,1}Prob. of f = Inner products of vectors (xS(f))S,f represent joint probabilities (give a psd moment matrix)Division corresponds to conditioning:Slide26

Previous Work on Lasserre HierarchyFew algorithmic results known before, including:[Chlamtac’07], [Chlamtac, Singh’08]

nΩ(1) approximation for 3-coloring and independent set on 3-uniform hypergraphs, [Karlin, Mathieu, Nguyen’10] (1+1/r) approximation of knapsack for r-rounds.Some known integrality gaps are:[Schoenebeck’08], [Tulsiani’09] Most NP-hardness results carry over to Ω(n) rounds of Lasserre.[

Bhaskara, Charikar

, G.,

Vijayaraghavan

, Zhou’12]

Densest k-

subgraph (n(1) integrality gap for (n) rounds of Lasserre

hierarchy)Slide27

Rounding Lasserre RelaxationFor regular SDP [Goemans, Williamson’95] showed that with

hyperplane rounding:No analogue known for rounding Lasserre RelaxationHere: an intuitive local propagation based rounding frameworkAnalysis via projection distance, and connections to ``column selection” in low-rank matrix approximationSlide28

General Rounding Framework

(Seed Selection) Choose appropriate seed set S.(Seed Labeling) Choose wp . (Propagation) Perform randomized rounding.so that the output satisfies:(i.e., match the conditional prob. for label for i, given S got labeling f)

Insp

ired

by

[

Arora

,

Kolla

, Khot, Steurer

, Tulsiani, V

ishnoi’08] algorithm for Unique Games on expanders: propagation from a single node chosen uniformly at randomSlide29

Talk OutlineIntroduction to problems we studyLaplacian eigenvalues

and our resultsLasserre hierarchyCase study: Minimum bisectionConcluding remarksSlide30

Case Study: Minimum BisectionWe will present some details of the analysis of rounding for the minimum bisection problem on d-regular unweighted graphs (for simplicity).We will show that it achieves factor .

Obtaining factor requires some additional ideas.Slide31

Lasserre SDP for Min -section

Vector xS(f)

for each S of size ≤ r, and each possible labeling f : S

 {0,1}

MinimizeSlide32

Rounding AlgorithmGiven optimal solution to r’=O(r) round Lasserre SDP:Choose suitable seed set

S of size rDetails laterPartition S by choosing f with probability Propagate to other nodes:For each node v independentlyWith probability include v in U.

Return U.Slide33

AnalysisPartition SizeEach node is chosen into U independentlyFixing S,f, expected size of U equalsBy Chernoff

, with high probability 33Slide34

AnalysisBy our rounding, for fixed seed set S,After some calculations, we have the following bound on number of edges cut:

Normalized Vector for xS(f)

≤ OPT

Call this matrix



SSlide35

Matrix ΠSRemember {x

S(f)}f are orthogonal. is a projection matrix onto span{xS(f)}f .For any

Let P

S

be the corresponding projection matrix.Slide36

Picking the seed setThe final bound is:Define X = matrix with columns {Xu = xu(1)}

Want seed set S to minimize Projection distance to span of columns { X

u : u  S}Slide37

Column selectionGiven matrix X  Rm x n, pick r columns S to minimize

Introduced by [Frieze, Kannan,Vempala’04]; studied in many works since.For any S of size r this is lower bounded by:[G.-Sinop] Can efficiently find set S of columns so thatAnd this bound is tight.

Error of best rank-r approximation of XSlide38

Relating Performance to Graph SpectrumCan show: Worst case is when best rank-r approximation of X is obtained by first r eigenvectors of graph Laplacian.Using Courant-Fischer theorem,ThereforeSlide39

Few words about column selectionGiven matrix X  Rm x n, pick t

columns S to minimize[G.-Sinop] Can efficiently find set S of columns so that

Error of best rank-r approximation of XSlide40

Proof Idea Goal: (Min. projection dist.)Observe Choose S with probability Volume Sampling [Deshpande

, Rademacher, Vempala, Wang’06]Converts sum-of-ratios to ratio-of-sums.Slide41

Proof Idea (contd.) where are eigenvalues of XT X, is rth

symmetric form. Expected projection distance achieved by volume sampling equalsSlide42

Schur Concavity is a Schur-Concave function.F()  F() if  majorizes  For a fixed prefix sum

F() is maximized bySubstituting back:

Best rank-r approximation

error.Slide43

Talk OutlineIntroduction to problems we studyLaplacian eigenvalues

and our resultsLasserre hierarchyCase study: Minimum bisectionConcluding remarksSlide44

SummaryRounding for Lasserre hierarchy SDPs for certain QIPs + analysis based on column selectionApproximation scheme-like guarantees for several graph partitioning problemsnO(r) time to solve r-levels of hierarchy. Rounding framework only looks at 2

O(r) nO(1) bits of solution. Can also make runtime 2O(r) nO(1) [G.-Sinop, FOCS’12]Lasserre SDP seems very powerfulOnly very weak integrality gaps known for the studied problems Slide45

Open questionsCan O(log n) rounds of Lasserre hierarchy refute SSE conjecture? Refute UGC? Currently no candidate hard instances for even 5 rounds

0.878 approx. for Max-Bisection? (0.85 [Raghavendra-Tan’12])Integrality gaps for Lasserre hierarchy beating NP-hardness (or matching UG/SSE-hardness) results [Tulsiani’09] For Max k-CSP, clique/coloring[G.-Sinop-Zhou’12] Balanced separator, uniform sparsest cut

[Bhaskara,

Charikar

, G.,

Vijayaraghavan

, Zhou’12]

Densest k-subgraph (n

(1) integrality gap for (n) rounds of

Lasserre hierarchy)