Lower Bounds for Depth Three Circuits with small bottom - PowerPoint Presentation

Slide1

Lower Bounds for Depth Three Circuits with small bottom fanin

Neeraj Kayal

Chandan

Saha

Indian Institute of ScienceSlide2

A lower bound

Theorem: Consider representations of a degree d polynomial

of the form

If the ’s have degree one and at most

variables each

then there is an explicit (family)

of polynomials on variables such that s is at least .

 Slide3

A lower bound

Theorem: Consider representations of a degree d polynomial

of the form If the ’s have degree one and at

most

variables each then there is an explicit (family)

of polynomials on variables such that s is at least .

Remark:

For a generic

, s must be at least

Any asymptotic improvement in the exponent will imply .

 Slide4

A lower bound

Corollary (via Kumar-

Saraf): Consider representations of a degree d polynomial

of the form If the

’s

have

degree one and at most variables each then there is an explicit (family) of polynomials on variables such for s is at

least

.

Remark:

Bad news. Good news.Slide5

Background/MotivationSlide6

Arithmetic Circuits

…

 Slide7

…

…

-1

Arithmetic CircuitsSlide8

…

…

…

-1

Arithmetic CircuitsSlide9

…

…

…

-1

Arithmetic CircuitsSlide10

…

…

…

-1

Arithmetic CircuitsSlide11

…

…

…

-1

 Slide12

…

…

…

-1

Size = Number of EdgesSlide13

…

…

…

-1

DepthSlide14

…

…

…

-1

This talk.

> Field is

.

> Gates have unbounded

fanin

>

 Slide15

Two Fundamental Questions

Can explicit polynomials be efficiently computed?

Can computation be efficiently parallelized? Slide16

Two Fundamental Questions

Can explicit polynomials be efficiently

computed?

Does VP equal VNP?  

Can computation be efficiently parallelized?

Can

every efficient computation be also done by small, shallow circuits? Slide17

Can computation be efficiently parallelized?

Question:

How efficiently can we simulate circuits of size by circuits of depth

?

 Slide18

Can computation be efficiently parallelized?

Theorem (Hyafil79, VSBR83, AV-08, Koiran-12, GKKS13, Tavenas-13,

Wigderson

/Tavenas

):

Any circuit of size s and degree

can be simulated by a -depth circuit of size

 Slide19

Can computation be efficiently parallelized?

Theorem (Hyafil79, VSBR83, AV-08, Koiran-12, GKKS13, Tavenas-13,

Wigderson

/Tavenas

):

Any circuit of size s and degree

can be simulated by a -depth circuit of size … also by a regular, homogeneous

-depth

circuit of size

 Slide20

Can computation be efficiently parallelized?

Theorem (Hyafil79, VSBR83, AV-08, Koiran-12, GKKS13, Tavenas-13,

Wigderson

/Tavenas

):

Any circuit of size s and degree

can be simulated by a -depth circuit of size … also by a regular, homogeneous

-depth

circuit of size

Question:

Is this optimal?Slide21

Can computation be efficiently parallelized?

Theorem (Hyafil79, VSBR83, AV-08, Koiran-12, GKKS13, Tavenas-13,

Wigderson

/Tavenas

):

Any circuit of size s and degree

can be simulated by a -depth circuit of size … also by a regular, homogeneous

-depth

circuit of size

Question:

Is this optimal?(KS15 + Ramprasad

):

For

yes, but with caveats.

 Slide22

Can computation be efficiently parallelized?

Theorem (Hyafil79, VSBR83, AV-08, Koiran-12, GKKS13, Tavenas-13,

Wigderson

/Tavenas

):

Any circuit of size s and degree

can be simulated by a -depth circuit of size … also by a regular, homogeneous

-depth

circuit of size

Corollary:

Strong enough lower bounds for low-depth circuits imply VP VNP.

 Slide23

A possible way to approach VP vs VNP

Strong enough

lower bounds for low-depth circuits imply VP VNP.

Low

Depth Circuit

(’s simple) Low-depth circuits are easy to analyze.. Slide24

A possible way to approach VP vs VNP

Strong enough

lower bounds for low-depth circuits imply VP VNP.

Low

Depth Circuit

(’s simple) Low-depth circuits are easy to analyze..

Lots of work on lower bounds for low depth arithmetic circuits in recent

years – hope to discover

general patterns

and

technical ingredients Slide25

’s are:

Lower Bound

has degree

~

in VNP

GKKS13+KSS14

has degree ~ IMMFLMS14 is sparse

~

in

VNP

KLSS14 is sparse~

IMM

KS14

has degree one and

arity

IMM

This work

have homogeneous depth three circuits of bottom

fanin

in VNP

This work

have homogeneous depth five circuits of bottom

fanin

IMM

Next talk

Lower Bound

in VNP

GKKS13+KSS14

IMM

FLMS14

in

VNP

KLSS14

IMM

KS14

IMM

This work

in VNP

This work

IMM

Next talkSlide26

A possible way to approach VP vs VNP

Prove

strong enough

lower bounds for low depth circuits.

Low

Depth Circuit

(’s simple)Low-depth circuits are easy to analyze.. Lots of work on low

depth arithmetic circuits

recently

This talk:

common pattern/proof strategy

Technical ingredients Slide27

A common Proof Strategy and some technical ingredientsSlide28

Proof Strategy

(

’s

simple ). Let .  

shallow circuit C

Find a geometric property GP of the

’s.Express the property GP in terms of rank of a “big” matrix M: if T has the property than rank(M(T)) is “relatively small” Show that rank(M()) is “relatively large”.

 Slide29

Proof Strategy

(

’s

simple ). Let .  

shallow circuit C

Find a geometric property GP of the

’s.Express the property GP in terms of rank of a “big” matrix M: if T has the property than rank(M(T)) is “relatively small” Show that rank(M(

)) is “relatively large”.

 Slide30

Lower Bounding rank of large matrices

If a matrix M(f) has a large upper triangular submatrix, then it has large rank

(

Alon): If the columns of M(f) are almost orthogonal then M(f) has large rank. Slide31

When the

’s have low degree

(’s have low degree). Let

.

shallow circuit CFind a geometric property GP of the ’s.

Express the property GP in terms of rank of a “big” matrix M: if T has the property than rank(M(T)) is “relatively small”

Show that rank(M(

)) is “relatively large”.

 Slide32

Finding

a geometric property GP of T

is

a product of low degree polynomials V(T) is a union of low-degree hypersurfacesV(T) has lots of high-order singularitiesSlide33

Finding

a geometric property GP of T

 Slide34

Finding

a geometric property GP of T

is

a product of low degree polynomials V(T) is a union of low-degree hypersurfacesV(T) has lots of high-order singularities

V(

T) has lots of points

 Slide35

When the

’s have low degree

(’s have low degree). Let

.

shallow circuit CFind a geometric property GP of the ’s.Express the property GP in terms of rank of a “big” matrix M: if T has the property than rank(M(T)) is “relatively small”

Show that rank(M(

)) is “relatively large”.

 Slide36

When the

’s have low degree

(’s have low degree). Let

.

shallow circuit CFind a geometric property GP of the ’s.Express the property GP in terms of rank of a “big” matrix M: if T has the property than rank(M(T)) is “relatively small”

Show that rank(M(

)) is “relatively large”.

 Slide37

Expressing largeness of a variety in terms of rank

is a variety.

Let

= set of degree-

polynomials.

 Slide38

Expressing largeness of a variety in terms of rank

is a variety.

Let

= set of degree-

polynomials.

Let = set of degree- polynomials which vanish at every point of V. Hilbert’s Theorem (Informal): If V is “large” then

has small dimension.

 Slide39

Expressing largeness of a variety in terms of rank

is a variety.

Let

= set of degree-

polynomials.

Let = set of degree- polynomials which vanish at every point of V. Hilbert’s Theorem (Informal): If V is “large” then

has small dimension.

Let

= { (

) of

deg

}

.

Hilbert’s Theorem (Formal):

If V has dimension r then

has asymptotic dimension

.

 Slide40

When the

’s have low degree

(’s have low degree). Let

.

shallow circuit CFind a geometric property GP of the ’s.Express the property GP in terms of rank of a “big” matrix M: if T has the property than rank(M(T)) is “relatively small”

Show that rank(M(

)) is “relatively large”.

 Slide41

Restrictions

Example:

is a function.

f(0, 1, z) is a restriction of f. Geometrically, f(0, 1, z) is same as restricting f to the axis-parallel line W = V(x, y-1). Algebraically, f(0, 1, z) is same as mod  

More generally:

Let

be any ideal and a polynomial. Call mod

as an

algebraic restriction of f.

Obs

: Hilbert’s theorem can be generalized for algebraic restrictions of polynomials. Slide42

Employing restrictions

Lemma (KLSS14):

If Q is a sparse polynomial then for a suitable random algebraic restriction

mod is a low degree polynomial. Yields lower bounds for homogeneous depth four (KLSS14 and KS14).Slide43

Lemma (KLSS14): If T is a

sparse polynomial a sum of product of low arity

polynomials then for a suitable random algebraic restriction

mod is a low degree polynomial. Employing RestrictionsSlide44

Lemma: If T is a

sparse polynomial a sum of product of low arity

polynomials then for a suitable random algebraic restriction

mod is a low degree polynomial. Employing RestrictionsYields lower bounds for homogeneous depth five with low bottom fanin (KS15 and BC15).Slide45

Lemma (Shpilka-Wigderson

): A depth three circuit C of size s can be converted to a homogeneous depth circuit C’ of size

. Further if C has bottom

fanin t then C’ also has bottom fanin t. A lemma by Shpilka and WigdersonYields lower bounds mentioned earlier.Slide46

Conclusion

Proving lower bounds for low depth circuits is a potential way to prove lower bounds for more general circuits.

There is a meta-strategy common to many recent (and older) lower bounds. We don’t understand the power or limitations of this meta-strategy.

Open: lower bounds for homogeneous depth three circuits for polynomials of degree .