On the Impossibility of Dimension Reduction for Doubling Su - PowerPoint Presentation

Slide1

On the Impossibility of Dimension Reduction for Doubling Subsetsof Lp

Yair

Bartal

Lee-Ad Gottlieb

Ofer

NeimanSlide2

Embedding and DistortionL

p

spaces:

Lpk is the metric space Let (X,d) be a finite metric spaceA map f:X→ Lpk is called an embeddingThe embedding is non-expansive and has distortion D, if for all x,yϵX :

 Slide3

JL Lemma

Lemma

: Any n points in L

2 can be embedded into L2k, k=O((log n)/ε2) with 1+ε distortionExtremely useful for many applications:Machine learningCompressive sensingNearest Neighbor searchMany others…Limitations: specific to L2, dimension depends on nThere are lower bounds for dimension reduction in L1, L∞Slide4

Lower bounds on Dimension Reduction

For general n-point sets in

L

p, Ω(logD n) dimensions are required for distortion D (volume argument)BC’03 (and also LN’04, ACNN’11, R’12) showed strong impossibility results in L1The dimension must be for distortion D Slide5

Doubling DimensionDoubling constant:

The minimal

λ

so that every ball of radius 2r can be covered by λ balls of radius rDoubling dimension: log2λA measure for dimensionality of a metric spaceGeneralizes the dimension for normed space: Lpk has doubling dimension Θ(k)The volume argument holds only for metrics with high doubling dimensionSlide6

Overcoming the Lower Bounds?One could hope for an analogous version of the JL-Lemma for doubling subsets

Question

: Does every set of points in L

2 of constant doubling dimension, embeds to constant dimensional space with constant distortion?More ambitiously: Any subset of L2 with doubling constant λ, can be embedded into L2k, k=O((log λ)/ε2) with 1+ε distortionSlide7

Our ResultSuch a dimension reduction is impossible in the

L

p

spaces with p>2Thm: For any p>2 there is a constant c, such that for any n, there is a subset A of Lp of size n with doubling constant O(1), and any embedding of A into Lpk with distortion at most D satisfies Slide8

Our Result

Thm

: For any p>2 there is a constant c, such that for any n, there is a subset A of

Lp of size n with doubling constant O(1), and any embedding of A into Lpk with distortion at most D satisfiesNote: any sub-logarithmic dimension requires non-constant distortionWe also show a similar bound for embedding from Lp into Lq, for all q≠2Lafforgue and Naor concurrently proved this using analytic tools, and their counterexample is based on the Heisenberg group Slide9

ImplicationsRules out a class of algorithms for NN-search, clustering, routing etc.

The first non-trivial result on

non-linear

dimension reduction for Lp with p≠1,2,∞Comment: For p=1, there is a stronger lower bound for doubling subsets, the dimension of any embedding with distortion D (into L1) must be at least (LMN’05) Slide10

The Laakso Graph

A recursive graph, G

i+1

is obtained from Gi by replacing every edge with a copy of G1A series-parallel graphHas doubling constant 6

G

0

G

1

G

2Slide11

Simple Case: p=∞

The

Laakso

graph lies in high dimensional L∞Assume w.l.o.g that there is a non-expansive embedding f with distortion D into L∞kProof idea:Follow the recursive constructionAt each step, find an edge whose L2 stretch is increased by some value, compared to the stretch of its parent edgeWhen stretch(u,v) > k, we will have a contradiction, as

 Slide12

vSimple Case: p=∞

Consider a single iteration

The pair

a,b is an edge of the previous iterationLet fj be the j-th coordinateThere is a natural embedding that does not increase stretch...But then u,v may be distortedabs

t

u

f

j

(a)

f

j

(b)Slide13

Simple Case: p=∞For simplicity (and

w.l.o.g

) assume

fj(s)=(fj(b)-fj(a))/4fj(t)=3(fj(b)-fj(a))/4fj(v)=(fj(b)-fj(a))/2Let Δj(u) be the difference between fj(u) and fj(v)The distortion D requirement imposes that for some j, Δj(u)>1/D (normalizing so that d(u,v)=1)v

a

b

s

t

u

f

j

(a)

f

j

(b)

Δ

j

(u)Slide14

Simple Case: p=∞

The stretch of

u,s

will increase due to the j-th coordinateBut may decrease due to other coordinates..Need to prove that for one of the pairs {u,s}, {u,t}, the total L2 stretch increases by at least Compared to the stretch of a,b v

a

b

s

t

f

j

(a)

f

j

(b)

Δ

j

(u)

u

v

a

b

s

t

f

h

(a)

f

h

(b)

-

Δ

h

(u)

uSlide15

Simple Case: p=∞

Observe that in

the j-

th coordinate:If the distance between u,s increases by Δj(u),Then the distance between u,t decreases by Δj(u) (and vise versa)Denote by x the stretch of a,b in coordinate jThe average of the L2 stretch of {u,s} and {u,t} (in the j-th coordinate alone) is:  v

a

b

s

t

f

j

(a)

f

j

(b)

Δ

j

(u)

uSlide16

Simple Case: p=∞

For one of the pairs {

u,s

}, {u,t}, the total L2 stretch (over all coordinates) increases by Continue with this edgeThe number of iterations must be at most kD2 (otherwise the stretch will be greater than k)But # of iterations ≈ log nFinally,  

u

s

a

b

t

vSlide17

Going Beyond Infinity

For p<∞, we cannot use the

Laakso

graphRequires high distortion to embed it into LpInstead, we build an instance in Lp, inspired by the Laakso graph The new points u,v will use a new dimensionParameter ε determines the (scaled) u,v distanceasuvtbεSlide18

Going Beyond Infinity

Problem

: the

u,s distance is now larger than 1, roughly 1+εpCauses a loss of ≈ εp in the stretch of each levelSince u,v are at distance ε, the increase to the stretch is now only (ε/D)2When p>2, there is a choice of ε for which the increase overcomes the lossasuvtb

εSlide19

ConclusionWe show a strong lower bound against dimension reduction for doubling subsets of L

p

, for any p>2

Can our techniques be extended to 1<p<2 ?The u,s distance when p<2 is quite large, ≈ 1+(p-1)ε2 , so a different approach is requiredGeneral doubling metrics embed to Lp with distortion O(log1/pn) (for p≥2)Can this distortion bound be obtained in constant dimension?