A Framework for Fast and - PowerPoint Presentation

Slide1

A Framework for Fast and Efficient Algorithms for Sparse Recovery Problems

Sidharth JaggiThe Chinese University of Hong Kong

Sheng

Cai

Mayank Bakshi

Minghua Chen

Slide2

1Sparse Recovery

Compressive Sensing

Network Tomography

Compressive Phase Retrieval

Group Testing

Slide3

a

i

2

x

2

ai3x

3ai1x1

yi=Σj

aijxj

2Problem 1: Compressive Sensing - Linear Measurement

n

Slide4

a

i

1

x

1

2

Problem 1: Compressive Sensing - Linear Measurement

a

i

2

x

2

a

i

3

x

3

y

i

=

Σ

j

a

ij

x

j

+

e

j

n

k

a

i

1

z

1

Slide5

Problem 1: Compressive Sensing - Linear Measurement?

k

n

?

A

x = y

x

A

y

m

3

Slide6

Problem 1: Compressive Sensing - Linear Measurement

?

k

n

?

A

x = y

x

A

y

m

n > m > k

3

Slide7

Problem 1: Compressive Sensing - Linear Measurement

Z

A

(x +

z

) = y +

e

+

x

A

y

e

Approximate sparsity

Measurement noise

4

Interference

Transmission noise

Slide8

Sparse Recovery

Z

A

(x +

z

) = y +

e

+

x

A

y

e

Approximate sparsity

Measurement noise

4

Linear or Non-Linear

Slide9

Bottleneck linkyi

=Σj ∈

path i

xj

x

2

x

3

x

1

5

Problem 2: Network Tomography - Restricted Linear Measurements

Slide10

6Problem 3: Compressive Phase Retrieval - Non-Linear (Intensity) Measurement

yi

=|Σj

aij

xj

|

Can we recover 3D information from intensity measurements?

Slide11

7

Problem 4: Group Testing - Non-Linear (OR) Measurement

n

k

y

i

=

˅

j

∈

test

i

x

j

Slide12

8Our Contribution: Fast and Efficient Algorithms

Compressive Sensing

Network Tomography

Compressive Phase Retrieval

Group Testing

Slide13

“Gold

Bar

” Puzzle I

100g

each

Sparse Recovery: Intuition

9

steals unknown

amount from

each gold bar

unknown thief

Which one?

How much?

Slide14

“Gold

Bar

” Puzzle I

100g

Sparse Recovery: Intuition

9

Which one?

How much?

Slide15

500g

490g

10g

“Gold Bar” Puzzle I

10

Step 1

Sparse Recovery: Intuition

1-sparse

Slide16

1,500g

1,460g

40g

“Gold Bar” Puzzle I

11

Step 2

40g

10g

=

4

Sparse Recovery: Intuition

Slide17

1,500g

1,460g

40g

“Gold Bar” Puzzle I

11

Step 2

Sparse Recovery: Intuition

Idea 1: Reconstruction

via

structured measurements

Slide18

“Gold

Bar

” Puzzle II

Bad group

12

Sparse Recovery: Intuition

Good group: Puzzle I

Slide19

“Gold

Bar

” Puzzle II

Bad group

12

Sparse Recovery: Intuition

110g

30g

∉

Z

structured measurements

Slide20

“Gold

Bar

” Puzzle II

12

Good group

Sparse Recovery: Intuition

structured measurements

30g

10g

=

3

Slide21

“Gold

Bar

” Puzzle II

12

Sparse Recovery: Intuition

80g

20g

=

4

0

0

0

-80

-20

Good group

Idea 2: Picking & Peeling

Slide22

y

1

y

2

x

2

x

5

x4

x3x1

Multiton

Singleton

Measurement Structure: Bipartite Graph

13

Sparse Recovery: Framework

Slide23

y

1

y

2

x

2

x

5

x4

x3x1

Multiton

Singleton

Decoding Process: Picking & Peeling

13

Sparse Recovery: Framework

Slide24

y

1

y

2

x

2

x

5

x4

x3x1

Multiton

Singleton

Decoding Process: Picking & Peeling

13

Sparse Recovery: Framework

Slide25

y1

y

2

x

2

x

5

x

4

x3x1

Decoding Process: Picking & Peeling

13

Sparse Recovery: Framework

Existence of useful nodes

--

Design of bipartite graphs

Decoding: iterative picking & peeling

--

Structured measurements

Multiton

Singleton

Slide26

14Our Contribution: Fast and Efficient Algorithms

Compressive Sensing

Network Tomography

Compressive Phase Retrieval

Group Testing

SHO-FA

Decoding Complexity

# Measurements

Slide27

Decoding complexity

# of measurements

°

RS’60

°

TG’07

°CM’06°C’08

°IR’08°

SBB’06°GSTV’06

°MV’12,KP’12°DJM’11

SHO-FA

Lower bound

Lower bound

SHO-FA:

Compressive

Sensing

15

O(

k

)

O(

k

)

Pawar, S.; Ramchandran, K., "A hybrid DFT-LDPC framework for fast, efficient and robust compressive sensing," 

50th Annual Allerton Conference on

 

Communication, Control, and Computing (Allerton)

, pp.1943,1950, 1-5 Oct. 2012

[SK12]

Slide28

16?

k ≤

m<

n

?

n

m

k

Compressive Sensing: Problem

Problem 1:

Compressive

Sensing

Slide29

17

4

3

4

n

4

3

4

k=2

Compressive Sensing: Intuition

A

(weighted) Biadjacency Matrix

Bipartite Graph → Measurement Matrix

Slide30

18

4

3

4

n

Compressive Sensing: Intuition

Multiton

Singleton

Zeroton

k=2

Picking

“Gold Bar” Puzzle Recap

Slide31

18

4

3

4

3

4

Multiton

Singleton

Zeroton

Compressive Sensing: Intuition

n

k=2

Peeling

Picking

“Gold Bar” Puzzle Recap

Slide32

≥2|

S|

|

S|

L+L’≥

2

|S|3|S|≥L+2L’

(

L+L’)/(

L+2L’) ≥2/319

L/

(

L+L’

)

≥

1/2

Compressive Sensing: Design of Bipartite graphs

n

ck

d=

3

L

: # singletons

Existence of Singletons

Slide33

Bipartite Graph → Measurement Matrix

n

ck

d=

3

20

A

Distinct weights

Compressive Sensing: Structured Measurements

Slide34

Bipartite Graph → Measurement Matrix 20

Compressive Sensing: Structured Measurements

n

ck

A

Slide35

21

Picking

& Peeling

Compressive Sensing: Decoding

Slide36

22

Picking

& Peeling

Compressive Sensing: Decoding

Slide37

23Our Contribution: Fast and Efficient Algorithms

Network Tomography

Compressive Phase Retrieval

Group Testing

SHO-FA

Slide38

23Our Contribution: Fast and Efficient Algorithms

SHO-FA

Network Tomography

Compressive Phase Retrieval

Group Testing

FRANTIC

Slide39

24

End-to-end delay

Edge delay

Sparse

Problem 2: Network Tomography

Network Tomography: Problem

Slide40

Challenge: Topology constraints

25

Slide41

,

Integer entries: “Gold bar” puzzle

Approach: Cancellation+“Loopy” measurements

25

Slide42

27Our Contribution: Fast and Efficient Algorithms

SHO-FA

Network Tomography

Compressive Phase Retrieval

Group Testing

FRANTIC

Slide43

27

Our Contribution: Fast and Efficient Algorithms

SHO-FA

Compressive Phase Retrieval

Group Testing

FRANTIC

SUPER

Slide44

?

n

x

1

x

=

?Problem 3: Compressive Phase Retrieval28

| | 

+

a

1

a

5

a

7

x

5

x

7

a

1

x

1

+

a

5

x

5

+

a

7

x

7

y

1

=

|

a

1

x

1

+

a

5

x

5

+

a

7

x

7

|

Intensity

detector

+

+

 

 

y

1

y

2

y

m

Decoder

k

x

Compressive Phase Retrieval: Problem

Slide45

b

Problem 1: Compressive Sensing

?

n

m

k

29

x

Compressive Phase Retrieval: Problem

Slide46

b

30

?

k

2

e

i

π

/

3n

2

2

Intensity

Problem 3: Compressive Phase Retrieval

x

Compressive Phase Retrieval: Problem

Slide47

b

Problem 3: Compressive Phase Retrieval

30

?

-

2

eiπ/3

n2

2

Intensity-x

Compressive Phase Retrieval: Problem

Slide48

b

Problem 3: Compressive Phase Retrieval

30

?

2

e

i(π/3+ θ)

n2

2

Intensityei

θ

x

Compressive Phase Retrieval: Problem

Slide49

31

4

3

4

n

ck

4

3

4

k=2

Multiton

Singleton

Compressive Phase Retrieval: Intuition

Challenges:

“Peeling” is not allowed

Magnitude + Phase

“Gold Bar” Puzzle Recap

Slide50

32Phases from Magnitudes?

n,k = 1

y1

=

|a

1 x1|

|x1| =|y

1/a1|

n,k = 2

y1 = |a1 x

1|

y2

=

|

a

2

x

2

|

|

x

1

|

=

|

y

1

/a

1

|

|

x

2

|

=

|

y

2

/a

2

|

y

3

=

|

x

1

+

x

2

|

y

4

=

|

x

1

+i

x

2

|

Δ

|

x

1

+

x

2

|

|

x

1

|

|

x

2

|

OR

Δ

|

x

1

+

x

2

|

|

x

1

|

|

x

2

|

x

1

=

|

y

1

/a

1

|

e

iθ

|

x

1

+

i

x

2

|

|

x

1

+

i

x

2

|

x

=

|

y

1

/

a

1

|

|

y

2

/a

2

|

e

iΔ

e

iθ

y

4

=

x

2

x

1

y

1

y

3,

y

4

y

2

“

Σ

” Structure

Compressive Phase Retrieval: Intuition

Slide51

x1x

6x

5

x

4

x3Phases from Magnitudes?

33x2

y

1

y

2

y

3

y

4

n

m

0

0

0

Compressive Phase Retrieval: Intuition

Slide52

x2x

6x

4

y

1

y2

y3y4Doubleton

Singleton

Phases from Magnitudes?

33Multiton

Compressive Phase Retrieval: Intuition

Slide53

x

2

x

6

x

5

x

4

x

3

y1

y

2

y

3

y

4

x

1

Graph properties

34

n

ck

log

k

with prob.

1/k

The probability that a

right node

is singleton

Compressive Phase Retrieval: Design of Bipartite graphs

Slide54

x

2

x

6

x

5

x

4

x

3

y1

y

2

y

3

y

4

x

1

Graph properties

34

with prob.

1/k

A

right node is singleton w.p.

e

-1

A

right node is doubleton w.p.

e

-1

/2

Compressive Phase Retrieval: Design of Bipartite graphs

n

ck

log

k

Slide55

x2x

6x

5

x

4

x3

x1

y

1

y

2

y

3

y

4

Bipartite Graph

→

Measurement Matrix

35

x

1

x

2

x

3

x

4

x

5

x

6

y

1

y

2

y

3

y

4

(weighted) Biadjacency Matrix

y

1

y

2

y

3

y

4

x

1

x

2

x

3

x

4

x

5

x

6

Compressive Phase Retrieval: Structured Measurements

n

ck

log

k

Slide56

x

2

x

5

y

1

x1

36

x

1x2

x

3

x

4

x

5

x

6

y

1

y

2

y

3

y

4

Compressive Phase Retrieval: Structured Measurements

Bipartite Graph

→

Measurement Matrix

Slide57

Identifying Singletons37

α

=

(π/

2)/n

unit phasey1,1

y1,2b1,3

b1,4b

1,5

x1

x2

x

3

x

4

x

5

x

6

y

1

y

2

y

3

y

4

x

2

x

5

y

1

x

1

Compressive Phase Retrieval: Decoding

Slide58

Identifying Singletons37

α

=

(π/

2)/n

unit phasey1,1

y1,2b1,3

b1,4b

1,5

x

1

x

2

x

3

x

4

x

5

x

6

y

1

y

2

y

3

y

4

x

2

y

1

0

0

Compressive Phase Retrieval: Decoding

Slide59

Identifying Singletons37

α

=

(π/

2)/n

unit phasey1,1

y1,2b1,3

b1,4b

1,5arctan(y1,2

/y1,1)/ α = 2

x2

≠ 0 and |x2|

= y

1,1

/cos2

α

y

1,1

= cos2

α

|x

2

|

y

1,2

= sin2

α

|x

2

|

x

2

y

1

0

0

Singleton!

Compressive Phase Retrieval: Decoding

Slide60

Singleton

Identifying Singletons38

Magnitude

recovery

x

6x

4x2

0

0

0

y

1

y

2

y

3

y

4

y

5

Compressive Phase Retrieval: Decoding

Slide61

Singleton

Singletons + Doubletons39

Magnitude

recovery

Phase

recovery Resolvable

x6x4x2

y

1

y

2

y

3

y

4

Doubleton

Useless

y

5

Compressive Phase Retrieval: Decoding

Slide62

Decoding Graph40

…

n

signal nodes

…

k

non-zero

components

k

nodes

G

H

Compressive Phase Retrieval: Decoding

Slide63

Decoding Graph40

n signal nodes

“

Σ

” Structure

x

1

x

2

Hx2

x

1

G

…

…

Compressive Phase Retrieval: Decoding

Transitive Property

Slide64

Decoding Graph40

n signal nodes

“

Σ

” Structure

x

1

x

2

Hx2

x

1

G

…

…

Compressive Phase Retrieval: Decoding

ck

log

k

Goal:

O

(

k

) measurements

O

(

k

log

k

) decoding complexity

Slide65

Decoding Graph40

n signal nodes

“

Σ

” Structure

x

1

x

2

Hx2

x

1

G

…

…

Compressive Phase Retrieval: Decoding

ck

Goal:

O

(

k

) measurements

O

(

k

log

k

) decoding complexity

Slide66

y

1

y

2

y

3

y4

Singleton

Decoding via Resolvable Multiton

41

Doubleton

Multiton

x

2

x

6

x

4

Compressive Phase Retrieval: Decoding

Slide67

Multiton

Singleton

Decoding via Resolvable Multiton

41

Resolvable

Solving a quadratic equation

“Peeling” process:

y

1

y2y3

y

4

x

2

x

6

x

4

Doubleton

Compressive Phase Retrieval: Decoding

M

M

P

P

M

P

Slide68

Two Phases42

n signal nodes

…

…

…

1

st

Phase

O

(

k

)

measurements

O

(

k

)

complexity

Singletons

&

Doubletons

2

nd

Phase

O

(

k

)

measurements

O

(

k

log

k

)

complexity

Multitons

…

…

Compressive Phase Retrieval: Design of Bipartite graphs

Slide69

1st Phase43

n

signal nodes

ck

measurement nodes

…

G

Iwith prob. 1/k

H ’

Many SingletonsMany Doubletons

H

…

…

Compressive Phase Retrieval: Design of Bipartite graphs

Slide70

2nd Phase44

n

signal nodes

ck/

2

measurement

nodes…GII,

1with prob. 2/k

Many

MultitonsH

H ’

…

…

Compressive Phase Retrieval: Design of Bipartite graphs

Slide71

2nd Phase44

n

signal nodes

ck/

4

measurement nodes

…GII,2

with prob. 4

/k

H

H ’

Many

Multitons

…

…

Compressive Phase Retrieval: Design of Bipartite graphs

Slide72

45

Our Contribution: Fast and Efficient Algorithms

SHO-FA

Compressive Phase Retrieval

Group Testing

FRANTIC

SUPER

Slide73

45

Our Contribution: Fast and Efficient Algorithms

SHO-FA

Group Testing

FRANTIC

SUPER

GROTESQUE

Slide74

Group Testing

IN

OUT

Negative

0

Positive

1

46

Group Testing: Problem

Slide75

GROTESQUE47Group Testing: Structured Measurements

Multiplicity

Localization

n

items

k

defectives

GROTESQUE

k

> 1

k

= 1

k

= 0

Slide76

Multiplicity

48

Group Testing: Structured Measurements

?

Half of 1’s

Distinct columns

n

m

1

Slide77

Localization

49

Group Testing: Structured Measurements

?

n

m

2

Binary

expression of index

Slide78

50

Our Contribution: Fast and Efficient Algorithms

SHO-FA

Group Testing

FRANTIC

SUPER

GROTESQUE

Slide79

y

1

y

2

x

2

x

5

x4

x3x1

Multiton

Singleton

“Good” Bipartite Graph + Structured Measurements

51

Sparse Recovery: Summary

Picking &

Peeling

Slide80

Thank you謝謝

Slide81

y1

y

2

x

2

x

5

x

4

x3x1

Decoding Process: Picking & Peeling

15

Sparse Recovery: Framework

y

3

“Useful”

Unknown support

Useful nodes

-- Existence (design of bipartite graphs)

-- Identification (structured measurements)

Decoding

Multiton

Singleton

Slide82

Decoding graph43

H

O

(

k

log

k

) right nodes are singletons

O

(

k

log

k

) right nodes are doubletons

(By Chernoff bound)

O

(

k

log

k

) different edges in graph

(By Coupon Collection)

O

(

k

log

k

) measurement nodes

The decoding graph is connected

(By connectivity of random graph)

Goal:

O

(

k

) measurements

O

(

k

log

k

) decoding complexity

March 20, 15

Compressive Phase Retrieval: Decoding

Slide83

Decoding graph45

H

O

(

k

) right nodes are singletons

O

(

k

)

right nodes are doubletons

(By Chernoff bound)

O

(

k

) different edges in graph

(By Coupon Collection)

O

(

k

) measurement nodes

Giant connected component

(By percolation results)

Goal:

O

(

k

) measurements

O

(

k

log

k

) decoding complexity

March 20, 15

Compressive Phase Retrieval: Decoding

Slide84

Singleton

Identifying doubletons39

Magnitude recovery

x

6

x4

x2

y

1

y

2

y

3

y

4

y

5

Compressive Phase Retrieval: Decoding

Slide85

Singleton

Identifying doubletons39

Magnitude recovery

x

6

x4

x2

x

2

x

2

y

1

y

2

y

3

y

4

y

5

Compressive Phase Retrieval: Decoding

Slide86

Singleton

Identifying doubletons39

Magnitude recovery

x

6

x4

x2

x

2

x

2

x

4

x

4

y

1

y

2

y

3

y

4

y

5

Compressive Phase Retrieval: Decoding

Slide87

Singleton

Identifying doubletons39

Magnitude recovery

x

6

x4

x2

x

2

x

2

x

4

x

4

x

6

y

1

y

2

y

3

y

4

y

5

Compressive Phase Retrieval: Decoding