Non-iterative Joint and Individual Variation Explained - PowerPoint Presentation

Slide1

Non-iterative Joint and Individual Variation Explained

Qing Feng

Joint Work with J.S. Marron, Jan HannigDate: 2014/09/25

1

Slide2

Era Challenge

2

Slide3

Data Challenges

Multi-Block data

XY

 

 

S

ubjects

Feature Set 1

Feature Set 2

R

apid growth of sources to obtain data

High volume of available feature information

A variety of feature sets

3

Slide4

Data Challenges

Multi-Block data challenges

High DimensionalityVariation Explanation

Heterogeneity

Singular value decomposition on concatenated data matrix

?

Singular value decomposition on Individual data blocks

?

Pre-transformation

?

4

Slide5

Data Challenges

Toy Example

==++

+

+

Rank=1

Rank=1

X

Y

Standard Gaussian Random

M

atrix

 

5

Slide6

Data Challenges

Simple concatenation

=

+

Low-rank SVD Approximation

Residual

Merged Data

6

Slide7

Goal

Insightful Variation Decomposition

X =

Y

 

=

 

 

+

+

 

 

 

 

+

+

Joint Structure

Individual Structure

Residual

7

Slide8

Essential Tool

Singular Value Decomposition (SVD) on

 

 

 

 

 

 

 

 

 

 

 

 

 

 

n

ull

 

 

 

 

 

 

Takeaway

Row space can be considered as ‘”Covariate” space

Singular values indicate “importance” of each covariate

8

Slide9

Definition

Joint Structure

 

 

=

=

 

 

 

 

 

 

 

 

 

 

 

9

Slide10

Definition

Individual structure

 

 

 

=

=

 

 

 

 

 

 

 

 

 

 

 

 

10

Slide11

Definition

Residual

 

 

 

=

=

 

 

 

 

 

 

 

 

 

 

11

Slide12

Outline of Implementation

Obtain covariate spaces of each structure

Recover structure matrices via projection

#

1 De-Noise

#

2

De-Noise

12

Slide13

#1 De-noise

Extract signal based on singular values Perform for each block individually

Data Block X

Data Block Y

# 1 thresholds for each data block

13

Slide14

#1 De-noise

#1 threshold selection[

Johnstone 2001] Limiting distribution of largest Eigen-value of covariance of standard Gaussian random matrix

In which,

follows Tracy-

Widom

law of distribution and

#1 threshold

 

14

Slide15

#2 De-noise

Union of null spacesFrom noiseless case

Union

contains everything

but

row space

 

 

 

 

 

Null space of data X, Null(X) contains

Individual row space of Y,

Grey noise

row space

Null space of data Y, Null(Y) contains

Individual row space of X,

Grey noise

row space

 

15

Slide16

#2 De-noise

Union of null spaces(UN) under noiseDirection spaces

Leaking joint signal becomes noiseEstimated signal direction deviates from direction in real row spaceSignal to ratio influences the angle

 

 

 

 

Joint in Y

Joint in X

Noise in X

Noise in

Y

Direction of leaking joint signal in UN

Direction of noise signal in UN

Paired in UN

16

Slide17

#2 De-noise

Union of null spaces under noiseIdentify components via singular value plot

Detect pairs of basis in two estimated row spacesAngle distinguish joint from individual components

Noisy direction

#2 threshold

Paired basis

Calculate angle

 

Individual direction

17

Slide18

#2 De-noise

#2 threshold selection

 

 

 

 

 

 

are estimation from [

Shablin

2010]

Suppose

as

,

In which,

 

18

Slide19

#2 De-noise

Connection between #1 and #2 thresholds

What if

are unknown ?

Multi-scale

aspect comes!

 

19

Slide20

Reconstruction

Project data blocks to each row spaces

Obtain estimations of each structure matricesTimes loading, singular value and basis in row together to recover each component

 

Data Block

Row space

Get Loading and singular value matrices

20

Slide21

Toy Example

Data visualization

==++

+

+

Rank=1

Rank=1

X

Y

Standard Gaussian Random

M

atrix

 

21

Slide22

Toy Example

#1 De-noise

Data Block XData Block Y

# 1 thresholds for each data block

22

Slide23

Toy Example

#2 De-noise

Noisy direction

#2 threshold

Individual direction

Joint direction

23

Slide24

Toy Example

Reconstruction

=++=

+

+

Joint

Individual

Residual

X

Y

24

Slide25

Spanish Mortality

Male block Versus Female block

Age as features

Age as features

Years as Subjects

Spanish Male

Spanish Female

+

-

25

Slide26

Spanish Male

26

Slide27

Spanish Female

27

Slide28

Spanish Mortality

#1 De-noise

Data Block MaleRank=7Data Block Female

Rank=6

(Log of singular value)

Eyeball # 1 thresholds

28

Slide29

Spanish Mortality

#2 De-noise

#2 thresholdRank of Joint structure=4

Joint directions

Noisy direction

Individual directions

29

Slide30

Joint Variation - Male

30

Slide31

Joint Variation - Female

31

Slide32

Individual Variation - Male

Spanish Civil War

32

Slide33

Individual Variation - Female

Flue epidemic

33

Slide34

Thank you!

34