/
Kernel Regularized Optimal Transport : Kernel Regularized Optimal Transport :

Kernel Regularized Optimal Transport : - PowerPoint Presentation

helene
helene . @helene
Follow
342 views
Uploaded On 2022-06-13

Kernel Regularized Optimal Transport : - PPT Presentation

Estimation amp Lifted Metrics J Saketha Nath IITH Joint Work with Pratik Jawanpuria Microsoft INDIA Piyushi Manupriya IITH Optimal Transport                 ID: 917466

optimal metric regularized transport metric optimal transport regularized amp mmd lifted ipm jawanpuria metrics nath ground manupriya application wasserstein

Share:

Link:

Embed:

Download Presentation from below link

Download Presentation The PPT/PDF document "Kernel Regularized Optimal Transport :" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Slide1

Kernel Regularized Optimal Transport :Estimation & Lifted Metrics

J. Saketha Nath (IITH)

Joint Work with Pratik

Jawanpuria

(Microsoft, INDIA),

Piyushi

Manupriya

(IITH)

Slide2

Optimal Transport

`

 

 

 

 

 

 

 

 

 

 

s

ource

t

arget

 

 

 

 

 

 

 

 

 

 

Slide3

Optimal Transport

`

 

 

 

 

 

 

 

 

 

 

s

ource

t

arget

2

3

 

Optimal

Transport Plan

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Slide4

Optimal Transport

`

 

 

 

 

 

 

 

 

 

 

Cost function

is given

 

 

Slide5

Optimal Transport

`

 

 

 

 

 

 

 

 

 

 

s

ource

t

arget

Optimal

Transport Plan

 

 

Slide6

Optimal Transport

`

 

 

 

 

 

 

 

 

 

 

s

ource

t

arget

 

Optimal

Transport Plan

 

Slide7

Optimal Transport

s

ource

target

 

Optimal

Transport Plan

 

Adapted from Gabriel

Peyré

and Marco

Cuturi

.

Computational Optimal Transport.

Slide8

Application: Color Transfer

https://optimaltransport.github.io/slides-peyre/Applications.pdf

Slide9

Application: Color Transfer

https://optimaltransport.github.io/slides-peyre/Applications.pdf

Slide10

Application: Color Transfer

https://optimaltransport.github.io/slides-peyre/Applications.pdf

 

Slide11

Application: Color Transfer

https://optimaltransport.github.io/slides-peyre/Applications.pdf

Slide12

ML Application: Domain Adaptation

[

Courty

et.al., PAMI’15]

Slide13

ML Application: Domain Adaptation

[

Courty

et.al., PAMI’15]

Slide14

ML Application: Domain Adaptation

Optimal Transport

[

Courty

et.al.

, PAMI’15]

Slide15

ML Application: Domain Adaptation

Optimal Transport

[

Courty

et.al.

, PAMI’15]

Slide16

ML Application: Domain Adaptation

Courty

et.al., PAMI’15Courty et.al., NIPS’17Redko et.al., ECML’17Gautheron et.al., ECML’18Yan et.al., IJCAI’18Redko et.al., AISTATS’19Lee et.al., CVPR’19Kerdoncuff et.al., IJCAI’20Dhouib et.al., ICML’20Taherkhani et.al., ECCV’20Xu et.al., CVPR’20Cheng et.al., NeuroComp’20Montesuma et.al., CVPR’21Nguyen et.al., UAI’21Luo et.al., CVPR’21

Slide17

-

Wasserstein Metric

 

 

needs to be a (ground) metric

 

Slide18

-

Wasserstein Metric

 

 

needs to be a (ground) metric

 

Slide19

Application: Generative Models

 

Slide20

Unsupervised Wasserstein loss

 

Slide21

Unsupervised Wasserstein loss

 

Arjovsky

et.al.

, ICML’17

I Gulrajani et.al., NIPS’17Petzka et al., ICLR’18

Genevay et.al., AISTATS’18Sanjabi et.al., NeurIPS’18

Tolstikhin et al., ICLR’18Dukler et.al., ICML’19Chen et.al., ICML’19Bunne et.al., ICML’19

Wu et.al., CVPR’19Liu et.al., ICCV’19Balaji et.al., NeurIPS’20

Xu et.al., NeurIPS’20Shen et.al., NeurIPS’20Fatras et.al., AISTATS’20Patrini et al.

, UAI’20Rout et.al., ICLR’22

Slide22

Wasserstein Interpolation

, Barycenter problem

 

Adapted from Gabriel

Peyré

and Marco

Cuturi. Computational Optimal Transport.

Slide23

Application: Shape interpolationSolomon et.al., Graphics’15

Slide24

-

Wasserstein Metric = Kantorovich Metric

 

 

needs to be a (ground) metric

 

 

Slide25

IPM metrics

 

needs to be a (ground) metric

 

 

 

Slide26

MMD Metric

 

needs to be a characteristic kernel

 

 

 

 

Slide27

Wasserstein vs MMD

Non-

Hilbertian

Lifts ground metricLocal minima issue less severeEstimation very challengingNumerical optimization

 

Hilbertian

Lifts similarity

Local minima issue more severe

Estimation is

easyClosed form

 

Slide28

Wasserstein vs MMD

Non-

Hilbertian

Lifts ground metricLocal minima issue less severeEstimation very challengingNumerical optimization

 

Hilbertian

Lifts similarity

Local minima issue more severe

Estimation is

easyClosed form

 

Slide29

Wasserstein vs MMD

Non-

Hilbertian

Lifts ground metricLocal minima issue less severeEstimation very challengingNumerical optimization

 

Hilbertian

Lifts similarity

Local minima issue more severe

Estimation is

easy

Closed form

 

Bottou

et.al., 2018

Slide30

Wasserstein vs MMD

Non-

Hilbertian

Lifts ground metricLocal minima issue less severeEstimation very challengingNumerical optimization

 

Hilbertian

Lifts similarity

Local minima issue more severe

Estimation is

easyClosed form

 

Bottou

et.al., 2018

Slide31

MMD regularized Optimal Transport

 

Slide32

MMD regularized Optimal Transport

 

 

Slide33

MMD regularized Optimal Transport

 

Slide34

MMD regularized Optimal Transport

 

Slide35

KL regularized Optimal Transport

 

Gaussian Hellinger-Kantorovich Metrics,

Liero

et.al.,

Inventiones

mathematicae

, 2018

needs to be Euclidean metric

 

Slide36

MMD regularized Optimal Transport

 

Slide37

MMD regularized OT: Statistical Estimation

Theorem:

Let

be a universal, normalized kernel and cost be continuous. Then,

.

 

[Nath &

Jawanpuria

, NeurIPS’20]

Slide38

MMD regularized OT: Statistical Estimation

Theorem:

Let

be a universal, normalized kernel and cost be continuous. Then,

.

 

Smoothness

:

[Hutter&Rigollet’20]

Low-rank

:

[Weed&Rigollet’19]

Entropic Regularization

:

[

Genevay

et.al.’19]

 

Slide39

MMD regularized OT: Statistical Estimation

Theorem:

Let

be a universal, normalized kernel and cost be continuous. Then,

.

 

A kernel embedding based viewpoint

More general regularization framework

Representer theorem

Dimension-free recovery of optimal plan/map

Out-of-sample estimation

[Nath & Jawanpuria, NeurIPS’20]

Slide40

IPM regularized OT: Lifted Metrics

 

[

Manupriya

, Nath &

Jawanpuria

, arXiv’22]

Slide41

IPM regularized OT: Lifted Metrics

[

Manupriya

, Nath & Jawanpuria, arXiv’22]Theorem: For any ground metric

, and any IPM,

is a norm-induced lifted metric over measures.

 

Slide42

IPM regularized OT: Lifted Metrics

[

Manupriya

, Nath & Jawanpuria, arXiv’22]Theorem: For any ground metric

, and any IPM,

is a norm-induced lifted metric over measures.

 

 

Slide43

IPM regularized OT: Lifted Metrics

[

Manupriya

, Nath & Jawanpuria, arXiv’22]Theorem: For any ground metric

, and any IPM,

is a norm-induced lifted metric over measures.

 

 

s.t.

 

Slide44

IPM regularized OT: Lifted Metrics

[

Manupriya

, Nath & Jawanpuria, arXiv’22]Theorem: For any ground metric

, and any IPM,

is a norm-induced lifted metric over measures.

 

 

s.t.

 

Slide45

IPM regularized OT: Lifted Metrics

[

Manupriya

, Nath & Jawanpuria, arXiv’22]Theorem: For any ground metric

, and any IPM,

is a norm-induced lifted metric over measures.

 

 

Slide46

IPM regularized OT: Lifted Metrics

[

Manupriya

, Nath & Jawanpuria, arXiv’22]

 

Slide47

IPM regularized OT: Lifted Metrics

[

Manupriya

, Nath & Jawanpuria, arXiv’22]

 

Slide48

IPM regularized OT: Lifted Metrics

[

Manupriya

, Nath & Jawanpuria, arXiv’22]

 

Theorem:

Let

be a bounded metric

, and the witness functions,

, in the IPM’s generating set satisfy the uniform bound

. Then, for any

,

is a lifted metric over measures.

 

Slide49

Simulations: Estimation

[Nath &

Jawanpuria

, NeurIPS’20]

Slide50

Simulations: Metrics

MMD

Wasserstein

Bottou et.al., 2018

Slide51

Simulations: Metrics

MMD

Wasserstein

Proposed

[Manupriya, Nath & Jawanpuria, arXiv’22]

Slide52

Simulations: Downstream Application

[

Manupriya

, Nath & Jawanpuria, arXiv’22]

Slide53

Summary

IPMs received hardly any attention compared to

divergences

MMD+OT alleviates the curse of dimensionality in estimationNew IPM+OT metrics show promise in initial ML applicationsFuture Work: Topological and Computational aspects 

Slide54

THANK YOU