/
Organizing Heterogeneous Scene Organizing Heterogeneous Scene

Organizing Heterogeneous Scene - PowerPoint Presentation

berey
berey . @berey
Follow
342 views
Uploaded On 2022-06-18

Organizing Heterogeneous Scene - PPT Presentation

Collections through Contextual Focal Points Kai Xu Rui Ma Hao Zhang Chenyang Zhu Ariel Shamir Daniel CohenOr Hui Huang Shenzhen VisuCA Key Lab ID: 920671

cluster focal clustering scene focal cluster scene clustering mining graph guided scenes points optimization extraction structural complex interleaving datasets

Share:

Link:

Embed:

Download Presentation from below link

Download Presentation The PPT/PDF document "Organizing Heterogeneous Scene" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Slide1

Slide2

Organizing Heterogeneous Scene

Collections through

Contextual Focal Points

Kai Xu, Rui Ma, Hao Zhang, Chenyang Zhu,Ariel Shamir, Daniel Cohen-Or, Hui Huang

Shenzhen VisuCA Key Lab / SIATSimon Fraser UniversityNational University of Defense TechnologyThe Interdisciplinary CenterTel Aviv University

Slide3

Rapid growth of 3D data

Trimble 3D Warehouse

over

2 million modelsOrganizing large 3D datasets

3

Slide4

Organization

Examples

Categorization tree

in ImageNet[Stanford]

Categorization trees of 3D shape set[Huang et al. 2013]

4

Slide5

The key to organization

Grouping & relating similar contents

How to compare complex things?5

For heterogeneous 3D shapes …qualitative analysis

[Huang et al. 2013]

Slide6

Objects vs. Scenes

heterogeneous

hybrid

bedroom

d

ining room

living room

office

studio

Slide7

London

Paris

Comparing complex things

7

Slide8

London

Paris

New York City

Milan

Comparing complex things

8

Slide9

London

Paris

Berlin

RomeMadrid

Comparing complex things

9

Slide10

The key point

Comparing complex things:

focal pointsDetermining focal points: within a context

10

Slide11

How to measure similarity of scenes?

Especially hard for hybrid scenes

?

?

distance

11

Slide12

How to measure similarity of scenes

3D scenes should be compared

wrt

a focal point

focal : a

subscene

12

Slide13

How to measure similarity of scenes

3D scenes should be

compared wrt a focal point

13

Slide14

How to determine focal points?

Contextual analysis: co-analyzing a set

Characterize a semantic scene typeFrequently appear in a category

… …

l

iving room

14

Slide15

How to determine focal points?

Frequency alone is not enough …

Trivially frequent subscene is meaningless

a single

chair may appear

in

most indoor

scenes

15

Slide16

How to determine focal points?

Discriminant

Frequent only within the set of semantically related scenesCharacterize the scene category

bedroom

l

iving room

unknown

unknown

16

Slide17

How to determine focal

points?

Focal point detection relies on clusteringFrequent only within some clusterScene clustering is guided by focal pointsCharacterized by representative focal points

Coupled problems …17

Slide18

Overview

Basic rep.: structural graphs

Interleaving optimization

18

Slide19

Overview

Interleaving optimization

Focal point detection

Focal-based clustering

19

Slide20

Overview

Focal-based organization

20

Slide21

Focal-based scene collection exploration

21

Slide22

Main ideas

How to measure similarity of complex things?

Based on focal pointsHow to determine focal points?Co-analysis of a collectionCoupled with clusteringFocal points provide a way to relate complex things

Slide23

Outline

Datasets

Structural graph & layout similarityFocal-driven scene co-analysisThe objectiveInterleaving optimization23

Slide24

Outline

Datasets

Structural graph & layout similarityFocal-driven scene co-analysisThe objective Interleaving optimization

24

Slide25

Dataset

Collection

#Scenes

#Objects

#Scene

categories

#Hybrid

scenesStanford[Fisher et al. 2012]1323,46150

Tsinghua

[Xu et al. 2013

]

792

13,365

6

102

25

Slide26

Outline

Datasets

Structural graph& layout similarityFocal-driven scene co-analysisThe objectiveInterleaving optimization

26

Slide27

Structural graph

Nodes: individual objects

Edge: relationships (support, proximity

) 27

Slide28

Layout similarity

Measuring edge similarity

Spatial arrangement of OBBs, not edge tags

28

Slide29

Outline

Datasets

Structural graph & layout similarityFocal-driven scene co-analysisThe objectiveInterleaving optimization

29

Slide30

Objective

Our core problem: clustering

Objective: overall compactness of clusters

Per-cluster compactness: focal-centric scene similarity

Optimize clustering

and focal points

 

30

Slide31

Interleaving optimization

Focal extraction: cluster-guided graph mining

Fix clustering, optimize focal pointsScene clustering: focal-induced scene clusteringFix focals, optimize clustering

31

Slide32

Interleaving optimization

Focal extraction: cluster-guided graph mining

Fix clustering, optimize focal pointsScene clustering: focal-induced scene clusteringFix focals

, optimize clustering32

Slide33

Focal extraction:

graph mining

……

……

……

……

33

Slide34

Focal extraction:

graph mining

……

……

……

……

34

Structural graph

Slide35

Focal extraction:

graph mining

Frequent pattern mining

……

……

……

……

35

Slide36

Focal extraction:

graph mining

Frequent pattern mining……

………………

36

Slide37

Focal extraction:

graph mining

…………

…………

37

Focal embedding

Slide38

Focal extraction

: cluster-guided

miningTrivially frequent substructuresE.g. a single chairFrequent but not discriminantCluster-guided mining

Mining substructures that characterize a clusterUsing clusters to weight frequency38

Slide39

Cluster-guided

mining: illustrative example

Total # of occurrence:

#

 

39

#

 

#

 

Frequent pattern

mining result:

 

 

Slide40

Cluster-guided

mining: illustrative example

40

Using clusters to weight frequency

cluster 1

cluster 2

cluster 3

Slide41

Cluster-guided

mining: illustrative example

cluster 1

cluster 2

cluster 3

is not discriminant

41

Slide42

Cluster-guided

mining: illustrative example

cluster 1

cluster 2

cluster 3

is representative focal of cluster 2

42

Slide43

Cluster-guided

mining: illustrative example

cluster 1

cluster 2

cluster 3

43

Cluster-guided mining result:

Slide44

Interleaving optimization

Focal extraction: cluster-guided graph mining

Fix clustering, optimize focal pointsScene clustering: focal-induced scene clusteringFix focals, optimize clustering

44

Slide45

Scene clustering

Representation: Bag-of-Word

(

BoW

)

feature

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

45

Slide46

Scene clustering

Subspace clustering [Wang et al. 2011]

 

 

 

46

Slide47

Scene clustering

Indicator feature: not very informative

 

 

 

Really similar?

47

Slide48

FCGK

Per-cluster compactness defined by

focal-centric graph kernel

(FCGK):Focal-induced scene clustering

Scaling factor

Root walk kernel

[Fisher et al. 2011]

48

Slide49

Focal-induced scene clustering

R

e

weight Bag-of-Word feature

 

 

 

Comp.: 0.2

Comp.: 1.0

Overall Comp.: 1.2

49

Slide50

Focal-induced scene clustering

Reweighted subspace

clustering

 

 

 

Comp.: 1.0

Comp.: 1.8

Overall Comp.: 2.8

50

Slide51

Interleaving optimization

Objective: maximize overall compactness

51

Slide52

Post-processing

Cluster attachment

Focal joining

Cluster overlapLarge-scale and non-local

focals52

Slide53

Co-analyzing scene collection

53

Slide54

54

Slide55

Results

Time

focal point extraction (~5%)scene clustering (~90% with a Matlab implementation)

CollectionTime (min) #Focal#Non local focal%Multi-focal scenes

Stanford (132)3.224450.4%

Tsinghua (792)10.5347

46.1%55

Slide56

Compactness Plot

Change of overall compactness while interleaving optimization

Stanford

Tsinghua56

Slide57

Detected focal points (Tsinghua)

……

Non local

focals

57

Slide58

Results – Comparing to graph kernel [Fisher et al. 2011]

FCGK is more discriminative;

New perspective to compare complex scenes 58

Slide59

Application – Comprehensive retrieval

Query

……

59

Slide60

Application

scene collection exploration

“gateway

60

Slide61

Limitations

Use object labels:

Handling of noisy or incomplete labels?Pure geometry analysis?Structural graphs model flat arrangements of objectsHierarchical organization potentially advantageous?

61

Slide62

Future work

Apply to other datasets

E.g. large and heterogeneous collections of imagesScene synthesisSubstitute sub-scenes instead of one object at a time62

Slide63

Acknowledgement

Anonymous reviewers

Datasets: [Fisher et al. 2012] and [Xu et al. 2013]Research grants:NSFC China, NSERC Canada, National 863 Program of China, Shenzhen Innovation Program, CPSF China,

Israel Science Foundation63

Slide64

Thank you!

An

exploratory

path through an overlap smoothly transits between two scene clusters.64