Harmonic Shape Analysis - PowerPoint Presentation

Slide1

Harmonic Shape Analysisfrom Fourier to Wavelets

Ming

Zhong

2012.9Slide2

Overview (1)

Harmonic analysis basics

Represent signals as the linear combination of basic overlapping, wave-like functions

Natural domain (space/time)frequency domainWide applicationssignal analysis/processingaudio, images, radio waves, seismic waves …smoothing, enhancing, equalization, denoising …reconstruction, approximation, compression … solving PDEsnumerical computation (FFT)Fourier analysisWavelet analysis

 Slide3

frequency

coefficients

edited coefficients

s

ignals

processed signals

space/time

transform / analysis / decomposition

inverse transform / synthesis / reconstruction

Harmonic analysis pipelineSlide4

Overview (2)Motivations

for

extending harmonic

analysis to 3D shapesSignal processing on shapesvertex coordinates, normals, texture seen as functions on surfaceDeformable shape analysisManifold Fourier bases are “shape aware”shape classification, symmetry detection, segmentation, matching, etcDiffusion geometry representationSlide5

Presentation Outline

Manifold Fourier

A

nalysis : TheoriesManifold Fourier Analysis : ApplicationsSpectral Geometry ProcessingLaplacian-based shape analysisDiffusion-based shape analysisManifold WaveletsPreliminary Work in Deformable Shape AnalysisHigh-order shape matchingHierarchical shape registrationBag-of-Feature-Graph shape retrievalConclusionSlide6

Manifold Fourier Analysis: Theoriesclassical Fourier analysis (1)

: space of periodic square-

integrable

functions on real lineBounded functions:

Inner product:

Fourier

basis:

Global

support

Oscillating

Distinct frequencies

Orthonormal

basis of

A

has unique decomposition on

 Slide7

Fourier basis functions of different frequencies

 Slide8

Manifold Fourier Analysis: Theoriesclassical Fourier analysis (2)

Fourier transform

Transform from

to Fourier space

-

th

Fourier coefficient:

Inner product in Fourier space

Reconstruction (Fourier series)

Parseval’s

equation

Preservation of energy

 Slide9

Approximating functions using Fourier seriesSlide10

Manifold Fourier Analysis: Theories manifold Fourier analysis (1)

Space of real valued functions defined on a 3D shape

Surface viewed as a compact Riemannian manifold

with metric

is a closed manifold or a manifold with Dirichlet boundary

Inner product of

What is the Fourier basis of

?

 Slide11

Origins of classical Fourier basis

Helmholtz equation on real line

: 1D Laplace operator

:

-

th

eigenfunction of Laplacian

:

-

th

eigenvalue, square of the frequency

Origins of manifold Fourier

basis

Helmholtz

equation on manifold

: Laplace-Beltrami operator on

:

-

th

eigenfunction of

, manifold harmonic basis

:

-

th eigenvalue, square of shape frequency Manifold Fourier Analysis: Theories manifold Fourier analysis (2)Slide12

Manifold Fourier Analysis: Theories manifold Fourier analysis (3)

Laplace-Beltrami operator

Self-adjoint

semi-positive definite

Admitting an orthonormal eigensystem by solving the Dirichlet problem

Eigenvalues

constitute a real non-negative sequence

Eigenfunctions

form an orthonormal basis of

 Slide13

Visualized manifold harmonic basis functions of different frequencies(value increases from blue to red)

 Slide14

Manifold Fourier Analysis: Theories

manifold Fourier analysis

(4)

Manifold Fourier transform (manifold harmonic transform)Scalar function defined on manifold (

)

Manifold Fourier coefficient

Manifold Fourier series (reconstruction)

 Slide15

Manifold Fourier Analysis: Theories classical and manifold Fourier comparisons

classical

Fourier analysis

Manifold Fourier analysis

domain

inner product

Laplacian equation

frequency

Fourier

basis (

Fourier transform

Fourier series

classical

Fourier analysis

Manifold Fourier analysis

domain

inner product

Laplacian equation

frequency

Fourier transform

Fourier seriesSlide16
Slide17

Manifold Fourier Analysis: Theories discrete settings (1)

Discrete Laplace-Beltrami operator

Discrete mesh

Cotangent formula [Belkin09]

Area matrix

,

is the

Voronoi

cell size around point

Weight matrix

Laplacian matrix

On general meshes,

.

So

is

not symmetric

.

 Slide18

Manifold Fourier Analysis: Theories

discrete settings

(2)

Generalized eigenvalue problem

Eigenvectors:

Orthogonal

w.r.t.

-inner

product

If

,

-inner

product becomes the standard dot product of vectors.

Fourier decomposition of

 Slide19

Presentation Outline

Manifold Fourier

A

nalysis : TheoriesManifold Fourier Analysis : ApplicationsSpectral Geometry ProcessingLaplacian-based shape analysisDiffusion-based shape analysisManifold WaveletsPreliminary Work in Deformable Shape AnalysisHigh-order shape matchingHierarchical shape registrationBag-of-Feature-Graph shape retrievalConclusionSlide20

Manifold Fourier Analysis : Applicationsspectral geometry processing (1)

Manifold Fourier analysis on the mesh geometry

Cartesian coordinates of vertices

Manifold Fourier transform of coordinates

Reconstruction

Level of detail control

 Slide21

Horse model reconstructed using indicated number of eigenvectors.[Zhang et al., 2010]Slide22

Manifold Fourier Analysis : Applications

spectral geometry processing

(1)

Mesh Filtering

: Frequency space filter/transfer function

Mesh smoothing / exaggeration [Vallet08]

Pose transfer / detail coating / shape blending [Vallet08, Lipman04]

Other applications

Mesh compression [Karni00, Sorkine03]

Mesh watermarking [Ohbuchi01]

 Slide23

Effect of mesh filters [Vallet and Lévy, 2008]

From left to right: original shape; smoothing; detail enhancing; band exaggeration.Slide24

Pose transfer filters [Vallet and Lévy, 2008]

(C) Is generated by copying the first 5 Fourier coefficients of (A) to (B)Slide25

Geometry detail coating [Sorkine 2005]Slide26

Manifold Fourier Analysis : Applications

Laplacian-based shape analysis

The Laplacian eigenfunctions and eigenvalues are

Intrinsic to shape geometryInvariant to rigid deformations and isometric deformationsGlobally shape awareStableLaplacian embeddingLaplacian-spectra (eigenvalues of the Laplace-Beltrami operator)

Global Point Signature

Shape analysis applications

Shape classification

[Reuter06,

Rustamov07]

Shape segmentation [Reuter09, Liu07,Goes08]

Intrinsic symmetry

detection [

Ovsjanikov08]

Mesh sequencing [Isenburg01]

 Slide27

An eigenfunction of the Laplace-Beltrami operator computed on different deformations, showing isomteric invarianceSlide28

Shape classification using the Laplacian eigenvalues. Plot by 2D MDS. [Reuter et al., 2006]Slide29

Intrinsic symmetry detection using Global Point Signature embedding

[

Ovsjanikov et at., 2008]Slide30

Shape segmentation using nodal sets. The nodal sets are the zero sets of Laplacian-eigenfunctions.[Reuter et al., 2009] Slide31

Diffusion-Based Shape Analysisheat diffusion and Fourier series (1)

Heat diffusion in a rod of length

Solution represented by Fourier series

 Slide32

Diffusion-Based Shape Analysis

heat diffusion and Fourier series

(2)

Heat diffusion on manifold

Heat kernel

fundamental solution to heat equation

Impulse response

Both heat kernel and general solution are represented by manifold Fourier series

 Slide33

Diffusion-Based Shape Analysisheat kernel (1)

Physical interpretation of heat kernel

:

Heat distribution at time

and point

given that an initial unit of heat energy is placed at point

at

Transition density function of Brownian motion on the manifold

small t: fast decay of

in space, slow decay in frequency

large t: slow decay of

in space, fast decay in frequency

Heat kernel embedding

Heat kernel signature

Heat diffusion distance

Heat Kernel Mapping (Single anchor, multiple timescales)

Heat Kernel Coordinates (Multiple anchors, single timescale)

 Slide34

Diffusion-Based Shape Analysisheat kernel (2)

Why use heat kernel for deformable shape analysis?

Intrinsic

Invariant to rigid and isometric deformations.InformativeGlobally shape-awareDetermines a shape up to isometry.Multi-scaleFor small ,

only reflects local properties of the shape around

.

For large values of

,

carry global structure of the shape from the point of view of

Heat kernel enables more flexible analysis by controlling the timescales.

Local objects

Contains both local and global information.

Feasible for analyzing partial deformable shape.

RobustRepresented as the aggregate of eigenfunctions.Insensitive to perturbations, including topological change.

 Slide35

Diffusion-Based Shape Analysisheat kernel (3)

Heat Kernel applications

Feature detection [Sun09

, Gebal09]Shape matching [Ovsjanikov10]Shape retrieval [Bronstein11]Slide36

Normalized HKS of the 4 marked points[Sun et al., 2009]Slide37

Feature detection and classification by HKS[Sun et al., 2009]Slide38

Two shapes have very different eigenfunctions (Left),while

their HKSs

(Right) are very

close, esp. when t is smallSlide39

Diffusion-Based Shape Analysisdiffusion kernels (1)

Heat kernel can be generalized to general diffusion kernels

Diffusion kernel function

should satisfySymmetry:

Positive preserving:

Positive-

semidefiniteness

:

Square

integrability

:

Conservation:

can be interpreted as transition probability from x to y

 Slide40

Diffusion-Based Shape Analysisdiffusion kernels (2)

Any diffusion kernel on

can be represented by the manifold harmonics

characterized by the real valued transfer function

satisfies

Positive semi-definiteness:

Square

integrability

:

Conservation:

 Slide41

Diffusion-Based Shape Analysisdiffusion kernels (3)

Multi-scale

Define transfer function as

Diffusion kernel has the scale parameter

Scale parameter can be interpreted as diffusion time or length of random walk

Diffusion distance

Diffusion kernel signature

 Slide42

Diffusion-Based Shape Analysis

diffusion

kernels (4)

Problem with heat kernel

High frequency information (corresponding to

with large

’s) are always suppressed

Undesirable for high-precision analysis where details matter.

Wave Kernel Signature

Physical interpretation: average probability that a particle of energy

is at point

Energy distribution

Signature vector is characterized by a collection

of band-pass filters; large energies (highly oscillatory particles) more influenced by local geometry; small energies more influenced by global

geometry.

 Slide43

Comparison of scaled-HKS and WKS of two different points[Aubry

et al., 2011]Slide44

Diffusion-based Shape Analysisdiffusion kernels (5)

Learning diffusion kernel

Train the optimal transfer function

based on specific dataset and applications 

Transfer functions of heat kernel

Transfer functions of wave kernel

Transfer functions of a learned diffusion

kerneSlide45

Presentation Outline

Manifold Fourier

A

nalysis : TheoriesManifold Fourier Analysis : ApplicationsSpectral Geometry ProcessingLaplacian-based shape analysisDiffusion-based shape analysisManifold WaveletsPreliminary Work in Deformable Shape AnalysisHigh-order shape matchingHierarchical shape registrationBag-of-Feature-Graph shape retrievalConclusionSlide46

Wave-like oscillationStart and end at zero; local supportA family of wavelet are generated through scaling and translation from a single mother wavelet

Manifold

Wavelets

wavelet basicsSlide47

Manifold Waveletscomparison of Fourier basis and wavelets

Fourier basis functions

wavelet functions

Localized in frequency but have global support in space/timeLocalized both in frequency and space/timeEach Fourier harmonic

has an exclusive frequency

Multiple wavelets localized at different locations represent the information of a single frequency.

Generally, we use fewer wavelets at lower-frequency

side

and more wavelets at higher-frequency

side.

A family of Fourier

harmonics are orthogonal to each other, forming a complete orthonormal basis

A family of wavelet functions form a frame, but are not necessarily orthogonal

Fourier basis functions

wavelet functionsLocalized in frequency but have global support in space/time

Localized both in frequency and space/timeMultiple wavelets localized at different locations represent the information of a single frequency.

Generally, we use fewer wavelets at lower-frequency side

and more wavelets at higher-frequency side.A family of Fourier

harmonics are orthogonal to each other, forming a complete orthonormal basis

A

family of wavelet

functions form a frame, but

are not necessarily

orthogonalSlide48

Distributions of Fourier basis and wavelet functions in space-frequencySlide49

Manifold Waveletsclassical wavelet analysis

Mother wavelet:

Square

integrable

:

Scaled by

and translated by

Continuous wavelet transform (CWT)

Admissibility condition

Inverse CWT

 Slide50

Manifold Waveletssubdivision wavelets [Lounsbery

et al

., 1997]

Based on subdivision connectivityApplied in surface and surface data approximation, hierarchical editingSlide51

Approximating color as a function over the

sphere using subdivision wavelets. [

Lounsbery

et al., 1997]Slide52

Manifold Waveletsdiffusion wavelets [Coifman and

Maggioni

, 2006]

Dyadic powers of a diffusion operator to expand nested subspaceLifting schemeBottom-up fashion Slow computation

 Slide53

Manifold Waveletsspectral manifold

w

avelets

Motivation: Scaling of mother wavelet cannot be explicitly expressed on manifold domain, but it can be easily defined on the frequency domainManifold: Laplace-Beltrami operator:

Manifold

Fourier transform

 Slide54

Manifold Waveletsspectral m

anifold wavelets

Operator valued function

Fourier transform of function

generator function/transfer function

Fourier transform of operator

 Slide55

Manifold Waveletsspectral manifold

w

avelets

Set

Dilate

by scaling Fourier coefficients

Scale by

and localized at

Bivariate kernel

Wavelet functions

Fourier transform

 Slide56

Manifold WaveletsSpectral Manifold Wavelets

Wavelet Transform

Inverse Transform

Admissibility condition

 Slide57

Manifold Waveletsspectral m

anifold

w

aveletsExamples: Mexican hat waveletTransfer function:

Wavelet functions:

For a larger scale, the Mexican hat wavelet has a wider windows in space, but

a narrower

window in frequency.

1D Mexican hatSlide58

Manifold Mexican hat wavelets in frequency domainSlide59

Presentation Outline

Manifold Fourier

A

nalysis : TheoriesManifold Fourier Analysis : ApplicationsSpectral Geometry ProcessingLaplacian-based shape analysisDiffusion-based shape analysisManifold WaveletsPreliminary Work in Deformable Shape AnalysisHigh-order shape matchingHierarchical shape registrationBag-of-Feature-Graph shape retrievalConclusionSlide60

High-order Shape Matchingoverview

Partial deformable feature matching

Diffusion-driven distance

Tensor-based high order graph matchingTwo-level matching hierarchyT. Hou, M. Zhong, H. Qin, "Diffusion-Driven High-Order Matching of Partial Deformable Shapes", ICPR’12 (accepted)Slide61

High-order Shape MatchingDiffusion-driven metric

On manifold

, the diffusion distance between point

and is given by

Heat kernel

When

,

converges to the geodesic between

and

 Slide62

High-order Shape Matching

heat kernel tensor

(1)

Two overlapping partial shapes (size

) and

(size

)

A pair

denotes a candidate match with

and

Assignment matrix

Consider a tuple of three candidate matches

for improved geometric compatibility

If both

and

form triangles through diffusion distance

Embed three-point tuple into a

space by the three angles of the triangle the tuple forms

Distance between

and

Affinity of

 Slide63

High-order Shape Matchingheat k

ernel tensor (2)

High-order score of assignment

if

matched to

Heat kernel tensor:

Tensor power iteration [

Duchenne

et al., 2009]

 Slide64

HKT for shape matching.Slide65

High-order Shape Matching2-level matching hierarchy

Segment shapes using low-frequency eigenfunctions

Cluster

subgraph centered at the local extreama of low-frequency eigenfunctionsFirst match the cluster centers, then match each subgraphMuch improved time performanceSlide66

Matching results with noise (left) and holes (right)Slide67

Matching features on similar objectsSlide68

Hierarchical Shape Registrationpipeline

Find and match initial features.

Construct multi-resolution structures of the two shapes in question.

Register points in the coarsest level using the matched features as seeds and anchors points for local parameterization.Select additional features from newly registered points to expand the set of matched features.Map registered points to the higher level.Register points in the current level with the help of the expanded set of matched features

M.

Zhong

,

T.

Hou

,

H. Qin, "A Hierarchical Approach to High-Quality Partial Shape Registration",ICPR’12 (accepted)Slide69

(a) Initial feature correspondences; (b) Coarse registration result (Third level); (c) Expanded feature

correspondences (Third level); (d) Final registration result.Slide70

Hierarchical Shape Registrationkey points

Feature driven. Features are used for

Seeding points for dense registration

Anchor points for local parameterizationDiffusion embeddingUse heat kernel coordinates to compare distances when searching for best matching candidatesProgressive registrationRegistrations are propagated from already registered points to their neighbors, achieving great time performance and geometric compatibilityHierarchicalCoarse-to-fine registrationFeature sets are augmented in each level for more accurate HKC embeddingSlide71

Comparison of multi-resolution (Left) and single-resolution (Right) shape registrationsSlide72

Bag-of-Feature-Graph Shape Retrievaloverview

Inspired

by Bag-of-Words (

BoW) and Spatial-Sensitive Bag-of-Words (SS-BoW)Feature-based representationExplicitly encode both spatial information and geometric word distribution among featuresGreat time performanceT. Hou, X. Hou, M. Zhong,

H.

Qin, "Bag-of-Feature-Graphs: A New Paradigm

for Non-rigid

Shape Retrieval

", ICPR’02Slide73

Bag-of-Feature-Graph Shape Retrievalformulation(1)

HKS point descriptor

A vector of HKS sampled at different values of t

Geometric word Representative HKS vector clustered in the HKS descriptor space by k-means algorithmVocabulary

Similarity of point

and word

 Slide74

Bag-of-Feature-Graph Shape Retrieval

formulation(2)

Feature set F of shape M

Feature graph associated with the i-th geometric word

K(

t,x,y

): heat kernel

Bag-of-Feature-Graph representation of M

 Slide75

 Slide76

Bag-of-Feature-Graph Shape Retrievalformulation (3)

Bag-of-Feature-Graph descriptor

Choose the sixe largest eigenvalues of each graph matrix, denoted by

vector

Shape distance

Retrieval by approximate nearest neighbor (ANN) search

Non-rigid shapes and their

BoFG

descriptorsSlide77

ConclusionManifold Fourier analysis

Spectral geometry processing

Laplacian-based shape analysis

Heat diffusion on manifold and diffusion based shape analysisManifold waveletsPreliminary research on deformable shape analysis Shape matchingShape registrationShape retrievalSlide78

Future work

Shape correspondence

Incorporating the similarity

of point descriptors when evaluating candidates for best matches

Experiment new diffusion kernels for more accurate local parameterization

Heat kernel not accurate enough for high-precision, small scale analysis

Applications such as space-time model completion

 Slide79

Future work

Spectral

manifold wavelet

Efficient computation of reconstructionWavelets design via the transfer function Multi-scale feature detectionShape approximatingAntistrophic mesh filtering

 Slide80

Thank you!

Questions?Slide81

Presentation Outline

Manifold Fourier

A

nalysis : TheoriesManifold Fourier Analysis : ApplicationsSpectral Geometry ProcessingLaplacian-based shape analysisDiffusion-based shape analysisManifold WaveletsPreliminary Work in Deformable Shape AnalysisHigh-order shape matchingHierarchical shape registrationBag-of-Feature-Graph shape retrievalConclusion