# Harmonic Shape Analysis PowerPoint Presentation

2016-08-10 65K 65 0 0

##### Description

from Fourier to Wavelets. Ming . Zhong. 2012.9. Overview (1). Harmonic analysis basics. Represent signals as the linear combination of basic overlapping, wave-like functions. Natural domain (space/time). ID: 441439

Embed code:

DownloadNote - The PPT/PDF document "Harmonic Shape Analysis" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

### Presentations text content in Harmonic Shape Analysis

Slide1

Harmonic Shape Analysisfrom Fourier to Wavelets

Ming

Zhong

2012.9

Slide2

Overview (1)

Harmonic analysis basicsRepresent signals as the linear combination of basic overlapping, wave-like functionsNatural domain (space/time)frequency domainWide applicationssignal analysis/processingaudio, images, radio waves, seismic waves …smoothing, enhancing, equalization, denoising …reconstruction, approximation, compression … solving PDEsnumerical computation (FFT)Fourier analysisWavelet analysis

Slide3

frequency

coefficients

edited coefficients

s

ignals

processed signals

space/time

transform / analysis / decomposition

inverse transform / synthesis / reconstruction

Harmonic analysis pipeline

Slide4

Overview (2)

Motivations

for

extending harmonic

analysis

to 3D shapes

Signal processing on shapes

v

ertex coordinates, normals, texture seen as functions on surface

Deformable shape analysis

Manifold Fourier bases are “shape aware”

shape

classification, symmetry detection, segmentation, matching,

etc

Diffusion geometry representation

Slide5

Presentation Outline

Manifold Fourier

A

nalysis : Theories

Manifold Fourier

A

nalysis : Applications

Spectral Geometry

P

rocessing

Laplacian-based shape analysis

Diffusion-based shape analysis

Manifold Wavelets

Preliminary Work in Deformable

S

hape

A

nalysis

High-order shape matching

Hierarchical shape registration

Bag-of-Feature-Graph shape retrieval

Conclusion

Slide6

Manifold Fourier Analysis: Theoriesclassical Fourier analysis (1)

: space of periodic square-integrable functions on real lineBounded functions: Inner product: Fourier basis: Global supportOscillatingDistinct frequenciesOrthonormal basis of Ahas unique decomposition on

Slide7

Fourier basis functions of different frequencies

Slide8

Manifold Fourier Analysis: Theoriesclassical Fourier analysis (2)

Fourier transformTransform from to Fourier space-th Fourier coefficient:Inner product in Fourier spaceReconstruction (Fourier series)Parseval’s equation Preservation of energy

Slide9

Approximating functions using Fourier series

Slide10

Manifold Fourier Analysis: Theories manifold Fourier analysis (1)

Space of real valued functions defined on a 3D shapeSurface viewed as a compact Riemannian manifold with metric is a closed manifold or a manifold with Dirichlet boundaryInner product of What is the Fourier basis of ?

Slide11

Origins of classical Fourier basis

Helmholtz equation on real line: 1D Laplace operator: -th eigenfunction of Laplacian : -th eigenvalue, square of the frequency

Origins of manifold Fourier basis

Helmholtz equation on manifold : Laplace-Beltrami operator on : -th eigenfunction of , manifold harmonic basis: -th eigenvalue, square of shape frequency

Manifold Fourier Analysis: Theories

manifold Fourier analysis (2)

Slide12

Manifold Fourier Analysis: Theories manifold Fourier analysis (3)

Laplace-Beltrami operator Self-adjointsemi-positive definiteAdmitting an orthonormal eigensystem by solving the Dirichlet problemEigenvalues constitute a real non-negative sequenceEigenfunctions form an orthonormal basis of

Slide13

Visualized manifold harmonic basis functions of different frequencies(value increases from blue to red)

Slide14

Manifold Fourier Analysis: Theories manifold Fourier analysis (4)

Manifold Fourier transform (manifold harmonic transform)Scalar function defined on manifold ()Manifold Fourier coefficientManifold Fourier series (reconstruction)

Slide15

Manifold Fourier Analysis: Theories classical and manifold Fourier comparisons

classical

Fourier analysisManifold Fourier analysisdomaininner productLaplacian equationfrequencyFourier basis (Fourier transformFourier series

classical

Fourier analysis

Manifold Fourier analysis

domain

inner product

Laplacian equation

frequency

Fourier transform

Fourier series

Slide16

Slide17

Manifold Fourier Analysis: Theories discrete settings (1)

Discrete Laplace-Beltrami operatorDiscrete mesh Cotangent formula [Belkin09]Area matrix , is the Voronoi cell size around point Weight matrix Laplacian matrix On general meshes, . So is not symmetric.

Slide18

Manifold Fourier Analysis: Theories discrete settings (2)

Generalized eigenvalue problemEigenvectors: Orthogonal w.r.t. -inner productIf , -inner product becomes the standard dot product of vectors.Fourier decomposition of

Slide19

Presentation Outline

Manifold Fourier

A

nalysis : Theories

Manifold Fourier

A

nalysis : Applications

Spectral Geometry

P

rocessing

Laplacian-based shape analysis

Diffusion-based shape analysis

Manifold Wavelets

Preliminary Work in Deformable

S

hape

A

nalysis

High-order shape matching

Hierarchical shape registration

Bag-of-Feature-Graph shape retrieval

Conclusion

Slide20

Manifold Fourier Analysis : Applicationsspectral geometry processing (1)

Manifold Fourier analysis on the mesh geometryCartesian coordinates of vertices Manifold Fourier transform of coordinatesReconstructionLevel of detail control

Slide21

Horse model reconstructed using indicated number of eigenvectors.[Zhang et al., 2010]

Slide22

Manifold Fourier Analysis : Applicationsspectral geometry processing (1)

Mesh Filtering: Frequency space filter/transfer functionMesh smoothing / exaggeration [Vallet08]Pose transfer / detail coating / shape blending [Vallet08, Lipman04]Other applicationsMesh compression [Karni00, Sorkine03]Mesh watermarking [Ohbuchi01]

Slide23

Effect of mesh filters [Vallet and Lévy, 2008]

From left to right: original shape; smoothing; detail enhancing; band exaggeration.

Slide24

Pose transfer filters [Vallet and Lévy, 2008]

(C) Is generated by copying the first 5 Fourier coefficients of (A) to (B)

Slide25

Geometry detail coating [Sorkine 2005]

Slide26

Manifold Fourier Analysis : ApplicationsLaplacian-based shape analysis

The Laplacian eigenfunctions and eigenvalues areIntrinsic to shape geometryInvariant to rigid deformations and isometric deformationsGlobally shape awareStableLaplacian embeddingLaplacian-spectra (eigenvalues of the Laplace-Beltrami operator)Global Point Signature Shape analysis applicationsShape classification [Reuter06, Rustamov07]Shape segmentation [Reuter09, Liu07,Goes08]Intrinsic symmetry detection [Ovsjanikov08]Mesh sequencing [Isenburg01]

Slide27

An eigenfunction of the Laplace-Beltrami operator computed on different deformations, showing isomteric invariance

Slide28

Shape classification using the Laplacian eigenvalues. Plot by 2D MDS. [Reuter et al., 2006]

Slide29

Intrinsic symmetry detection using Global Point Signature embedding [Ovsjanikov et at., 2008]

Slide30

Shape segmentation using nodal sets. The nodal sets are the zero sets of Laplacian-eigenfunctions.[Reuter et al., 2009]

Slide31

Diffusion-Based Shape Analysisheat diffusion and Fourier series (1)

Heat diffusion in a rod of length Solution represented by Fourier series

Slide32

Diffusion-Based Shape Analysisheat diffusion and Fourier series (2)

Heat diffusion on manifold Heat kernel fundamental solution to heat equationImpulse response Both heat kernel and general solution are represented by manifold Fourier series

Slide33

Diffusion-Based Shape Analysisheat kernel (1)

Physical interpretation of heat kernel :Heat distribution at time and point given that an initial unit of heat energy is placed at point at Transition density function of Brownian motion on the manifoldsmall t: fast decay of in space, slow decay in frequencylarge t: slow decay of in space, fast decay in frequencyHeat kernel embeddingHeat kernel signatureHeat diffusion distanceHeat Kernel Mapping (Single anchor, multiple timescales)Heat Kernel Coordinates (Multiple anchors, single timescale)

Slide34

Diffusion-Based Shape Analysisheat kernel (2)

Why use heat kernel for deformable shape analysis?IntrinsicInvariant to rigid and isometric deformations.InformativeGlobally shape-awareDetermines a shape up to isometry.Multi-scaleFor small , only reflects local properties of the shape around . For large values of , carry global structure of the shape from the point of view of Heat kernel enables more flexible analysis by controlling the timescales.Local objectsContains both local and global information. Feasible for analyzing partial deformable shape.RobustRepresented as the aggregate of eigenfunctions.Insensitive to perturbations, including topological change.

Slide35

Diffusion-Based Shape Analysisheat kernel (3)

Heat Kernel applications

Feature detection [Sun09

,

Gebal09]

Shape

matching [

Ovsjanikov10]

Shape retrieval [Bronstein11]

Slide36

Normalized HKS of the 4 marked points[Sun et al., 2009]

Slide37

Feature detection and classification by HKS[Sun et al., 2009]

Slide38

Two shapes have very different eigenfunctions (Left),while their HKSs (Right) are very close, esp. when t is small

Slide39

Diffusion-Based Shape Analysisdiffusion kernels (1)

Heat kernel can be generalized to general diffusion kernelsDiffusion kernel function should satisfySymmetry: Positive preserving: Positive-semidefiniteness: Square integrability: Conservation: can be interpreted as transition probability from x to y

Slide40

Diffusion-Based Shape Analysisdiffusion kernels (2)

Any diffusion kernel on can be represented by the manifold harmonics characterized by the real valued transfer function satisfiesPositive semi-definiteness: Square integrability: Conservation:

Slide41

Diffusion-Based Shape Analysisdiffusion kernels (3)

Multi-scaleDefine transfer function as Diffusion kernel has the scale parameter Scale parameter can be interpreted as diffusion time or length of random walkDiffusion distanceDiffusion kernel signature

Slide42

Diffusion-Based Shape Analysisdiffusion kernels (4)

Problem with heat kernel High frequency information (corresponding to with large ’s) are always suppressedUndesirable for high-precision analysis where details matter.Wave Kernel SignaturePhysical interpretation: average probability that a particle of energy is at point Energy distributionSignature vector is characterized by a collection of band-pass filters; large energies (highly oscillatory particles) more influenced by local geometry; small energies more influenced by global geometry.

Slide43

Comparison of scaled-HKS and WKS of two different points[Aubry et al., 2011]

Slide44

Diffusion-based Shape Analysisdiffusion kernels (5)

Learning diffusion kernelTrain the optimal transfer function based on specific dataset and applications

Transfer functions of heat kernel

Transfer functions of wave kernel

Transfer functions of a learned diffusion

kerne

Slide45

Presentation Outline

Manifold Fourier

A

nalysis : Theories

Manifold Fourier

A

nalysis : Applications

Spectral Geometry

P

rocessing

Laplacian-based shape analysis

Diffusion-based shape analysis

Manifold Wavelets

Preliminary Work in Deformable

S

hape

A

nalysis

High-order shape matching

Hierarchical shape registration

Bag-of-Feature-Graph shape retrieval

Conclusion

Slide46

Wave-like oscillationStart and end at zero; local supportA family of wavelet are generated through scaling and translation from a single mother wavelet

Manifold

Waveletswavelet basics

Slide47

Manifold Waveletscomparison of Fourier basis and wavelets

Fourier basis functions

wavelet functionsLocalized in frequency but have global support in space/timeLocalized both in frequency and space/timeEach Fourier harmonic has an exclusive frequencyMultiple wavelets localized at different locations represent the information of a single frequency. Generally, we use fewer wavelets at lower-frequency side and more wavelets at higher-frequency side.A family of Fourier harmonics are orthogonal to each other, forming a complete orthonormal basisA family of wavelet functions form a frame, but are not necessarily orthogonal

Fourier basis functions

wavelet functions

Localized

in frequency but have global support in space/time

Localized

both in frequency and space/time

Multiple wavelets localized at different locations represent the information of a single frequency.

Generally,

we use fewer wavelets at

lower-frequency

side

and

more wavelets at higher-frequency

side.

A family of Fourier

harmonics are orthogonal to each other, forming

a complete

orthonormal basis

A

family of wavelet

functions form a frame, but

are not necessarily

orthogonal

Slide48

Distributions of Fourier basis and wavelet functions in space-frequency

Slide49

Manifold Waveletsclassical wavelet analysis

Mother wavelet: Square integrable: Scaled by and translated by Continuous wavelet transform (CWT)Admissibility conditionInverse CWT

Slide50

Manifold Waveletssubdivision wavelets [Lounsbery et al., 1997]

Based on subdivision connectivityApplied in surface and surface data approximation, hierarchical editing

Slide51

Approximating color as a function over the

sphere using subdivision wavelets. [

Lounsbery

et al., 1997]

Slide52

Manifold Waveletsdiffusion wavelets [Coifman and Maggioni, 2006]

Dyadic powers of a diffusion operator to expand nested subspaceLifting schemeBottom-up fashion Slow computation

Slide53

Manifold Waveletsspectral manifold wavelets

Motivation: Scaling of mother wavelet cannot be explicitly expressed on manifold domain, but it can be easily defined on the frequency domainManifold: Laplace-Beltrami operator: Manifold Fourier transform

Slide54

Manifold Waveletsspectral manifold wavelets

Operator valued function Fourier transform of function generator function/transfer functionFourier transform of operator

Slide55

Manifold Waveletsspectral manifold wavelets

Set Dilate by scaling Fourier coefficientsScale by and localized at Bivariate kernel Wavelet functions Fourier transform

Slide56

Manifold WaveletsSpectral Manifold Wavelets

Slide57

Manifold Waveletsspectral manifold wavelets

Examples: Mexican hat waveletTransfer function: Wavelet functions: For a larger scale, the Mexican hat wavelet has a wider windows in space, but a narrower window in frequency.

1D Mexican hat

Slide58

Manifold Mexican hat wavelets in frequency domain

Slide59

Presentation Outline

Manifold Fourier

A

nalysis : Theories

Manifold Fourier

A

nalysis : Applications

Spectral Geometry

P

rocessing

Laplacian-based shape analysis

Diffusion-based shape analysis

Manifold Wavelets

Preliminary Work in Deformable

S

hape

A

nalysis

High-order shape matching

Hierarchical shape registration

Bag-of-Feature-Graph shape retrieval

Conclusion

Slide60

High-order Shape Matchingoverview

Partial deformable feature matchingDiffusion-driven distanceTensor-based high order graph matchingTwo-level matching hierarchy

T.

Hou

,

M.

Zhong

, H. Qin, "Diffusion-Driven High-Order Matching of Partial Deformable Shapes", ICPR’12 (accepted)

Slide61

High-order Shape MatchingDiffusion-driven metric

On manifold , the diffusion distance between point and is given by Heat kernel When , converges to the geodesic between and

Slide62

High-order Shape Matchingheat kernel tensor (1)

Two overlapping partial shapes (size ) and (size )A pair denotes a candidate match with and Assignment matrix Consider a tuple of three candidate matches for improved geometric compatibilityIf both and form triangles through diffusion distance Embed three-point tuple into a space by the three angles of the triangle the tuple formsDistance between and Affinity of

Slide63

High-order Shape Matchingheat kernel tensor (2)

High-order score of assignment if matched to Heat kernel tensor: Tensor power iteration [Duchenne et al., 2009]

Slide64

HKT for shape matching.

Slide65

High-order Shape Matching2-level matching hierarchy

Segment shapes using low-frequency eigenfunctionsCluster subgraph centered at the local extreama of low-frequency eigenfunctionsFirst match the cluster centers, then match each subgraphMuch improved time performance

Slide66

Matching results with noise (left) and holes (right)

Slide67

Matching features on similar objects

Slide68

Hierarchical Shape Registrationpipeline

Find and match initial features.

Construct multi-resolution structures of the two shapes in question.Register points in the coarsest level using the matched features as seeds and anchors points for local parameterization.Select additional features from newly registered points to expand the set of matched features.Map registered points to the higher level.Register points in the current level with the help of the expanded set of matched features

M.

Zhong

,

T.

Hou

,

H.

Qin, "A Hierarchical Approach to High-Quality Partial

Shape Registration

",

ICPR’12 (accepted)

Slide69

(a) Initial feature correspondences; (b) Coarse registration result (Third level); (c) Expanded feature correspondences (Third level); (d) Final registration result.

Slide70

Hierarchical Shape Registrationkey points

Feature driven. Features are used for

Seeding points for dense registration

Anchor points for local parameterization

Diffusion embedding

Use heat kernel coordinates to compare distances when searching for best matching candidates

Progressive registration

Registrations are propagated from already registered points to their neighbors, achieving great time performance and geometric compatibility

Hierarchical

Coarse-to-fine registration

Feature sets are augmented in each level for more accurate HKC embedding

Slide71

Comparison of multi-resolution (Left) and single-resolution (Right) shape registrations

Slide72

Bag-of-Feature-Graph Shape Retrievaloverview

Inspired by Bag-of-Words (BoW) and Spatial-Sensitive Bag-of-Words (SS-BoW)Feature-based representationExplicitly encode both spatial information and geometric word distribution among featuresGreat time performance

T.

Hou

,

X.

Hou

,

M.

Zhong

,

H.

for Non-rigid

Shape Retrieval

", ICPR’02

Slide73

Bag-of-Feature-Graph Shape Retrievalformulation(1)

HKS point descriptor A vector of HKS sampled at different values of tGeometric word Representative HKS vector clustered in the HKS descriptor space by k-means algorithmVocabulary Similarity of point and word

Slide74

Bag-of-Feature-Graph Shape Retrieval

formulation(2)

Feature set F of shape MFeature graph associated with the i-th geometric wordK(t,x,y): heat kernelBag-of-Feature-Graph representation of M

Slide75

Slide76

Bag-of-Feature-Graph Shape Retrievalformulation (3)

Bag-of-Feature-Graph descriptorChoose the sixe largest eigenvalues of each graph matrix, denoted by vector Shape distanceRetrieval by approximate nearest neighbor (ANN) search

Non-rigid shapes and their

BoFG

descriptors

Slide77

Conclusion

Manifold Fourier analysis

Spectral geometry processing

Laplacian-based shape analysis

Heat diffusion on manifold and diffusion based shape analysis

Manifold wavelets

Preliminary research on deformable shape analysis

Shape matching

Shape registration

Shape retrieval

Slide78

Future work

Shape correspondenceIncorporating the similarity of point descriptors when evaluating candidates for best matchesExperiment new diffusion kernels for more accurate local parameterizationHeat kernel not accurate enough for high-precision, small scale analysisApplications such as space-time model completion

Slide79

Future work

Spectral manifold waveletEfficient computation of reconstructionWavelets design via the transfer function Multi-scale feature detectionShape approximatingAntistrophic mesh filtering

Slide80

Thank you!

Questions?

Slide81

Presentation Outline

Manifold Fourier

A

nalysis : Theories

Manifold Fourier

A

nalysis : Applications

Spectral Geometry

P

rocessing

Laplacian-based shape analysis

Diffusion-based shape analysis

Manifold Wavelets

Preliminary Work in Deformable

S

hape

A

nalysis

High-order shape matching

Hierarchical shape registration

Bag-of-Feature-Graph shape retrieval

Conclusion