for Geometry Processing Justin Solomon Princeton University David Bommes RWTH Aachen University This Mornings Focus Optimization Synonym ish Variational methods This Mornings Focus ID: 430214
Download Presentation The PPT/PDF document "Optimization Techniques" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Optimization Techniquesfor Geometry Processing
Justin Solomon
Princeton University
David BommesRWTH Aachen UniversitySlide2
This Morning’s FocusOptimization.
Synonym(-
ish): Variational methods.Slide3
This Morning’s FocusOptimization.
Synonym(-
ish): Variational methods.Caveat: Slightly different connotation in MLSlide4
More SpecificallySlide5
Two RolesClientWhich optimization tool is relevant?
Designer
Can I design an algorithm for this problem?Slide6
Our Bias
Optimization is a
huge field.Patterns, algorithms, & examples common in geometry processing.Slide7
Rough PlanVocabularySimple examples
Unconstrained optimization
Equality-constrained optimizationPart I (Justin)Slide8
Rough PlanInequality constraintsAdvanced algorithms
Discrete problems
ConclusionPart II (David)Slide9
Rough PlanVocabulary
Simple examples
Unconstrained optimizationEquality-constrained optimizationPart I (Justin)(basic material!)Slide10
Optimization Terminology
Objective (“Energy Function”)Slide11
Optimization Terminology
Equality ConstraintsSlide12
Optimization Terminology
Inequality ConstraintsSlide13
Optimization TerminologyGradient
https://en.wikipedia.org/?title=GradientSlide14
Optimization TerminologyHessian
http://math.etsu.edu/multicalc/prealpha/Chap2/Chap2-5/10-3a-t3.gifSlide15
Optimization TerminologyJacobian
https://en.wikipedia.org/wiki/Jacobian_matrix_and_determinantSlide16
Optimization TerminologyCritical point
(unconstrained)
Saddle point
Local min
Local maxSlide17
Common MistakeCritical pointsmay not be minima.Slide18
Neither Sufficient nor NecessarySlide19
Except…
Numerical Algorithms
, SolomonMore laterSlide20
Rough PlanVocabulary
Simple examples
Unconstrained optimizationEquality-constrained optimizationPart I (Justin)Slide21
Encapsulates Many ProblemsSlide22
How effective are
generic
optimization tools?Slide23
Generic AdviceTry thesimplest solver first.Slide24
Quadratic with Linear Equality
(assume A is symmetric
and positive definite)Slide25
Special Case: Least-Squares
Normal equations
(better solvers for this case!)Slide26
Example: Mesh Embedding
G.
Peyré, mesh processing course slidesSlide27
Linear Solve for Embedding
:
Tutte embedding from mesh: Harmonic embeddingAssumption: symmetric. Slide28
More Explicit Form
Laplacian matrix!Slide29
Differential Geometry Perspective
K. Crane, brickisland.net
Leads to famous cotangent weights!Useful for interpolation.Slide30
Linear Solver ConsiderationsNever construct
explicitly(if you can avoid it)Added structure helpsSparsity, symmetry, positive definite Slide31
Two Classes of SolversDirect (explicit matrix)
Dense: Gaussian elimination/LU, QR for least-squaresSparse: Reordering (SuiteSparse
, Eigen)Iterative (apply matrix repeatedly)Positive definite: Conjugate gradientsSymmetric: MINRES, GMRESGeneric: LSQRSlide32
Very Common: Sparsity
Induced by the
connectivity of the triangle mesh.Iteration of CG has local effect Precondition! Slide33
Returning to Parameterization
What if
? Slide34
Nontriviality Constraint
Prevents
trivial solution .Extract the smallest eigenvalue. Slide35
Common Situation
Prevents
trivial solution .N contains basis for null space of A.Extract the smallest nonzero eigenvalue. Slide36
Back to Parameterization
Mullen et al. “Spectral Conformal Parameterization.” SGP 2008.Slide37
Continuous Story
2
3
4
5
6
7
8
9
10
“Laplace-Beltrami
Eigenfunctions
”Slide38
Basic Idea of EigenalgorithmsSlide39
Combining Tools So FarRoughly:Extract Laplace-Beltrami
eigenfunctions
:Find mapping matrix (linear solve!):
Ovsjanikov et al. “Functional Maps.” SIGGRAPH 2012.Slide40
Rough PlanVocabularySimple examples
Unconstrained optimization
Equality-constrained optimizationPart I (Justin)Slide41
Unconstrained OptimizationSlide42
Unconstrained Optimization
Unstructured.Slide43
Basic AlgorithmsGradient descentSlide44
Basic AlgorithmsGradient descent
Line searchSlide45
Basic AlgorithmsGradient descent
Multiple optima!Slide46
Basic AlgorithmsAccelerated gradient descent
Quadratic convergence on convex problems!
(Nesterov 1983)Slide47
Basic Algorithms
Newton’s Method
1
2
3
Line search for stabilitySlide48
Basic AlgorithmsQuasi-Newton: BFGS and friends
Hessian approximation
(Often sparse) approximation from previous samples and gradientsInverse in closed form!Slide49
Geometric FlowsOften continuous
gradient descent
M. KazhdanSlide50
Example: Shape Interpolation
Fr
öhlich and Botsch. “Example-Driven Deformations Based on Discrete Shells.” CGF 2011.Slide51
Interpolation PipelineRoughly:
Linearly
interpolate edge lengths and dihedral angles. Nonlinear optimization for vertex positions.Sum of squares: Gauss-NewtonSlide52
SoftwareMatlab:
fminunc or
minfuncC++: libLBFGS, dlib, othersTypically provide functions for function and gradient (and optionally, Hessian).Try several!Slide53
Rough PlanVocabularySimple examples
Unconstrained optimization
Equality-constrained optimizationPart I (Justin)Slide54
Lagrange Multipliers: IdeaSlide55
Lagrange Multipliers: Idea
- Decrease
f
:
- Violate constraint:
Slide56
Lagrange Multipliers: Idea
Want:Slide57
Example: Symmetric EigenvectorsSlide58
Use of Lagrange MultipliersTurns constrained optimization into
unconstrained root-finding.Slide59
Many OptionsReparameterizationEliminate
constraints to reduce to unconstrained case
Newton’s methodApproximation: quadratic function with linear constraintPenalty methodAugment objective with barrier term, e.g. Slide60
Schur Complement Reduction
(assume A is symmetric and positive definite)
Recall:Slide61
Schur Complement Reduction
No longer positive definite!Slide62
Trust Region Methods
Example:
Levenberg
-Marquardt
Fix (or adjust)
damping parameter
.
Slide63
Example: Polycube Maps
Huang et al. “L1-Based Construction of
Polycube Maps from Complex Shapes.” TOG 2014.Slide64
Nonlinear Constrained Problem
Align with coordinate axes
Preserve areaNote: Final method includes several more terms!Slide65Slide66
Convex Optimization ToolsTry lightweight options
versusSlide67
Convex Optimization ToolsTry lightweight options
versus
Sometimes work for non-convex problems…Slide68
Iteratively Reweighted Least Squares
Repeatedly solve linear systems
“Geometric median”Slide69
Alternating Projection
d
can be
a
Bregman
divergenceSlide70
Iterative Shrinkage-Thresholding
FISTA combines with
Nesterov descent!Decompose as sum of hard part f and easy part g.https://blogs.princeton.edu/imabandit/2013/04/11/orf523-ista-and-fista/Slide71
Augmented LagrangiansAdd constraint to objective
Does nothing when constraint is satisfiedSlide72
Alternating DirectionMethod of Multipliers (ADMM)
https://web.stanford.edu/~boyd/papers/pdf/admm_slides.pdfSlide73
The Art of ADMM “Splitting”Want two easy
subproblems
Takes some practice!Augmented partSolomon et al. “Earth Mover’s Distances on Discrete Surfaces.” SIGGRAPH 2014.Slide74
Frank-Wolfe
https://en.wikipedia.org/wiki/Frank%E2%80%93Wolfe_algorithm
Linearize objective, not constraints