Li Xu 1 Jiaya Jia 1 Yasuyuki Matsushita 2 1 The Chinese University of Hong Kong 2 Microsoft Research Asia Conventional Optical Flow Middlebury Benchmark Baker et al 07 Dominant Scheme CoarsetoFine Warping ID: 830189
Download The PPT/PDF document "Motion Detail Preserving Optical Flow Es..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Motion Detail Preserving Optical Flow Estimation
Li Xu
1
,
Jiaya
Jia
1
, Yasuyuki Matsushita
2
1
The Chinese University of Hong Kong
2
Microsoft Research Asia
Slide2Conventional Optical Flow
Middlebury Benchmark [Baker et al. 07]
Dominant Scheme: Coarse-to-Fine Warping
Slide3Large Displacement Optical Flow
Region Matching [
Brox
et al.
09, 10]
Discrete Local Search [
Steinbrucker
et al. 09]
Slide4Both Large and Small Motion Exist
Capture large motion
Preserve sub-pixel accuracy
Slide5Our Work
Framework
Extended coarse-to-fine motion estimation for both large and small displacement optical flow
ModelA new data term to selectively combine constraintsSolverEfficient numerical solver for discrete-continuous optimization
Slide6Outline
Framework
Extended coarse-to-fine motion estimation for both large and small displacement optical flow
Model
A new data term to selectively combine constraintsSolverEfficient numerical solver for discrete-continuous optimization
Slide7The Multi-scale Problem
Slide8The Multi-scale Problem
Ground truth
Ground truth
Ground truth
Slide9The Multi-scale Problem
Ground truth
Ground truth
Ground truth
Slide10Ground truth
…
Estimate
Estimate
Estimate
Ground truth
Ground truth
Slide11The Multi-scale Problem
Large discrepancy between initial values and optimal motion vectors
Our solution
Improve flow initialization to reduce the reliance on the initialization from coarser levels
Slide12Extended Flow Initialization
Sparse feature matching for each level
Slide13Extended Flow Initialization
Identify missing motion vectors
Slide14Extended Flow Initialization
Identify missing motion vectors
Slide15Extended Flow Initialization
…
Slide16…
Extended Flow Initialization
Fuse
Slide17Outline
Framework: extended initialization for coarse-to-fine motion estimation
Model: selective data term
Efficient numerical solver
Slide18Data Constraints
Average
Gradient constancy
Color constancy
Slide19Pixels moving out of shadow
Problems
Color constancy is violated
Average:
: ground truth motion of
p
1
Gradient constancy holds
Slide20Pixels undergoing rotational motion
Problems
Color constancy holds
Gradient constancy is violated
: ground truth motion of
p
2
Average:
Slide21Our Proposal
Selectively combine the constraints
where
Slide22Comparisons
Slide23Outline
Framework: extended initialization for coarse to fine motion estimation
Model: selective data term
Efficient numerical solver
Slide24Energy Functions and Solver
Total energy
Probability of a particular state of the system
Slide25Mean Field Approximation
Partition function
Sum over all possible values of
α
. . .
The
effective
potential
E
eff
(u)
[Geiger &
Girosi
, 1989]
Slide26Optimal condition (Euler-Lagrange equations)
It decomposes to
{
Slide27{
Slide28Algorithm Skeleton
For each level
Extended Flow Initialization (QPBO)
Continuous Minimization (Iterative reweight)
Update
Compute flow field (
Variable Splitting
)
{
Slide29Results
Averaging constraints
Ours
Difference
Slide30Middlebury Dataset
EPE=0.74
Slide31Results from Different Steps
Coarse-to-fine
Extended coarse-to-fine
Slide32EPE=0.15 rank =1
EPE=0.24 rank =1
Slide33Large Displacement
Overlaid Input
Slide34Large Displacement
Motion Estimates
Coarse-to-fine
Our Result
Warping Result
Slide35Comparison
Motion Magnitude Maps
LDOP [
Brox
et al. 09 ]
[
Steinbrucker
et al. 09
]
Ours
Slide36More Results
Overlaid Input
Slide37Conventional Coarse-to-fine
Our Result
Slide38More Results
Overlaid Input
Slide39Coarse-to-fine
Our Result
Slide40Conclusion
Extended initialization (Framework)
Selective data term (Model)
Efficient numerical scheme (Solver)LimitationsFeatureless motion details
Large occlusions
Slide41Thank you!
Slide42More Results
Overlaid Input
Slide43Coarse-to-fine
Our Results