Engin Tola Vincent Lepetit Pascal Fua Computer Vision Laboratory EPFL 20080610 Motivation Narrow baseline Pixel Difference Graph Cuts groundtruth pixel difference input frame ID: 255143
Download Presentation The PPT/PDF document "A Fast Local Descriptor for Dense Matchi..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
A Fast Local Descriptor for Dense Matching
Engin Tola, Vincent Lepetit, Pascal FuaComputer Vision LaboratoryEPFL
2008-06-10Slide2
Motivation
Narrow baseline : Pixel Difference + Graph Cuts*groundtruthpixel difference
input frame
input frame
* Y. Boykov et al.
Fast Approximate Energy Minimization via Graph Cuts
. PAMI’01.Slide3
Motivation
Wide baseline : Pixel Difference + Graph Cutsgroundtruth
USE A DESCRIPTOR
input frame
input frame
pixel differenceSlide4
Motivation
Wide baseline : SIFT Descriptor*+ Graph Cuts
groundtruth
SIFT
250 Seconds
* D. Lowe.
Distinctive Image Features from Scale-Invariant Keypoints
. IJCV’04
input frame
input frameSlide5
Motivation
Wide baseline : DAISY Descriptor+ Graph Cuts
groundtruth
DAISY
5 Seconds
input frame
input frameSlide6
MotivationHistogram Based Descriptors: SIFT, GLOH, SURF…
Perspective robustnessProven good performanceRobustness to many image transformationsCons No efficient implementation exists for dense computation- Do not consider occlusionsDesign a descriptor that is as robust as SIFT or GLOH but can be computed much more effectively and handle occlusions. Slide7
Problem Definition
epipolar
line
epipolar
line
Virtual Camera
Input FramesSlide8
descriptor
Histogram based Descriptors…SIFT Computation
…Slide9
Histogram based Descriptors…
SIFT ComputationSlide10
SIFT -> DAISY
SIFT
+ Good Performance
Not suitable for dense
computationSlide11
SIFT -> DAISY
SIFT
Sym.SIFT
+ Gaussian Kernels : Suitable for Dense Computation
GLOH*
+ Good Performance
+ Better Localization
Not suitable for dense
computation
+ Good Performance
Not suitable for dense
computation
* K. Mikolajczyk and C. Schmid. A Performance Evaluation of Local Descriptors. PAMI’04.Slide12
SIFT -> DAISY
DAISY+ Suitable for dense computation + Improved performance:*+ Precise localization+ Rotational Robustness
Sym.SIFT
+ Suitable for Dense Computation
GLOH
+ Good Performance
+ Better Localization
Not suitable for dense
computation
* S. Winder and M. Brown.
Learning Local Image Descriptors
in CVPR’07Slide13
DAISY Computation
…
…
…Slide14
DAISY Computation
…
…
…Slide15
DAISY Computation
DAISY : 5sSIFT : 250s
-
Rotating the descriptor only involves reordering the histograms.
-
The computation mostly involves 1D convolutions, which is fast.
Slide16
Depth Map Estimation
Descriptors
Occlusion
Depthmap
Evidence
Smoothness Prior
Occlusions should be handled explicitly!Slide17
Depth Map Estimation
Evidence
P. of a specific Occlusion Mask
Occlusion MasksSlide18
Depth Map Estimation
EvidenceOcclusion Masks
P. of a specific Occlusion MaskSlide19
Experiments
DAISY
SIFT
SURF
NCC
Pixel Diff
Laser Scan
Comparing against other DescriptorsSlide20
100
908070605040
30
20
10
0
Correct Depth % for Image Pairs
Experiments
Comparison with other Descriptors
DAISY
SIFT
SURF
NCC
PIXELSlide21
100
908070605040
30
20
10
0
Correct Depth % for Image Pairs
Experiments
Comparison with other Descriptors
DAISY
SIFT
SURF
NCC
PIXEL
Correct Depth %
vs
Error ThresholdSlide22
Herz-Jesu Sequence
87.4 %
83.9
%
83.8
%
84.9
%
91.8
%
91.8
%
90.8 %
83.2 %
93.5
%
89.4
%
80.2
%
90.7
%
Truly Occluded
Missed Depths
Missed OcclusionsSlide23
Herz-Jesu Sequence
Ground TruthDAISYSlide24
Comparison with Strecha’05
Strecha’05: Wide baseline stereo from Multiple Views: A probabilistic Account Strecha: 3072x2048Slide25
Comparison with Strecha’05
Strecha’05: Wide baseline stereo from Multiple Views: A probabilistic Account
768x512Slide26
Image TransformsContrast Change
ScaleBlurry Webcam ImagesSIFTNCCSlide27
Image TransformsContrast Change
ScaleBlurry Webcam ImagesDAISYNCCSlide28
Conclusion
DAISY: Efficient descriptor for dense wide baseline matching. Handles occlusions correctly. Robust to perspective distortions. Robust to lighting changes. Can handle low quality imagery. Future work: Image-based rendering from widely spaced cameras. Object detection and recognition.Slide29
DAISY
Source Codehttp://cvlab.epfl.ch/softwareStereo Data and Ground Truthhttp://cvlab.epfl.ch/dataC. Strecha et al. On Benchmarking Camera Calibration and Multi-View Stereo for High Resolution Imagery, CVPR’08Source Code & DataSlide30
Questions
DAISY Source Codehttp://cvlab.epfl.ch/softwareImageshttp://cvlab.epfl.ch/datahttp://cvlab.epfl.ch/~tolaEngin TolaSlide31
DAISY Source Code
http://cvlab.epfl.ch/softwareImageshttp://cvlab.epfl.ch/data
http://cvlab.epfl.ch/~tola
Engin
Tola
QUESTIONS ?Slide32
Parameter Selection
THQ=2
THQ=4
R: 5->30
R: 5->30
R: 5->30
THQ=8
HQ=2
HQ=4
HQ=8
RQ:2->5
RQ:2->5
RQ:2->5Slide33
R: 5->30
R: 5->30
R: 5->30
THQ=2
THQ=4
THQ=8
HQ=2
HQ=4
HQ=8
RQ:2->5
RQ:2->5
RQ:2->5
Parameter Selection
R: 5->30
R: 5->30
R: 5->30
THQ=2
THQ=4
THQ=8
HQ=2
HQ=4
HQ=8
RQ:2->5
RQ:2->5
RQ:2->5
Wide Baseline
Narrow Baseline
Max: 87 %
> 86 %
V:328
R=15, RQ=5, THQ=8, HQ=8
V:52
R=10, RQ=3, THQ=4, HQ=4
V:104
R=10, RQ=3, THQ=4, HQ=8
Max: 78%
V:328
R=15, RQ=5, THQ=8, HQ=8
V:200
R=15, RQ=3, THQ=8, HQ=8
V:104
R=10, RQ=3, THQ=4, HQ=8
> 77%Slide34
Parameter SelectionWide Baseline
Narrow BaselineR: 5->30R: 5->30R: 5->30
TQ=2
TQ=4
TQ=8
Q:1->
5
Q:1->
5
Q:1->
5
H=2
H=4
H=8
R: 5->30
R: 5->30
R: 5->30
TQ=2
TQ=4
TQ=8
Q:1->
5
Q:1->
5
Q:1->
5
H=2
H=4
H=8
0
100