Kurt Konolige Willow Garage Stanford University Patrick Mihelich JD Chen James Bowman Helen Oleynikova Freiburg TORO group Giorgio Grisetti Cyrill Stachness Rainer Kummerle ID: 513824
Download Presentation The PPT/PDF document "Registration for Robotics" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Registration for Robotics
Kurt KonoligeWillow GarageStanford University
Patrick
Mihelich
JD Chen
James Bowman
Helen
Oleynikova
Freiburg TORO
group:
Giorgio
Grisetti
Cyrill
Stachness
Rainer
KummerleSlide2
Outline
ApplicationsOverview of RegistrationFront end and image matchingVisual OdometryPlace recognitionGlobal SBA
ExtensionsSlide3
Applications of Registration
Photo tourism [Snavely, Seitz,
Szeliski
2006]
Slide4
Applications of Registration
Object Modeling [Lovi et al. 2010]
[
Newcombe
and Davison 2010]
Slide5
Applications of Registration
Face Tracking [WATSON: Morency 2003]
Slide6
Applications of Registration
Visual Odometry [Willow Garage] Slide7
Applications of Registration
Visual SLAM [Willow Garage] Slide8
Outline
ApplicationsOverview of RegistrationFront end, image matchingVisual OdometryPlace recognitionGlobal SBA
ExtensionsSlide9
Registration Elements
Incremental
Extract
keypoints
Compute features
Match
against local views
Estimate pose
Optimize
Global
Add
new view to map
Match
against global views
[Place recognition]
Estimate pose
OptimizeSlide10
Mapping Systems
MonoSLAM [Davison 2003]
Parallel Tracking and Mapping
[Klein and Murray 2007]
View-based Maps
[
Konolige
et al. 2009]
Map type
World points
Views and world points
Views [and implicit world points]Slide11
Map Representations
Map type
World points
Views and world points
Views [and implicit world points]
Covariance Matrix over points
p0 p1 p2 …
p0 p1 p2 …
Extended
Kalman
Filter updateSlide12
MonoSLAM
[Davison 2003]Slide13
Map Representations
Map type
World points
Views and world points
Views [and implicit world points]
Information matrix over views
and world points
Nonlinear least squares update
H
D
x
= -g
c0 c1 c2 … p0 p1 p2 …
c0c1c2…
p0 p1 p2 …Slide14
Parallel Tracking and MappingSlide15
Map Representations
Map type
World points
Views and world points
Views [and implicit world points]
Information matrix over views
c0 c1 c2 …
c0c1c2…
Nonlinear least squares update
H
D
x
= -gSlide16
View-based MapsSlide17
Comparison of Mapping Methods
Map type
World points
Useful
in small spaces
- World points for global matching
- World point
Covar
matrix update
Not as accurate [
Strasdat
et al. 2010]
Views and world points
Useful
in small spaces
- World points for global matching
- View and point Info matrix update
Highly accurate
Views [and implicit world points]
Useful
in larger spaces
- View matching
- View Info matrix update
Not as accurate Slide18
Outline
ApplicationsOverview of RegistrationFront end and image matchingVisual OdometryPlace recognition
Global SBA
ExtensionsSlide19
Interest points
FAST [Rosten and Drummond 2006]
Harris
SIFT [Lowe 1999]
SUSAN,
CenSure
, MSER, …
Adaptive threshold
Gridding for spatial diversitySlide20
Feature matching
SIFT / SURF featuresPlanar patchesRandom-tree signatures
[
Calonder
et al. 2009]
…
Tracking with motion model
Windowed brute-force
KD tree nearest-neighbor
930 features
354 matched
183 inliersSlide21
Outlier detection and pose estimation
RANSAC to estimate pose3 points for stereo (3D-3D)3-point PnP for monocular (2D-3D)
Outlier rejection
Refinement
Least-squares
Reprojection
error
Scavenging
930 features
354 matched
183 inliersSlide22
Outline
ApplicationsOverview of RegistrationFront end and image matchingVisual Odometry
Place recognition
Global SBA
ExtensionsSlide23
Visual Odometry
– Sliding Window2-view pose estimates are unreliableTriangulation is narrow-baseline
Multi-view estimates are more accurate
Triangulation is wide-baseline
More points, wider baseline => more accuracy
Keyframes
Many close-together views don’t add much
Too far apart leads to low
inlier
countSliding window of viewsTrack points as long as possible~20 – 40
keyframes
in windowSlide24
Visual Odometry
– Bundle Adjustment
Measurement model
:
For Gaussian distributions ( is covariance)
[Gauss-Newton /
Normal]
Sparseness
of
H
H
20 camera views
5000 features
H is
(20x6
+
5000x3)
2
Efficient - ~10ms
c0 c1 c2 … p0 p1 p2 …
c0c1c2…
p0 p1 p2 …Slide25
Visual
Odometry examples[courtesy Andrew Comport, INRIA]
Outdoor sequence in Versailles
1 m stereo baseline, narrow FOV
~400 m sequence
Average frame distance: 0.6 m
Max frame distance: 1.1 mSlide26
26
Visual Odometry examples
[courtesy Andrew Comport, INRIA]
Indoor Willow Garage sequence
10 cm stereo, wide
FOV
~100
m sequence
Average frame distance:
0.3 mSlide27
Outline
ApplicationsOverview of RegistrationFront end and image matchingVisual Odometry
Place recognition
Global SBA
ExtensionsSlide28
Place
RecognitionK-nearest neighbor feature matching across database of images
KD tree
[Lowe 1999,
Eade
and Drummond 2008,
Williams et al. 2007
]
Bag-of-wordsvisual vocabulary [Stewenius and Nister 2006, Cummins and Newman 2008]
DB
test
DB
testSlide29
Place Recognition: Vocabulary Trees
[Nister and Stewenius CVPR06]
“Bag of words” retrieval
Vocab
tree created offline
For recognition:
Image
keypoints
extracted
Tree encodes approximate NN search
Inverted index of images at leaves
[Image from
Nister
and
Stewenius
CVPR06]
[Cummins and Newman ICRA07
Cullmer
et al. ACRA08
Fraundorfer
et al.
IROS07]
Slide30
Place Recognition: Vocabulary
TreesPerformance on Indoor datasetSlide31
View-based Maps
[Konolige et al. 2009]
Performance on Indoor datasetSlide32
View-based Maps
[Konolige et al. 2009]
Performance on Indoor datasetSlide33
Trajectory synthesisSlide34
Outline
ApplicationsOverview of RegistrationFront end and image matchingVisual Odometry
Place recognition
Global SBA
ExtensionsSlide35
Comparison of Mapping Methods
Map type
World points
Useful
in small spaces
- World points for global matching
- World point
Covar
matrix update
Not as accurate [
Strastdat
2010
]
Views and world points
Useful
in small spaces
- World points for global matching
- View and point Info matrix update
Highly accurate
Views [and implicit world points]
Useful
in larger spaces
- View matching
- View Info matrix update
Not as accurate Slide36
Fast SBA
[Konolige BMVC 2010, Jeong et al. CVPR 2010]
Take advantage of sparse secondary structure of Hessian
Use fast linear solvers
Davis’ CHOLMOD
Block preconditioned conjugate gradientSlide37
Fast SBA in VSLAMSlide38
WG
Projected Texture Stereo
Device
Paint the
scene with texture from a projector
vs. single camera with structured light
Advantages:
Simple projector Standard algorithms Full frame rates (640x480)
Dynamic scenesSlide39
Fast SBA in ReconstructionSlide40
Open-Source in ROS
sba package for fast SBAframe_common
and
posest
for front-end and pose estimation
vocabulary_tree
for place recognition
vslam_system for Visual Odometry and VSLAMSlide41
Solved vs. Unsolved
Front endKeypoint matching
Pose estimation (2D and 3D)
Visual
odometry
Stereo and monocular
Real time
Place recognition
Visual scene static
>10K imagesLarge areasSBAFrame-based Stereo and monocular
Object modeling
Registration of textured objects
Surface reconstruction
Front end
Faster, more robust
descripters
Blur, low texture, lines
Visual
odometry
RGB-D devices (3D + 2D)
Place recognition
Dynamic scenes
Large areas
Manifolds
Object modeling
Untextured
objects
Realtime
Surface reconstruction