kinect Yoni Choukroun Elie Semmel Advisor Yonathan Afflalo Agenda Projects Goals Project Development Kinect Registration Reconstruction Performances Demonstration Projects Goals ID: 583497
Download Presentation The PPT/PDF document "3D full object reconstruction from" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
3D full object reconstruction from kinect
Yoni
Choukroun
Elie
Semmel
Advisor
:
Yonathan
AfflaloSlide2
Agenda
Project’s Goals
Project Development
Kinect
Registration
Reconstruction
Performances
DemonstrationSlide3
Project’s Goals
Understand Kinect working
C++ development
Rigid Iterative closest point implementation
Unknown libraries integration on Windows Platform
Deal with Reconstruction problemsSlide4
Development of the project
Papers read
Kinect depth maps acquisition
Simple rigid ICP implementation
Body reconstructionSlide5
Papers
The
model Project
3D SELF-PORTRAITS
Hao
Li, Etienne
Vouga
, Anton
Gudym
, Jonathan T. Barron,
Linjie
Luo,
Gleb
Gusev
A
Method for Registration of 3-D Shapes
Besl
, Paul J.; N.D. McKay (1992).
Efficient Variants of the ICP Algorithm
Szymon
Rusinkiewicz
, Marc
LevoySlide6
The model Project
3D SELF-PORTRAITS
Hao
Li, Etienne
Vouga
, Anton
Gudym
, Jonathan T. Barron,
Linjie
Luo,
Gleb Gusev
Full body reconstruction in 3-D with multiples scan frames of the Kinect
Pipelined reconstruction
First pipeline station:
scanning and fusion
Rigid ICP of all the scans to get the entire body
Poisson reconstruction for a better quality Slide7Slide8
KinectSlide9
Kinect
Some facts:
The Kinect moves from [+27 ; -27 ]
Our Kinect motor is controlled by an OPENNI 1.5 code.
The Kinect provides
a
480X640
pixels
resolution depth map .
The depth map returned by the Kinect is of a poor quality.Slide10
Kinect
Intrinsic
Matrix Calibration
application
requires maximum
accuracy rigorous
calibration.
different
users with different Kinect
average
calibration with an average intrinsic matrix calibration.
Pros / Cons:
More precise calibration will improve the performances and the final result but
it asks
from the user a tiresome calibration step
.Slide11
Registration Problem
Given:
Two shapes
P
and
Q
which partially overlap.
Goal
:
Using only
rigid transforms
,
register
Q
against
P
by minimizing the squared distance between them
.Slide12
Simple rigid ICP theory
Mathematical preliminaries
Calculation tools
Algorithm
Performances
Optimization
A Method for Registration of 3-D Shapes
Besl
, Paul J.; N.D. McKay (1992).Slide13
ICP Problematic
Given set of corresponding points it is easy to find the transformation which stitch the two images together.
Here, we do not have any previous prior about the shapes (maybe just good initial conditions enough).
We need then to find the best correlation between points:
ITERATIVE CLOSEST POINT!Slide14
Iteratively find the pairs of closest points
Solve:
Correspondence Space
P
QSlide15
A Method for Registration of 3-D Shapes
Besl
, Paul J.; N.D. McKay (1992).
Sampling (~1:30) the cloud (sparse matrix array of coordinates)
Slide16
A Method for Registration of 3-D Shapes
Besl
, Paul J.; N.D. McKay (1992).
Euclidean Metric used to find the Closest pointSlide17
Transformation
Calculus
:
Find
rotation and translation:
Apply
registration
until
threshold
is
reached
.Slide18
ICP Algorithm
Init the error to ∞
Compute correspondences
Compute alignment
Apply alignment
Update error
If error > threshold
Y =
CP(P,Q)
(
Rot,Trans,d
)
Q`= Rot(Q)+TransSlide19
Efficient Variants of the ICP
Algorithm
Szymon
Rusinkiewicz
, Marc
Levoy
Rigid ICP variant performances
Selection of points
Matching points
Weighting pairs
Rejecting pairsSlide20
Selection of points
Simple variants give good enough results: we implemented both random and uniform samplingSlide21
Matching points
Simple implementation for good enough results (suppose to be small Kinect angles): we chose the Closest Point for pair correspondingSlide22
Reconstruction
Methods
:
Perform ICP between a frame and the result of the all precedent frames bond together
bad results because of noise accumulation.
Improvement: Perform ICP between pair of frames and apply transitivity to get the registrations to the first frame:
Fn
, the n-
th
frame
Tr
, the transformation between Fn+1 and
Fn
Rn and
Tn
, the corresponding Rotation and Translation.Slide23
Project Algorithm
Kinect
Depth maps acquisition
OpenNi
interface
OpenCV
conversion
Filtering
Registration
Displaying
After
#scans
registrations
Body reconstructionSlide24
ICP Performances
Bad performances for big moves
Not too big rotations required
Number of iterations: 1-3 for small registrations to about 10 for big moves (in general converges to wrong local minimum )
Noisy depth map
Bad
Kinect
resolution affects image’s acquisition: holes in the target shape
.
For complexity purpose, search the closest point only in small radius near the point: finding closest point complexity Slide25
Performances Good fusion result vs time
Tradeoff: Fusion result Vs Execution time
Number of sampling between 8000 and 12000 (static memory allocation)
Time for ICP running :
1
~ 3 min (
depended on number of samples)
Scanning and reconstructing time less than one minute.
Total time between
1.2
mn
to 3mnSlide26
Fusion Results
No pre/post processing used (noise filtering etc.)
Conversion to
MeshLab
and
Matlab
files for displaying (and for post processing like texturing…)
Matlab
Implementation for triangulation.(Not used because of poor quality: points cloud looks better) Slide27
Demonstration
https
://
www.dropbox.com/s/ep2r8ajsodmhw7v/My%20Movie.mp4?dl=0Slide28
References
" 3D SELF-PORTRAITS "
Hao
Li, Etienne
Vouga
, Anton
Gudym
, Jonathan T.
Barron
, Linjie Luo, Gleb Gusev ACM Transactions on Graphics, Proceedings of the 6th ACM SIGGRAPH Conference and Exhibition in Asia 2013, 11/2013 – SIGGRAPH ASIA 2013
"A Method for Registration of 3-D Shapes"
Besl
, Paul J.; N.D. McKay (1992).
IEEE
Trans
. on Pattern
Analysis
and Machine Intelligence (Los
Alamitos
, CA, USA: IEEE Computer Society)
"Efficient Variants of the ICP Algorithm"
Szymon
Rusinkiewicz
, Marc
Levoy
Stanford UniversitySlide29