Hao Du 3 Peter Henry 1 Xiaofeng Ren 2 Marvin Cheng 1 Dan B Goldman 4 Steven M Seitz 1 Dieter Fox 12 1 University of Washington Computer Science amp Engineering ID: 514095
Download Presentation The PPT/PDF document "Interactive 3D Modeling of Indoor Enviro..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Interactive 3D Modeling of Indoor Environments with a Consumer Depth Camera
Hao Du3, Peter Henry1, Xiaofeng Ren2, Marvin Cheng1,Dan B Goldman4, Steven M. Seitz1, Dieter Fox1,21University of Washington: Computer Science & Engineering2Intel Labs Seattle (now ISTC at UW)3Google4Adobe SystemsSlide2
Outline
Introduce RGB-D (Red, Green, Blue + Depth) mapping with a Kinect cameraShow how an interactive mapping system improves usability and resultsDemonstrate potential applications to localization, measurement, and interactive visualizationSlide3
The
KinectGuinness record: “fastest selling consumer device”$150, more than 10 million sold as of March 2011OpenNI and Kinect SDK allow for use by developersSlide4
RGB-D Camera
Red Green BlueDepth[PrimeSense.com]Slide5
RGB-D DataSlide6
Interactive MappingSlide7Slide8
RGB-D Frames
Local Alignment (RANSAC)
Global Alignment
Loop Closure
Detection
Update Model
RGB-D Mapping*
RGB-D frames are recorded
The sequence is then processed to obtain the map
*
RGB-D Mapping: Using Depth Cameras for Dense 3D Modeling of Indoor Environments
.
Henry et al. International Symposium on Experimental Robotics (ISER), 2010.Slide9
Frame Matching with RANSAC(
Random Sample Consensus)Visual features (from image) in 3D (from depth)Figure out how the camera moved by matching these features to the previous frameSlide10Slide11Slide12Slide13
Limitations of PreviousRGB-D Mapping
Limited to offline processing after recordingEven a small number of missing alignments result in a uselessly disconnected mapExpert knowledge of the system is required to get good resultsNo feedback on coverage of the environment Slide14
Interactive 3D MappingSlide15
RGB-D Frames
Failure Detection
Local Alignment (RANSAC)
Global Alignment
Loop Closure
Detection
CompletenessAssessment
User Moves and Resumes Mapping
User Verifies and Adds Loop Closures
User Acquires Additional Frames
Visualization
User Controls Viewpoint
Update Model
Suggest Places to Visit
Interactive MappingSlide16
RGB-D Frames
Local Alignment (RANSAC)
Global Alignment
Loop Closure
Detection
Update Model
RGB-D Frames
Failure Detection
Local Alignment (RANSAC)
Global Alignment
Loop Closure
Detection
CompletenessAssessment
User Moves and Resumes Mapping
User Verifies and Adds Loop Closures
User Acquires Additional Frames
Visualization
User Controls Viewpoint
Update Model
Suggest Places to Visit
Interactive
RGB-D MappingSlide17
User InterfaceSlide18Slide19Slide20Slide21
Examining Model CompletenessSlide22
Completeness HintsSlide23
User Study
2 novice and 2 expert users were instructed to map three walls of a room both with and without the interactive systemAll experienced failures without the interactive systemThe proposed system catches failures allowing all users to interactively go back and create accurate mapsSlide24
Interactive System Evaluation
4 users (2 novice, 2 experienced) were instructed to map three walls of a room both with and without the interactive systemAll experienced failures without the interactive system (after a mean count of 40.75 frames)The proposed interactive system caught 26 potential failures allowing all users to interactively go back and create accurate mapsNon-interactiveInteractiveFinal Position Error (m)1.130.08Slide25
Benefits of Interactive 3D Mapping
RobustnessReal-time feedback of mapping status and failuresRewind / Recover / ResumeDelete frames (if a person walks in front of you, etc.)Remove false-positive loop closuresPropose additional loop closuresCompletenessSee the map as it is builtShow unexplored areasSlide26
Large Scale MapSlide27Slide28
Espresso Room ModelSlide29
Applications
LocalizationMeasuring DimensionsInteractive VisualizationSlide30
LocalizationSlide31
Localization
Centimeter level accuracy within an existing mapProvides full 3D translation and rotationRequires no instrumentation of the environmentAn alternative to RFID / 802.11 / GSM / PowerLine based localization in GPS denied environments Slide32
Measuring DimensionsSlide33Slide34
Online real estate sales
Virtual remodelingOnline furniture shoppingVirtual walkthrough for educationRemote telepresenseInteractive VisualizationInteractive gesture controlled visualizationSlide35
Conclusion
Incorporating user interaction into real-time 3D mapping allows non-experts to create accurate maps using RGB-D cameras (such as the Kinect)These maps have applications to localization, measurement, and visualizationMaking this system publicly available will allow for a proliferation of 3D mapping and the potential for new context-aware applications peter@cs.washington.edu