Miguel Bordallo López Jari Hannuksela Olli Silvén Machine Vision Group University of Oulu miguelbordalloeeoulufi 3 Hypothesis The latency and computing bottlenecks present in current mobile designs can be removed via software and hardware developments ID: 430738
Download Presentation The PPT/PDF document "Designing for image-based 3D interactivi..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Designing for image-based 3D interactivity
Miguel Bordallo López, Jari Hannuksela, Olli
Silvén
,
Machine Vision
Group,
University of Oulu {miguel.bordallo@ee.oulu.fi}
3.
Hypothesis
The latency and computing bottlenecks present in current mobile designs can be removed via software and hardware developments.
Decreasing sensor/camera start-up latencies improves the usability
Requires careful balancing with energy efficiency demands
Fusing the sensor data improves the user interactivity and performance of camera-based applicationsIncreases robustness in bad light conditions
2.
Objective
of the research
4
. Development of interactive applications
Mobile panorama builder
Selected frames
Final panorama
Mobile
handheld
scanner
Motion
based
image-browser
Face-tracking
virtual
3D
Display
”
Support
”
frames
Frame
of
interest
Video-based
point
&
find
5. Algorithm
implementation
and
evaluation
Face
and
eye
detection and tracking
Image
blending
Quality
assesment
Image
correction
FFT-based registration
Feature-based registration
Block matching and motion estimation
Moving object detection
7.
Development
of new interaction methods
Interactive image capture
Head movement triggers
5. Mobile
platforms
evaluation and analysis
Performance
analysis
Energy efficiencyLatency analysisSensor data fusionSmall footprint processorsDSP accelerationMobile GP-GPU
To gain understanding on how to build the future 3D mobile platforms in order to satisfy their interactivity requirements.
To provide insight into the computing needs and characteristics of the future camera-based applications.
1. Multimodal 3D mobile platforms and UIs
F
uture multimodal 3D mobile platforms will integrate highly interactive applications and UIs.
Mobile devices integrate touch screens, cameras and several sensors
The small size of handheld devices and their multiple cameras and sensors are under-exploited assets
The problems:
The usability of 3D user interfaces in mobile devices critically rests on its latency Interactive camera-based applications require computationally costly algorithms Solutions need to be energy-efficient Sensors required to be always-on The use of power hungry application processors compromises responsivenessThe possible solutions: Parallel distribution of tasks among the available processing resources. Improving reactiveness by reducing the number of operations on the main application processor needs to execute Deriving the most expensive tasks to accelerators or specific processors
Using cameras and as an input modality enables recognizing the 3D context in real time Computer vision can use camera data in user interfaces to analyse the context User intentions automatically detected