2D Warping and Block Matching Shinjini Kundu Anand Kamat Tarcar EE398A Final Project 1 EE398A Compression of Light Fields using 2D Warping and Block Matching Outline Motivation and ID: 566490
Download Presentation The PPT/PDF document "Light Field Compression Using" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Light Field Compression Using 2-D Warping and Block Matching
Shinjini KunduAnand Kamat TarcarEE398A Final Project
1
EE398A - Compression of Light Fields using 2-D Warping and Block MatchingSlide2
OutlineMotivation and
GoalsOverview of Our MethodResults and AnalysisSummaryFuture WorkReferences2
EE398A - Compression of Light Fields using 2-D Warping and Block MatchingSlide3
MotivationLight field images are used in computer graphics to compute new views of a scene without need for scene geometry model
1. Need to compress large set of imagesExploit inter-view coherence to achieve compression.1. M. Levoy and P. Hanrahan, “Light field rendering,” in
Computer Graphics (Proceedings SIGGRAPH 96), August 1996, pp. 31-42.
3EE398A - Compression of Light Fields using 2-D Warping and Block MatchingSlide4
Light Fields
Represents a 3D scene or object from all viewing positions and directions2D array of 2D imagesDifficult to AcquireVery LargePerfect representation requires images of the order of the resolution Slide5
Light Field Views
EE398A - Compression of Light Fields using 2-D Warping and Block Matching5Slide6
Light Field Data Set
8.4 MB uncompressed data sets
http://lightfield.stanford.edu/aperture.swf?lightfield=data/lego_lf/preview.zip&zoom=1
Credit: Andrew Adams
6
EE398A - Compression of Light Fields using 2-D Warping and Block MatchingSlide7
Related Work
Intra-frame codingVector quantization, DCT coding, transform coding yield compression ratios of less than 30:1Inter-frame coding (compression in the hundreds, thousands)Disparity compensation3D geometry modelsBlockwiseCompression ideal: maximally use coherence between two images
7EE398A - Compression of Light Fields using 2-D Warping and Block MatchingSlide8
Our Method: 2-D Warping
Each consecutive view is a projection of the previous view due to constant predictable movement of cameraFind this relation between the views by obtaining projection matrix for each pair of viewsPredict the view and encode the residual8
EE398A - Compression of Light Fields using 2-D Warping and Block MatchingSlide9
Our Encoding Scheme
9EE398A - Compression of Light Fields using 2-D Warping and Block MatchingReconstructed Previous View
Previous Frame 2-D Warped
Lagrangian
Cost Function
Cost=R1+
λ
D1
Cost=R2+
λ
D2
2D Warping Algorithm
2-D DCT for the Residual
Residual and MV
?
Input View
--
Use for ReconstructionSlide10
Notes
DCT used on 8x8 blocks to encode residualLaplacian distribution assumed for motion vectorsProjection matrix was encoded by normalizing values with respect to 10, and assuming Laplacian distribution of bitrate. The min and max values are encoded separately using binary encoding. H = -0.578 0.005
-0.720 -0.003 -
0.572 0.007
0.000
0.000
-
0.582
EE398A - Compression of Light Fields using 2-D Warping and Block Matching
10Slide11
1. Feature match by correlation
2. Projective matrix computedLagrangian Mode Decision using two references3. Clipped edges are interpolated using motion compensation
11EE398A - Compression of Light Fields using 2-D Warping and Block MatchingSlide12
Getting a predicted projection:Step 1: Feature matching by Correlation
Features detected by Harris corner detection algorithm, and matching points identified by maximum correlation
12
EE398A - Compression of Light Fields using 2-D Warping and Block MatchingSlide13
Computing the Homography Matrix
A homography is an invertible transformation from the real projective plane to the projective plane that maps straight lines to straight lines EE398A - Compression of Light Fields using 2-D Warping and Block Matching
13Slide14
Results for 2-D Projection WarpingEE398A - Compression of Light Fields using 2-D Warping and Block Matching
14Slide15
Results for 2-D Projective WarpingEE398A - Compression of Light Fields using 2-D Warping and Block Matching
15Slide16
Results for 2D Projective WarpingEE398A - Compression of Light Fields using 2-D Warping and Block Matching
16Slide17
Compression RatiosEE398A - Compression of Light Fields using 2-D Warping and Block Matching
17Slide18
ConclusionAdvantages: decreased coding complexity, and increased rate/PSNR as well as compression
Experimental results demonstrate improved coding efficiency with our 2D warp method when compared with MVC.18EE398A - Compression of Light Fields using 2-D Warping and Block MatchingSlide19
Future Work PossibleOptimize the code to give better PSNR values and check performance by introducing extra modes like copy mode
Explore other methods of using inter-view redundancy in detail like disparity compensation at sub-pel accuracyRun for larger data sets and optimize complexity of the algorithmEE398A - Compression of Light Fields using 2-D Warping and Block Matching19Slide20
SummaryLight fields represent a 3D scene using sequence of 2-D images
Large amounts of dataCan use redundancy between images using 2-D warping with motion compensated block matchingResults in a sleek method for compressionPerformance wise..EE398A - Compression of Light Fields using 2-D Warping and Block Matching
20Slide21
AcknowledgementProf.
Girod for pointing us in the right directionMina Makar for his helpChuo-Ling Chang for DAPBT codeHuizhong Chen and Derek Pang for their helpProf. Peter Kovesi for open source matlab function libraryProf. Levoy’s
group and Andrew Adams for access to light field images21
EE398A - Compression of Light Fields using 2-D Warping and Block MatchingSlide22
Questions?
EE398A - Compression of Light Fields using 2-D Warping and Block Matching22Slide23
Other ProjectsUse Motion Compensation with Directional Transforms
Result: Gain in PSNR due to directionality is approximately 0.1dB at high Quantization; almost nil increase seen at low quantizationSo, We adapted the direction of out project to study a new approach of compression presented next.EE398A - Compression of Light Fields using 2-D Warping and Block Matching23Slide24
Results with Motion Compensation and DAPBT for Crystal light field
24EE398A - Compression of Light Fields using 2-D Warping and Block MatchingSlide25
Results with Motion Compensation and DAPBT for Lego light field
EE398A - Compression of Light Fields using 2-D Warping and Block Matching25Slide26
This is how blocking is done and direction selection happens!IAP(DAT)+IRP(DCT) for QP=44, Crystal Light Field
EE398A - Compression of Light Fields using 2-D Warping and Block Matching26Slide27
For Lego light field IAP(DAT)+IRP(DCT) for QP=44, Crystal Light Field
EE398A - Compression of Light Fields using 2-D Warping and Block Matching27