Supervised by Margrit Gelautz and Carsten Rother Matting and compositing Matting and compositing Outline Talk Outline Introduction amp previous approaches Our matting model Evaluation strategy ID: 1006375
Download Presentation The PPT/PDF document "Interactive Matting Christoph Rhemann" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
1. Interactive MattingChristoph RhemannSupervised by:Margrit Gelautz and Carsten Rother
2. Matting and compositing
3. Matting and compositing
4. OutlineTalk Outline: Introduction & previous approaches Our matting model Evaluation strategy
5. =+ Cr,g,b = α Fr,g,b + (1 - α) Br,g,b●●●●Inverse process of compositing:Determine: F, B, αGiven: C Matting is ill posed
6. =+Underconstrained problem:7 Unknowns in only 3 Equations●● Cr = α Fr + (1 - α) Br Cg = α Fg + (1 - α) Bg Cb = α Fb + (1 - α) Bb Cr,g,b = α Fr,g,b + (1 - α) Br,g,b●●Matting is ill posed
7. TrimapScribblesBackgroundBackgroundUnknownForegroundUnknownForegroundUser interaction
8. Previous approaches C = α F + (1 – α) B●●Recall compositing equation:
9. Previous approaches C = α F + (1 – α) B●●Recall compositing equation:Closed Form Matting [Levin et al. 06]RBG
10. Previous approaches C = α F + (1 – α) B●●Recall compositing equation:RBGClosed Form Matting [Levin et al. 06]Assumption: F and B colors in a local window lie on color line
11. Previous approaches C = α F + (1 – α) B●●Recall compositing equation:RBGClosed Form Matting [Levin et al. 06]Assumption: F and B colors in a local window lie on color lineAnalytically eliminate F,B.Alpha can be solved in closed form
12. Result of [Levin et al 06]True SolutionInput image + TrimapResult of Closed Form Matting [Levin et al. 06]:Result imperfect: Hairs cut offProblem: Cost function has large solution spacePrevious approaches
13. What are the reasons for pixels to be transparent?Segmentation – based mattingDefocus Blur
14. LensCamera sensorPoint spread functionPoint Spread FunctionFocal planeLens’ aperture Lens and defocusSlides by Anat Levin
15. LensObjectCamera sensorPoint spread functionLens’ aperture Focal planeSlides by Anat LevinLens and defocusPoint Spread Function
16. What are the reasons for pixels to be transparent?Segmentation – based mattingDefocus BlurMotion BlurPSF forMotion Blur
17. What are the reasons for pixels to be transparent?Segmentation – based mattingDefocus BlurMotion BlurDiscretization
18. What are the reasons for pixels to be transparent? Observation: Apart from translucency mixed pixels are caused by camera’s Point Spread Function (PSF)Segmentation – based mattingDefocus BlurMotion BlurDiscretizationTranslucency
19. Basic idea:Model alpha as convolution of a binary segmentation with PSFApproach taken [Rhemann et al. 08]:Use this model as prior in framework of [Levin et al. 06]Model for alphaBinary segmentationPSFObserved alphaInput image + Trimap
20. Matting processInitial alpha using [Wang et al. ´07](Result is imperfect)Initialize PSF/deblur alphaDeblured (sparse) alphaBinarized (sparse) alpha using gradient preserving MRF priorIterate a few timesInput image
21. Matting processBinarized (sparse) alpha using gradient preserving MRF priorSegmentation priorFinal alphaGround truth
22. Result for [Levin et al. ’06]Input imageInput image + trimapComparison
23. Result of [Wang et al. ’07]Input imageInput image + trimapComparison
24. Input imageInput image + trimapResult of [Rhemann et al. ’08]Comparison
25. Input image + trimap[Levin et al. ’06] [Wang et al. ’07][Rhemann et al. ’08]Ground truth alpha[Levin et al. ’07]Comparison – Close up
26. Evaluation of matting algorithmsHow to compare performance of algorithms?Showing some qualitative resultsORQuantitative evaluation using reference solutions
27. Evaluation of matting algorithmsKey Factors for a good quantitative evaluationGround truth datasetOnline evaluationPerceptual error functions
28. 35 natural imagesHigh resolutionHigh qualityTriangulation Matting [Smith, Blinn 96] - Photograph object against 2 different backgrounds True solution to matting problem Input imageGround truthZoom inGround truth dataset
29. Data and evaluation scripts onlineAdvantages:Investigate resultsUpload novel results www.alphamatting.comOnline evaluation
30. Motivation: Simple metrics not always correlated with visual qualityInput imageZoom inResult 1SAD: 1215 Result 2SAD: 806 Perceptually motivated error functions
31. Develop error measures for two properties:Connectivity of foreground objectGradient of the alpha mattePerceptually motivated error functionsInput imageZoom inResult 1SAD: 312Result 2SAD: 83
32. User Study:Goal: Infer visual quality of image compositionsTask: Rank to according to how realistic they appearPerceptually motivated error functionsGradient artifactsConnectivity artifacts
33. Correlation of error measures to average user rankingPerceptually motivated error functions
34. Model for alpha overcomes ambiguitiesModel-based algorithm: Performs better than competitorsPerceptual motivated evaluationMessage to you: Evaluation of your algorithm is importantUse ground truth data to make quantitative comparisonsUse a large datasetUse a training / test splitConclusions
35.
36. Previous approaches C = α F + (1 – α) B●●Recall compositing equation:RBGModel of FModel of BObserved colorData driven approaches (e.g. [Wang et al. 07])Model color distribution of F and B (from the user defined trimap)Observed color more likely under F or B model?Use likelihood in framework of [Levin et al 06]
37. Result of data driven approaches [Wang et al. 07]:Hair is better capturedMany artifacts in the backgroundPrevious approachesResult of [Levin et al 06]True SolutionInput image + TrimapResult of [Wang et al 07]