By Georg Petschnigg Maneesh Agrawala Hugues Hoppe Richard Szeliski Michael Cohen Kentaro Toyama Microsoft Corporation Presented by Yael Amsterdamer Advanced Topics in Visual Computing Spring 2012 ID: 285025
Download Presentation The PPT/PDF document "Digital Photography with Flash and No-Fl..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Digital Photography with Flash and No-Flash Image Pairs
By: Georg Petschnigg Maneesh Agrawala Hugues Hoppe Richard Szeliski Michael Cohen Kentaro Toyama, Microsoft CorporationPresented by: Yael AmsterdamerAdvanced Topics in Visual Computing, Spring 2012Slide2
The Dilemma: to Flash or not to flash?
Introduction
2
Digital Photography with Flash and No-Flash Image Pairs
Natural lighting
Low signal-to-noise ratio (SNR)
Loss of details
Longer exposure – motion blur
Harsh, unnatural lighting
High SNR
More details
May cause unwanted artifacts (red eye, shadows, specularities)Slide3
Why not both?
The idea: use the good features of each photo to create a better image Introduction3Digital Photography with Flash and No-Flash Image PairsSlide4
Other works on combining images
Introduction4Digital Photography with Flash and No-Flash Image Pairs
Different exposures
Panoramic view
Relighting
Current work
handles 2 images with specific characteristics,
Envisioned for camera firmware Slide5
Different Applications
DenoisingDetail transferSide requirement: handling flash induced shadows and specularitiesWhite BalancingContinuous Flash AdjustmentRed Eye RemovalIntroduction
5Digital Photography with Flash and No-Flash Image PairsSlide6
Camera Adjustment to light
A certain amount of light is needed for a photoTaking pictures of a dark sceneIncreasing exposure timeOpening the apertureIncreasing ISO
(sensor sensitivity)The problem – high ISO noise affects high-frequency details
Background
6
Digital Photography with Flash and No-Flash Image PairsSlide7
The Input
F – the photo with flashA – the no-flash (ambient) photoComposed of 3 RGB intensity values for each pixelAp
– one pixel in ASame camera position, focus, apertureOtherwise, registration is requiredAdjusted ISO and exposure time
Undergo non-linear tone-mapping operations
Background
7
Digital Photography with Flash and No-Flash Image PairsSlide8
1st Application: Denoising
A classic problem in image processingThe target: analytically reduce SNR One type of solution: smoothing filtersBilateral Filter [Tomasi and Manduchi 1998]Fast, non-iterativeAveraging close-by pixelsLow-pass filter, combined with
Edge-stopping function
Denoising8
Digital Photography with Flash and No-Flash Image PairsSlide9
Bilateral filter
However, results still have noise or blur (or both)Denoising9
Digital Photography with Flash and No-Flash Image Pairs
ambient
flash
Bilateral filterSlide10
Improvement: Joint Bilateral Filter
In the flash image there are much more detailsWhy not use F to find edges?Denoising10
Digital Photography with Flash and No-Flash Image Pairs
Bilateral filter
Joint Bilateral filter
The differenceSlide11
Not all edges are real …
May cause over- or under-blur in joint bilateral filterWe need to eliminate their effectDenoising
11Digital Photography with Flash and No-Flash Image PairsSlide12
Detecting shadows
Observation: the pixels in the flash shadow should be similar to the ambient imageNot identical:NoiseInter-reflected flashCompute a shadow mask
Take pixel p if is manually adjusted
Mask is smoothed and dilated
Denoising
12
Digital Photography with Flash and No-Flash Image Pairs
Pixels for which we will not use FSlide13
Detecting specularities
Take pixels where sensor input is close to maximum (very bright)Over fixed threshold Create a specularity maskAlso smoothed and dilatedM – the combination of shadow and specularity masks
Where Mp=
1, we use ABase.
For other pixels we use
A
NR
.
Denoising
13
Digital Photography with Flash and No-Flash Image PairsSlide14
There is so much denoising can do
It cannot add details missing in the ambient imageExist in flash image because of high SNRWe use a quotient image:Multiply with A
NR to add the detailsMasked in the same way
Detail Transfer
14
Digital Photography with Flash and No-Flash Image Pairs
Bilateral filtered
Reduces the effect of noise in F Slide15
Unified process
Denoising + detail transfer + masking shadows and specularitiesDetail Transfer15
Digital Photography with Flash and No-Flash Image PairsSlide16
More examples
Detail Transfer16Digital Photography with Flash and No-Flash Image PairsSlide17
The opposite of preserving the atmosphere
The goal: simulate white light in the sceneWhat color is the light in the scene?Flash image can help us:The difference between F and A is only the flash lightAssume it is proportional to the
surface albedoInherent property of the surfaceAssume the surface reflects light in the same color (will not work for plastic, glass…)
White Balancing
17
Digital Photography with Flash and No-Flash Image PairsSlide18
Computing the ambient light
We cancel the color of the surfaceThe average c of Cp(col) for pixels with certain properties is the ambient light colorThen we scale A to cancel the ambient light
White Balancing
18Digital Photography with Flash and No-Flash Image PairsSlide19
Example
White Balancing19Digital Photography with Flash and No-Flash Image Pairs
Estimated ambient light
Computed color
Ambient original
White-balancedSlide20
Combining flash and no-flash images
Providing complementary details about the sceneUse for denoising and detail transferUse for white balancingFuture workMore robustnessAutomatization of configuration
Summary
20
Digital Photography with Flash and No-Flash Image PairsSlide21
Thank you!