Digital Photography with Flash and No-Flash Image Pairs - Presentation

149K - views

Digital Photography with Flash and No-Flash Image Pairs

By: . Georg Petschnigg Maneesh Agrawala Hugues Hoppe Richard Szeliski. Michael Cohen Kentaro Toyama,. Microsoft Corporation. Presented by. : Yael Amsterdamer. Advanced Topics in Visual Computing, Spring 2012.

Embed :
Presentation Download Link

Download Presentation - The PPT/PDF document "Digital Photography with Flash and No-Fl..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

Digital Photography with Flash and No-Flash Image Pairs






Presentation on theme: "Digital Photography with Flash and No-Flash Image Pairs"— Presentation transcript:

Slide1

Digital Photography with Flash and No-Flash Image Pairs

By: Georg Petschnigg Maneesh Agrawala Hugues Hoppe Richard Szeliski Michael Cohen Kentaro Toyama, Microsoft CorporationPresented by: Yael AmsterdamerAdvanced Topics in Visual Computing, Spring 2012Slide2

The Dilemma: to Flash or not to flash?

Introduction

2

Digital Photography with Flash and No-Flash Image Pairs

Natural lighting

Low signal-to-noise ratio (SNR)

Loss of details

Longer exposure – motion blur

Harsh, unnatural lighting

High SNR

More details

May cause unwanted artifacts (red eye, shadows, specularities)Slide3

Why not both?

The idea: use the good features of each photo to create a better image Introduction3Digital Photography with Flash and No-Flash Image PairsSlide4

Other works on combining images

Introduction4Digital Photography with Flash and No-Flash Image Pairs

Different exposures

Panoramic view

Relighting

Current work

handles 2 images with specific characteristics,

Envisioned for camera firmware Slide5

Different Applications

DenoisingDetail transferSide requirement: handling flash induced shadows and specularitiesWhite BalancingContinuous Flash AdjustmentRed Eye RemovalIntroduction

5Digital Photography with Flash and No-Flash Image PairsSlide6

Camera Adjustment to light

A certain amount of light is needed for a photoTaking pictures of a dark sceneIncreasing exposure timeOpening the apertureIncreasing ISO

(sensor sensitivity)The problem – high ISO noise affects high-frequency details

Background

6

Digital Photography with Flash and No-Flash Image PairsSlide7

The Input

F – the photo with flashA – the no-flash (ambient) photoComposed of 3 RGB intensity values for each pixelAp

– one pixel in ASame camera position, focus, apertureOtherwise, registration is requiredAdjusted ISO and exposure time

Undergo non-linear tone-mapping operations

Background

7

Digital Photography with Flash and No-Flash Image PairsSlide8

1st Application: Denoising

A classic problem in image processingThe target: analytically reduce SNR One type of solution: smoothing filtersBilateral Filter [Tomasi and Manduchi 1998]Fast, non-iterativeAveraging close-by pixelsLow-pass filter, combined with

Edge-stopping function

Denoising8

Digital Photography with Flash and No-Flash Image PairsSlide9

Bilateral filter

However, results still have noise or blur (or both)Denoising9

Digital Photography with Flash and No-Flash Image Pairs

ambient

flash

Bilateral filterSlide10

Improvement: Joint Bilateral Filter

In the flash image there are much more detailsWhy not use F to find edges?Denoising10

Digital Photography with Flash and No-Flash Image Pairs

Bilateral filter

Joint Bilateral filter

The differenceSlide11

Not all edges are real …

May cause over- or under-blur in joint bilateral filterWe need to eliminate their effectDenoising

11Digital Photography with Flash and No-Flash Image PairsSlide12

Detecting shadows

Observation: the pixels in the flash shadow should be similar to the ambient imageNot identical:NoiseInter-reflected flashCompute a shadow mask

Take pixel p if is manually adjusted

Mask is smoothed and dilated

Denoising

12

Digital Photography with Flash and No-Flash Image Pairs

Pixels for which we will not use FSlide13

Detecting specularities

Take pixels where sensor input is close to maximum (very bright)Over fixed threshold Create a specularity maskAlso smoothed and dilatedM – the combination of shadow and specularity masks

Where Mp=

1, we use ABase.

For other pixels we use

A

NR

.

Denoising

13

Digital Photography with Flash and No-Flash Image PairsSlide14

There is so much denoising can do

It cannot add details missing in the ambient imageExist in flash image because of high SNRWe use a quotient image:Multiply with A

NR to add the detailsMasked in the same way

Detail Transfer

14

Digital Photography with Flash and No-Flash Image Pairs

Bilateral filtered

Reduces the effect of noise in F Slide15

Unified process

Denoising + detail transfer + masking shadows and specularitiesDetail Transfer15

Digital Photography with Flash and No-Flash Image PairsSlide16

More examples

Detail Transfer16Digital Photography with Flash and No-Flash Image PairsSlide17

The opposite of preserving the atmosphere

The goal: simulate white light in the sceneWhat color is the light in the scene?Flash image can help us:The difference between F and A is only the flash lightAssume it is proportional to the

surface albedoInherent property of the surfaceAssume the surface reflects light in the same color (will not work for plastic, glass…)

White Balancing

17

Digital Photography with Flash and No-Flash Image PairsSlide18

Computing the ambient light

We cancel the color of the surfaceThe average c of Cp(col) for pixels with certain properties is the ambient light colorThen we scale A to cancel the ambient light

White Balancing

18Digital Photography with Flash and No-Flash Image PairsSlide19

Example

White Balancing19Digital Photography with Flash and No-Flash Image Pairs

Estimated ambient light

Computed color

Ambient original

White-balancedSlide20

Combining flash and no-flash images

Providing complementary details about the sceneUse for denoising and detail transferUse for white balancingFuture workMore robustnessAutomatization of configuration

Summary

20

Digital Photography with Flash and No-Flash Image PairsSlide21

Thank you!