Ioannis Gkioulekas 1 Shuang Zhao 2 Kavita Bala 2 Todd Zickler 1 Anat Levin 3 1 Harvard 3 Weizmann 2 Cornell 1 Most materials are translucent 2 jewelry skin architecture Photo credit ID: 296653
Download Presentation The PPT/PDF document "Inverse Volume Rendering with Material D..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Inverse Volume Rendering with Material Dictionaries
Ioannis
Gkioulekas1
Shuang Zhao2
Kavita Bala2
Todd Zickler1
Anat Levin3
1Harvard
3Weizmann
2Cornell
1Slide2
Most materials are translucent
2
jewelry
skin
architecture
Photo credit:
Bei
Xiao, Ted
Adelson
foodSlide3
We know how to render them
3
Monte-Carlo rendering
material parameters
Veach
1997,
Dutré
et al. 2006
?
rendered imageSlide4
We show how to measure them
4
inverse rendering
material parameters
rendered image
captured photographSlide5
Our contributions
5
material
1.
exact
inverse volume rendering
with
arbitrary
phase functions!
2. validation with
calibration materials
known parameters
3. database of
broad range
of materials
thin
thick
non-
dilutable
solidsSlide6
material sample
Why is inverse rendering so hard?
6
radiative
transfer
r
andom walk of photons inside volume
volume light transport
has
very complex dependence
material parameters
thin
thick
non-
dilutable
solidsSlide7
thin
thick
non-
dilutable
solids
Light transport approximations
7
Previous approach:
single-scattering
r
andom walk of photons inside volume
single-bounce random walk
Narasimhan
et al.
2006
Slide8
Light transport approximations
8
Previous approach:
diffusion
Jensen
et al.
2001
Papas
et al.
2013
…
…
…
…
isotropic distribution of photons
parameter ambiguity
≈
≠
material 1
material 2
r
andom walk of photons inside volume
thin
thick
non-
dilutable
solids
Slide9
Inverse rendering without approximations
9
r
andom walk of photons inside volume
exact inversion of random walk
thin
thick
non-
dilutable
solids
Slide10
Our approach
10
appearance matching
ii. operator-theoretic analysis
i. material representation
iii. stochastic optimizationSlide11
Background
11
phase function
p(
θ)
scattering coefficient
σ
s
extinction coefficient
σ
t
θ
m = (
σ
t
σ
s
p
(θ)
)
random walk of photons inside mediumSlide12
Papas
et al. 2013
Phase function parameterization
12
not general enough
Henyey
-Greenstein lobes
Chen et
al.
2006
Donner et
al.
2008
Fuchs et
al.
2007
Goesele
et
al.
2004
Gu
et
al.
2008
Hawkins et
al.
2005
Holroyd et
al.
2011
McCormick et
al.
1981
Pine et
al.
1990
Prahl
et
al.
1993
Wang et
al.
2008
Gkioulekas
et
al.
2013
Narasimhan
et al.
2006
Jensen et
al.
2001
Previous approach:
single-parameter familiesSlide13
m
=
Σ
q
π
q
mq
p = Σq
πq p
q
D = {m1, m
2
, …,
m
Q
}
Dictionary parameterization
13
tent phase functions
D = {p
1
, p
2
, …,
p
Q
}
p
1
p
2
p
3
p
4
p
5
p
6
p
7
p
8
p
9
p
10
p
11
dictionary of
arbitrary
p
similarly for
σ
t
and
σ
s
π
1
π
2
π
3
π
4
π
5
π
6
π
7
π
8
π
9
π
10
π
11
D
phase functions
phase functions
materials
materials
σ
t
=
Σ
q
π
q
σ
t,q
σ
s
=
Σ
q
π
q
σ
s
,qSlide14
Our approach
14
appearance matching
ii. operator-theoretic analysis
i. material representation
iii. stochastic optimization
m
=
Σ
q
πq m
qSlide15
Operator-theoretic analysis
15
m = (
σ
t
σ
s
p
(θ)
)
τ
τ
τ
τ
random walk of photons inside medium
discretized random walk paths
propagation step
τSlide16
total
radiance
K(π
) = Σq πq
Kq
Operator-theoretic analysis
16
m = (
σ
t
σ
s
p
(θ)
)
discretized random walk paths
propagation step
τ
L(x,
θ
)
radiance at
all
medium points and directions
L
n+1
(x,
θ
)
=
L
n
(x,
θ
)
K
rendering
operator R
= (I - K)
-1
L
input
L
=
Σ
n
L
n
L(x,
θ
)
=
R
L
input
(x,
θ
)
radiance after n steps
radiance after n+1 steps
R(
π
)=
(I -
Σ
q
π
q
K
q
)
-1
dictionary representation:
m =
Σ
q
π
q
m
qSlide17
Our approach
17
appearance matching
ii. operator-theoretic analysis
i. material representation
iii. stochastic optimization
m
=
Σ
q
πq m
q
R(
π
)=
(I -
Σ
q
π
q
K
q
)
-1Slide18
Stochastic optimization
18
appearance matching
analytic operator expression for gradient!
=
R(
π
)
render(π
)single-
stepq
·
·
render(
π
)
R(
π
)
K
q
gradient descent optimization for inverse rendering
min ǁ photo - render(
π
) ǁ
2
πSlide19
Stochastic optimization
19
exact gradient descent
for k = 1, …, N,
π
k
= πk - 1 -
ak
end
N = a few hundreds
several CPU hours
*
=
intractable
exactSlide20
Stochastic optimization
20
Monte-Carlo rendering to compute
10
2
samples
noisy + fast
10
4
samples
10
6
samples
accurate + slowSlide21
Stochastic optimization
21
exact gradient descent
for k = 1, …, N,
πk
= πk - 1
- ak
end
N = a few hundreds
several CPU hours
*
=
intractable
stochastic
gradient descent
for k = 1, …, N,
π
k
=
π
k -
1
-
a
k
end
N = a few hundreds
few CPU seconds
*
=
solvable
exact
noisySlide22
Theory wrap-up
22
appearance matching
ii. operator-theoretic analysis
i. material representation
iii. stochastic optimization
m
=
Σ
q
πq m
q
R(
π
)=
(I -
Σ
q
π
q
K
q
)
-1
noisy
min ǁ photo - render(
π
) ǁ
2
πSlide23
Our contributions
23
material
1.
exact
inverse volume rendering
with
arbitrary
phase functions!
2. validation with
calibration materials
known parameters
3. database of
broad range
of materials
thin
thick
non-
dilutable
solidsSlide24
Measurements
24
multiple lighting multiple viewpoints
appearance matching
min ǁ photo - render(
π
) ǁ
2
πSlide25
Acquisition setup
25
material sample
frontlighting
backlighting
cameraSlide26
Acquisition setup
26
bottom rotation stage
top rotation stage
material sample
frontlighting
backlighting
material sample
frontlighting
camera
backlighting
bottom rotation
stage
top rotation
stage
cameraSlide27
Validation
27
Frisvad
et al. 2007
polystyrenemonodispersions
aluminum oxide
polydispersions
very precise dispersions
(NIST Traceable Standards)
calibration materials
known parameters
Mie theory
size
%
particle material
medium materialSlide28
Parameter accuracy
28
polystyrene 1
polystyrene 2
polystyrene 3
aluminum oxide
all parameters
estimated within 4% error
comparison of ground-truth and measured parameters
ground-truth
measured
Henyey
-Greenstein fit
θ
-π
π
0
p(
θ)Slide29
Matching novel measurements
29
captured
rendered
rendered with HG
profiles
polystyrene 3
comparison of captured and rendered
images
images under
unseen geometries
predicted within 5% RMS error
ground-truth
measured
Henyey
-Greenstein fitSlide30
Our contributions
30
material
1.
exact
inverse volume rendering
with
arbitrary
phase functions!
2. validation with
calibration materials
known parameters
3. database of
broad range
of materials
thin
thick
non-
dilutable
solidsSlide31
thin
thick
non-
dilutable
solids
Measured materials
31
mustard
whole milk
shampoo
hand cream
coffee
wine
robitussin
olive oil
curacao
mixed soap
milk soap
liquid clay
reduced milkSlide32
Measured phase functions
32
whole milk
reduced milk
mustard
shampoo
hand cream
liquid clay
milk soap
mixed soap
glycerine
soap
robitussin
coffee
olive oil
curacao
wine
θ
-π
π
0
p(
θ)
measured
Henyey
-Greenstein fitSlide33
whole milk
reduced milk
mustard
shampoo
hand cream
liquid clay
milk soap
mixed soap
glycerine
soap
robitussin
coffee
olive oil
curacao
wine
Measured phase functions
33
θ
-π
π
0
p(
θ)
measured
Henyey
-Greenstein fitSlide34
Synthetic images
34
mixed soap
glycerine
soap
olive oilcuracao
whole milk
rendered imageSlide35
Synthetic images
35
chromaticitySlide36
Synthetic images
36
mixed soap
glycerine
soap
olive oilcuracao
whole milk
rendered imageSlide37
Effect of phase function
37
mixed soap
measured phase function
Henyey
-Greenstein fit
θ
-π
π
0
p(
θ)
rendered image
chromaticity
measured
Henyey
-Greenstein fitSlide38
Discussion
38
faster capture and convergence: trade-offs between accuracy, generality, mobility, and usability
more interesting materials: more general solids, heterogeneous volumes, fluorescing materials
other setups: alternative lighting (basis, adaptive, high-frequency), geometries, or imaging (transient imaging)Slide39
Take-home messages
39
material
1.
exact
inverse volume rendering
with
arbitrary
phase functions!
2. validation with
calibration materials
known parameters
3. database of
broad range
of materials
thin
thick
non-
dilutable
solidsSlide40
Acknowledgements
40
Henry
Sarkas (Nanophase)
Wenzel Jakob (Mitsuba)
Funding:National Science Foundation
European Research CouncilBinational Science
FoundationFeinberg Foundation
IntelAmazon
http://tinyurl.com/sa2013-inverse
Database of measured materials:Slide41
Error surface
41
appearance matching
min ǁ photo - render(
π) ǁ
2πSlide42
Light generation
42
MEMS light switch
RGB combiner
blue (480 nm) laser
green (535 nm)
laser
red (635 nm)
laser