Lei Yang Pedro V Sander The Hong Kong University of Science and Technology Jason Lawrence University of Virginia Hugues Hoppe Microsoft Research Overview Introduction Nonlinear filtering and edge artifacts ID: 499013
Download Presentation The PPT/PDF document "Antialiasing Recovery" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Antialiasing Recovery
Lei Yang, Pedro V. SanderThe Hong Kong University of Science and TechnologyJason LawrenceUniversity of VirginiaHugues HoppeMicrosoft ResearchSlide2
Overview
IntroductionNonlinear filtering and edge artifactsPrior workAntialiasing recoveryA 1D illustrationFinding the edge color model in 2DCorrecting the filtered imageA real-time GPU implementationResults and discussionSlide3
Smooth edge and nonlinear filtersAntialiased edges in imagesSmooth transition pixels along object boundaries
Blending of the two meeting colorsNonlinear filtersCan map edge pixels to abrupt colorsVery common in image processing
Nonlinear Filter
Image
scanline
LuminanceSlide4
Image
AbstractionBilateral
FilteringSlide5
Detail
EnhancementIntensity
ThresholdingSlide6
Repairing abrupt edges Morphological antialiasing (MLAA) [Reshetov 2009]Slide7
Repairing abrupt edges Morphological antialiasing (MLAA) [Reshetov 2009]
Designed to repair point sampling artifactsNot suitable for fixing partially antialiased edgesFilteredMLAADesiredSlide8
Antialiasing recoveryExtract the edge blending info from the original imageApply it to the filtered image
Preserve filtered colors except at edgesOriginalFilteredSlide9
Antialiasing recoveryA postprocess to existing nonlinear filters
Filtered (
F)
Original (O)
Result (R)
Nonlinear
filter
Antialiasing recoverySlide10
A 1D illustration
For each edge pixel:
Resolve the contribution of the smooth regions in “
O
”
Apply the same contribution factors in “F” to obtain “R”
Smooth regions
Edge
Filtered:
Incorrect
6
0%
4
0%
Re-blended
6
0%
4
0%
Contributions of smooth regionsSlide11
Generalizing to 2D imagesDifficulty: each pixel has 8 neighborsIdentify the two contributing colorsSelect two pixels
pa and pbMost representative of the two smooth regions(Closest to their colors)
p
a
p
b
pSlide12
The color line model
Blendings of two colors are collinear in RGB color spaceThe two generating colors ca and cb are endpoints (extrema)
R
G
c
c
a
c
b
p
a
p
b
p
BSlide13
Identifying
extrema colorsThe local 33 patch may contain noise and other colorsFind the direction of maximum variance “x” in color space Compute the first principal component using EM [Roweis 97]Very efficient (2-3 iterations)
p
R
G
c
xSlide14
Identifying extrema colorsForm a datum line
l = c + xt passing through the center color c along the principal direction xProject each pixel color ci onto lDistance: di, parametric coord.: tiDiscard all pixels with di > 3dd: a user specified tolerance
p
R
G
c
l
x
6
d
d
i
t
iSlide15
Identifying extrema colors
p
R
G
c
l
t
i
t
j
c
a
p
a
c
b
p
b
Find the
two extrema
colors along
l
Equivalent to
finding:
where
such that
Slide16
Determine pixel coverage values
p
R
G
c
c
b
p
a
p
b
c
a
1
-
p
:
p
d
p
Project
c
onto the line segment [
c
a
,
c
b
], compute the linear blending factor
p
Equivalent to solving a linear least-square system that minimizes:
subject to
Slide17
Correcting the filtered imageCorrect pixels only at edgesDetermining edge strength
epCompute and combine Sobel edge strength in O and F
Edges in
O
Edges in
FEdge strength
e
p
=Slide18
Correcting the filtered imageBlending confidence
p:combine edge strength ep with interpolation error dp:
where
G is the Gaussian function
e is a user specified parameter controlling edge sensitivity
Slide19
Correcting the filtered imagePixels in the recovered image R
are computed as:
Blended colors are mutually dependent
Form a sparse linear system
Non-edge pixels use
F[p] to condition the solution
Blended color
Unchanged
When the blending
confidence is high
When confidence
is lowSlide20
Examples of dependency chains Thick edges Thin features Slide21
Real-time GPU ImplementationEmbarrassingly parallel algorithmImplemented using pixel shaders
on the GPUApply Jacobi iterations to solve the linear systemUse R = F as the initial solutionConverges quickly (fixed 3 iterations suffices)Extremely fastOver 500MP/s (= 270 1080p images/s) on AMD HD5870Slide22
Parameterse
for controlling edge sensitivityd for controlling noise sensitivityWe use a fixed d = 0.1 and e = 0.01 for resultsInsensitive to changing parameters in a rangeCan be interactively adjusted for special casesSlide23
ResultsApplication to a number of image processing tasksImage abstractionBilateral filter
Intensity thresholdingDetail enhancementGradient mappingColor to grayColor replacementSlide24
ResultsImage abstraction
Original (
O)
Filtered (F)
Recovered(
R)
Images courtesy from J. E.
Kyprianidis
et al. [2009]Slide25
ResultsBilateral filter
Original (
O
)
Filtered (
F)
Recovered(
R
)
Image courtesy of
paullew@Flickr
Slide26
ResultsIntensity thresholding
Original (
O
)
Filtered (
F)Recovered(R
)Slide27
ResultsGradient mapping
Original (
O
)
Filtered (F
)Recovered(R
)Slide28
ResultsColor replacement
Original (O)
Filtered (F)
Recovered(
R)Slide29
LimitationsOur method is not suitable forFilters with geometric distortions
(E.g. image warping)Filters that intentionally change the edge fidelity(E.g. Gaussian blur)Filters that have artifacts in the interior regions(E.g. Unsharp masking)Slide30
LimitationsWhen the color line assumption is violatedPixels that receive more than two scene elements
May lead to artifacts in extreme casesOriginal (O
)Filtered (
F)
Recovered(
R)
Images courtesy of M.
Čadík
and Y. KimSlide31
ConclusionAn algorithm to repair antialiased edges damaged by certain types of nonlinear filtersSimple to implement
Very fast (500 MP/sec)Useful for a number of filtersCode will be available within a monthFuture workConsider filters with geometric distortionsRelax the color line modelSlide32
ThanksAcknowledgement:
Diego Nehab and Tian Fang for fruitful discussionsHK RGC GRF grant #619509Image credits:Yongjin Kim, Martin Čadík, Johannes Kopf, Jan Eric
Kyprianidis, Giuseppe Papari, Phillip Greenspun,
paullew@Flickr, shoshonasnow@Flickr and thaneeya
@Flickr