Antoni B Chan Rynson WH Lau City University of Hong Kong Automatic Stylistic Layout Background Manga layout is crucial for manga production with unique styles AYOYAMA Gosho Shogakukan ID: 509555
Download Presentation The PPT/PDF document "Ying Cao" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Ying Cao
Antoni B. Chan
Rynson W.H. Lau
City University of Hong Kong
Automatic Stylistic
LayoutSlide2
Background
Manga layout is crucial for manga production, with unique styles
©
AYOYAMA
Gosho
/
Shogakukan
Inc
.
Manga pages
Their layoutsSlide3
Background
Effective manga layout can benefit
Storytelling
Attention guidance
Visual attractiveness
It is a difficult taskSlide4
Goal
To create
high-qualit
y manga layout
with ease
Resulting layout
1
2
2
3
3
3
Semantics
A
rtworksSlide5
Challenge
Not a well-studied problem
Our solution
:
d
ata-driven strategy to
learn stylistic aspects from existing manga pages
No explicit rules Slide6
Related Work
General layout problem:
global optimization
[ Yu et al. 2011]
[Merrell
et al. 2011
]Slide7
Related Work
Comic layout:
heuristic rules or
templates
[
Kurlander
et al. 1996
]
[Shamir et al. 2006
]
[
Preu
et al. 2007]Slide8
Related Work
Computational
Manga
[
Qu
et al. 2006
]
[
Qu
et al. 2008]Slide9
OverviewSlide10
OverviewSlide11
OverviewSlide12
OverviewSlide13
Manga Database
4,000 scanned manga pages from two manga series
Panel annotation
Page clustering
One manga series
3
-panel pages
10-panel pages
4
-panel pages
…Slide14
OverviewSlide15
Style Models
Represent stylistic aspects of manga layout
Learned from manga examples
3) Panel shape
…
2) Panel importance (size)
1
2
3
1) Layout structure (i.e., spatial arrangement of panels)
…Slide16
A probabilistic generative model:
Synthesize
novel
plausible layout structures
Layout structure ModelSlide17
Root
©
AYOYAMA
Gosho
/
Shogakukan
Inc
.
Layout structure Model
Generative process:
r
ecursive spatial division
R1
R2
R3
C
1
C
2
C1
R2
R1
C3
C2
C1Slide18
Layout structure Model
Parameterization: spatial division instance
:
I
nstance label
[
Root-R]
:
Number of rows [3]
:
S
plitting configuration [
(
)
]
©
AYOYAMA
Gosho
/
Shogakukan
Inc
.Slide19
Layout structure Model
:
I
nstance label
:
N
umber of rows
: Splitting configuration
Probabilistic
graphical model
Parameterization: spatial division instanceSlide20
Layout structure Model
Sample splitting configuration
:
I
nstance label
:
N
umber of rows
:
Splitting configuration
Probabilistic
graphical model Slide21
Layout structure Model
Sample splitting configuration
:
Instance label
:
Number of rows
:
Splitting configuration
Probabilistic
graphical model
Sample
Slide22
L
ayout structure Model
Layout structures sampled from our model
Training example
Sampling:
r
ecursive splitting using sampled
Slide23
Panel clustering
Width
Heigh
t
Panel Importance
3
1
2
S
ize
I
mportance
Shape
?
A shape-to-importance classifier
Slide24
Panel Shape Variation Model
Captures panel shape variability
Active
Shape
Model [Cootes et al. 1995]
…
…
…Slide25
OverviewSlide26
Semantic Specification
Single-panel semantics
I
nter-panel semantics
Image geometry
Group of related panels
3
I
mportanceSlide27
OverviewSlide28
Initial Layout Generation
A layout structure
Maximum a posteriori (MAP) inference
Fitness between
and
Our generative model
Input semantics
E
xisting ones
m
atches
resemblesSlide29
Initial Layout Generation
Likelihood term
Penalize panel-wise mismatch
in aspect ratio & importance
Single-panel Likelihood
Image geometry
panel geometrySlide30
Initial Layout Generation
Likelihood term
Inter-panel LikelihoodSlide31
Initial Layout Generation
Likelihood term
Inter-panel Likelihood
Measure the smoothness of path through panelsSlide32
Initial Layout Generation
Likelihood term
Inter-panel Likelihood
Align group boundary with layout boundary Slide33
Initial Layout Generation
Estimate optimal initial layout
Exact
MAP inference is
computationally expensive
…
Generative Model
Maximum PosterioriSlide34
Layout Optimization
Unoptimized
Fit
to
and reproduce panel shape irregularity
Slide35
Layout Optimization
Energy function
Collinearity
constraint
Boundary constraint
Regularization termSlide36
Layout Optimization
Minimize
via an
a
lternating solver
Slide37
Results
(1)
(2)
(2)
(3)
(2)
(2)
(1)
(2)
(2)
(2)
(3)
(3)
(1)Slide38
Comparison with existing manga page
Input
Our result
Existing
manga page
(
3)
(
3)(1)
(3)(3
)
(
2
)
(
3
)
©
AYOYAMA
Gosho
/
Shogakukan
Inc
.Slide39
Layouts of different styles
(1)
(2)
(3)
(1)
(3)
(2)
Input
Style of “Fairy Tail”
Style of “Detective Conan”Slide40
Layouts
of Western comic styleSlide41
User Study
10 participants: manual tool + our tool
10 Evaluators: pairwise comparisonSlide42
Summary
First
attempt
to computationally reproduce layout styles of manga
A
data-driven approach for automatic generation of stylistic manga layoutEasy and
quick production of
professional-looking and stylistically rich manga layouts Slide43
Limitations & Future Work
Story pacing
Art
composition & balloon placement
Generic framework for
other
layout problems Slide44
Thanks