Download presentation

## Presentation on theme: "Motion Estimation I"— Presentation transcript

Embed / Share - Motion Estimation I

Slide1

Motion Estimation I

What affects the induced image motion?

Camera motion

Object motion

Scene structureSlide2

Example Flow Fields

This lesson – estimation of general flow-fields

Next lesson – constrained by global parametric transformationsSlide3

The Aperture Problem

So how much information is there locally…?Slide4

The Aperture Problem

Copyright, 1996 © Dale Carnegie & Associates, Inc.

Not enough info in local regionsSlide5

The Aperture Problem

Copyright, 1996 © Dale Carnegie & Associates, Inc.

Not enough info in local regionsSlide6

The Aperture Problem

Copyright, 1996 © Dale Carnegie & Associates, Inc.Slide7

The Aperture Problem

Copyright, 1996 © Dale Carnegie & Associates, Inc.

Information is propagated from regions with high certainty (e.g., corners) to regions with low certainty.Slide8

Such info propagation can cause optical illusions…

Illusory cornersSlide9

1.

Gradient-based (differential) methods

(Horn &Schunk, Lucase

&

Kanade

)

2.

Region-based methods

(Correlation, SSD, Normalized correlation)

Direct (intensity-based) Methods Feature-based MethodsSlide10

Image

J

(taken at time

t

)

Brightness Constancy Assumption

Image

I

(taken at time

t+1

)Slide11

Brightness Constancy Equation:

The Brightness Constancy Constraint

Linearizing (assuming small

(u,v)

):Slide12

* One equation, 2 unknowns

* A line constraint in (u,v) space.* Can recover Normal Flow.Observations:

Need additional constraints…Slide13

Horn and Schunk (1981)

Add global smoothness term

Smoothness error

Error in brightness constancy equation

Minimize:

Solve by using calculus of variationsSlide14

Horn and Schunk (1981)

Problems…* Smoothness assumption wrong at motion/depth discontinuities over-smoothing of the flow field.* How is Lambda determined…?Slide15

Lucas-Kanade (1984)

Assume a single displacement (u,v) for all pixels within a small window (e.g., 3x3, 5x5)

Minimize

E(u,v):

Geometrically -- Intersection of multiple line constraints

Algebraically -- Slide16

Lucas-Kanade (1984)

Differentiating w.r.t

u

and

v

and equating to

0:

Solve for (u,v)

[ Repeat this process for each and every pixel in the image ]

Minimize

E(u,v):Slide17

Problems…

* Still smoothes motion discontinuities (but unlike Horn & Schunk, does not propagate error across the entire image)* Singularities (partially solved by coarse-to-fine)

Lucas-Kanade (1984)Slide18

Singularites

Where in the image will this matrix be invertible and where not…?

HomeworkSlide19

Linearization approximation iterate & warp

x

x

0

Initial guess:

Estimate:

estimate updateSlide20

x

x

0

estimate update

Initial guess:

Estimate:

Linearization approximation

iterate & warpSlide21

x

x

0

Initial guess:

Estimate:

Initial guess:

Estimate:

estimate update

Linearization approximation

iterate & warpSlide22

x

x

0

Linearization approximation

iterate & warpSlide23

Revisiting the small motion assumption

Is this motion small enough?Probably not—it’s much larger than one pixel (2nd order terms dominate)

How might we solve this problem?Slide24

==> small

u

and

v

...

u=10 pixels

u=5 pixels

u=2.5 pixels

u=1.25 pixels

image I

image J

iterate

refine

+

Pyramid of image J

Pyramid of image I

image I

image J

Coarse-to-Fine Estimation

Advantages:

(i) Larger displacements. (ii) Speedup.

(iii) Information from multiple window sizes.Slide25

Optical Flow ResultsSlide26

Optical Flow ResultsSlide27

1.

Gradient based methods (Horn &Schunk, Lucase & Kanade, …)

2.

Region based methods

(SSD, Normalized correlation, etc.)

Copyright, 1996 © Dale Carnegie & Associates, Inc.

But… (despite coarse-to-fine estimation)

rely on B.C.

cannot handle very large motions (no more than 10%-15% of image width/height) small object moving fast…?Slide28

Region-Based Methods

* Define a small area around a pixel as the region.* Match the region against each pixel within a search area in next image.* Use a match measure (e.g., SSD=sum

of-squares difference, NC=normalized correlation, etc.)* Choose the maximum (or minimum) as the match.

Advantages:

Can avoid B.C. assumption

Can handle large motions (even of small objects)

Disadvantages:

Less accurate (smaller sub-pixel accuracy)

Computationally more expensiveSlide29

SSD Surface – Textured areaSlide30

SSD Surface -- EdgeSlide31

SSD – homogeneous area

[Anandan’89 - Use coarse-to-fine SSD of local windows to find matches.

- Propagate information using

directional

confidence measures

extracted from each local SSD surface]Slide32

B.C. + Additional constraints:

Copyright, 1996 © Dale Carnegie & Associates, Inc.

Increase aperture:

[e.g., Lucas & Kanade]

Local singularities at degenerate image regions.

Increase analysis window to large image regions

=> Global model constraints:

Numerically stable, but requires prior model selection:

Planar (2D) world model

[e.g., Bergen-et-al:92, Irani-et-al:92+94, Black-et-al]

3D world model[e.g., Hanna-et-al:91+93, Stein & Shashua:97, Irani-et-al:1999]

Spatial smoothness: [e.g., Horn & Schunk:81, Anandan:89] Violated at depth/motion discontinuities

Download Presentation - The PPT/PDF document "Motion Estimation I" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

Please download the presentation after appearing the download area.

What affects the induced image motion Camera motion Object motion Scene structure Example Flow Fields This lesson estimation of general flowfields Next lesson constrained by global parametric transformations ID: 247102 Download Presentation

pptx
60 views

pptx
104 views

pptx
113 views

pdf
168 views

pptx
98 views

pptx
1 views

pptx
13 views

pptx
77 views

pptx
44 views

pptx
39 views