Optical flow A tutorial of the paper KH Wong Optical Flow v5a beta 1 G Farneback Twoframe Motion Estimation based on Polynomial Expansion 13th Scandinavian Conference SCIA 2003 Halmstad ID: 769763
Download Presentation The PPT/PDF document "Optical flow , A tutorial of the paper:" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Optical flow ,A tutorial of the paper: KH Wong Optical Flow v.5a (beta) 1 G. Farneback , “Two-frame Motion Estimation based on Polynomial Expansion”, 13th Scandinavian Conference, SCIA 2003 Halmstad , Sweden, June 29 – July 2, 2003.
IntroductionOptical flow : Based on two frames sampled at different slight different time, find the change of position of each pixel in the first image to the second image. Reference:G. Farneback , “Two-frame Motion Estimation based on Polynomial Expansion”, 13th Scandinavian Conference, SCIA 2003 Halmstad , Sweden, June 29 – July 2, 2003. Optical Flow v.5a 2
Problem definition and method Problem:Input: Two imagesOutput : motion vectors (v) and xx( c )? of each pixel in the first image.The Farneback method [1] is fast and linear.Method: Step1: Approximate the image using a polynomialStep2: Use the model of polynomial to approximate the motion for each pixel. Optical Flow v.5a 3
Example Optical Flow v.5a 4 Image1 Image2 Optical flow result , vectors showing the direction and intensity of flow of each pixel. Plotting all vectors is too messy, only plot one vector in a window of 10x10 pixels.
Ideal and method We use a polynomial (with parameters A,b,c) to model the intensity change of an image (small window). Assume x is dimension2 here, but it can be larger. In image f1 (x) is taken at t , and f2 (x) at t+dt f 1 (x)=x T A 1 x+b 1 x+c 1 x is 2x1, A 1 is 2x2, b 1 is 2x1, c1 is 1x1Assume f 2 is f1 with a global displacement by d=[d1,d2]’ (shape no change, just displacement) f2 (x)=xTA2x+b 2 x+c 2By comparing f1 and f2, we getA2=A1, b2=b1-2A1dC2=dTA1d-bT1d+c1We observe that (when A1 is non-singular , i.e. invertible)2A1d=-(b2-b1)d=-(1/2)(A1)-1(b2-b1)So if we can measure f1 and f2, can may find d=[d1,d2]’. d is the 2-D displacement and hence the flow of the image pixel. Optical Flow v.5a 5 Intensity of image f 1 (x)) x =[ x,y ]’x1 (just show one dimension of x) d Image 1, at t Image 2, at t= dt x Use a polynomial (with parameters A,b,c) to model the intensity change of an image (assume a small window). f 2 (x)
Step: 1a Model an image using a polynomial Optical Flow v.5a 6 For a polynomial with parameters A,b,c : x= is 2x1 A is 2x2 b is 2x1 C is 1x1 d is 2x1
Modeling images using polynomials Read the Ph.D thesisOrientation and Velocity, Estimation , Ph.D https://www.facebook.com/download/333523813505207/farneback99-phd.pdf (page22 the example) and also the code make_Abc_fast.m in Code, https://github.com/GunnarFarneback/spatial_domain_toolbox Optical Flow v.5a 7
step1b Assume all pixels in a small window I() behave the same. If we can measure A1, b1, c1 from f1 for each pixel in I() ; and A2, A2 c2 from f2 we can calculate d. Optical Flow v.5a 8 Sum over an window of I() Pointwise is too noisy
Step2 : find displacement d=[dx,dy]’ (=optical flow) We assume d is not a constant over the window I(), but a function according to its position x,y based on affine transformation. Optical Flow v.5a 9 Affine model of d S is 2x8 S T is 8x2 p is 8x1 d is 2x1 w i = is 1x1 A , p ,b can be measured. Put them in (19) and minimize (19), or set it to [0,0]’. We can solve for p using (20), if p is found d can be found by (15) 0
Some modification Optical Flow v.5a 10
Result Optical Flow v.5a 11 Image: yos2 Image: yos8 Optical flow (averaged) using a sequence of 15 pictures. Plotting all vectors is too messy, only plot one vector in a window of 10x10 pixels. Using the code https://github.com/GunnarFarneback/spatial_domain_toolbox Image: yos16
SummaryStudied a linear method to find the optical flow based on two frames of image. Optical Flow v.5a 12
References G. Farneback , “Two-frame Motion Estimation based on Polynomial Expansion”, 13th Scandinavian Conference, SCIA 2003 Halmstad, Sweden, June 29 – July 2, 2003. Publications of G. Farneback , http://lmi.bwh.harvard.edu/papers/gunnar.html G. Farneback , Spatial Domain Methods for Orientation and Velocity, Estimation , Ph.D https://www.facebook.com/download/333523813505207/farneback99-phd.pdf Code, https://github.com/GunnarFarneback/spatial_domain_toolbox Optical Flow v.5a 13