2Consider the above calibration patternitconsists of twoorthogonal gridsequally spaced black squares drawn on white perpendicular planesassume that the world reference frame is centered at t ID: 301700
Download Pdf The PPT/PDF document "Camera Calibration(Trucco, Chapter 6)" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Camera Calibration(Trucco, Chapter 6)What is the goal of camera calibration?-Toproduce an estimate of the extrinsic and intrinsic camera parameters.Procedure-Giv enthe correspondences between a set of point features in the world (Xw,Yw,Zw)and their projections in an image (xim,yim), compute the intrinsic and extrinsic cam-era parameters.PYXccYXZwwwCameraFrameWorldFrameyxxyimimo ,oxyoptical axisZcprincipal point center ofperspective projectionpixel frameimageplaneframeEstablishing the correspondences-Calibration methods rely on one or more images of a calibration pattern:(1) a 3D object of known geometry.(2) it is located in a known position in space.(3) it is generating image features which can be located accurately. -2--Consider the above calibration pattern:*itconsists of twoorthogonal grids.*equally spaced black squares drawn on white, perpendicular planes.*assume that the world reference frame is centered at the lower left corner of theleft grid, with axes parallel to the three directions identied by the calibrationpattern.*giv enthe size of the planes, their angle, the number of squares etc.(all knownby construction), the coordinates of each vertexcan be computed in the worldreference frame using trigonometry.*the projection of the vertices on the image can be found by intersecting theedge lines of the corresponding square sides (or through corner detection). -3-Methods(1) Direct parameter calibration.Direct recovery of the intrinsic and extrinsic camera parameters.(2) Camera parameters through the projection matrixM=MinMex=éêêëm11m21m31m12m22m32m13m23m33m14m24m34ùúúû(2.1) Estimate the elements of the projection matrix.(2.2) Compute the intrinsic/extrinsic as closed-form functions of the entries ofthe projection matrix. -4-Method 1: Direct Parameter Calibration-Weassume that the world reference frame is known (e.g., the origin is the middlelower corner of the calibration pattern).Review of basic equations-From world coordinates to camera coordinates (note that we have changed the orderof rotation/translation):Pc=R(Pw-T)orPc=RPw-RTorPc=RPw-T¢-Inthe rest of this discussion, I will replaceT¢withT:éêêëXcYcZcùúúû=éêêër11r21r31r12r22r32r13r23r33ùúúûéêêëXwYwZwùúúû+éêêëTxTyTzùúúû-From camera coordinates to pixel coordinates:xim=-x/sx+ox=-fsxXcZc+oxyim=-y/sy+oy=-fsyYcZc+oy-Relating world coordinates to pixel coordinates:xim-ox=-f/sxr11Xw+r12Yw+r13Zw+Txr31Xw+r32Yw+r33Zw+Tzyim-oy=-f/syr21Xw+r22Yw+r23Zw+Tyr31Xw+r32Yw+r33Zw+Tz -5-Independent intrinsic parameters-The veintrinsic parametersf,sx,sy,ox,oyare not independent.-Wecan dene the following four independent parameters:fx=f/sx,the focal length in horizontal pixels =sy/sx(or =fx/fy), aspect ratio(ox,oy), image center coordinatesMethod 1: main steps(1) Assuming thatoxandoyare known, estimate all the remaining parameters.(2) EstimateoxandoyStep 1: estimatefx, ,R,andT-Tosimplify notation, consider (xim-ox,yim-oy)=(x,y)x=-fxr11Xw+r12Yw+r13Zw+Txr31Xw+r32Yw+r33Zw+Tzy=-fyr21Xw+r22Yw+r23Zw+Tyr31Xw+r32Yw+r33Zw+Tz-Using the fact that the above two equations have the same denominator,weget thefollowing equation:xfy(r21Xw+r22Yw+r23Zw+Ty)=yfx(r11Xw+r12Yw+r13Zw+Tx)Problem StatementAssuming thatoxandoyare known, computefx, ,R,andTfromNcorrespond-ing pairs of points (Xwi,Ywi,Zwi), (xi,yi),i=1, . . . ,N. -6-Der iveasystem of equations-Each pair of corresponding points leads to an equation:xify(r21Xwi+r22Ywi+r23Zwi+Ty)=yifx(r11Xwi+r12Ywi+r13Zwi+Tx)-Rewrite the above equation as follows (i.e., divide byfy):xiXwiv1+xiYwiv2+xiZwiv3+xiv4-yiXwiv5-yiYwiv6-yiZwiv7-yiv8=0wherev1=r21v5=r11v2=r22v6=r12v3=r23v7=r13v4=Tyv8=Tx-Ncorresponding points lead to a homogeneous system ofNequations with 8unknowns:Av=0where:A=éêêêëx1Xw1x2Xw2...xNXwNx1Yw1x2Yw2...xNYwNx1Zw1x2Zw2...xNZwNx1x2...xN-y1Xw1-y2Xw2...-yNXwN-y1Yw1-y2Yw2...-yNYwN-y1Zw1-y2Zw2...-yNZwN-y1-y2...-yNùúúúûSolving thesystem-Itcan be shown that ifN³7, thenAhas rank 7.-IfA=UDVT,wehav ediscussed in class that the system has a nontrivial solu-tionvwhich is proportional to the column ofVcorresponding to the smallest sin-gular value ofA(i.e., the last column ofVwhich we denote asv):v=v(is the scale factor) orv=v(=1/)-Using the components ofvandv:(v1,v2,v3,v4,v5,v6,v7,v8)=(r21,r22,r23,Ty,r11,r12,r13,Tx) -7-Deter mineand||Ö` ``````v21+v22+v23=Ö` ``````````2(r221+r222+r223)=||(r221+r222+r223=1)Ö` ``````v25+v26+v27=Ö` ```````````22(r211+r212+r213)=||(r211+r212+r213=1and�0)Deter miner21,r22,r23,r11,r12,r13,Ty,Tx-Wecan determine the above parameters, up to an unknown common sign.r21=1/||v1r11=1/||v5r22=1/||v2r12=1/||v6r23=1/||v3r13=1/||v7Ty=1/||v4Tx=1/||v8Deter miner31,r32,r33-Can be estimated as the cross product ofR1andR2:R3=R1xR2-The sign ofR3is already xed (the entries ofR3remain unchanged if the signsof all the entries ofR1andR2are reversed).Ensur ingthe orthogonality ofR-The computation ofRdoes not takeinto account explicitly the orthogonalityconstraints.-The estimate^RofRcannot be expected to be orthogonal (e.g.,^R^RT=I).-Wecan "enforce" the orthogonality on^Rby using its SVD:^R=UDVT-ReplaceDwithI,e.g.,^R¢=UIVT(^R¢^R¢T=I) -8-Deter minethe sign of-Consider the following equations again:x=-f/sxr11Xw+r12Yw+r13Zw+Txr31Xw+r32Yw+r33Zw+Tz=-f/sxXcZcy=-f/syr21Xw+r22Yw+r23Zw+Tyr31Xw+r32rw+r33Zw+Tz=-f/syYcZc-IfZc�0, thenxandr11Xw+r12Yw+r13Zw+Txmust have opposite signs (it issufcient to check the sign for one of the points).ifx(r11Xw+r12Yw+r13Zw+Tx�)0,thenreverse the signs ofr11,r12,r13,andTxelseno further action is required-Similarly,ifZc�0, thenyandr21Xw+r22Yw+r23Zw+Txmust have oppositesigns (it is sufcient to check the sign for one of the points).ify(r21Xw+r22Yw+r23Zw+Ty�)0,thenreverse the signs ofr21,r22,r23,andTyelseno further action is required -9-Deter mineTzandfx:-Consider the equation:x=-f/sxr11Xw+r12Yw+r13Zw+Txr31Xw+r32Yw+r33Zw+Tz-Let'srewrite it in the form:x(r31Xw+r32Yw+r33Zw+Tz)=-f/sx(r11Xw+r12Yw+r13Zw+Tx)-Wecan obtainTzandfxby solving a system of equations likethe above,writ-ten forNpoints:AéêëTzfxùúû=bwhereA=éêêêëx1x2...xN(r11Xw1+r12Yw1+r13Zw1+Tx)(r11Xw2+r12Yw2+r13Zw2+Tx)...(r11XwN+r12YwN+r13ZwN+Tx)ùúúúûb=éêêêë-x1(r31Xw1+r32Yw1+r33Zw1+Tx)-x2(r31Xw2+r32Yw2+r33Zw2+Tx)...-xN(r31XwN+r32YwN+r33ZwN+Tx)ùúúúû-Using SVD, the (least-squares) solution is:éêëTzfxùúû=(ATA)-1ATbDeter minefy:-Fromfx=f/sxandfy=f/sywe have:fy=fx/ -10-Step 2: estimateoxandoy-The computation ofoxandoywill be based on the following theorem:Orthocenter Theorem:LetTbe the triangle on the image plane dened by the threevanishing points of three mutually orthogonal sets of parallel lines in space. Theimage center (ox,oy)isthe orthocenter ofT.-Wecan use the same calibration pattern to compute three vanishing points (use threepairs of parallel lines dened by the sides of the planes).Note 1:it is important that the calibration pattern is imaged from a viewpoint guaran-teeing that none of the three mutually orthogonal directions will be near parallel tothe image plane !Note 2:to improve the accuracyofthe image center computation, it is a good idea toestimate the center using several views of the calibration pattern and average theresults. -11-Method 2: Camera parameters through the projection matrixReview of basic equationséêêëxhyhwùúúû=MinMexéêêêëXwYwZw1ùúúúû=MéêêêëXwYwZw1ùúúúû=éêêëm11m21m31m12m22m32m13m23m33m14m24m34ùúúûéêêêëXwYwZw1ùúúúûx=xhw=m11Xw+m12Yw+m13Zw+m14m31Xw+m32Yw+m33Zw+m34y=yhw=m21Xw+m22Yw+m23Zw+m24m31Xw+m32Yw+m33Zw+m34(Note:Ihav ereplacedximwithxandyimwithyfor simplicity)Step 1: solveformijs-The matrixMhas 11 independent entries (e.g., divide every entry bym11).-Wewould need at leastN=6 world-image point correspondences to solvefor theentries ofM.m11Xwi+m12Ywi+m13Zwi+m14-m31xiXwi-m32xiYwi-m33xiZwi+m34=0m21Xwi+m22Ywi+m23Zwi+m24-m31yiXwi-m32yiYwi-m33yiZwi+m34=0-These equations will lead to a homogeneous system of equations:Am=0whereA=éêêêêêêêëXw10Xw20...XwN0Yw10Yw20...YwN0Zw10Zw20...ZwN01010...100Xw10Xw2...0XwN0Yw10Yw2...0YwN0Zw10Zw2...0ZwN0101...01-x1Xw1-y1Xw1-x2Xw2-y2Xw2...-xNXwN-yNXwN-x1Yw1-y1Yw1-x2Yw2-y2Yw2...-xNYwN-yNYwN-x1Zw1-y1Zw1-x2Zw2-y2Zw2...-xNZwN-yNZwN-x1-y1-x2-y2...-xN-yNùúúúúúúúû -12--Itcan be shown thatAhas rank 11 (forN³11).-IfA=UDVT,the system has a nontrivial solutionmwhich is proportional to thecolumn ofVcorresponding to the smallest singular value ofA(i.e., the last column ofVdenoted here asm):m=m(is the scale factor) orm= m( =1/)Step 2: nd the intrinsic/extrinsic parameters usingmijs-The full expression forMis as follows:M=éêêë-fxr11+oxr31-fyr21+oyr31r31-fxr12+oxr32-fyr22+oyr32r32-fxr13+oxr33-fyr23+oyr33r33-fxTx+oxTz-fyTy+oyTzTzùúúû-Let'sdene the following vectors:q1=(m11,m12,m13)Tq2=(m21,m22,m23)Tq3=(m31,m32,m33)Tq4=(m14,m24,m34)T-The solutions are as follows (see book for details):ox=qT1q3oy=qT2q3fx=Ö` `````qT1q1-o2xfy=Ö` `````qT2q2-o2y-The rest parameters are easily computed ....Question: howwould you estimate the accuracyofacalibration algorithm? -13-Some comments-The precision of calibration depends on howaccurately the world and image pointsare located.-Studying howlocalization errors "propagate" to the estimates of the camera parame-ters is very important.-Although the twomethods described here should produce the same results (at leasttheoretically), we usually obtain different solutions due to different error propaga-tions.-Method 2 is simpler and should be preferred if we do not need to compute theintrinsic/extrinsic camera parameters explicitly.