nY arctannX nY2andnTNtan 1tan WithoutlossofgeneralitytheprojectioncenterZ00ZTischosenTheoriginoftheobjectcoordinatesystemliesinthereferenceplanetheZaxisrunsthroughtheprojec ID: 324561
Download Pdf The PPT/PDF document "angle ,therollangle andtheheightZoftheca..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
angle ,therollangle andtheheightZofthecamera(cf.g.1).Sincetheazimuthoftheviewingdirectionisatourdisposal,therotationmatrixfromtheobjecttothecamerasystemreadsR=RZ( )RX(=2+ )andwiththenormaltotheplaneintheobjectcoordinatesys-temE=(0;0;1)Tthenormalinthecameracoordinatesystemsbecomesn=RE=(nX;nY;nZ)T:(1)Therelationshipsbetweenthenormalandtheanglesis =arctannZ nY; =�arctannX nY(2)andnT=N(tan(� );1;tan( )):Withoutlossofgenerality,theprojectioncenterZ=(0;0;Z)Tischosen.Theoriginoftheobjectcoordinatesystemliesinthereferenceplane,theZ-axisrunsthroughtheprojectioncenterofthecamera.TheY-axisisdenedbytheprojectionoftheopticalaxisontotheplane,theX-axisisperpendiculartoboth(cf.g.1). Z XYZX00X0HEFigure1:showsthedenitionoftheinvolvedcoordinatesystemsandtheprojectionofaheightintotheimage.CameraModel.Forthecameraastraightlinepreserv-ingpinholemodelisintroducedwiththeprincipaldistancec,thescalefactorm,theshearsandtheprincipalpoint(x00;y00)astheintrinsiccameraparameters.Withtheho-mogeneouscalibrationmatrixK=0@cscx000mcy000011Athehomogeneous34–projectionmatrixP=KR(I3j�Z)projectsanobjectpointXiintotheimagepointx0iviathelineartransformationx0i=PXi.Withthepresentedapproachandacameraingeneralposi-tion,twooftheveintrinsicparameterscanbedetermined—preferablytheprincipaldistanceandthescalefactor.Therefore,initiallytheusedcalibrationmatrixhasdiago-nalshape:K=Diag(c;mc;1):Observations.Foreachobjectthefourcoordinatesx0b,y0t,y0bandy0t(bottom,top)ofthefootandheadpointsareavailableasobservations.2.2ConceptoftheVirtualCameraThemappingofanobjectfootpointx0i=(x0b;y0b)Tiintothecorrespondingheadpointx00i=(x0t;y0t)Ticanbeex-pressedbytheprojectivetransformationx00i=Hx0i;(3)calledahomographywitheightindependentparametersduetothehomogenity.The33–transformationmatrixHisconstantforobjectsofequalheightandcanbedeter-minedbyfourpointcorrespondences.InthefollowingweshowhowHcanbeexpressedasafunctionoftheunknownparametersandhowagiventransformationmatrixcanbedecomposedaccordingly.2.2.1VirtualHomography.Withthenotionofcorre-spondingheadandfootpointsbeingidenticalinspace,thesituationcanalsobedescribedwiththehelpofasecondvirtualcamera(cf.g.2).Aplaneinducedhomographyresultsfromtheimagesoftwocamerasobservingthesameobjectonaplane.WiththecalibrationmatricesK0andK00ofthesetwocameras,thedistanceZoftherstcameratotheplaneandthebaselinevectortthehomographyreadsH=K00R00�1 ZtnTK0�1;(4)withtherotationmatrixR00ofthesecondcamerainre-specttotherstcameracoordinatesystem;cf.(FaugerasandLustman,1988)or(HartleyandZisserman,2000)foranalternativederivation.TheterminbracketsiscalledthemotionmatrixM.Inourcasewehaveonecameraobserv-ingthescenefromtwoaltitudeswithunchangedviewingdirection,thusK0=K00=K,R00=I3,andt=Hn(cf.g.2).Thehomography(4)becomesH=KI3�H ZnnTK�1:(5)Observethatthebaselinelengthktkisidenticaltotheob-jectheightH.Thetransformation(5)isaso-calledplanarhomology(HartleyandZisserman,2000,p.585)sincewiththehori-zonlinel0=K�Tnandthevanishingpointv0=Kn—normallythenadir—equation(5)readsH=I3+(�1)v0l0T v0Tl0with(�1)=H=(Zv0Tl0).Theplanarhomologyhasvedegreesoffreedom—thevertexv0(2dof),theaxisl0(2dof)andthecharacteristicratio(SempleandKnee-bone,1952)andcanthereforebedeterminedby2.5pointcorrespondences.AsHcontains5dof,wecandeterminetwointrinsicpa-rametersinadditiontothethreeparameters , andZof O0O00X00X0HO00O0X0=X00HZFigure2:showsthetrueconguration(top)andtheequiv-alentsituationwithasecondvirtualcamera(bottom).InbothcasestheobservationsoftheobjectfootandheadpointsX0andX00areidentical.theexteriororientation.WiththespecialcalibrationmatrixK=Diag(c;mc;1)theplanarhomologyexplicitlyreadsH=0B@Hn2X�Z ZHnXnY ZmcHnXnZ ZmHnXnY ZHn2Y�Z ZmcHnYnZ ZHnXnZ ZcHnYnZ ZmcHn2Z�Z Z1CA:(6)Observethatform=1therelationH12H21holds.Inmanypracticalcasestherollangle equalszero,sothatnX=0holdsandthehomography(6)becomesH=0BBB@�1000Hn2Y�Z ZcmHnYnZ Z0HnYnZ ZmcHn2Z�Z Z1CCCA:asacommonspecialization.Inthiscaseonlytheprincipaldistanceorthescalefactorisdeterminable.2.2.2DecompositionofH.Parameterestimationre-quiresapproximationvaluesfortheunknowncalibrationparameters.Thesevaluescanbededucedbyadirectes-timationoftheeightparametersofthecommonhomogra-phy(3),ifareal-valueddecompositionaccordingto(5)isavailable.(1)Intrinsiccameraparameters.Fromeq.(6)theprin-cipaldistanceandthescaledifferencearec=p H13=H31andm=p H21=H12;,butforthefrequentcaseoftherollangle =0theelementH31becomeszero.Inthiscasetheprincipaldistancemustbecomputedviac=p H23=H32=mwiththeknownscalefactorm.(2)Exteriororientation.Oncetheintrinsicparameterscand,whereapplicable,mhavebeendetermined,themo-tionmatrixM=K�1HKcanbecomputedwithK=Diag(c;mc;1).Theeigenvalue-eigenvector-decomposi-tionofMhasthreereal-valuedeigenvectorsei,i=1;2;3,withtwoidenticalreal-valuedeigenvalues2=3andanindividualeigenvalue1.Thenormalvectoroftheplaneresultsfromtheeigenvectorsn=e1=N(e2e3)andtheratioofcameraandobjectheightfromtheeigen-values:H Z=1�3 1=1�2 1:Notethatthesolutionisunambiguousexceptforacom-monsignofcandnZandthesignofn.Buttherequire-mentnY�0isreasonableformostcamerainstallations.Withtheorientationparametersdeterminedinthismannerweareabletomeasuretheobjectheightandposition.2.33DObjectMeasurementSimilarformulasforthecomputationoftheheightofanobjecthavebeendevelopedindependentlyin(Criminisi,2001)and(Rennoetal.,2002,Jonesetal.,2002)—ontheonehandgeometricandontheotherhandmorealgebraic.Belowtheequivalenceofbothisshown.Westartfromtheformulationofthetransformation(3)asaconditionS(x00i)Hx0i=0:(7)Withtheverticalvanishingpointv0=Kn(thexedpointofthetransformation)andthehorizonline(xedline)l0=K�Tn(HartleyandZisserman,2000)in(5)thecondition(7)leadstotheformuladevelopedin(Criminisi,2001)Hi=�Z l0Tx0ikS(x00i)x0ik kS(x00i)v0k:bytakingthenormofthecondition.Withthedirectionsmi=N(K�1x0i)thehomographyfordirectionsm00i=Mm0icanbeexpressedS(m00i)Mm0i=0andfortheobjectheightthesecondexpressionresultsHi=�Z nTm0ikS(m00i)m0ik kS(m00i)nk:(8)Thepositionoftheobjectontheplaneresultsfromsub-stitutingtheangulardistance0i=�Z=(nTm0i)fromtheprojectioncentertothefootpointX0intothepoint-direction-formX0i=(X0;Y0;Z0)Ti=Z+0iRTm0i(9)forwhichZ0i=0holds.Theformulas(8)and(9)providethebasisfortheobjectmeasurement.Thecalibrationprocedureisdescribedinthefollowingsection. 3REALISATION3.1CalibrationProcedureTheproposedcalibrationprocedureconsistsoftwostages:Aftertheinitialcalibrationwithobjectsofequalandknownheights,theparameterscanbecheckedand—ifneeded—updatedinthecontinuousoperationphasewithnewobjectsofunknownheight:(1)InitialCalibration.Aftertheinstallationofthecam-erathefootandheadpointsoftheobjectshavetobemea-sured.Dependingonthespeciccalibrationobjectthiscanbedonemanuallyorwiththehelpoffeatureextrac-tion.Theobservedheightsmaynotbearrangedonasin-glestraightlineintheobjectspace(cf.section3.3,de-terminabilityoftheparameters).Whiletheheightoftheobjectshastobeknown,theheightofthecameraZmaybeintroducedasanunknownparameteror—ifaccessible–asameasuredquantity.Approximationvaluesfortheunknownparametercanbedeterminedasdescribedbeloworbyaroughguess,e.g.fortherollanglezeroisalwaysagoodassumption.(2)ParameterUpdate.Forthecontinuousoperationweassumethattheheightofthecameradoesnotchange,whiletheotherparametersmayvaryduetoenvironmen-talinuences,forinstancetemperature.ForeverynewscenetanunknownheightHtisintroducedintothead-justmentprocedure.SincetheunknownobjectheightsHtcanvary,relinearisationwithfewiterationsisadvisableforeverynewscene—slightlyincreasingthecomputingtime.Atthesametimethemeasurementsyieldthepositionandheightoftheobjectsforeachimage.Furthermore,thead-justmentprovidestheaverageheightforeverytypeofob-ject.3.2ApproximationValuesMinimizingalgebraicdistances.Thetransformationparameterscanpossiblybedeterminedwithouttheknowl-edgeofapproximationvalues(HartleyandZisserman,2000).Withtheprojectivetransformationx00i=Hx0iwrit-teninhomogeneouscoordinates0@u00v00w001A=0@abcdefghi1A0@u0v0w01Awithx0=(u0;v0;w0)T=(x0b;y0b;1)Tandx00=(u00;v00;w00)T=(x0t;y0t;1)Twerstofallgetthecon-straintsbetweentheimagecoordinatesandthehomogra-phyelementsu00i(gu0i+hv0i+i)�w00i(au0i+bv0i+cw0i)=0(10)v00i(gu0i+hv0i+i)�w00i(du0i+ev0i+fw0i)=0:(11)IncompactformaT1ih=0andaT2ih=0withthe9-vectorsaT1i=(�w00ix0iT;0T;�u00ix0iT)aT2i=(0T;�w00ix0iT;v00ix0iT)andtheunknownparametersh=(a;b;c;d;e;f;g;h;i)TwegetthehomogeneousequationsystemAh=0.TherighteigenvectorofAforthesmallesteigenvaluelisagoodestimationforh.Withthesingularvaluedecomposi-tionA=UDVTthesolutionishk=Vkl;withk=1;:::;9(12)Fornumericalreasonsaconditioningoftheproblemisad-visable.Enforcingthehomologyconstraints.Theestimation(12)ofHdoesnotpossessthepropertiesofaplanarhomo-logypresentedinsection2.2.1.Therefore,aleastsquaresadjustmentcanbedoneassumingtheelementsh=vec(H)asi.i.d.observations.Theexplicitmodelofthisobserva-tionprocessreadsh=f(c; ; ;Z)with(0)hh=20I9(13)withtheaprioricovariancematrix(0)hhoftheobserva-tionsandtheunknownvariancefactor20.ThesolutionbHminimizestheFrobeniusnormkH�bHk.Approxima-tionvaluesaretakenfromthedecompositionexplainedinsection2.2.2.Althoughthesolutionbhfulllstheconstraintsofthepla-narhomology,itisstillanapproximationsincepotentialindividualweightsoftheobservationshavenotbeentakenintoconsideration.Therefore,asubsequentstringentad-justmentisnecessary.3.3ParameterEstimationDeterminabilityoftheParameters.Ifthepitchangle iszeroor90—i.e.theviewingdirectionishorizontalortowardsthenadir—theelementnZofthenormalvector(1)becomeszero.Inthiscasethe2D-homography(6)de-generatestoa1D-homographyandtheprincipaldistancecisnotdeterminable.Ifthepitchangleisapproximatezeroor,thedeterminationoftheparametersisveryweak.Inthiscasepriorinformationabouttheparametershastobeprovided.Thiscaneasilybedonebyintroducingthesevaluesasadditional,ctitiousobservationsintotheadjust-mentprocessexplainedinthefollowing.Onecriticalarrangementofthecalibratingobjectscanbeobserved:ifthefootandheadpointsintheimagearecollinear,thehomographydegeneratesandtheparametersarenotdeterminable.Thusnotallobjectsmaybesituatedonasinglestraightline.AdjustmentModel.Forthecalibrationphases(initialandupdate)thegeneralnon-linearmodelg(l;p)=0with(0)ll=20P�1ll(14)withtheconstraintsbetweentheobservationsl,theparam-eterspandtheaprioricovariancematrixoftheobserva-tions(0)llisarranged,cf.forinstance(Mikhail,1976).Theconstraintsofthemodelaretheeqs.(10)and(11).Fortechnicalconveniencewithp=(c; ; ;Z;H)Tve parametershavebeenintroducedalthoughjustthefractionH=Zisdeterminable.Dependingontheactualcalibrationphase(initialorupdate)eitherHorZhavetobexedbypriorinformation.Becauseoftheassumptionofi.i.d.observationgroupsthenormalequationsystemforthead-justmentmodel(14)canbebuilt-upsequentially.Tomakesure,thatthenecessarypriorinformationhasaconstantcontributiontothesolution,therelativeweightingbetweentheobservationsandthepriorinformationcanbecontrolledbyaregularizationfactor.Anad-hocsolutionis=tr(N)=tr(Ppp)(Pressetal.,1992)withthetracesofthenormalequationmatrixNandthepriorweightsPppforthe'observed'parameters.Again,aconditioningoftheproblemisadvisablebyatranslationandscalingoftheimagequantitiesandtheprincipaldistancerespectively.KalmanFilter.Thesequentialbuild-upofthenormalequationsystemoffersthepossibilityofintroducingadis-creteKalmanlter(WelchandBishop,2002)forthecali-brationupdatephase.Thisisequivalenttoarecursivepa-rameterestimationprocess.Topreventanumericalover-owandthesolutiontobite,amemorylengthtermkcanbeintroduced,whichcontrolstheamountofmemoryusedfortheactualsolution.Withk=0:9forinstance,90%ofthepastobservationswillbeusedatthepresenttime.Af-tereveryevaluationstepthenormalequationmatrix,theright-hand-sidevector,thesumofsquaredresidualsandthenumberofconditionshavetobeupdated.Thelatterbecomesreal-valuedwhichisasyetpracticallyirrelevant.TheparameterkmaynotaffecttheunknownobjectheightsHtasthisparametercanvaryfromscenetoscene.4EXPERIMENTALRESULTS4.1ObservationsandReferenceCalibrationObservations.Fortheevaluationoftheapproachanim-ageofalectureroomwasrecorded,showingaseatingar-rangementofchairsofindenticalheights(cf.g.3).Thecamerausedhasanimageformatof9601280pictureel-ements.Theimagemeasurementofthefootpointsofthechairlegsandthetoppointsofthechairbackswasdonebyanoperator.ReferenceCalibration.Fortheevaluationoftheap-proachareferencecalibrationhasbeencarriedoutfortheintrinsiccameraparametersaswellasfortheexteriorori-entation.Aftertherecordingoftheimageacalibrationeldhasim-mediatelybeencapturedonlocation.Theintrinsicparam-etersarethentakenfromabundleadjustment.Table1summarizestheresultsoftheparameterestimationfortheintrinsicparameters.Forthedeterminationoftheexteriorcameraorientationtheimagepointsrepresentingthecornersofthetableshavebeenmeasured.Togetherwiththeworldcoordinatesofthecorrespondingpoints0.74mabovethegroundplane Figure3:showstheobservedcorrespondingfootandheadpointsaswellastheestimatedhorizonline,itspointofgravityanditshyperbolicerrorband(3intervals).andtheinteriororientationgivenintable1aspatialresec-tionhasbeenaccomplishedassumingastandarddeviationof0.02mfortheobjectcoordinatesand2pelfortheim-agescoordinates.Fromtheestimatedmatrixfortherota-tionfromtheobjecttothecameracoordinatesystemtherollandpitchangleresultfrom(1)and(2).Theestimatedaccuraciesresultfromerrorpropagationandarelistedintable2.Theestimatedheightofthecameraabovegroundhasbeenveriedwiththehelpofameasuringtape.parameter estimation estim.std.dev. principaldist.c 1328.86pel 2.577pelscalefactorm 0.9962 3.37710�4principalpt.x00 -1.35pel 1.458pelprincipalpt.y00 -4.90pel 1.389pelTable1:summarizestheresultsfromtheintrinsiccameracalibrationwithatesteld.parameter estimation estim.std.dev. pitchangle 31.2324deg 0.4479degrollangle 0.4847deg 0.5341deg camerapositionX 3.0611m 0.0961mcamerapositionY -2.2095m 0.0397mcameraheightZ 2.5583m 0.0830mTable2:summarizestheresultsoftheexteriorreferencecalibration.4.2CalibrationResultsAheightofH=0:77mhavebeendeterminedforthechairsinthescene.Theresultsofthedirectsolution(12)andoftheconstrainedadvancementwith(13)aresumma-rizedintable3.Forthefollowingcalibrationspriorinformationhastobeusedinordertointroducemetricinformation.Forthe parameter directsol. constrained principaldist.c 1157.8pelpel 1160.5pelpitchangle +29.99deg +30.01degrollangle -2.52deg +0.08degcameraheightZ 2.49m 2.49mTable3:showstheresultsofthedirectsolutionanditsconstrainedadd-on.heightofthechairsH=0:77m,H=0:02mhasbeenintroduced.Table4summarizestheresultsoftheinitialcalibrationwitharedundancyof38.Thepro-cessconvergedafterfouriterations.Theestimatedfactorb0=3:75liesintheexpectedmagnitudefortheprecisionsoftheimagepoints.Figure3showstheresultsqualitative.Drawnintheimageistheestimatedhorizonlinewithitshyperbolicerrorband.Thepositionandorientationofthehorizonlinecanbeeasilycheckedbyvisualinspectionofthevanishinglines.parameter estim. est.std.dev. principaldist.c 1196.8pel 32.3pelpitchangle +29.37deg 0.48degrollangle -1.96deg 0.37degcameraheightZ 2.53m 0.05mTable4:showstheresultsoftheinitialcalibration. 4.3ObjectMeasurementTheobservedandmeasuredchairlegsareillustrateding.4inanuprightprojection,togetherwiththeprojectioncenter,thefootprintoftheprincipalpointandtheprojec-tionofanimageraster.Thepositionsandheightsofnew,unknownobjectscanbedeterminedby(8)and(9). Figure4:showsthefootprintsofaimagerasterandthepositions()ofthechairlegsonthegroundplane.5CONCLUSIONSANDOUTLOOKConclusions.Aneasycameracalibrationprocedurehasbeenpresentedfortheobservationofobjectsofequalheightsonagroundplane.Theprocedureusesaminimalparametrizationforthecameraitselfanditsexteriororien-tation.Feweffortsareassociatedwiththeinstallation;thefootandheadpointsoftheobjectsserveasobservations.Afteraninitializationphasewitharstscenetheapproachallowswithinthecontinuousoperationaparametercheckandupdateifnecessary.Forthecalibrationresultsthesin-gleparametervaluesarelessimportantthanthespecicparametercombination;thechangeofoneparametercantosomedegreebecompensatedbytheothers.Duetothesequentialbuild-upofthenormalequations,thedemandofstoragespaceisminimal.Fortheset-upofthecamerasystemapitchangle�20andalargeapertureangle(orsmallprincipaldistance)areadvisable.Otherwisepriorinformationhasbebeintroducedtocopewiththeweakgeometricconguration.Thepriorinformationguaranteesbutalsodominatesthesolution.Theheightofthecamerashouldbemeasuredwhereverpossibleinordertoimposemoregeometricconstraintsontothesolution.Outlook.Inordertoeliminatetheinuenceofgrossob-servationalerrors,arobustestimationisdesirable.Fur-thermore,theintegrationofothereasilyavailablemeasure-ments—suchasdistancesintheobjectspace—isadvan-tageous,dependingonthepreciselocationtoberecorded.REFERENCESCriminisi,A.,2001.AccurateVisualMetrologyfromSin-gleandMultipleUncalibratedImages.DistinguishedDis-sertations,Springer,London,Berlin,Heidelberg.Faugeras,O.andLustman,F.,1988.MotionandStructurefromMotioninapiecewiseplanarEnvironment.Interna-tionalJournalofPatternRecognitioninArticialIntelli-gence2,pp.485–508.Hartley,R.andZisserman,A.,2000.MultipleViewGe-ometryinComputerVision.CambridgeUniversityPress,Cambridge.Jones,G.A.,Renno,J.andRemagnino,P.,2002.Auto-CalibrationinMultiple-CameraSurveillanceEnvi-ronments.In:3rdIEEEWorkshoponPerformanceEvalua-tionofTrackingandSurveillance(PETS02),Copenhagen,pp.40–47.Mikhail,E.M.,1976.ObservationsandLeastSquares.WithContributionsbyF.Ackerman.UniversityPressofAmerica,Lanham.Press,W.H.,Teukolsky,S.A.,Vetterling,W.T.andFlan-nery,B.P.,1992.NumericalRecipiesinC.CambridgeUniversityPress.Renno,J.,Orwell,J.andJones,G.,2002.Learn-ingSurveillanceTrackingModelsfortheSelf-CalibratedGroundPlane.In:BritishMachineVisionConference.PosterSession.Semple,J.G.andKneebone,G.T.,1952.AlgebraicPro-jectiveGeometry.OxfordUniv.Press,NewYork.Welch,G.andBishop,G.,2002.AnIntroductiontotheKalmanFilter.TechnicalReportTR95-041,DepartmentofComputerScience,Univ.ofNorthCarolinaatChapelHill. -3-2-10123012345678X [m]Y [m]