An AsShortAsPossible Introduction to the Least Squares Weighted Least Squares an - PDF document

Figure1:Fittingbivariate,quadraticpolynomialsto2Dscalarelds:thetoprowshowsthetwosetsofninedatapoints(seetext),thebottomrowshowstheleastsquarestfunction.Thecoefcientvectors[c1;:::;c6]Tare[�0:834;�0:25;0:75;0:25;0:375;0:75]T(leftcolumn)and[0:334;0:167;0:0;�0:5;0:5;0:0]T.MethodofNormalEquations.Foradifferentbutalsoverycom-monnotation,notethatthesolutionforcinEqn.(3)solvesthefollowing(generallyover-constrained)LSE(Bc=f)intheleast-squaressense264bT(x1)...bT(xN)375c=264f1...fN375;(5)usingthemethodofnormalequationsBTBc=BTfc=(BTB)�1BTf:(6)PleaseverifythatEqns.(4)and(6)areidentical.2WLSApproximationProblemFormulation.Intheweightedleastsquaresformulation,weusetheerrorfunctionalJWLS=åiq(k x�xik)kf(xi)�fik2foraxedpoint x2Rd,whichweminimizeminf2Õdmåiq(k x�xik)kf(xi)�fik2;(7)similarto(1),onlythatnowtheerrorisweightedbyq(d)wherediaretheEuclidiandistancesbetween xandthepositionsofdatapointsxi.Theunknowncoefcientswewishtoobtainfromthesolutionto(7)areweightedbydistanceto xandthereforeafunctionof x.Thus,thelocal,weightedleastsquaresapproximationin xiswrittenasf x(x)=b(x)Tc( x)=b(x)
c( x);(8)andonlydenedlocallywithinadistanceRaround x,i.e.kx� xkR.WeightingFunction.Manychoicesfortheweightingfunctionqhavebeenproposedintheliterature,suchasaGaussianq(d)=e�d2 h2;(9)wherehisaspacingparameterwhichcanbeusedtosmoothoutsmallfeaturesinthedata,see[Levin2003;Alexaetal.2003].AnotherpopularweightingfunctionwithcompactsupportistheWendlandfunction[Wendland1995]q(d)=(1�d=h)4(4d=h+1):(10)Thisfunctioniswelldenedontheintervald2[0;h]andfurther-more,q(0)=1,q(h)=0,q0(h)=0andq00(h)=0(C2continuity).Severalauthorssuggestusingweightingfunctionsoftheformq(d)=1 d2+e2:(11)Notethatsettingtheparameteretozeroresultsinasingularityatd=0,whichforcestheMLStfunctiontointerpolatethedata,aswewillseelater.Solution.AnalogoustoSection1,wetakepartialderivativesoftheerrorfunctionalJWLSwithrespecttotheunknowncoefcientsc( x)åiq(di)2b(xi)[b(xi)Tc( x)�fi]=2åi[q(di)b(xi)b(xi)Tc( x)�q(di)b(xi)fi]=0;wheredi=k x�xik.Wedividebytheconstantandrearrangetoobtainåiq(di)b(xi)b(xi)Tc( x)=åiq(di)b(xi)fi;(12)andsolveforthecoefcientsc( x)=[åiq(di)b(xi)b(xi)T]�1åiq(di)b(xi)fi:(13)Obviously,theonlydifferencebetweenEqns.(4)and(13)aretheweightingterms.Noteagainthough,thatwhereasthecoef-cientscinEqn.(4)areglobal,thecoefcientsc( x)arelocalandneedtoberecomputedforevery x.IfthesquarematrixAWLS=åiq(di)b(xi)b(xi)T(oftentermedtheMomentMatrix)isnonsingular(i.e.det(AWLS)6=0),substitutingEqn.(13)intoEqn.(8)providesthetfunctionf x(x).GlobalApproximationusingaPartitionofUnity(PU).Byt-tingpolynomialsatj2[1:::n]discrete,xedpoints xjinthepa-rameterdomainW,wecanassembleaglobalapproximationtoourdatabyensuringthateverypointinWiscoveredbyatleastoneapproximatingpolynomial,i.e.thesupportoftheweightfunctionsqjcenteredatthepoints xjcoversWW=[jsupp(qj):Properweightingoftheseapproximationscanbeachievedbycon-structingaPartitionofUnity(PU)fromtheqj[Shepard1968]jj(x)=qj(x) ånk=1qk(x);(14)whereåjjj(x)1everywhereinW.Theglobalapproximationthenbecomesf(x)=åjjj(x)b(x)Tc( xj):(15) ANumericalIssue.Toavoidnumericalinstabilitiesduetopossi-blylargenumbersinAWLSitcanbebenecialtoperformthettingprocedureinalocalcoordinatesystemrelativeto x,i.e.toshift xintotheorigin.Wethereforerewritethelocaltfunctionin xasf x(x)=b(x� x)Tc( x)=b(x� x)
c( x);(16)theassociatedcoefcientsasc( x)=[åiq(di)b(xi� x)b(xi� x)T]�1åiq(di)b(xi� x)fi;(17)andtheglobalapproximationasf(x)=åjjj(x)b(x� xj)Tc( xj):(18)3MLSApproximationandInterpolationMethod.TheMLSmethodwasproposedbyLancasterandSalka-uskas[LancasterandSalkauskas1981]forsmoothingandinterpo-latingdata.Theideaistostartwithaweightedleastsquaresfor-mulationforanarbitraryxedpointinRd,seeSection2,andthenmovethispointovertheentireparameterdomain,whereaweightedleastsquarestiscomputedandevaluatedforeachpointindividu-ally.Itcanbeshownthattheglobalfunctionf(x),obtainedfromasetoflocalfunctionsf(x)=fx(x);minfx2Õdmåiq(kx�xik)kfx(xi)�fik2(19)iscontinuouslydifferentiableifandonlyiftheweightingfunctioniscontinuouslydifferentiable,seeLevinswork[Levin1998;Levin2003].SoinsteadofconstructingtheglobalapproximationusingEqn.(15),weuseEqns.(8)and(13)(or(16)and(17))andcon-structandevaluatealocalpolynomialtcontinuouslyovertheen-tiredomainW,resultingintheMLStfunction.Aspreviouslyhintedat,using(11)astheweightingfunctionwithaverysmalleassignsweightsclosetoinnityneartheinputdatapoints,forcingtheMLStfunctiontointerpolatetheprescribedfunctionvaluesinthesepoints.Therefore,byvaryingewecandirectlyinuencetheapproximatimg/interpolatingnatureoftheMLStfunction.4ApplicationsLeastSquares,WeightedLeastSquaresandMovingLeastSquares,havebecomewidespreadandverypowerfultoolsinComputerGraphics.Theyhavebeensuccessfullyappliedtosurfacerecon-structionfrompoints[Alexaetal.2003]andotherpointsetsurfacedenitions[AmentaandKil2004],interpolatingandapproximatingimplicitsurfaces[Shenetal.2004],simulating[Belytschkoetal.1996]andanimating[M¨ulleretal.2004]elastoplasticmaterials,PartitionofUnityimplicits[Ohtakeetal.2003],andmanyotherresearchareas.In[Alexaetal.2003]apoint-set,possiblyacquiredfroma3Dscanningdeviceandthereforenoisy,isreplacedbyarepresenta-tionpointsetderivedfromtheMLSsurfacedenedbytheinputpoint-set.Thisisachievedbydown-sampling(i.e.iterativelyre-movingpointswhichhavelittlecontributiontotheshapeofthesurface)orup-sampling(i.e.addingpointsandprojectingthemtotheMLSsurfacewherepoint-densityislow).Theprojectionproce-durehasrecentlybeenaugmentedandfurtheranalyzedintheworkofAmentaandKil[AmentaandKil2004].Shenet.al[Shenetal.2004]useanMLSformulationtoderiveimplicitfunctionsfrompolygonsoup.Insteadofsolelyusingvalueconstraintsatpoints(asshowninthisreport)theyalsoaddvalueconstraintsintegratedoverpolygonsandnormalconstraints.ReferencesALEXA,M.,BEHR,J.,COHEN-OR,D.,FLEISHMAN,S.,LEVIN,D.,ANDT.SILVA,C.2003.Computingandrenderingpointsetsurfaces.IEEETransactionsonVisualizationandComputerGraphics9,1,3–15.AMENTA,N.,ANDKIL,Y.2004.Deningpoint-setsurfaces.InProceed-ginsofACMSIGGRAPH2004.BELYTSCHKO,T.,KRONGAUZ,Y.,ORGAN,D.,FLEMING,M.,ANDKRYSL,P.1996.Meshlessmethods:Anoverviewandrecentdevel-opments.ComputerMethodsinAppliedMechanicsandEngineering139,3,3–47.FRIES,T.-P.,ANDMATTHIES,H.G.2003.Classicationandoverviewofmeshfreemethods.Tech.rep.,TUBrunswick,GermanyNr.2003-03.LANCASTER,P.,ANDSALKAUSKAS,K.1981.Surfacesgeneratedbymovingleastsquaresmethods.MathematicsofComputation87,141–158.LEVIN,D.1998.Theapproximationpowerofmovingleast-squares.Math.Comp.67,224,1517–1531.LEVIN,D.,2003.Mesh-independentsurfaceinterpolation,toappearin'ge-ometricmodelingforscienticvisualization'editedbybrunnett,hamannandmueller,springer-verlag.M¨ULLER,M.,KEISER,R.,NEALEN,A.,PAULY,M.,GROSS,M.,ANDALEXA,M.2004.Pointbasedanimationofelastic,plasticandmelt-ingobjects.InProceedingsof2004ACMSIGGRAPHSymposiumonComputerAnimation.OHTAKE,Y.,BELYAEV,A.,ALEXA,M.,TURK,G.,ANDSEIDEL,H.-P.2003.Multi-levelpartitionofunityimplicits.ACMTrans.Graph.22,3,463–470.PRESS,W.,TEUKOLSKY,S.,VETTERLING,W.,ANDFLANNERY,B.1992.NumericalRecipesinC-TheArtofScienticComputing,2nded.CambridgeUniversityPress.SHEN,C.,O'BRIEN,J.F.,ANDSHEWCHUK,J.R.2004.Interpolatingandapproximatingimplicitsurfacesfrompolygonsoup.InProceedingsofACMSIGGRAPH2004,ACMPress.SHEPARD,D.1968.Atwo-dimensionalfunctionforirregularlyspaceddata.InProc.ACMNat.Conf.,517–524.WENDLAND,H.1995.Piecewisepolynomial,positivedeniteandcom-pactlysupportedradialbasisfunctionsofminimaldegree.AdvancesinComputationalMathematics4,389–396. Figure2:TheMLSsurfaceofapoint-setwithvaryingdensity(thedensityisreducedalongtheverticalaxisfromtoptobottom).ThesurfaceisobtainedbyapplyingtheprojectionoperationdescribedbyAlexaet.al.[2003].ImagecourtesyofMarcAlexa.