Circulant/SkewcirculantMatricesasPreconditionersforHermitianToeplitzSy - PDF document

Circulant/SkewcirculantMatricesasPreconditionersforHermitianToeplitzSystemsThomasHuckleInstitutf¨urAngewandteMathematikundStatistikUniversit¨atW¨D-8700W¨urzburg,F.R.G.Abstract.WestudythesolutionsofHermitianpositivede“niteToeplitzsystemsbythepreconditionedconjugategradientmethod.Forpreconditionertheconvergencerateisknowntobegovernedbythedistributionoftheeigenvaluesofthepreconditioned.NewpropertiesofthecirculantpreconditionersintroducedbyStrang,R.Chan,T.Chan,Szeg¨o/GrenanderandTyrtyshnikovarederivedconcerningthepositivede“nitenessofandthespectrumof.Furthermore,weintroduceanewclassofToeplitzmatrices,similartotheWienerclass.Forthisclassweconsidernewprecon-ditionersasproductsofcirculantandskewcirculantmatrices,thatarebestapproximationsofintheFrobeniusnorm,andstudythespectraofthepreconditionedmatrices.KeyWords.Toeplitzmatrix,circulantmatrix,preconditionedconjugategradientmethod.AMS(MOS)SubjectClassiÞcations. Circulant/SkewcirculantMatricesasPreconditionersforHermitianToeplitzSystems0.IntroductionSystemsoflinearequationswithHermitianpositivede“niteToeplitzmatrices,...,t):=············ariseinmanyapplicationssuchastimeseriesanalysisanddigitalsignalprocessing.Directfastandsuperfastmethodsforsolving(1)arebasedonSchuralgorithmorSzeg¨orecursionandneed),resp.)),operations[15,18,10,1,2].Morerecently,Strang[17]proposedtousethepreconditionedconjugategradientmethod,andsincethattime,thedesignofeectivepreconditionershasreceivedmuchattention[3,4,19,5].Thepcg-methodrequiresthatispositivede“nite,linearequationsarefastcomputable,andthespectrumofshouldbeclusteredaround1.First,circulantmatricesareusedaspreconditioners[6,7,20],andlateralsoskewcirculantmatrices[11,12].In[13,14],KuoandKuintroducednewaproximationstowhichareHankelplusToeplitzmatricesandsubmatricesofcirculantmatrices.Linearequationswithcirculantmatrices[8]canbesolvedbyFastFourierTransformationin)).Thisisaconsequenceoftheeigendecompositionwithadiagonalmatrixand  ikj/nk,j =1  1111)(i/ntheFouriermatrixoforder.Thereby,iscomplexsymmetric,unitary,andful“llstheequation.Hence,theeigenvaluesofagivencirculantmatrix(3)are1(6)isthepolynomialassociatedwith.Thecorrespondingeigenvectorsare,...,nWiththenotation(2)forHermitiancirculant,(3)canbewrittenintheform,...,c,...,c)foreven,...,c,...,c)foroddSkewcirculantmatricesdierfromcirculantmatricesbyachangeofthesigninthesubdiagonalelements.Thus,skewcirculantmatricesareoftheform···ŠUsingtheunitarydiagonalmatrix:=diag(1,,i/niscirculantforeveryskewcirculantmatrix.Becauseof(4)and(5),theeigenvaluesandeigenvetorsofaregivenby,w,...,n InconnectionwithToeplitzmatricesitisoftenassumed,thattheHermitainmatricesare“nitesectionsofa“xedsinglyin“niteToeplitzmatrix,...).Then,everymatrixisassociatedwithafunctionik,2];fissaidtobeintheWienerclassifthesequence,isin,andtherefore0,thenispositivede“nte,andthusthesameholdsforall,...ThepreconditionersofStrangandofSzeg¬o/GrenanderForagivenToeplitzmatrixStrang[17]proposedthecirculantmatrix,...,aspreconditionerfor(1).isobtainedbypreservingthecentralhalfdiagonalsofusingthemtoformacirculantmatrix.Inthesamewayyougettheskewcirculantmatrix,...,IthasbeenshownbyR.ChanandStrang[6],thatthespectrumofisclusteredaround1,excepta“nitenumberofoutliers,forToeplitzmatricesgeneratedbypositivefunctionsintheWienerclass.Thesameistruefor[11].Intuitively,itisclearthatforanearcirculantmatrixonechosesaspreconditionerandforanearskewcirculant.Togetabettercriterionletusconsideranotherelemtaryproof[4]fortheclusteringpropertyofthespectrumof,withbelongingtoapositivefunctionintheWienerclass.Letusassume.ThenthereexistsanintegersuchthatforagivenitholdsSet()thematrixthatyougetbyremovingthelastcolumnsandrowsofBecauseofallrowsumsof()areboundedby.Then,theTheoremofGershgorinstatesthattheeigenvaluesof()lieintheinterval[,].()isaprincipalsubmatrixof andthus,inviewoftheinterlaceproperty[16],the-neighbourhoodof0containsatleasteigenvaluesof.Note,thatisindependof,andhence,theeigenvaluesofareclusteredaround1forlargeFortheskewcirculantmatrixinsteadof(10)wegetNow,(10)and(10)giveacriterionforchoosingthepreconditioner.Forgivenlet)bethemaximalindexwith,resp..Then,atleasteigenvaluesofliein[,]andatleasteigenvaluesof.Hence,weuseaspreconditionerifforasmall.If,forexample,allhavethesamesign,thentheleftsideof(10)issmallerthentheleftsidein(10)andthusleadstoabetterclustering.In[9]Szeg¨oandGrenanderintroducedanotherclassofcirculantapproximationsto.For1 ,...,nthediagonalmatrixwithelements,and.Inordertoanalysetheconvergenceofthesequencefor,theyconsideredthenorm nn jj(Tn)2=1 denotestheFrobeniusnormof.ForthecirculantmatricesweproveTheorem1.TheeigenvaluesoffulÞlltheinequalitiesminminminmaxmaxmaxProof:Itholds=(1,w,...,wHence,theeigenvaluesofarecontainedintherangeoftheleadingprincipalsubmatrixRemarks:1.Theorem1guaranteesthatforpositivede“nitetheapproximationsarepositivede“nite,too.Therefore,canbeusedaspreconditioner.withitseigenvalues(11)isdierentfromthebestFrobeniusnormapproximation ,...,t,...,0).From(6)itfollowsthattheeigenvaluesofare(seealso(13)inthenextsection) ,...,n3.Inviewof(7)-(9),thecorrespondingskewcirculantapproximationisgivenby2.TheoptimalandsuperoptimalFrobeniusnormapproximationTheoptimalcirculantpreconditionerwas“rstconsideredbyT.Chan[7].de“nedasthebestcirculantapproximationtointheFrobeniusnormandthus,isgiven ,..., Here,wewillconsideramoregeneralapproachtothisapproximationproblem.ForagivenunitarymatrixwesetdiagonalmatrixoverThenitholds(see[5]forthecirculantcaseTheorem2.Foramatrixthesolutionofisgivenbydiag(UAUisHermitianthentheeigenvaluesoflieintheintervallminmaxFurthermore,theoperatorhasthefollowingproperties:isalinearmappingoftheBanachalgebraofcomplexmatricesinthesubalgebralub)=lubforallmatricesProof:WiththenotationUAU,foranelementtheexpressionUAU takesitsminimalvalueifisthediagonalofFurthermore,itholdslub)=maxdiag( i,imax maxlub)=maxdiag( .Thereby,maxdenotesthemaximalsingularvalueofAsaCorollaryofTheorem2weget,that,foragivenHermitianToeplitzmatrixof(13)isequaltodiag(andtheskewcirculantapproximationful“llsdiag(andde“nedin(5)and(8).Inaddition,fromTheorem2itisobvious,thatforpositivede“nitearepositivedeinite,too.ThesameanalysiscanbeappliedtoTyrtyshnikovssuperoptimalpreconditioner[20,5]thesolutionofoverallcirculantmatricesWiththesetde“nedasabove,wecanproveTheorem3.(a)Foragivennonsingularmatrixthesolutionofisgivenby):=Theeigenvaluesofcanbeorderedsuchthat))=))=0Furthermore,itholds(b)ForanHermitianpositivedeÞniteisnonsingular.Therefore,wecandeÞne):= Then,theeigenvaluesfulÞllminminmaxandwithanappropiatenumberingProof:UAU,...,b.Then,thesolutionof=minisgivenby=diag(,...,)andi,i Becauseof=diag(diag(itfollowsdiag()diag(UAATheeigenvaluesof)arethenumbersi,i.Hence,(14)showsthateitheri,i=0or i,iisHermitianpositivede“nite,thenhasaorthonormalbasisofeigenvectorsandthus,and=diag(,...,0.Therefore,itholds))=i,i i,i qHiqi=nj2j|q2 Thisprovesminmax))=i,i i,ii,iRemarks:1.Part(a)ofTheorem3remainstrueifwereplacetheassumptionthatAisnonsingularbythecondition=0foreachrow,becausethenitalsofollowsIfthisconditionisnotful“lled,thenthesolutionisnomoreunique. 2.Theoperatorisnotlinear.Moreover,theeigenvaluesof)aregiven+2Re(i,i i,iIfforanHermitianinde“nitematrixthereexistsanumberi,i i,iforalli,ithenforthisthematrixispositivede“nite.Wecanassume,thatwith)alsowillbeagoodapproximationtotheidentitymatrix.Hence,for“xedthematrix(15)seemstobeagoodchoiceforapreconditionerintheinde“nitecase.AsadirectconclusionfromTheorem3wegetCorollary1.ForanHermitianpositivedeÞniteToeplitzmatrixthesuperoptimalcirculant,resp.skewcirculant,Frobeniusnormapproximationhastheform)andWithanappropriatenumberingtheeigenvaluesoffulÞll)andminminmin)andmaxmaxmaxminminmin)andmaxmaxmaxHence,forpositivedeÞnitearepositivedeÞnite,too.3.NewclassesofoptimalcirculantapproximationsIn[9]Szeg¨oandGrenanderprovedthatfor,associatedwithan-functionde“nedonthecirclewithradius1,theapproximations(11)convergetotendtoin“nitywithrespecttothenorm(12).Thisisalsoobviousforthecirculantandskewcirculantapproximations,and.Theconvergenceinthenorm(12)isequivalenttothefollowingproperty:Foreverygivenpositivethenumbers,n)ofeigenvalues)withabsolutevaluesexceedingful“ll,n 0for IfwestrengthentheconditionsonandconsiderToeplitzmatricesassociatedwithfunctionsoutoftheWienerclass,thenStrangandR.Chanshowed,thattheeigenvaluesareasymptoticlyclusteredaround0andtherefore,,n)isboundedbyaconstanttLetusintroduceanewclassbyde“ningThenitiseasytoverifythat-testexamplesthatappearintheliteratureoncirculantpreconditionersareelements,too.Forassociatedwithafunctionfrom(16)itfollowsHencetheeigenvaluesofareclusteredaround0for.Thisgivesanelementaryprooffortheclusteringpropertyofforatmostallknowncirculantandskewcirculantpreconditioners.ForthefollowingwewillassumethatisassociatedwithafunctionInordertoreducethenumberofiterationsinthepcg-algorithm,itseemstobeworth-wile,tospendmorecomputationaleortin“ndingbetterpreconditioners.Oneapproachistheuseofapproximationsthataresubmatricesofcirculant2matrices[13,14].Here,wewillproposeotherapproximationsthatusebothcirculantandskewcirculantma-trices.Togetasymmetricpositivede“niteapproximationofapositivede“niteToeplitzwewillconsiderWecan“ndanoptimalchoiceofbythefollowingLemma1.beasequenceofHermitianpositivedeÞniteToeplitzmatricesthatfulÞll(16).Then,withitholdsthatalsotheeigenvaluesofareclusteredaround0forProof:Letusconsideronlythenontrivialcase=0.Withthenotation,...,u)weonlyhavetoshowthat isboundedfor.Inviewofi,j=spur()=spur(i,ji,jitholdsi,ji,ji,ji,ji,jInthesamewaywegeti,ji,jNow,arecontainedintheinterval[minmax)].Therefore,itexistsapositivenumberfori,j,...,n.Hence,itfollowsForallpreconditioner(17)wegetthathasthesamespectrumasHence,inviewofLemma1andtheTheoremofCourant-Fischer[16],theeigenvaluesareingeneralclusteredaround1onlyif2=1.Thus,withoutfurtherinformationtheoptimalchoiceis2and4,becausethenthecirculantandtheskewcirculantparthavethesameweight.Allinall,weget Theorem4.beanHermitianpositivedeÞniteToeplitzmatrix,associatedwithafunction,andthebestFrobeniusnormapproximationsto.ThenthepreconditionerisHermitianpositivedeÞnite,uniformlybounded,andthespectrumofisasymp-toticlyclusteredaround1.Remarks:1.Inthesamewayonecande“neapreconditioner,orreplacebyothercirculant/skewcirculantapproximations,e.g.by2.Ineverystepofthepreconditionedconjugategradientmethodthenumberofoperationsdoublesifweusetheapproximation(18)insteadof.Ineachiterationwehaveinadditiontosolveonecirculantandoneskewcirculantlinearequation.Inordertoimprovetheapproximation,wecanreplace(18)bysolvesanewFrobeniusnormapproximationproblem.Apossiblechoice,forex-ample,is=minskewcirculantThesolutionof(20)isgivenbyandispositivede“niteanduniformlyboundedforintheclass(16).de“nedin(20)hasasmallerFrobeniusnormthentheapproximation(18).Hence,thespectrumofisagainclusteredaround1.Similarapproximationsoftheform(19)wegetbysolvingskewcirculantskewcirculantskewcirculantThesolutionsof(20)-(23)canbecomputedexplicitely,butthenumberofoperationsforthatisingeneralmuchtolargeincomparisonwiththecaseofusing.Hence,thepreconditioner(18)seemstobethemostcompetetiveoneinthisclassofapproximations,becauseonehastocomputeonlyboth 4.NumericalexamplesForcomparingthedierentpreconditionersweconsiderthefollowingexamples:+1)for,...,n1[17,7].=cos(+1),...,n1[7];theassociatedfunctionisneitherin=2and (1+for,...,n1[4,5].First,forexample1and2wewilldisplaythespectrumoffortheprecondi-tioners(18)and(20)-(23).Thereby,1)denotes,and5denote(18),Fig.1.Example1with=16 Fig.2.Example2with=16Thefollowingtablesdisplaythenumberofiterationsinthepreconditionedconjugategradientmethodforsolvingapositivede“nteToeplitzsystem.Thereby,thestartvector,andthealgorithmstops,iftheresiduumissmallerthan10 Example2 =16 =32 =64 =128 =256 F 7 8 8 9 9 SF 7 8 8 8 9 C1/4FS1/2FC1/4F 6 6 6 6 6 6 6 6 6 Table1.Iterationsofthepcg-algorithmforexample2 Example1 =16 =32 =64 =128 =256 F 6 5 6 6 6 SF 6 6 6 6 6 C1/4FS1/2FC1/4F 5 5 4 4 4 Table2.Iterationsofthepcg-algorithmforexample1 Example3 =16 =32 =64 =128 =256 F 7 7 7 7 7 SF 7 8 7 7 8 C1/4FS1/2FC1/4F 7 6 6 6 6 Table3.Iterationsofthepcg-algorithmforexample3Thefollowingtableshowsforexample2with=16thebehaviourofthepcg-algorithmforallpreconditionersintroducedintheprevioussection. : CF SF Numberofiterations: 7 C1/4FS1/2FC1/4F (21) (22) (23) 5 5 5 5 Table4.Iterationsofthepcg-algorithmforexample2ThisexamplesshowthatpreconditionersoftheformCSCreducethenumberofiterationsinthepcg-algorithm.Theadvantageislargestiftheassociatedfunctionisnotin [1]Ammar,G.S.,Gragg,W.B.:Superfastsolutionofrealpositivede“niteToeplitzsystems,SIAMJ.Anal.Appl.9,61-76,1988.[2]Ammar,G.S.,Gragg,W.B.:NumericalexperiencewithasuperfastrealToeplitzsolver,LinearAlg.Appl.121,185-206,1989.[3]Chan,R.:ThespectrumofafamilyofcirculantpreconditionedToeplitzsystems,SIAMJ.Numer.Anal.26(2),503-506,1989.[4]Chan,R.:CirculantpreconditionersforHermitianToeplitzsystems,SIAMJ.MatrixAnal.Appl.10(4),542-550,1989.[5]Chan,R.,Jin,Xiao-Qing,Yueng,Man-Chung:ThespectraofsuperoptimalcirculantpreconditionedToeplitzsystems,toappear.[6]Chan,R.,Strang,G.:Toeplitzequationsbyconjugategradientswithcirculantprecon-ditioner,SIAMJ.Sci.Stat.Comput.10,104-119,1989.[7]Chan,T.:AnoptimalcirculantpreconditionerforToeplitzsystems,SIAMJ.Sci.Stat.Comput.9(4),766-771,1988.[8]Davis,P.J.:CirculantMatrices,JohnWiley,NewYork[9]Grenander,U.,Szeg¨o,G.:ToeplitzFormsandtheirApplications,UniversityPress,Cambridge[10]deHoog,F.:AnewalgorithmforsolvingToeplitzsystemsofequations,LinearAlg.Appl.88/89,123-138,1987.[11]Huckle,T.K.:CirculantandskewcirculantmatricesforsolvingToeplitzmatrixprob-lems,inCopperMountainConferenceonIterativeMethods,CopperMountain,Col-orado,1990[12]Huckle,T.K.:Krylovmethodenf¨urnormaleMatrizenundToeplitzmatrizen,Habilita-tionsschrift,Universit¨atW¨urzburg,Juli1990[13]Ku,T.,Kuo,C.:DesignandanalysisofToeplitzpreconditioners,Tech.Rep.155,USC,SignalandImageProcessingInstitute,May1990.ToappearinIEEETrans.Acoust.,Speech,SignalProcessing.[14]Ku,T.,Kuo,C.:SpectralPropertiesofPreconditionedRationalToeplitzmatricesTech.Rep.163,USC,SignalandImageProcessingInstitute,September1990.[15]Levinson,N.:TheWienerRMS(rootmeansquare)errorcriterionin“lterdesignandprediction,J.Math.Phys.25,261-278,1947.[16]Parlett,B.N.:TheSymmetricEigenvalueProblem,Prentice-Hall,Inc.,EnglewoodClis,N.J.[17]Strang,G.:AproposalforToeplitzmatrixcomputations,StudiesinAppliedMathe-matics74,171-176,1986.[18]Trench,W.:Analgorithmfortheinversionof“niteToeplitzmatrices,SIAMJ.Appl.Math.12,515-522,1964.[19]Trefethen,L.N.:Approximationtheoryandnumericallinearalgebra,toappear.[20]Tyrtyshnikov,E.:Optimalandsuper-optimalcirculantpreconditioners,toappear.