Chapter2NewDemiclosednessPrinciplesfor(Firmly)NonexpansiveOperatorsHeinzH.BauschkeDedicatedtoJonathanBorweinontheoccasionofhis60thbirthdayAbstractThedemiclosednessprincipleisoneofthekeytoolsinnonlinearanalysisand - PDF document

Chapter2NewDemiclosednessPrinciplesfor(Firmly)NonexpansiveOperatorsHeinzH.BauschkeDedicatedtoJonathanBorweinontheoccasionofhis60thbirthdayAbstractThedemiclosednessprincipleisoneofthekeytoolsinnonlinearanalysisandÞxedpointtheory.Inthisnote,thisprincipleisextendedandmademoreßexiblebytwomutuallyorthogonalafÞnesubspaces.VersionsforÞnitelymany(Þrmly)nonexpansiveoperatorsarepresented.Asanapplication,asimpleproofoftheweakconvergenceoftheDouglas-Rachfordsplittingalgorithmisprovided.Keywords:Demiclosednessprinciple¥Douglas-Rachfordalgorithm¥Firmlynonexpansivemapping¥Maximalmonotoneoperator¥Nonexpansivemapping¥Proximalalgorithm¥Resolvent¥SplittingalgorithmMathematicsSubjectClassiÞcations(2010):Primary47H05,47H09;Secondary47J25,49M27,65J15,65K05,65K15,90C252.1IntroductionThroughoutthispaper,weassumethatisarealHilbertspacewithinnerproductandinducednorm·.(2.1) OMMUNICATEDBYICHELERAH.H.Bauschke(DepartmentofMathematics,UniversityofBritishColumbia,Kelowna,BCV1V1V7,Canadae-mail:heinz.bauschke@ubc.caD.H.Baileyetal.(eds.),ComputationalandAnalyticalMathematics,SpringerProceedingsinMathematics&Statistics50,DOI10.1007/978-1-4614-7621-4 ©SpringerScience+BusinessMediaNewYork2013 H.H.BauschkeWeshallassumebasicnotationandresultsfromÞxedpointtheoryandfrommonotoneoperatortheory;see,e.g.,[].Thegraphamaximallymonotoneoperatorisdenotedbygra,itsresolvent,itssetofzerosbyzer,andwesetId,whereIdistheidentityoperator.Weakconvergenceisindicatedby.RecallthatÞrmlynonexpansiveItiswellknowthatisÞrmlynonexpansiveifandonlyifIdisnonexpansive,i.e.,Clearly,everyÞrmlynonexpansiveoperatorisnonexpansive.BuildingonworkbyMinty[],EcksteinandBertsekas[]clearlylinkedÞrmlynonexpansivemappingstomaximallymonotoneoperatorsÑthekeyresultisthefollowing:Þrmlynonexpansiveifandonlyifforsomemaximallymonotoneoperator(namely,Id).Thisimpliesalsoacorrespondencebetweenmaximallymonotoneoperatorsandnonexpansivemappings(see[]).Thus,ÞndingazeroofisequivalenttoÞndingaÞxedpointof.Furthermore,thegraphofanymaximallymonotoneoperatorisbeautifullydescribedbytheassociatedMintyparametrizationThemostprominentexampleofÞrmlynonexpansivemappingsareprojectors,i.e.,resolventsofnormalconeoperatorsassociatedwithnonemptyclosedconvexsubsetsof.Despitebeing(Þrmly)nonexpansiveandhenceLipschitzcontinuous,evenprojectorsdonotinteractwellwiththeweaktopologyaswasÞrstobservedbyZarantonello[Example2.1.Supposethat,set,anddenotethesequenceofstandardunitvectorsin.Set.Then Thefollowingclassicaldemiclosednessprincipledatesbacktothe1960sandworkbyBrowder[].Itcomessomewhatasasurpriseinviewofthepreviousexample.Fact2.2(Demiclosednessprinciple).beanonemptyclosedconvexsubset,letbenonexpansive,letbeasequenceinconvergingweaklyto,andsupposethat.Then 2NewDemiclosednessPrinciplesfor(Firmly)NonexpansiveOperators21Remark2.3.Onemightinquirewhetherornotthefollowingevenlessrestrictivedemiclosednessprincipleholds:However,thisgeneralizationisfalse:indeed,supposethat,andasinExample,andset,whichis(evenÞrmly)nonexpansive.Then 2e00Š=PCe0=e0 1 Theaimofthisnoteistoprovidenewversionsofthedemiclosednessprincipleandillustratetheirusefulness.Theremainderofthispaperisorganizedasfollows.Sectionpresentsnewdemiclosednessprinciplesforone(Þrmly)nonexpansiveoperator.Multi-operatorversionsareprovidedinSect..TheweakconvergenceoftheDouglas-Rachfordalgorithmisrederivedwithaverytransparentproofin2.2DemiclosednessPrinciplesFact2.4(Brezis).(See[,Proposition2.5onP.27],[,Lemma4],or[Corollary20.49].)Letbemaximallymonotone,let,andbeasequenceinsuchthat .ThenTheorem2.5(Seealso[,Proposition20.50]).LetAXbemaximallymonotone,letX,andletCandDbeclosedafÞnesubspacesofXsuchthatD.Furthermore,letbeasequenceinAsuchthatThenAandProof.,whichisaclosedlinearsubspace.Since0,wehaveandthus.Likewise,andhenceItfollowsthatTherefore,sinceareweaklycontinuous, H.H.Bauschke(2.10i)TheresultnowfollowsfromFactRemark2.6.generalizes[,Theorem2],whichcorrespondstothecasewhenisaclosedlinearsubspaceand.Arefereepointedoutthatmaybeobtainedfrom[,Theorem2]byatranslationargument.However,theaboveproofofTheoremisdifferentandmuchsimplerthantheproofof[,Theorem2].Corollary2.7(FirmNonexpansivenessPrinciple).LetFXbeÞrmlynon-expansive,letbeasequenceinXsuchthatconvergesweaklytozX,andsupposethatFzXandthatCandDareclosedafÞnesubspacesofXsuchthatD,Fz,andThenxC,zD,andxFz.Proof.Idsothat.By(ismaximallymonotoneandisasequenceingrathatconvergesweaklyto.Thus,byTheorem,and.Therefore,,i.e.,Corollary2.8(NonexpansivenessPrinciple).LetTXbenonexpansive,letbeasequenceinXsuchthatzz,andsupposethatTzyandthatCandDareclosedafÞnesubspacesofXsuchthatD,andz.Then 2z+1 2y 2zŠ1 D,andyTz.Proof. 21 ,whichisÞrmlynonexpansive.Then 2z+1 SinceisafÞne,weget0(2.12a) 2PCzn+1 0(2.12b) 2NewDemiclosednessPrinciplesfor(Firmly)NonexpansiveOperators23 2zn+1 0(2.12c)0(2.12d)Likewise,since 2znŠ1 2=1 2znŠ1 ,wehave0(2.13a) 2PDzn+1 0(2.13b) 2zn+1 0(2.13c)Thus,byCorollary,and,i.e., 2z+1 2yC,z1 2z+1 ,and 2z+1 2y=1 2z+1 ,i.e., 2z+1 2yC,1 2zŠ1 ,andCorollary2.9(ClassicalDemiclosednessPrinciple).LetSbeanonemptyclosedconvexsubsetofX,letTXbenonexpansive,letbeasequenceinSconvergingweaklytoz,andsupposethatzx.ThenzProof.Wemayanddoassumethat(otherwise,considerinsteadof).Setandnotethat.Nowset.Then0,and0.Corollaryimplies,i.e.,2.3Multi-operatorDemiclosednessPrincipleswhereisanintegergreaterthanorequalto2.(2.14)WeshallworkintheproductHilbertspacewithinducedinnerproduct ,wheredenotegenericelementsinWestartwithamulti-operatordemiclosednessprincipleforÞrmlynonexpan-sivemappings,whichwederivefromthecorrespondingtwo-operatorversion H.H.Bauschke(Corollary).ArefereepointedoutthatTheoremisalsoequivalenttoto1,Corollary3](seealso[,Lemma5]foraBanachspaceextensionof[Corollary3]).Theorem2.10(Multi-operatorDemiclosednessPrincipleforFirmlyNonex-pansiveOperators).beafamilyofÞrmlynonexpansiveoperatorsonX,andlet,foreachibeasequenceinXsuchthatforalliandjinI,ThenFx,foreveryiProof.)=(,andisaclosedsubspaceofFurthermore,wesetsothatandalso.ThenisÞrmlynonexpansiveon,and.Now)implies whichÑwhenviewedinÑmeansthat0.Similarly,using( Therefore,byCorollaryTheorem2.11(Multi-operatorDemiclosednessPrincipleforNonexpansiveOperators).beafamilyofnonexpansiveoperatorsonX,andlet,foreachibeasequenceinXsuchthatforalliandjinI, 2NewDemiclosednessPrinciplesfor(Firmly)NonexpansiveOperators25ThenT,foreachiProof. 21 .ThenisÞrmlynonexpansiveand 2zi+1 ,forevery.By(),0,forall.Itfollowsthat 2zi+1 independent.Furthermore, 2zi,nŠTizi,niI1 2ziŠyiiI1 2ziŠxŠ1 Therefore,theconclusionfollowsfromTheorem2.4ApplicationtoDouglas-RachfordSplittingInthissection,weassumethataremaximallymonotoneoperatorsonsuchthat)=(Weset 21 )+(whichistheDouglas-RachfordsplittingoperatorandwhereIdandIdaretheÒreßectedresolventsÓalreadyconsideredinSect..(ThetermÒreßectedresolventÓismotivatedbythefactthatwhenisaprojectionoperator,thenisthecorrespondingreßection.)See[]forfurtherinformationonthisalgorithmandalso[]forsomeresultsforoperatorsthatarenotmaximallymonotone.Onehas(see[,Lemma2.6(iii)]or[,Proposition25.1(ii)]) H.H.BauschkeNowletanddeÞnethesequenceThissequenceisveryusefulindeterminingazeroofasthenextresultillustrates.Fact2.12(LionsÐMercier[Thesequenceconvergesweaklytosomepointsuchthat.Moreover,thesequenceisbounded,andeveryweakclusterpointofthissequencebelongstoSinceisingeneralsequentiallyweaklycontinuous(seeExample),itisnotobviouswhetherornot.However,recentlySvaiterprovidedarelativelycomplicatedproofthatinfactweakconvergencedoeshold.Asanapplication,werederivethemostfundamentalinstanceofhisresultwithaconsiderablysimplerandmoreconceptualproof.Fact2.13(Svaiter[ThesequenceconvergesweaklytoProof.ByFactSinceis(Þrmly)nonexpansiveandisbounded,thesequenceisboundedaswell.Letbeanarbitraryweakclusterpointof,saybyFact.Set.ThenSincetheoperatorisÞrmlynonexpansiveandFix,itfollowsfrom[]that0(i.e.,isÒasymptoticallyregularÓ);thus,0(2.28)andhenceNext, 2NewDemiclosednessPrinciplesfor(Firmly)NonexpansiveOperators27Tosummarize,)+(ByTheorem.Hence.Sincewasanarbitraryweakclusterpointoftheboundedsequence,weconcludethatMotivatedbyarefereeÕscomment,letusturntowardsinexactiterationsof.Thefollowingresultunderlinestheusefulnessofthemulti-operatordemiclosednessprinciple.Theorem2.14.SupposethatisasequenceinXsuchthatzandzz,wherezT.ThenJProof.ArgueexactlyasintheproofofFactWenowpresentaprototypicalresultoninexactiterations;see[]formanymoreresultsinthisdirectionaswellas[]andalso[Corollary2.15.SupposethataresequencesinXsuchthatThenthereexistszTsuchthatzzandJProof.CombettesÕ[,Proposition4.2(ii)]yields0whiletheexistencesuchthatisguaranteedbyhis[,Theorem5.2(i)].NowapplyUnfortunately,theauthorisunawareofanyexistingactualnumericalimplemen-tationguaranteeingsummableerrors;however,thesetheoreticalresultscertainlyincreaseconÞdenceinthenumericalstabilityoftheDouglas-Rachfordalgorithm.AcknowledgementsHHBthanksPatrickCombettesandJonathanEcksteinfortheirpertinentcomments.HHBwaspartiallysupportedbytheNaturalSciencesandEngineeringResearchCouncilofCanadaandbytheCanadaResearchChairProgram. H.H.BauschkeReferences1.Bauschke,H.H.:AnoteonthepaperbyEcksteinandSvaiterongeneralprojectivesplittingmethodsforsumsofmaximalmonotoneoperators.SIAMJ.ControlOptim.,2513Ð25152.Bauschke,H.H.,Combettes,P.L.:ConvexAnalysisandMonotoneOperatorTheoryinHilbertSpaces.Springer,NewYork(2011)3.Borwein,J.M.,Sims,B.:TheDouglas-Rachfordalgorithmintheabsenceofconvexity.In:Bauschke,H.H.,Burachik,R.S.,Combettes,P.L.,Elser,V.,Luke,D.R.,Wolkowicz,H.(eds.)Fixed-PointAlgorithmsforInverseProblemsinScienceandEngineering,SpringerOptimizationandItsApplications,vol.49,pp.93Ð109.Springer,NewYork(2011)4.Borwein,J.M.,Vanderwerff,J.D.:ConvexFunctions.CambridgeUniversityPress,Cambridge5.Brezis,H.:OperateursMaximauxMonotonesetSemi-GroupesdeContractionsdanslesEspacesdeHilbert.Elsevier,NewYork(1973)6.Browder,F.E.:SemicontractiveandsemiaccretivenonlinearmappingsinBanachspaces.Bull.Am.Math.Soc.,660Ð665(1968)7.Bruck,R.E.,Reich,S.:NonexpansiveprojectionsandresolventsofaccretiveoperatorsinBanachspaces.HoustonJ.Math.,459Ð470(1977)8.Burachik,R.S.,Iusem,A.N.:Set-ValuedMappingsandEnlargementsofMonotoneOperators.Springer,NewYork(2008)9.Combettes,P.L.:Quasi-Fejeriananalysisofsomeoptimizationalgorithms.In:Butnariu,D.,Censor,Y.,Reich,S.(eds.)InherentlyParallelAlgorithmsinFeasibilityandOptimizationandTheirApplications,pp.115Ð152.Elsevier,NewYork(2001)10.Combettes,P.L.:Solvingmonotoneinclusionsviacompositionsofnonexpansiveaveragedoperators.Optimization,475Ð504(2004)11.Combettes,P.L.:Iterativeconstructionoftheresolventofasumofmaximalmonotoneoperators.J.ConvexAnal.,727Ð748(2009)12.Combettes,P.L.,Svaiter,B.F.:Asymptoticbehaviorofalternating-directionmethodofmultipliers(2011,preprint)13.Eckstein,J.,Bertsekas,D.P.:OntheDouglas-Rachfordsplittingmethodandtheproximalpointalgorithmformaximalmonotoneoperators.Math.Program.,293Ð318(1992)14.Eckstein,J.,Ferris,M.C.:Operator-splittingmethodsformonotoneafÞnevariationalinequal-ities,withaparallelapplicationtooptimalcontrol.Inform.J.Comput.,218Ð235(1998)15.Goebel,K.,Kirk,W.A.:TopicsinMetricFixedPointTheory.CambridgeUniversityPress,Cambridge(1990)16.Goebel,K.,Reich,S.:UniformConvexity,HyperbolicGeometry,andNonexpansiveMappings.MarcelDekker,NewYork(1984)17.Lawrence,J.,Spingarn,J.E.:OnÞxedpointsofnonexpansivepiecewiseisometricmappings.Proc.LondonMath.Soc.,605Ð624(1987)18.Lions,P.-L.,Mercier,B.:Splittingalgorithmsforthesumoftwononlinearoperators.SIAMJ.Numer.Anal.,964Ð979(1979)19.Minty,G.J.:Monotone(nonlinear)operatorsinHilbertspaces.DukeMath.J.,341Ð34620.Rockafellar,R.T.,Wets,R.J-B.:VariationalAnalysis.Springer,NewYork(1998)21.Simons,S.:MinimaxandMonotonicity.Springer,Berlin(1998)22.Simons,S.:FromHahn-BanachtoMonotonicity.Springer,NewYork(2008)23.Svaiter,B.F.:OnweakconvergenceoftheDouglas-Rachfordmethod.SIAMJ.ControlOptim.,280Ð287(2011)24.Zalinescu,C.:ConvexAnalysisinGeneralVectorSpaces.WorldScientiÞcPublishing,RiverEdge(2002)25.Zarantonello,E.H.:ProjectionsonconvexsetsinHilbertspaceandspectraltheoryI.Projectionsonconvexsets.In:Zarantonello,E.H.(ed.)ContributionstoNonlinearFunctionalAnalysis,pp.237Ð341.AcademicPress,NewYork(1971)