4CONTENTSChapter7.InvariantdistributionsforMarkovprocessesingeneralspa - PDF document

4CONTENTS3.CouplingandconvergenceintotalvariationundertheDoeblincondition87Chapter9.Takingthetopologyintoaccount911.FellerpropertyandexistenceofinvariantmeasurestheviaKrylov{Bogolyubovapproach912.ApplicationstoSDEs,Stochasticheatequation,stochasticNavier{Stokesequation933.StrongFellerpropertyanduniqueness.93Chapter10.Randomdynamics951.Stabilityinstochasticdynamics952.Markovprocessesandrandomtransformations983.Invariantmeasures1024.ErgodicityforMarkovprocessesandsystemsofi.i.d.transformations1075.Auxiliarystatements110Chapter11.InvariantdistributionsforMarkovchainsindiscretespaces1151.AnotherproofofergodictheoremforMarkovchain1152.Entropyapproach1163.Coupling118Chapter12.InvariantdistributionsforMarkovprocessesingeneralspaces1211.ExistenceviaKrylov{Bogolyubovapproach,movedup1212.Doeblincondition,movedup1213.Harrispositiverecurrencecondition,movedup121Bibliography123 61.INTRODUCTIONidealtruevalue.Inotherwords,collectingmoreandmorestatisticsisusefulonlyifaversionofthelawoflargenumbersholds,namely,thatthereisanumberfsuchthatf(X1)+:::+f(Xn) n!fasn!1.Moreover,thatinnitetimehorizonaverageshouldbeindependentofthespecicinitialstateofthesystemoratleastbestablewithrespecttoaclassofperturbationsoftheinitialstate.Otherwiseitishardtoassignameaningtothislimitingvalue.So,somequestionsthatnaturallyariseare:Doesthesystempossessanystatisticalregularity,i.e.,doesitmakesensetocollectstatisticsaboutthesystem?Howdothestatisticalpropertiesofthesystemdependontheinitialcondition?Doesthesystemtendtoremembertheinitialconditionordoesitforgetitinthelongrun?Whatisrememberedandwhatisforgotteninthelongrun?Onewaytolookattheseissuesisthestabilitypointofview.Iwouldliketoillustratethatwithacoupleofstandardmathematicalexamples.Therstverysimpleexampleisadeterministiclineardynamicalsystemwithonestablexedpoint.AdiscretedynamicalsystemisgivenbyatransformationofaphasespaceX.Forourexample,wetakethephasespaceXtobethereallineRanddenethetransformationby(x)=ax;x2R,whereaisarealnumberbetween0and1.Toanypointx2Xonecanassociateitsforwardorbit(Xn)1n=0,asequenceofpointsobtainedfromX0=xbyiterationsofthemap,i.e.,Xn=(Xn�1)foralln2N:X0=x=0(x)X1=(X0)=(x)=1(x);X2=(X1)=(x)=2(x);X3=(X2)=(x)=3(x);::::Weareinterestedinthebehavioroftheforwardorbit(Xn)1n=0ofxasn!1,wherenplaystheroleoftime.Inthissimpleexample,theanalysisisstraightforward.Namely,zeroisauniquexedpointofthetransformation:(0)=0,andsinceXn=anx,n2Nanda2(0;1)weconcludethatasn!1,Xnconvergestothatxedpointexponentiallyfast.Therefore,0isastablexedpoint,oraone-pointglobalattractorforthedynamicalsystemdenedby,i.e.,itsdomainofattractioncoincideswithR.So,duetothecontractionandintrinsicstabilitythatispresentinthemap,thereisafastlossofmemoryinthesystem,andnomatterwhattheinitialconditionis,itgetsforgotteninthelongrunandthepointsXn=n(x)approachthestablexedpoint0asn!1.Acompletelydierentexampleisthefollowing:letX=[0;1)andletbedenedby(x)=f2xg.Thissystemexhibitsnostabilityatall.Toseethat,itisconvenienttolookatthissystemusingbinaryrepresentationsofnumbersin[0;1):foreachx2[0;1)thereisasequence(xi)1i=1ofnumbers 81.INTRODUCTIONSimilarquestionsarenaturaltoaskifoneconsidersrandommapsin-steadofdeterministicones.Onenaturalwaytherandommapsemergeisviarandomperturbationsofdeterministicdynamics.Letusdescribeoneexampleofthiskind.Itisa`noisy'modicationofourrstexampleabove.Recallthatinthatexampleweworkedwiththemap:x7!x,and0wasastableequilibriumpoint.Letusperturbthisdynamicalsystemwithnoise,i.e.,arandomperturbationthatkickthesystemoutofequilibrium.Sup-posewehaveasequence(n)n2ZofindependentGaussianrandomvariableswithmean0andvariance2denedonsomeprobabilityspace( ;F;P).Foreveryn2Zwewillnowdenearandommapn;!:R!Rbyn;!(x)=ax+n(!):Thismodelisknownasanautoregressive-moving-average(ARMA)modeloforder1.Anaturalanalogueofaforwardorbitfromourrstexamplewouldbeastochasticprocess(Xn)n0emittedfromapointx02R,i.e.,satisfyingX0=xand,foralln2N,(1.2)Xn=aXn�1+n:However,thestabilityissueitisnotasstraightforwardhereasinthede-terministiccase.Itisclearthatthereisnoxedequilibriumpointthatwouldserveallmapsn;!atthesametime.Thesolutionoftheequa-tionn;!(y)=yforsomenmaybeirrelevantforallothervaluesofn.Stillthissystemallowsforanergodicresultsimilarto(1.1).LetbetheGaussianmeasurewithmean0andvariance2=(1�a2).Then,foranyBorelfunctionfintegrablewithrespectto,wehavethatforalmostevery(x0;!)2R withrespecttoP,limn!1f(X0)+f(X1)+f(X2)+:::+f(Xn�1) n=ZRf(x)(dx):Theunderlyingreasonforthisresultisthatisauniqueinvariantdistri-butionfortheMarkovsemigroupassociatedwiththeprocess.Thisexampledemonstratesasituationwherethereisnostabilityinthestraightforwarddeterministicsense,butthereisstatisticalstability.Roughlyspeakingtheergodicapproachtostochasticstabilityistobasethestudyofthesystemonthedescriptionofthesetofinvariantdistributionsandtheirproperties,andsothisisthemainthemeofthesenotesinthecontextofrandomdynamics.Nevertheless,wewillbeginwiththebasicsofergodictheoryofdeterministictransformations.Thefulltentativeplanistostudy(1)Generalitiesonergodictheoryofdeterministictransformations.(2)GeneralitiesonergodictheoryofrandomtransformationsandMarkovprocesses.(3)Finiteandcountablestatespace.Markovchains.(4)RandomdynamicsinEuclideanspaces 122.MEASURE-PRESERVINGTRANSFORMATIONSTheorem2.1.Letbeirrational.Thenforanypoint!2S1,andanyintervalI=(a;b)S1,(2.2)limn!1Pn�1j=01n!2I n=b�a:Thesumintheleft-handsidecomputesthetotalnumberofvisitsofthesequence(j!)1j=0toIbeforetimen,andtheratiocomputesthefrequencyofthosevisitsrelativetothetotaltimen.Noticethatthelimitingvalueontheright-handsideequalstoLeb(I),theLebesguemeasureofI.AlsonoticethattheLebesguemeasureplaysaspecialroleforthisdynamicalsystem,namelyitisinvariantunderrotation.Itshouldbeintuitivelyclearwhatthismeans.First,imaginemassuniformlyspreadoverthecircleandthenrotatethecircle.Thedistributionofmassinthecircleaftertherotationisstilluniform,soitcoincideswiththeinitialmassdistribution.Wewillseesoonthatthefactthatthestatisticsofvisitstointervalsisdescribedinthelongrunbyaninvariantdistributionisageneralphenom-enonexplainedbyergodictheorems.Herewenotethatitiseasytoextendthisresulttothefollowing:Theorem2.2.Letbeirrational.Thenforanypoint!2S1andanycontinuousfunctionf:S1!R,(2.3)limn!1Pn�1j=0f(j!) n=ZS1f(x)dx:DerivationofTheorem2.2fromTheorem2.1:Identity(2.2)isaspeciccaseofidentity(2.3)forf=1I.Given(2.2),wecanuselinearityofsumsandintegralstoderive(2.3)forfunctionsfrepresentableasf(!)=mXk=1ck1Ik(!);forsomesystemofintervalsIkS1;k=1;:::;m.Supposenowfiscontinuous.Therefore,itisuniformlycontinuousandforevery"�0,wecanndastepfunctionf"suchthatjf(!)�f"(!)j"forall!2S1.ThenPn�1j=0f(j!) n�ZS1f(x)dxPn�1j=0f"(j!) n�ZS1f"(x)dx+2";n2N;solimsupn!1oftheleft-handsidedoesnotexceed2".Since"�0isarbitrary,weconcludethatthelimitoftheleft-handsideequalszero.Theorem2.2canbeinterpretedsothatforeveryobservableffromabroadclassoffunctions,thetimeaveragesapproximatethespaceaverages.Ofcourse,thispropertyfailsforrationalvaluesof=m=ndespitethefactthattheLebesguemeasureisstillinvariant.Thereasonisthatinthatcase,forany!2S1,theorbit(j!)j0exploresonlyaverysmallpartofS1|aniteset,infact,|whereastheintegralintheright-handsideof(2.3)dependsonthebehavioroffontheentireS1.Thisalsocanbe 142.MEASURE-PRESERVINGTRANSFORMATIONSDefinition2.2.Let( ;F)beameasurablespace.Amaportrans-formation: ! iscalledmeasurableif,foreveryA2F,wehave�1A=f!:!2Ag2F.Hereandofteninthesenoteswefollowthetraditiontodenotethevalueassignedbythetransformationtoargument!by!.Ameasurabletransformationdenesasemigroupofmeasurabletrans-formations(n)n2Z+.Herenisinductivelydenedby0=Id(identicaltransformation,i.e.,Id(!)!)andforn1,n=n�1.Ifatrans-formationisinvertible,i.e.,isabijectionof ontoitself,then�1iswell-denedandinasimilarwaygeneratesagroupoftransformations(n)n2Z.Identitymn=m+nisvalidforallm;n2Z+inthecaseofthesemigroupandforallm;n2Zinthecaseofthegroup.Ofteninthesenoteswewillomitthecompositionsign\"forbrevity.Forexample,thisconventionallowstorewritetheabovegroupidentitywrittenasmn=m+n.Definition2.3.Let( ;F;P)beaprobabilityspaceand(~ ;~F)beameasurablespace.LetamapX: !~ be(F;~F)-measurable.ThepushforwardofPunderXisameasureon( ;~F)denotedbyP�1anddenedbyPX�1(A)=P(X�1A)=Pf!:X(!)2Ag;A2F:Definition2.4.Let( ;F;P)beaprobabilityspaceandsupposetrans-formation: ! bemeasurable.IfthepushforwardP�1coincideswithP,i.e.,(2.5)P(�1A)=P(A);A2F;thenwesaythatPisinvariantunder,orthatpreservesP,orthatisP-preserving.Itisimportanttonoticethatthedenitionofinvarianceisbasedonpre-imagesunder,andnotforwardimages.Thereasonforthatisthatthepushforwardsofmeasuresaredenedintermsofpre-imagesandthereisnonaturalnotionofpullbacksofmeasures.Problem2.1.SupposetransformationpreservesameasureP.Provethatforanyn2Z+,nalsopreservesP.Supposeadditionallythatisinvertibleand�1ismeasurable.Provethatthenastrongerstatementholds:foranyn2Z,nalsopreservesP(inparticular,�1isP-preserving).Letusconsidersomebasicexamplesofprobabilityspaceswithmeasure-preservingtransformations.Let =f0;1;:::;n�1g,F=2 (-algebraofallsubsetsof ),andletPbetheuniformdistributionon .ThenatransformationisP-preservingifandonlyifitisapermutation,i.e.,aone-to-onemap.Forinstance,thecyclicmap!=!+1(modn)isP-preserving. 162.MEASURE-PRESERVINGTRANSFORMATIONSProof:Letusproverstthatthesets�nT(A),n2Naremutuallydisjoint.Infact,if!2�nT(A)\�mT(A)andmn,thenwedenote!0=m!andnoticethat!02T(A),andatthesametimen�m!0=n�mm!=!2T(A)whichisacontradictionwiththedenitionofT(A).Dueto-invarianceofP,P(�nT(A))=P(T(A))foralln,soXn2NP(T(A))=Xn2NP(�nT(A))=P [n2N�nT(A)!P( );whichcanholdtrueonlyifP(T(A))=0.Itispossibletogiveaquantitativestrengtheningofthistheorem.Thefollowingisanadaptationfrom[Tao08].Theorem2.4.UndertheconditionsofTheorem2.3,limsupn!1P(�nA\A)P2(A);A2F:Proof:TheinvarianceofPimpliesthatforanyN2N,E"NXn=11�nA#=NP(A):UsingtheCauchy{SchwartzinequalityorLyapunovinequality,weobtain(2.8)E"NXn=11�nA#2 E"NXn=11�nA#!2=N2P(A)2:Theleft-handsideofthisinequalitycanberewrittenas(2.9)E"NXn=11�nA#2=NXn;m=1P(�nA\�mA)=NXn;m=1P(A\�jm�njA):IntroducingL=limsupn!1P(A\�nA),forany"&#x-382;0,wecanndn0=n0(")suchthatP(A\�nA)L+";n&#x-278;n0:Letusnowsplitthesumontheright-handsideof(2.9)intotwo:overn;msatisfyingjn�mjn0andovern;msatisfyingjn�mjn0.Noticingthattherearelessthan2n0Ntermsinthesecondsumandtheyallareboundedby1,weobtain1 N2NXn;m=1P(A\�jm�njA)L+"+2n0(")N N2:Combiningthiswith(2.8)and(2.9),weobtainP(A)2L+"+2n0(") N:TakingN!1andthen"!0,weobtainthedesiredresult. 182.MEASURE-PRESERVINGTRANSFORMATIONSItisoftenusefultoconsideran(X;X)-valuedprocessXasoneXT-valuedrandomvariable,andsoweneedtointroduceanappropriate-algebraonXT.Definition2.7.SupposeTR.Form2N,n1:::nm,andA1;:::;Am2X,wedenote(2.10)Cn1;:::;nm(A1;:::;Am)=fx2XT:xn12A0;:::;xnm2Amg:Setsofthisformarecalledelementarycylinders.Letusstressthatallcylindershavenon-uniquerepresentationsoftheform(2.10).Forexample,Cn1;:::;nm;nm+1(A1;:::;Am;X)=Cn1;:::;nm(A1;:::;Am):Definition2.8.Thecylindric-algebraXTisgeneratedbyallelemen-tarycylinders.Foranytimen2T,theprojectionn:XT!Xisdenedbyn(x)=xn.Theorem2.5.Thecylindric-algebraXTisgeneratedbymapsn.Thefamilyofallelementarycylindersisa-system,i.e.,itisclosedunderintersection.Theorem2.6(see[Shi96,LemmaII.2.3]).Iftherestrictionsoftwoprobabilitymeasuresona-systemEcoincide,thenthemeasurescoincideon(E).Thefollowingcorollarysaysthataprobabilitymeasureon(XT;XT)isuniquelydenedbyitsvaluesonelementarycylinders.Theorem2.7.LetPandQbetwomeasureson(XT;XT)suchthatforanyelementarycylinderC,P(C)=Q(C).ThenP=Q.Toformulateanexistenceresult,weneedtointroducegeneralcylinders.Definition2.9.Letm2Nandn1;:::;nmTsatisfyn1:::nm,wedenotebyXn1;:::;nm=(Cn1;:::;nm(A1;:::;Am);A1;:::;Am2X):ElementsofXn1;:::;nmarecalledcylinders.Theorem2.8(Kolmogorov{Daniellextensiontheorem).If(X;X)isaBorelspaceandPisanonnegativefunctiononallcylinderssuchthatforanym2Nandanyn1;:::;nmTsatisfyingn1:::nm,therestrictionofPtoXn1;:::;nmisaprobabilitymeasure,thenthereisauniquemeasureonXTsuchthatitsrestrictiononcylinderscoincideswithP.OfteninthesenoteswewillworkwithBorelspaces,asinthistheorem.Letusintroducethecorrespondingdenitions. 202.MEASURE-PRESERVINGTRANSFORMATIONSProof:Letuscheckthedenitionofstationarityforshiftr=1:Pf!:Xi1+1(!)2A1;:::;Xim+1(!)2Amg=Pf!:X(i1+1!)2A1;:::;X(im+1!)2Amg=Pf!:X(i1!)2A1;:::;X(im!)2Amg=Pf!:X(i1!)2A1;:::;X(im!)2Amg=Pf!:Xi1(!)2A1;:::;Xim(!)2Amg;wherethethirdidentityfollowsfromthe-invarianceofP.Onecanalsomakesenseofaconversestatement:everystationarypro-cesscanberepresentedintheformdescribedinLemma2.2.Suppose(Xn)n2Z+isan(X;X)-valuedstationaryprocessonaprobabilityspace( ;F;P).OnewaytolookattheprocessXistoviewitasamapfrom tothespaceoftrajectories~ =XZ+,i.e.,thespaceofallfunctionsx:Z+!X,i.e.,sequencesx=(x0;x1;:::),equippedwithcylindric-algebra~F=XZ+.Infact,theprocessXis(F;~F)-measurable(toseethis,itissucienttocheckthatX-pre-imagesofallcylindersareinF).Therefore,wecanconsiderthepushforward~P=PX�1ofPto(~ ;~F)underthismap.Theresultisaprobabilityspace(~ ;~F;~P).Theprocess(~Xn)n2Z+denedby~Xn(x)=nx=xn,n2Z+,isastationaryprocessunder~P,becauseitsdistributionisthesameasthatoftheoriginalprocessX.Letusprovethat~Pisinvariantundertheshiftmap:~ !~ denedby(x0;x1;:::)=(x1;x2;:::).Tocheckthis,letusrecallthat,duetotheKolmogorov{Daniellextensiontheorem,measureson(~ ;~F)=(XZ+;XZ+)areuniquelydeterminedbytheirvaluesoncylindersCn(A0;:::;An�1)=fx2XZ+:x02A0;:::;xn�12An�1g;wheren2N,A0;:::;An�12X.Soourclaimfollowsfrom~P(�1Cn(A0;:::;An�1))=~P(Cn+1(X;A0;:::;An�1))=PfX12A0;:::;Xn2An�1g=PfX02A0;:::;Xn�12An�1g=~P(Cn(A0;:::;An�1)):Nowwecantaketherandomvariable0:~ !Xdenedby0(x)=x0andnoticethattheprocess~Xnon(~ ;~F;~P)canberepresentedas~Xn(x)=0(nx).Thisiswhatwewanted,becausethisrepresentationisofform(2.12).However,toobtainthisrepresentationwehadtoswitchfromtheoriginalprobabilityspacetothecanonicalspaceoftrajectories.ThesameprocedureappliestoprocessesindexedbyNandZ.Itisoftenconvenienttoworkwithstationaryprocessesdirectlyontheirtrajectoryspaces.Forexample,whenworkingwithi.i.d.(X;X)-valuedran-domvariables,itisconvenienttoworkwithproductmeasureon(XZ+;XZ+). 222.MEASURE-PRESERVINGTRANSFORMATIONS(2.15)BnA=1[n=11\k=n�kAnA1[n=1(�nAnA):Nowourclaimfollowsfrom(2.13),(2.14),and(2.15).ItiseasytocheckthatthefamiliesIandIofinvariantandalmostinvariantsetsare-algebras.However,theforwardinvariantsetsdonotforma-algebraingeneralandforthisreasontheyarenotconvenienttoworkwithinthecontextofmeasuretheory.Thisistightlyrelatedtothefactthattakingpre-imageisamorewell-behavedoperationthantakingimages,andthusitiseasytodenepushforwardsof-algebrasandmeasuresbytakingpre-images,andthereisnonaturalwaytodenepullbacksoftheseobjectsviaforwardimages.Problem2.3.Giveanexampleofametricdynamicalsystemsuchthatthecollectionofforward-invariantsetsdoesnotforma-algebra.Hint:complements.Definition2.14.Atransformationiscalledergodicifeveryinvariantsethasmeasure0or1.Thisdenitionmeansthatonecannotsplitourmetricdynamicalsystemintotwo.Lemma2.4.Atransformationisergodicifandonlyifeveryalmostinvariantsethasmeasure0or1.Proof:The\if"partisobvioussinceII.Theconverseisadirectconsequenceofpart(c)ofLemma2.3.Definition2.15.ArandomvariableXiscalledinvariantifX(!)=X(!)forall!2 .Definition2.16.ArandomvariableXiscalledalmostinvariantorinvariantmod0ifX(!)=X(!)forP-almostall!2 .Thefollowingobviousstatementgivesonemorereasontodeneinvari-antandalmostinvariantsetsasinDenition2.13.Lemma2.5.AsetAisinvariantifandonlyif1Aisinvariant.AsetAisalmostinvariantifandonlyif1Aisalmostinvariant.Definition2.17.Wesaythata(real-valued)randomvariableXisa.s.-constantifthereisanumberc2RsuchthatPfX=cg=1.Theorem2.9.ThefollowingthreeconditionsareequivalentifisaP-preservingtransformation:(a)isergodic;(b)everyalmostinvariantrandomvariableisa.s.-constant;(c)everyinvariantrandomvariableisa.s.-constant; 242.MEASURE-PRESERVINGTRANSFORMATIONSTheorem2.10.LetbeashiftonthetorusTddenedby(2.6).Itisergodicifandonlyiftheidentityr11+:::+rdd=misimpossibleforanyr1;:::;rd;m2Zsuchthatnotallofthemareequalto0.Problem2.4.ProveTheorem2.10.Theorem2.11(AvariationonKolmogorov's0-1Law,REFERENCE????).Let(X;X;P)beaprobabilityspaceand( ;F;P)=(XN;XN(X);PN),whereXNisthecylindric-algebraandPNistheproductmeasureonitwithmar-ginaldistributionP.Thentheshifttransformationdenedby(!1;!2;:::)=(!2;!3;:::)ismeasurepreservingandergodic.Proof:TheinvarianceofPunderfollowsfromthediscussionrightafterLemma2.2.Toproveergodicity,consideranalmostinvariantsetA.ItissucienttoprovethatP(A)=P2(A)whichcanberewrittenasP(A\A)=P(A)P(A)or,usingthealmostinvarianceofA,asP(A\�nA)=P(A)P(�nA)foralln2N.ThismeansthatAisindependentofitselforitspre-imagesundern.Toprovethis,wewillapproximateAand�nA(forlargen)bycylindersthatdependondisjointcoordinatesetsandthusareindependent.SinceA2XN,itcanbeapproximatedbycylinders,i.e.,forany"�0thereisasetA"2XN,anumberk(")2NandasetB"2Xk(")suchthatA"=f!:(!1;:::;!k("))2B"g,andP(A"4A)".Letustakeanyn2Nsatisfyingnk(").SetsA"and�nA"=f!:(!n+1;:::;!n+k("))2B"gareindependentbythedenitionoftheproductmeasure,so(2.17)P(A"\�nA")=P(A")P(�nA")=P2(A"):Todealwiththeleft-handside,wenoticethatA"isveryclosetoanalmostinvariantset:jP(A"\�nA")�P(A)j=jP(A"\�nA")�P(A\�nA)jjP(A"\�nA")�P(A"\�nA)j+jP(A"\�nA)�P(A\�nA)jjP(�n(A"4A)j+jP(A"4A)j2":Fortheright-handsideof(2.17),wehavejP2(A")�P2(A)j=(P(A")+P(A))
jP(A")�P(A)j2jP(A"4A)j2";sojP2(A)�P(A)j4".Since"&#x-278;0isarbitrary,wehaveP2(A)=P(A),soP(A)=0or1. 263.ERGODICTHEOREMSFORMEASURE-PRESERVINGTRANSFORMATIONSmeasurablesets,andcj2Cforj=1;:::;m:kUfk2=mXi;j=1cicjhU1Ai;U1Aji=mXi;j=1cicjh1�1Ai;1�1Aji=Xi=1jcij2P(Ai)=kfk2:Forageneralf,wecanndasequenceofsimplefunctionsfnsuchthatf2n(!)isincreasingtof2(!)asn!1forall!2X.Then(Ufn)2alsoincreasesto(Uf)2pointwise,andwecanusetheaboveidentityalongwiththemonotoneconvergencetheoremtowrite:kUfk2=limn!1kUfnk2=limn!1kfnk2=kfk2:Problem3.1.ThisproblemisageneralizationofLemma3.1.Let( ;F;P;)beametricdynamicalsystem.Letp2[1;1].ProvethatifXisarandomvariablebelongingtoLp( ;F;P),thenUX2Lp( ;F;P),andkUXkLp( ;F;P)=kXkLp( ;F;P).Lemma3.1showsthatstudyingtheconvergenceofergodicaveragesinL2ispossibleviatheasymptoticanalysisofoperators1 nPn�1k=0Uk,whereUisanisometryofacomplexHilbertspace.TherstversionofthefollowingL2ergodictheoremappearedin[vN32].Theorem3.1.LetUbeanisometryofacomplexHilbertspaceH.LetIbetheorthogonalprojectionontothespaceI=ff2H:Uf=fg.Then,foreveryf2H,(3.2)Anf=1 nn�1Xk=0UkfconvergestoIfasn!1.Proof:LetusdenoteG=fg�Ug:g2HgandprovethatIandtheclosureofGareorthogonalcomplementsofeachother.Ifhf;g�Ugi=0forsomefandallg,thenhf;gi=hf;Ugiforallg.Inparticular,hf;fi=hf;Ufi.Usingthispropertyandtheisometryproperty,weobtainhUf�f;Uf�fi=hUf;Ufi�hUf;fi�hf;Ufi�hf;fi=kfk2�kfk2�kfk2+kfk2=0;soUf=f.ThusG?I.SincetheisometrypropertycanberewrittenashUf;Ugi=hf;giforallf;g2H,wecantakeanyf2I,g2Handwritehf;g�Ugi=hf;gi�hf;Ugi=hf;gi�hUf;Ugi=hf;gi�hf;gi=0:Therefore,IG?,andthusI=G?.So,nowwecandecomposeanyf2Hasf=fI+fG,wherefI2IandfG2Gareorthogonalprojectionsoff 283.ERGODICTHEOREMSFORMEASURE-PRESERVINGTRANSFORMATIONSnumberaandanorthogonalrandommeasureZon[�;)suchthatXn=a+Z[�;)ein'Z(d');n2Z:Therefore,wecanwrite1 n(X0+:::+Xn�1)=Z(f0g)+Z[�;)nf0gein'�1 n(ei'�1)Z(d');n2N;andconcludethat1 n(X0+:::+Xn�1)L2!Z(f0g);n!1:Therefore,thelimitisnonrandomandequalszeroifthespectralmeasurehasnoatomatzero,i.e.,EjZ(f0g)j2=0.2.Birkho'spointwiseergodictheoremThefollowingtheoremwasrstprovedbyG.Birkhoin[Bir31].Theorem3.2.Let( ;F;P;)beametricdynamicalsystem,Ithe-algebraof-invariantsets,andf2L1( ;F;P).Then,withprobability1,Anf!E[fjI];n!1;wheretheergodicaveragesAnaredenedby(3.1)or,equivalently,by(3.2)andUf(!)=f(!).Ifinadditionisergodicthenwithprobability1,(3.3)Anf!Ef;n!1:Thereexistvariousproofsofthepointwiseergodictheorem.TheyalldonotseemastransparentastheproofoftheL2version.Here,wegiveoneofthesimplestandmostconceptualproofsconstructedbyY.KatznelsonandB.Weiss[KW82]andbasedontheideaofthenonstandardanalysisproofofT.Kamae[Kam82].Proof:Duetolinearityoftimeaveragesandconditionalexpectations,itissucienttoprovetheconvergencefornonnegativefunctionf.Letusintroducef(!)=limsupn!1Anf(!);f (!)=liminfn!1Anf(!);!2 :Aprioriwedonotknowiffandf arenite,buttheyarealmostinvariant.Toseethat,wewritejAnf(!)�An(!)j1 nf(!)+1 nf(n!):Thersttermontheright-handsideclearlyconvergesto0asn!1forall!.Thesecondtermconvergesto0almostsurely.Thelatterfollowsfrom 303.ERGODICTHEOREMSFORMEASURE-PRESERVINGTRANSFORMATIONSThepurposeofthisistosplitthesumfrom0toL�1intointervalssuchthatoneachofthemwecanapply(3.6):wedenek(!)=maxfk:nk(!)L�1gandwriteL�1Xj=0fM(j!)=k(!)Xk=1nk(!)�1Xj=nk�1(!)fM(j!)+L�1Xj=nk(!)(!)fM(j!):Applying(3.6)toeachofthek(!)termsandestimatingthelastspecialtermbyNM(hereweusethefactthat~n(!)isuniformlyboundedbyN,andsoisL�nk(!)(!)),weobtainL�1Xj=0fM(j!)L�1Xj=0fM;"(j!)+L"+NM:IntegratingbothsidesoverBanddividingbyLgives(3.7)EfM1B1 LL�1Xj=0EfM;"(j!)1B+"+NM LEfM;"1B+2";whereweusedinvarianceofBandmeasure-preservingpropertyof:EfM;"(j!)1f!2Bg=EfM;"(!)1f!2�jBg=EfM;"(!)1f!2Bg:SincefM;"f+M1Cc,wecanestimatetheright-handsideof(3.7)withEfM;"1BEf1B+MP(Cc)Ef1B+";weobtain(3.8)EfM1BEf1B+3":Letting"!0andthenM!1,weobtaintherstinequalityin(3.4).Toprovethesecondinequalityin(3.4),wedonotevenneedtointroduceaatruncationanalogoustofM.Wesimplyx"�0,denen(!)=infn:f (!)Anf(!)�" ;!2 ;introduceC=fn(!)�NgwhereNischosensothatEf1Cc",denef"=f1Cand~n=n1C+1Cc,andproceedinasimilarwaytotheproofoftherstinequalityin(3.4).Toprove(3.3),itissucienttonoticethatintheergodiccase,Icontainsonlysetsofprobability0and1,soE[fjI]a:s:=Ef.Problem3.3.Checkthedetailsoftheproofofthesecondinequalityin(3.4).Problem3.4.Let( ;F;P)beaprobabilityspaceandsupposethat-algebrasG0;G1Fsatisfythefollowingcondition:foreveryi2f0;1g,everysetA2GithereisasetB2G1�isuchthatP(A4B)=0.ThenforeveryrandomvariableX2L1( ;F;P),E[XjG1]a:s:=E[XjG2].Inparticular, 323.ERGODICTHEOREMSFORMEASURE-PRESERVINGTRANSFORMATIONSthestatementsofthoseresults,infact,holdforall!2S1withoutexception.Thisgapcanbeclosedeasily.Problem3.5.FollowtheselinestoconstructacompleteproofofThe-orems2.1and2.2.Onecanusetheergodictheoremtogiveequivalentdenitionofergod-icity:Theorem3.5.Letbeameasurepreservingtranformationon( ;F;P).Thefollowingstatementsareequivalent:(a)isergodic;(b)Foranyf2L1( ;F;P),(3.3)holds;(c)ForanyA2F,(3.9)holds.Proof:Condition(a)impliescondition(b)bytheergodictheorem,(c)isaspeciccaseof(b).Toderive(a)from(c),wetakeanyinvariantsetA,usetheinvarianceofAandofActowriteAn1A=1A,andsinceAn1AconvergestoP(A)a.s.,weconcludethat1Aa:s:=P(A),andthiscanhappenonlyifP(A)=0or1. 343.ERGODICTHEOREMSFORMEASURE-PRESERVINGTRANSFORMATIONSSo,bis-invariantand,duetoergodicity,hastobea.s.-constantwithrespecttoP.Constantsaandbarebothnonpositive.Letusprovethatifa��1,thenba.Supposethatba�"forsome"&#x-277;0.IfBm=!:min1nmXn(!)=na�";m2N;then,forany&#x-277;0,wecanndm2NsuchthatP(Bm)&#x-277;1�.Letusx!andtakeanarbitraryn2N.LetR=fj2f1;:::;nmg:j!2Bmg.BydenitionofR,foreveryj2Rthereisk=k(j)2f1;:::;mgsuchthatXk(j!)k(a�").AssumingR6=;,letusnowdenetwonitesequencesj1;:::;jrandk1;:::;krdependingon!.First,weletj1bethesmallestnumberinRandk1=k(j1).ThenweinductivelysetjitobethesmallestnumberinRexceedingji�1+ki�1�1andki=k(ji)untilwegettoanumberjrnmsuchthatkr=k(jr)satisesjr+kr&#x-277;maxR.Wehave(3.15)Rr[i=1fji;:::;ji+ki�1g;wheretheintervalsintheunionontheright-handsidearemutuallydisjoint.Applying(3.12)tothesequencej1;k1;j2�(j1+k1);k2;j3�(j2+k2);:::;kr;nm+m�(jr+kr);andthrowingoutnonpositivetermscorrespondingtoj1;j2�(j1+k1);:::;nm+m�(jr+kr),weobtainXnm+m(!)Xk1(j1!)+Xk2(j2!)+:::+Xkr(jr!):WehaveXki(ji!)ki(a�")bydenitionofji;ki,i=1;:::;r,soXnm+m(!)(k1+:::+kr)(a�")(a�")nmXi=11Bm(i!);wherethelastinequalityfollowsfrom(3.15)anda�"0.Notethat(3.16)Xnm+m(!)(a�")nmXi=11Bm(i!);alsotriviallyholdstrueifR=;.Letusnowdividebothsidesof(3.16)bynm+mandtakeexpectations:EXnm+m nm+m(a�")nm nm+mP(Bm):RecallingthatP(Bm)�1�,lettingn!1andusingthedenitionofa,weseethata(a�")(1�);andweobtainacontradictionifwechoosesucientlysmall. 384.INVARIANTMEASURESTheorem4.1.Let beametricspaceequippedwithBorel-algebraB( )andameasurabletransformationon( ;B( )).SupposethereisacompactsetKthatisforwardinvariantunder,andiscontinuousonK.ThenthereisaninvariantprobabilitymeasuresupportedonK.Proof:FirstletusnoticethatitissucienttoconsiderthecasewhereK= .LetnowPbeanyprobabilitymeasureon .LetPk=P�k,k2Z+andPn=1 nPn�1k=0Pk.Wewouldliketoextractaweaklyconvergentsubsequencefrom(Pn)n2Z+.Letusrecallthatweakconvergencen)ofprobabilitymeasures(n)n2Z+toaprobabilitymeasuremeansthatforeveryboundedcontinuousfunctionf: !RZ f(!)n(!)!Z f(!)(!);n!1:TheProkhorovtheorem(see[Bil68]foraclassicalintroductiontothetheoryofweakconvergenceofmeasures)statesthatifforany"�0thereisacompactsetCsuchthatn(C)�1�"foralln,thenonecanextractaweaklyconvergentsubsequencen0fromn.Inourcase,forany"�0wecanchooseC= ,soanysequenceofmeasureson containsaconvergentsubsequence.Soletuschooseasequenceni!1suchthatPni)PforsomeprobabilityP,i!1.LetusprovethatPis-invariant.ItissucienttocheckZ f(!)P(d!)=Z f(!)P(d!);forallcontinuousboundedfunctionsfsincethatintegralsoffunctionsfromthatsetdeterminesameasureuniquely.Forsuchafunctionf,Z f(!)P(d!)=limi!1Z f(!)Pni(d!)=limi!11 nini�1Xk=0Z f(!)Pk(d!)=limi!1"1 niniXk=1Z f(!)Pk(d!)+1 niZ f(!)P(d!)�1 niZ f(!)Pni(d!)#=limi!11 nini�1Xk=0Z f(!)Pk(d!)=Z f(!)P(d!);whichcompletestheproof.2.Structureofthesetofinvariantmeasures.Theorem4.2.Letbeameasurabletransformationon( ;F).SupposeP1andP2aretwodistinct-invariantandergodicmeasureson( ;F).Thentheyaresingulartoeachother. 404.INVARIANTMEASURESIntegratingthisidentitywithrespecttoQ1weobtainZ f(!)P(d!)=Z f(!)Q1(d!);andsincefwasanarbitraryboundedmeasurablefunction,weconcludethatP=Q1.Similarly,weobtainP=Q2,soPisanextremepoint.Ingeneral,every-invariantprobabilitymeasurecanberepresentedasamixtureofergodic-invariantmeasures:(4.3)P=ZEP(dP);whereisameasureonE,thesetofallergodicmeasures.Intheremainderofthissectionwewillmakesenseofthisstatement,butletusrstconsiderthefollowingexample.LetbearotationofthecylinderS1R1byanirrationalangle:(x;y)=(fx+g;y):Sinceforeveryy2R,thecircleS1fygisinvariant,everyergodicmeasurehastobeconcentratedononeofthecircles.Restrictedtoanycircle,actasacirclerotation,soforeveryy2R,themeasureLebyisauniqueinvariantmeasureonS1fyganditisergodicunder.Weseethatallergodicmeasuresareindexedbyy2R,andrepresentation(4.3)may2beinterpretedasP(A)=Zy2R(dy)Leb(Ay);whereAy=fx2S1:(x;y)2Agforeveryy2R.Inotherwords,aninvariantmeasuredistributesthemassovermultiplebers(ergodiccompo-nents),butwithinindividualbers,themassisdistributedaccordingtoanergodicmeasure.OnecoulduseanabstractChoquettheoremthatsaysthateverypointofacompactconvexsubsetofalocallyconvextopologicalvectorspacecanberepresentedasamixture(integralconvexcombination)ofitsextremepoints.Wewilltakeanother,moregeneralandconstructiveapproach.Wefollow[Sar09]and[EW11].Definition4.1.A-algebraGiscountablygeneratedifG=(Ek;k2N)forsomecountablefamilyofsetsEk2F;k2N.Definition4.2.Fortwo-algebrasG;H,wewriteGP=HifforeverysetA2GthereisasetB2HsuchthatP(A4B)=0,andforeverysetA2HthereisasetB2GsuchthatP(A4B)=0.Theorem4.4.(1)LetPbeaprobabilitymeasureonaBorelspace( ;F)andletGFbea-algebraon .Thenthereisaset 02GsatisfyingP( 0)=1andaprobabilitykernelPG(
;
)from 424.INVARIANTMEASURESandalimitofG-measurablefunctionsisaG-measurablefunction.LetAbethecollectionofsetssuchthattheirindicatorscanberepre-sentedaspointwiselimitsofcontinuousuniformlyboundedfunctions.ThenAisasetalgebraandamonotoneclass(see[Shi96,SectionII.2]forthesenotions).Thereforeitisa-algebra.SinceAcontainsopensets,itcoincideswithBorel-algebra.Therefore,ourclaimthatforeveryB2F,PG(
;B)isaG-measurablefunctionhodstrue.Problem4.1.LetX2L1( ;F;P).ProvethatthereisasequenceXn2C( );n2NsuchthatXa:s:=P1n=1XnandP1n=1kXnkL1( ;F;P)1.Usingthisproblem,wecanwriteE(XjG)(!)a:s:=1Xn=1E(XnjG)(!)a:s:=1Xn=1Z Xn()PG(!;d)a:s:=Z 1Xn=1Xn()PG(!;d)a:s:=ZX()PG(!;d):TherstidentityholdssinceconditionalexpectationisacontinuouslinearoperatorinL1.Thesecondidentityfollowsfrom(4.5)andcontinuityofXn.ThelastidentityfollowsfromthedenitionofXn,soitremainstoexplainthechangeoforderofintegrationandsummationinthethirdidentity.ThiswillbejustiedifwecanprovethatR P1n=1jXn()jPG(!;d)isa.s.-nite.Thiswillfollowfrom(4.6)EZ 1Xn=1jXn()jPG(!;d)1:IdentityEZ Y()PG(!;d)=EYholdsforallY2RbydenitionandthusforallY2C( ),soweuseitalongwiththemonotoneconvergencetheoremtoseethattheleft-handsideof(4.6)equalslimm!1EZ mXn=1jXn()jP(!;d)limm!1mXn=1EjXn()j1Xn=1kXnkL11;whichcompletestheproofofpart1with 0=A.Wewillneedtoadjust 0intheproofofpart2thatfollows.First,P(EijG)(!)a:s:=1Ei(!)a:s:=PG(!;Ei);i2N;whereP(EijG)(!)a:s:=1Ei(!)followsbydenitionofconditionalexpectation,andP(EijG)(!)a:s:=PG(!;Ei)isaspeciccaseof(4.4). 444.INVARIANTMEASURESmeasurePI(!;
)is-invariantandergodic,andPisamixtureorconvexcombinationofthoseergodicmeasures:P=Z P(d!)PI(!;
);i.e.,(4.10)P(A)=Z P(d!)PI(!;A);A2F;and,moregenerally,(4.11)EX=Z P(d!)Z PI(!;d)X();X2L1( ;F;P):Proof:Checking(4.10)or(4.11)isstraightforward:accordingtoTheo-rem4.4,(4.12)EX=EE(XjI)=EZ X()PI(!;d);X2L1( ;F;P):FromnowonweusetheBorelisomorphismtoassumethat( ;F)isacompactsegmentonRwithBorel-algebra.WechooseadensesetD=fXngn2NinC( ).LetusproveinvarianceofPI(!;
).ByBirkho'sergodictheorem,thereisaset 12FsuchthatP( 1)=1, 1 0(where 0isintroducedinTheorem4.6),andforalln2Nand!2 1,Z PI(!;d)Xn()=E(XnjI)(!)=limN!11 NN�1Xk=0Xn(k!)andZ PI(!;d)Xn()=E(XnjI)(!)=limN!11 NN�1Xk=0Xn(k+1!):SinceXnisbounded,theright-handsidesofthelasttwoidentitiescoincidefor!2 1.SinceeveryX2C( )canbeuniformlyapproximatedbyfunctionsfromD,weobtainthat(4.13)Z PI(!;d)X()=Z PI(!;d)X()holdsforX2C( )and!2 1.ForeverysetB2Fthereisasequenceofuniformlyboundedcontinuousfunctionsconvergentpointwiseto1B.Boundedconvergenceimpliesthat(4.13)holdsforX=1BwhichprovesthatpreservesPI(!;
)for!2 1.Toprovetheergodicitypartofthetheorem,wewillneedthefollowingstatements: 464.INVARIANTMEASURESSo,if!2 4,thenforevery2[!]H\B\ 3(whichisaPH(!;
)-fullmeasureset),wehave1 NN�1Xi=0Xn(i)!Z PH(!;d)Xn();N!1:WeseethatthecriterionofLemma4.1issatised,someasuresPH(!;
)areergodicfor!2 4.3.AbsolutelycontinuousinvariantmeasuresSupposenowthat =(Rd;B(Rd))andisadierentiabletransforma-tion.InsomecasesitisreasonabletoexpectthatthereisanabsolutelycontinuousinvariantmeasurePfor.AbsolutecontinuityofPwithrespecttoLebesguemeasuremeansthatthereisanL1(Rd;B(Rd);Leb)functioncalleddensitysuchthatforanysetB2B(Rd),P(B)=ZBf(!)d!:Densitiesarenotuniquelydenedsincetheycanbemodiedonazeromeasureset.Inwhatfollows,foradierentiablefunctionf:Rm!RnwedenotebyDf(x)theJacobimatrixoffatpointx:(@ifj(x))i=1;:::;mj=1;:::;n.Theorem4.7.LetP(dx)=(x)dxbeameasureon(Rd;B(Rd)).Sup-pose:Rd!Rdisdierentiableandnondegenerateforalmostallx(thelattermeansLebfx:detD(x)=0g=0).ThenP�1isalsoabsolutelycontinuouswithrespecttoLebesguemeasureandthesocalledtransferop-eratororPerron{FrobeniusoperatorLappliedto:(4.14)L(x)=Xy2�1x(y) jdetD(y)j;x2Rd;givesadensityofP�1.Hereweadopttheusualconventionthatsummationoveranemptysetiszero.Sincedensitiesthatdieronlyonasetofzeromeasuredenethesamemeasure,itmakessensetoconsidertheirequivalenceclassesandlookatthetransferoperatorasatransformationinL1understoodasaspaceofequivalenceclasses.Inparticular,wehavethefollowingstatement.Theorem4.8.UndertheconditionsofTheorem4.7,anabsolutelycon-tinuousmeasurePisinvariantundertransformationifandonlyifidentity(4.15)L(x)=(x)holdsforLeb-almostallx2Rd.Inotherwords,isaxedpointofthetransferoperatorLor,equivalently,itseigenfunctionwitheigenvalue1. 484.INVARIANTMEASURESTheright-handsideequalsXm2N1 1+1 x+m1 (x+m)2=Xm2N1 (x+m+1)(x+m)=Xm2N1 x+m�1 x+m+1=1 1+x=(x):Problem4.6.SupposeanopensetURd,apointx02U,andamap:Rd!Rdsatisfylimn!1nx=x0forallx2U.Theneveryabsolutelycontinuousinvarantmeasuremustsatisfy(U)=0.Problem4.7.Supposeadierentiablemap:Rd!RdsatisesjD(x)j1forallxinaboundedopenforwardinvariantsetU.Theneveryabsolutelycontinuousinvarantmeasuremustsatisfy(U)=0.Onecaninterprettheseresultsasnonexistenceofabsolutelycontinuousinvariantmeasuresdueto\deregularizing"propertiesofthetransferopera-torL.Namely,duetocontractioninthephasespace,undertheiterationsofLthedensitiesbecomelargeandtendstoabnormallyconcentrate.Infact,insituationswhereLissucientlyregularizingorsmoothening,onecanproveexistence.Theorem4.10(A.Renyi,1953).Suppose( ;F)=([0;1);B).Letm2N,m&#x]TJ/;༕ ; .96;& T; -3;P.9; -;.9;U T; [0;1andf:[0;1]![0;m]satisesf(0)=0;f(1)=mandf0(x)&#x]TJ/;༕ ; .96;& T; -3;P.9; -;.9;U T; [0;1forallx2[0;1].Thenthetransformationdenedbyx=ff(x)ghasanabsolutelycontinuousinvariantmeasure.Iff2Crforsomer2,thentheinvariantdensityhasaCr�1version.Theorem4.11(A.LasotaandJ.A.Yorke,1973).Let:[0;1]![0;1]beapiecewiseC2functionsuchthatinfj0j&#x]TJ/;༕ ; .96;& T; -3;P.9; -;.9;U T; [0;1.Thenthereisa-invariantmeasureabsolutelycontinuouswithrespecttoLebesguemeasureandsuchthatitsdensityhasaversionwithboundedvariation.Ideaoftheproof:Itturnsoutthatthefollowing\Lasota{Yorke"inequalityholds.TherearenumbersN2N,&#x]TJ/;༕ ; .96;& T; -3;P.9; -;.9;U T; [0;0and2(0;1)suchthatforeveryfwithboundedvariationV(f),V(LNf)kfkL1+V(f):So,takinganyf0anddenotingfk=Lkf,weobtainV(fNk)kfN(k�1)kL1+V(fN(k�1))kfkL1+V(fN(k�1));whereweusedthefactthatiff0andf2L1,thenLf0andkLfkL1=kfkL1.Iteratingthisinequality,weobtainV(fNk)(1++2+:::+k�1)kfkL1+kV(f):Therefore,limsupk!1V(fNk)(1�)�1kfkL1; CHAPTER5MarkovProcessesThegoalofthischapteristodevelopgeneralergodictheoryofstochasticprocesseswithinstantaneouslossofmemory.1.BasicnotionsHerewebeginstudyingsituationswherethestateofthesystemattimen+1isnotuniquelydeterminedbythestateattimen.Wewillmostlybeinterestedinprocesseswithinstantaneouslossofmemory.TheyareusuallycalledMarkovprocessesandaredenedbythefollowingproperty:giventhehistoryoftheprocessXuptotimen,thestateattimen+1israndomandhasconditionaldistributionthatdependsonlyonthevalueXn.Definition5.1.Let(X;X)beameasurablespace.AfunctionP:XX!Riscalledatransitionprobabiliy,ortransitionkernel,orprobabilitykernelifforeachx2X,P(x;
)isaprobabilitymeasure,andforeachB2X,P(
;B)isanX-measurablefunction.Throughoutthissectionweassumethat(X;X)isaBorelspace.WealsoxatransitionkernelP(
;
)on(X;X).Foranymeasureon(X;X)(thatwillserveastheinitialdistribution)andtimen2Z,wearegoingtodeneameasureonpathstartingatn,i.e.,onthespace(Xfn;n+1;:::g;Xfn;n+1;:::g).Definition5.2.Let(X;X)beaBorelspace.Letbeaprobabilitymeasureona(X;X)andletP(
;
)beatransitionkernelon(X;X).An(X;X)-valuedprocessXdenedonaprobabilityspace( ;F;P)withtimeindexedbyfn;n+1;n+2;:::gforsomen2ZiscalledahomogeneousMarkovprocesswithinitialdistributionandone-steptransitionprobabilityP(
;
)ifforanyk0andanysetsA0;A1;:::;Ak2X,(5.1)PfXn2A0;:::;Xn+k2Akg=ZA0(dx0)ZA1P(x0;dx1):::ZAk�2P(xk�2;dxk�1)ZAk�1P(xk�1;Ak):TheexistenceanduniquenessofameasureP=Pn;1onthecanoni-calspace(Xfn;n+1;:::g;Xfn;n+1;:::g)withnite-dimensionaldistributionsde-scribedby(5.1)followsfromtheKolmogorov{Daniellconsistencytheorem.Problem5.1.Showthatformula(5.1)denesaconsistentfamilyofnite-dimensionaldistributions.51 1.BASICNOTIONS53Theorem5.3.Foranymeasure,anym;k0,anyA0;A1;:::;Ak2X,P(Xm2A0;:::;Xm+k2AkjFm)a:s:=P(Xm2A0;:::;Xm+k2AkjXm)a:s:=1Xm2A0ZA1P(Xm;dx1)ZA2P(x1;dx2):::ZAk�2P(xk�2;dxk�1)ZAk�1P(xk�1;Ak)a:s:=Pm1Xm(Xm2A0;:::;Xm+k2Ak):Proof:Thisfollowsdirectlyfromtheprevioustheoremandthefactthat1Xm2A0is(Xm)-measurableTheorem5.4.Foranymeasure,anym;k0,andanyboundedmeasurablef:Xk+1!R,EP(f(Xm;:::;Xm+k)jFm)a:s:=EP(f(Xm;:::;Xm+kjXm))a:s:=ZXP(Xm;dx1)ZXP(x1;dx2):::ZXP(xk�2;dxk�1)ZXP(xk�1;dxk)f(Xm;x1;:::;xk):a:s:=EPm1Xmf(Xm;x1;:::;xk):Proof:TobeinsertedlaterInthefollowingtheoremweusethestandardshiftoperatoron(XZ+;XZ+).Theorem5.5.Letm0,andletA2XZ+.ThenP(�mAjFm)a:s:=P(�mAjXm)a:s:=PXm(A):Proof:IfA2XZ+,then�mA=f(x0;x1;:::):(xm;xm+1;:::)2Ag2Xm1:Thetheoremfollowsfromapproximating�mAbycylindersetsbasedoncoordinatesstartingwiththem-thone,andusingTheorem5.4.Onecaneasilyprioveanextensionofthistheoremwhereindicatorsarereplacedbyarbitraryboundedfunctions:Theorem5.6.Letm0,andletH:XZ+!Rbeaboundedrandomvariable.ThenEP(HmjFm)a:s:=EP(HmjXm)a:s:=EPXmH:Forexample,foreverysetB2XZ+,(5.3)P(B)=ZX(dx0)Px0(B):ThisidentityfollowsfromP(B)=EP1B=EPEP(1BjF0)=EPPX0(B)=ZX(dx0)Px0(B): 2.STATIONARYMARKOVPROCESSESANDINVARIANTDISTRIBUTIONS55Proof:IfPdenesastationaryprocess,thenthedistributionsofX0andX1mustcoincide,so(5.5)holds.Nowsupposethat(5.5)holds.Thenforallm,wehavePm=,i.e.,thedistributionofXmisalso.Weneedtocheckthatforallm2Z+,allk2N,(5.6)PfXm2A0;:::;Xm+k2Akg=PfX02A0;:::;Xk2Akg:WecanwritePfXm2A0;:::;Xm+k2Akg==EPP(Xm2A0;:::;Xm+k2AkjXm)=EP1Xm2A0P(Xm+12A1;:::;Xm+k2AkjXm)=EP1Xm2A0ZA1P(Xm;dx1)ZA2P(x1;dx2):::ZAk�2P(xk�2;dxk�1)ZAk�1P(xk�1;Ak):Denotingfory2Xf(y)=1x2A0ZA1P(y;dx1)ZA2P(x1;dx2):::ZAk�2P(xk�2;dxk�1)ZAk�1P(xk�1;Ak);wecanwritePfXm2A0;:::;Xm+k2Akg=EPf(Xm)=ZX(dy)f(y);sincethedistributionofXmis.Thisexpressionontheright-handsidedoesnotdependonm,sotheproofof(5.6)iscomplete.Definition5.3.Anydistributionthatsatises(5.5)iscalledP-invariant.IfXisnite,sayX=f1;:::;NgforsomeN2N,thenanymeasureandanytransitionkernelPareuniquelydenedbytheirvaluesonsingle-pointsets.Denotingi=figandPij=P(i;fjg)foralli;j2X,weobtainthatisP-invarianti(5.7)Xi2XiPij=j:Inotherwords,theidentityP=canbeunderstoodinthelinearalgebrasense.HereisarowvectorandPisasquarematrix.Forexample,letX=f1;2;3gwithX=2X,andP(x;A)=jAnfxgj=2foranyAXwhichmeansthat,giventhatatanytimestepthesystemchangesitscurrentstatetoanewonechosenuniformlyfromtheremainingtwostates.DenotingPij=P(i;fjg)=(1 2;j6=i;0;j=i;weseethatthesystem(5.7)hasalineofsolutions1=2=3.Sinceweareinterestedinprobabilitymeasures,wehavetoset1=2=3=1=3,andsothistransitionkernelhasauniqueinvariantdistribution. 2.STATIONARYMARKOVPROCESSESANDINVARIANTDISTRIBUTIONS57Wealreadyknowthatdeterministictransformationscanfailtohaveanyinvariantdistributionsatall.ThesameappliestoMarkovtransitionker-nels.Infact,anydeterministictransformationonaspace(X;X)naturallygeneratesaMarkovtransitionkernelgivenbyP(x;
)=(x),i.e.,foranymeasureon(X;X),P=�1.Sobasicexamplesofdeterministictrans-formationswithoutinvariantdistributionsnaturallygenerateexamplesofMarkovkernelswithoutinvariantdistributions.IfthesetofinvariantdistributionsforPisnonempty,thenithasastructuresimilartothatofthesetofinvariantmeasuresfordeterministictransformations.Tomakemoreconcretestatementsweneedtointroduceseveralnewnotions.Definition5.4.LetP(
;
)beaprobabilitykernelon(X;X)andletbeaP-invariantprobabilitymeasureon(X;X).AsetAiscalled(P;)-invariantiffx2A:P(x;Ac)�0g=0:Thesetofall(;P)-invariantsetsisdenotedbyI(;P).Lemma5.1.LetP(
;
)beaprobabilitykernelon(X;X)andletbeaP-invariantprobabilitymeasureon(X;X).ThesetI(;P)isa-algebra.Proof:Itisobviousthat 2I(;P)sinceP(x; )=1forallx.IfA1;A2;:::2I(;P),thenforA=Si2NAi,wehavefx2A:P(x;Ac)�0gXifx2Ai:P(x;Ac)�0gXifx2Ai:P(x;Aci)�0g=0:SupposenowA2I(;P).LetusprovethatAc2I(;P).Wehave(A)=ZX(dx)P(x;A)=ZA(dx)P(x;A)+ZAc(dx)P(x;A):Thersttermontherightequals(A)becausefor-almostallx2A,P(x;A)=1.Therefore,thesecondtermiszero,whichimpliesthatP(x;A)=0for-almostallx2Ac.Definition5.5.LetP(
;
)beaprobabilitykernelon(X;X)andletbeaP-invariantprobabilitymeasureon(X;X).Wesaythatthepair(;P)isergodicifforevery(;P)-invariantsetA,(A)2f0;1g.Ergodicitymeansthatonecannotdecomposethesystemintotwosys-temsthatcanbestudiedindependently.WewillalsooftensaythatisP-ergodicif(;P)isanergodicpair.Theorem5.8.SupposeisaninvariantmeasureforaMarkovkernelP(
;
)on(X;X).AsetB2XZ+belongstothe-algebraI(XZ+;XZ+;P;) 2.STATIONARYMARKOVPROCESSESANDINVARIANTDISTRIBUTIONS59and(5.15)f(nx)1Bc(x)L1�!1B(x)
1cB(x)=0:UsingtheinvarianceofB,wecanrewrite(5.14)and(5.15)as(5.16)f(nx)1B(x)L1�!1B(x);and(5.17)f(nx)1Bc(x)L1�!0;SinceEPf(nx)1B(x)=EPf(x)1B(x);theconvergence(5.16)impliesEPf(x)1B(x)=EP1B(x),and(5.12)fol-lows.SinceEPf(nx)1Bc(x)=EPf(x)1Bc(x);theconvergence(5.17)impliesEPf(x)1B(x)=0,and(5.13)follows.Theproofof(5.11)iscompleted,and(5.10)isestablished.Wecannowuse(5.10)towrite1Ba:s:=(Xn)foralln2Z+.Weseethattakesvalues0and1withprobability1.Therefore,(Xn)=1Xn2AforasetA2X.So,BP=fXn2Agforalln2Z+and,moreover,BP=TfXn2Ag=AZ+.LetusprovethatAis(;P)-invariant.WehaveZA(dx0)P(x0;A)=PfX02A;X12Ag=P(B)=PfX02Ag=(A):Therefore,P(x0;A)=1for-a.e.x02A.Definition5.6.LetusdenotebyI0the-subalgebraofXZ+generatedbysetsfX02Ag;A2I(;P).Remark5.1.Theorem5.8maybeinterpretedas(5.18)I(XZ+;XZ+;P;)P=I(XZ+;XZ+;P;)P=I0:Lemma5.2.Letg:X!RbeX-measurableandbounded.ForP-almosteveryx2X,(5.19)ZXZ+P;I0(x;dy)g(y0)=ZXI(x0;dy0)g(y0);whereI(
;
)=I(;P)isaregularconditionalprobabilitywithrespecttoI(;P).Proof:Sincebothsidesof(5.19)areI0-measurable,itsucestocheckthatforeveryB2I0,(5.20)ZBP(dx)ZXZ+P;I0(x;dy)g(y0)=ZBP(dx)ZXI(x0;dy0)g(y0): 2.STATIONARYMARKOVPROCESSESANDINVARIANTDISTRIBUTIONS61ProofofTheorem5.10:Supposethat1and2arenotmutuallysingular.Let;1;2bedenedaccordingtoLemma5.3.ThenP1=P+P1;(5.23)P2=P+P2:(5.24)DuetoLemma5.9,measuresP1andP2areergodicandhencemutuallysingular.However,thiscontradictsLemma5.3anddecompositions(5.23){(5.24).Theorem5.11.LetbeaprobabilitymeasureonaBorelspace(X;X),invariantunderP(
;
).LetI(
;
)bearegularconditionalprobabilitywithrespecttoI(;P).Thenfor-almosteveryx0,themeasureI(x0;
)isP-invariant,formsanergodicpairwithP,andisamixtureorconvexcombinationofthoseergodicmeasures:=ZX(dx0)I(x0;
);i.e.,(5.25)(A)=ZX(dx0)I(x0;A);A2X;and,moregenerally,(5.26)ZX(dx0)f(x0)=ZX(dx0)Z I(x0;dy0)f(y0);f2L1(X;X;):Proof:WestartwiththeergodicdecompositionforthemeasurePde-scribedinTheorem4.6.WeknowthatforP-almosteveryx2XZ+,themea-sureP;I=P;I(XZ+;XZ+;P;)(x;
)is-invariantandergodic.Dueto(5.18)andpart3ofTheorem4.4,wealsohavethatforP-almosteveryx2XZ+,P;I(XZ+;XZ+;P;)(x;
)=P;I0(x;
):Letusprovethatforalmosteveryx,P;I0(x;
)isaMarkovmeasureon(XZ+;XZ+).Wecanassumethat(X;X)istheunitsegmentwithBorel-algebra.LetustakeacountablesetDdenseinC[0;1].ThevaluesofEP[f0(X0)f1(X1):::fn(Xn)];n2Z+;f0;f1;:::;fn2D;uniquelydenethemeasurePon(XZ+;XZ+).SoletuscomputeEP;I0(x;
)[f0(X0)f1(X1):::fn(Xn)]=EP[f0(X0)f1(X1):::fn(Xn)jI0](x)=EP[EP[f0(X0)f1(X1):::fn(Xn)jF0]jI0](x): 3.FILTRATIONS.STOPPINGTIMES.STRONGMARKOVPROPERTY63Thestandardinterpretationofltrationsisthatforeveryn2Z+,Fnisthecollectionofeventsobservedonthetimeintervalf0;1;:::;ng.OnecanfullydecidewhetheraneventA2Fnistruebyobservingallavailableinformationuptotimen.Definition5.8.Let(Xn)n0beastochasticprocess.Thenthel-tration(Fn)n0denedbyFn=(X0;:::;Xn)isthenaturalltrationof(Xn)n0orltrationgeneratedby(Xn)n0.InthiscaseFncosnsistsofallsetsthatcanbedescribedintermsofrandomvariablesX0;:::;Xn,n0.OnecandecidewhetheraneventA2FnistruebyobservingthetrajectoryoftheprocessXuptotimen.Definition5.9.Let(Xn)n0beastochasticprocess.Wesaythat(Xn)n0isadaptedtoaltration(Fn)n0ifforeveryn,therandomvariableXnismeasurablewithrespecttoFn.Inparticular,anyprocessisadaptedtoitsownnaturalltration.Definition5.10.Arandomvariable: !Z+[f+1giscalledastoppingtimewithrespectto(Fn)n2Z+iffng2Fnforeveryn0.Problem5.5.Checkthatinthedenitionaboveonecanreplacefng2Fnbyf=ng2Fnforeveryn0.Thefollowingisoneofthemostusefulexamplesofhittingtimes.Let(Xn)n0bean(X;X)-valuedstochasticprocessadaptedtoaltration(Fn)n0.Thismeansthatforeveryn,therandomvariableXnismeasurablewithrespecttoFn.LetAbeanysetinX,andlet(5.27)A=inffn2N:Xn2Ag2Z+[f+1g:ThenAisastoppingtimesincefA=ng=n�1\k=1fXk=2Ag\fXn2Ag:Definition5.11.The-algebraassociatedtoaltration(Fn)n0andastoppingtimewithrespectto(Fn)n0isdenedbyF=fA2F:A\f=ng2Fng.Problem5.6.ProvethatFisa-algebra.Giveanexampleofaltrationandarandomvariable(thatisnotastoppingtime)suchthatFdenedaboveisnota-algebraThe-algebraFisinterpretedasthe-algebraofeventsobservedonarandomtimeintervalf0;1;:::;g.Inotherwords,everyeventfromFcanbedescribedintermsoftheinformationavailableuptotime.If(Fn)n0isthenaturalltrationof(Xn)n0,theneventsfromFcanbedescribedintermsoftherealizationofXuptotime,i.e.,oncandecidewhetheraneventA2FistrueornotbasedonthetrajectoryX0;X1;:::;X(ofrandomlength). CHAPTER6MarkovProcessesonFiniteStateSpacesThisisnotquiteproofread.Proceedwithcaution.1.ApproachbasedontheabstracttheoryInthissectionweapplythepowerfulresultswehaveobtainedtoarelativelysimplecasewhereXisnite.WexN2NandassumethatX=f1;:::;Ng,andX=2X.Everyprobabilitymeasureon(X;X)inthissectionwillbeidentiedwithaprob-abilityvector(i)i2Xwithi=fig.EveryMarkovkernelPon(X;X)willbeidentiedwithatransitionmatrix(Pij)Ni;j=1denedbyPij=P(i;fjg).LetusstudythesetofallinvariantdistributionswithrespecttoP.Ofcourse,theproblemisequivalenttondingallsolutionsof(6.1)P=satisfying2
N,where
N=p2RN:p1+:::+pn=1;andpi0foralli2f1;:::;Ng TheanalysiscanbeperformedwiththehelpofthePerron{Frobeniustheo-rem,butletususethetheoryofergodicdecompositioninstead.Thesimplex
Niscompactandconvex.Thevectorsubspacedenedby(6.1)isalsoconvex.Therefore,theintersectionisalsocompactandcon-vex(wewillshortlyseethatitisnon-empty)andcanbeseenastheconvexhullofitsextremepoints.Weknowfromtheabstractergodicdecomposi-tionthattheseextremepointsareergodicdistributions.Soletusestablishseveralusefulfacts.WesaythatasetAXisabsorbingwithrepecttoPifP(i;A)=1foralli2A.Lemma6.1.IfAXisabsorbingwithrespecttoP,thenthereisaP-invariantdistributionsatisfying(A)=1.Proof:LetususetheKrylov{Bogolyubovapproach.Letustakeanyinitialstatei2Aandconsidertheinitialdistributioniconcentratedati.Considerasequenceofmeasures(orprobabilityvectors)n=1 nn�1Xk=0iPk;n2N:65 1.APPROACHBASEDONTHEABSTRACTTHEORY67Lemma6.7.Leti2XandPi(B)�0,whereB=fXn6=iforalln2Ng:Theni=0foreveryinvariantmeasure.Proof:Duetotheergodicdecomposition,itissucienttoprovethatforeveryergodic,i=0.Supposei�0.Duetotheergodictheorem,P(A)=1,whereA=(1 nn�1Xk=01fXk=ig!i):Since1=P(A)=ZX(dj)Pj(A)=Xj2XjPj(A);weconcludethatPi(A)=1.Clearly,A\B=;,soPi(A)=1andPi(B)�0contradicteachother.Summarizingtheresultsaboveweobtainthefollowingtheorem.Theorem6.1.Leti2X.Ifthereisjsuchthati!jandj6!i,thenthereisnoinvariantdistributionsatisfyingi�0.Ifthereisnosuchstatej,thenthereisauniqueergodicdistributionsatisfyingi�0.ThesetA=fj2X:j�0gcoincideswithB=fj2X:i!jg.Proof:TherstpartfollowsfromLemma6.7.Toprovethesecondpart,wenotethatthesetBisanabsorbingclassofcommunicatingstates.Lemma6.5impliesthatthereisauniqueergodicmeasuresupportedbyB.Lemma6.3impliesj�0foreveryj2B.Ifallstatesformoneclass,thekernelPiscalledirreducubleorergodic.Thenthereisexactlyoneinvariantdistribution.Thissituationisoftencalleduniqueergodicity.Often,toanalyzetheinvariantdistributionthereisnobetterwaythetosolveequation(6.1).However,thereareusefulrepresentationsfortheinvariantdistributions.Onesuchrepresentationisthefollowing,takenfrom[FW12].WethinkofXasthecompletedirectedgraphwhereeachedge(ij)isassignedaweightPij.ToeachstateiweassociateacollectionGiofdirectedsubgraphsgwiththefollowingproperties:thereisnoarrowcomingoutofi,foreachj6=i,thereisexactlyonearrowcomingoutofj,andtherearenocycles.Foreverycollectiongofarrowswedene(g)=Y(jk)2gPjk:Foreveryi2X,wedeneQi=Pg2Gi(g).Wealsoset(6.2)qi=Qi Pj2XQj: 1.APPROACHBASEDONTHEABSTRACTTHEORY69Theorem6.3.Suppose(Pij)isanirreduciblekernel.Thenitisre-versiblewithrespecttosomemeasureiforeverynandeverysequenceofverticesi1;i2;:::;in,(6.4)Pi1i2Pi2i3:::Pin�1inPini1=Pi1inPinin�1:::Pi2i1Proof:Toderive(6.4)fromreversibility,itissucienttowrite(6.3)fori1;i2,fori2;i3,etc,taketheproductoftheseidentitiesandcancelQkikonbothsides.Ifweassumethat(6.4)holdstrueforallsequencesofstates,thenwecantakeanarbitraryi02Xandforeveryidenei=1 ZPi0i1Pi1i2:::Pin�1inPini PiinPinin�1:::Pi2i1Pi1i0;wherethesequence(i0;i1;:::;in)ischosensothatPiinPinin�1:::Pi2i1Pi1i0�0,andthenormalizationconstantZindependentofiistobechosenlater.Condition(6.4)impliesthatthisdenitiondoesnotdependonthechoiceofaspecicsequence(i0;i1;:::;in):If(i0;i01;:::;i0n)isanothersuchse-quence,weobtainPi0i1Pi1i2:::Pin�1inPini PiinPinin�1:::Pi2i1Pi1i0=Pi0i01Pi01i02:::Pi0n�1i0nPi0ni Pii0nPi0ni0n�1:::Pi02i01Pi01i00:Also,itisclearfrom(6.4)thatexistenceofacyclerealizingi0!i!i0impliesthati�0foralli.Similarly,weseethatifPij�0,thenPji�0,sotocheck(6.3),wewriteiPij=1 ZPi0i1Pi1i2:::Pin�1inPini
Pij PiinPinin�1:::Pi2i1Pi1i0
Pji Pji=1 ZjPji:Inapplications,onecanusetheideaofreversibilitytoconstructMarkovchainswithagiveninvariantdistribution.Ofteninsystemswithstatisticalmechanics avor,thereisnoeasyaccesstothevaluesithemselves,buttheratiosi=jareeasytocompute.Thishappens,forexample,forBoltzmann{Gibbsdistributionswherei=e�H(i) Z;i2X:HereH:X!Ristheenergyfunctionand0istheinversetemperature.SuchdistributionsforvariouschoicesofHareubiquitousinstatisticalme-chanics.LetusxsomeH.Somecharacteristicfeaturesofsuchafamilyofdistributionscanbeseenforlimitingcases!0and!1.When=0andthetemperatureisinnite,thenallstatesareequallyprobable,andthiscanbeinterpretedasacompletedisorder.Ontheotherhand,ifislarge(thetemperatureissmall),thestateswithminimalenergy(\ground"states)aremuchmoreprobablethanotherstates,andas!1,thesystem\freezes"tothegroundstates. 2.PERRON{FROBENIUSTHEOREM71andclosed,andsoistheirintersection
0.SinceTk+1
Tk
forallk2N,weobtain
06=;.Theconditionsofthetheoremimplythat(6.5)Tm
Int
;mn;whereInt
denotestherelativeinteriorof
.Therefore,
0Int
.Letusshowthat
0hasatmostdextremepoints.Namely,letuschooseanincreasingsequence(nk)k2Nsuchthatlimk!1Tnkei=xiforsomexi2
0,i=1;:::;d,andprovethatallext
0fx1;:::;xdg.Ifx2
0thenx2Tnk
forallk2NandthereforexisaconvexcombinationofextremepointsofTnk
.Alltheseareimagesofextremepointsof
underTnk,soweobtainarepresentationx=Pdi=1(k)iTnkeiforsome(k)i0satisfyingPdi=1(k)i=1,k2N.Choosinganincreasingsequence(k(m))m2Nsuchthatlimm!1(nk(m))i=iforsomei,weobtainx=Pdi=1ixi,soifxisanextremepointithastocoincidewithoneofxi,i=1;:::;d.Since
0=T
0andext
0isnite,weobtainext
0=Text
0,inotherwordsTactsapermutationonext
0.Thereforethereisanumberm2NsuchthatTmx=xforeveryx2ext
0.So,everyx2ext
0isaneigenvectorofAmwithapositiveeigenvalue.Letusprovethat,infact,therecannotbetwodistincteigenvectorswiththoseproperties.Therearetwocasestoexclude:(i)therearex;y2
0and�0suchthatx6=yAmx=xandAmy=y;(ii)therearex;y2
0and��0suchthatandAmx=xandAmy=y.Incase(i),thereisanumbersuchthatz=x+(y�x)2@
.ThereforeAmnz=nz2@C,soTmnz2@
whichcontradictsTmn
Int
.Incase(ii),wecanusey2Int
tochoose"�0smallenoughtoensurethatz=y�"x2IntC.ThenAkmz=ky�"kx,soforsucientlylargek,Akmz=2Cwhichisacontradiction.Weconcludethat
0containsauniquepointx0.Recallingthat
0isT-invariant,weobtainthatTx0=x0,i.e.,Ax0=x0forsome�0.LetusshowthatifAx=xforanyx2Cthatisnotamultipleofx0andsatisesx6=0,thenjj.If=,thenwecannd2Rsuchthatz=x0+x2@Cnf0g.ThenA2kz=2kzforallk2Nwhichcontradicts(6.5).If2Randjj&#x-278;,thenwehavealreadyprovedthatx62C.Alsowecannd"&#x-278;0smallenoughtoguaranteethatz=x0+"x2IntC.Thenforlargevaluesofm,thedirectionofA2m+1zwillbeclosetothedirectionofx,sowewillhaveA2m+1z=2CwhichcontradictsA-invarianceofC.Ingeneral,if=ei2forsome&#x-278;0and2R,thenthereisaplaneLsuchthattheactionofAonLismultiplicationbyandrotationbyangle. 3.HILBERTPROJECTIVEMETRICAPPROACH.73yr�1x.Thelatterisequivalenttoryx,sothesetwoinequalitiescanholdtogetheronlyifx=ry.WecannowdenetheHilbertprojectivemetricby(x;y)=�ln�(x;y)=�ln(x;y)�ln(y;x)2[0;1]:Noticethat(x;y)isinniteifx;y2@Candx6y.Thefollowingtriangleinequalityfollowsfrom(6.7):(x;z)(x;y)+(y;z);x;y;z2C:Fromthepropertiesof�,wederivethat(x;y)=(x;ry)foranyx;y2Candr�0,and(x;y)=0ixy,soisapseudo-metriconCandametricon
=C=.Otherwaystorepresentare(x;y)=�lnminixi yi
minjyj xj=lnmaxiyi xi
maxjxj yj=lnmaxi;jxjyi yjxi:LetusprovethatthemapT:
!
isnonexpanding.First,wenoticethatx�ry2CimpliesA(x�ry)=Ax�rAy2C.Therefore(Ax;Ay)(x;y),so(Ax;Ay)(x;y).AssumingnowthatalltheentriesofAarepositive,letusprovethatthecontractioncoecientisactuallystrictlylessthan1.FirstwerecallthatinthiscaseT
isacompactsubsetofInt
.Therefore,expressionsln(xjyi=(yjxi))areuniformlyboundedwithrespecttox;y2
,andD=diam(T
)=supf(x;y):x;y2
g1;or,equivalently=inff�(x;y):x;y2Cg&#x]TJ/;༷ ;.9;‘ ;&#xTf 1;
.51; 0 ;&#xTd [;0:Thesenumbersarerelatedby=e�D.Thefollowingtheoremestablishesthestrictcontractionpropertyof.Theorem6.5.IfAhasallpositiveentries,thenforallx;y2
,(6.8)(Tx;Ty)1�p  1+p (x;y):Proof:Letustaketwodistinctpointsx;y2
.Wecanassumethat(x;y)1,i.e.,bothnumbers1=(x;y)and2=(y;x)arenonzero.Theny�1x2Candx=2�y2C.Applyingthedenitionoftothosevectors,weobtainthattherearetwonumbers1;20suchthat12andA(x=2�y)1A(y�1x);A(y�1x)2A(x=2�y);i.e.,Ax1+1 1 2+11Ay;Ay1+2 2 1+2Ax: CHAPTER7CountablestatespaceMarkovchains1.StrongMarkovpropertywithrespecttostatehittingtimesAveryusefulwaytolookatMarkovchainsistostudyreturntimestoastate.InthissectionweassumethatX=f1;:::;NgorX=N(inthelattercasewesetN=1).Letusrecallthedenition(5.27).Foranarbitraryi2X,wedenei=figandnotethatPXi=Pionfi1g.Theorem7.1.LetPfi1g=1.ThenforanyboundedrandomvariableH:XZ+!R,andeverysetB2Fi,i.e.,(7.1)EP[Hi
1B]=EP[Hi]P(B):Proof:Forbrevitywedenote=iThestrongMarkovpropertyimpliesthatEP[H
1B]=EP[EP[H
1BjF]]=EP[1BEP[HjF]]=EP[1BEPXH]=EP[1BEPiH]=P(B)EPiHThisidentitywithB=XZ+givesEP[H]=EPiH:Thelasttwoidentitiestogetherimply(7.1).Theorem7.1meansthatthefutureandthepastofaMarkovprocesswithrespecttoavisittoastateiareindependentofeachother.Iteratingthisargument,weobtainthattherealizationoftheMarkovprocesscanbesplitinexcursions,eachexcursionbeingapathbetweentwoconsecutivevisitstoi.Ofcourse,thisislooselyformulated.Ihopetoaddmoretothislater.2.InvariantmeasuresandclassicationofstatesSomenotionsandresultsforcountablestateMarkovchainscanbeadaptedfromthenitestatespacesituation.Inparticular,Lemmas6.2{6.7stillholdtrueforcountableX.However,Theorem6.1doesnotholdtrueforcountablestatespace.Themaindierenceisthefollowing.Theonly75 2.INVARIANTMEASURESANDCLASSIFICATIONOFSTATES77Ifi=h,thetheright-handsideof(7.3)equals1Xk=0PhfXk+1=h;k+1=hg=Pfh1g:Theinvarianceoffollows,andsodoesergodicityanduniquenessoftheinvariantmeasure.whichcompletestheproof.Corollary7.1.Ifh2Xispositiverecurrentandistheergodicinvariantdistributionsatisfyingi&#x]TJ/;༷ ;.9;‘ ;&#xTf 1;
.51; 0 ;&#xTd [;0,thenh=1=Eii,or,equivalently(7.4)Ehh=1=h:Letusprovetheconversestatement.Theorem7.4.Supposeanergodicinvariantdistributionsatisesh&#x]TJ/;༷ ;.9;‘ ;&#xTf 1;
.51; 0 ;&#xTd [;0forsomeh2X.Then(7.4)holds.Proof:Letusdenote0h=inffk2N:Xk=hgandthenrecursively,n+1h=inffk2N:Xn+k=hg;n2N:Inotherwords,nhisthelengthofthen-thexcursionbetweentwocinsecutivevisitstoh.WealsosetS0=0andSn=1h+:::+nh;n2N:DuetothestrongMarkovproperty,randomvariables(nh)n2formani.i.d.sequence.Therefore,thestronglawoflargenumbersapplies,andweobtainSn na:s:�!Ehh2(0;1]:Thenumberofexcursionsorvisitstohuptotimem2NisgivenbyN(m)=maxfn0:Snmg;m0:WehaveSN(m)mSN(m)+1;m0:Therefore,SN(m) N(m)m N(m)SN(m)+1 N(m);m0:Bythestronglawoflargenumbers,bothl.h.s.andr.h.s.convergetoEhha.s.,andsodoesm N(m)a:s:�!Ehh:Ontheotherhand,ergodicityofimpiestheergodicityofP,andtheergodictheoremimpliesN(m) m=1 mm�1Xk=01fXk=hga:s:�!E1fX0=hg=PfX0=hg=h:Comparingtwolastdisplays,weobtainthestatementofthetheorem. 2.INVARIANTMEASURESANDCLASSIFICATIONOFSTATES79Therefore,V(x)1Xn=0PxfA�ng=1Xn=01Xk=n+1PxfA=kg=1Xk=1k�1Xn=0PxfA=kg=1Xk=1kPxfA=kg=ExA:Inparticular,PxfA=1g=0,andwealsoobtainA6=;.Sowehaveprovedthatforallx2Ac,(7.6)ExA1Infact,itimmediatelyfollowsthat(7.6)holdsforallx2X.LetusnowprovethattherearepositiverecurrenctstatesinA.WewilldosobyconstructingaP-invariantnitemeasureonXsuchthat(A)&#x]TJ/;༷ ;.9;‘ ;&#xTf 1;
.51; 0 ;&#xTd [;0.LetusconsideratransitionmatrixQijonAdenedbyQij=PifXA=jg;i;j;2A:ThistransitionmatrixdenesaMarkovkernelonanitesetA.ByLemma6.1,thereisaQ-invariantprobability.Foralli2X,wedeneitobetheaveragetimespentatibytheprocessXduringanexcursionbetweentwoconsecutivevisitstoA,initiatedaccordingtotheinitialdistribution.i=E1Xk=01fXk=i;kg=Xh2Ah1Xk=0PhfXk=i;kg:WeclaimthatthisisaninvariantmeasureforP.Forj=2A,(P)j=Xi2XXh2Ah1Xk=0PhfXk=i;kgPij=Xh2Ah1Xk=0PhfXk+1=j;k+1g=j:Forj2A,(P)j=Xi2XXh2Ah1Xk=0PhfXk=i;kAgPij=Xh2Ah1Xk=0PhfXk+1=j;A=k+1g=Xh2AhPhfXA=jg=Xh2AhQhj=j:Thisnishestheproofoftheclaimsincej=jforj2A.Since(A)&#x-278;0andallstatescommunicate,i&#x-278;0foralli.Therefore,allstatesarepositiverecurrentbyTheorem7.4. CHAPTER8Moregeneralstatespace.Minorizationconditions.Inthissectionweconsideranarbitrarystatespace(X;X)withoutmak-inganyassumptionsonthecardinalityofX.1.TheDoeblinconditionWesaythatatransitionprobabilityP(
;
)on(X;X)satisestheDoeblinconditionifthereisaprobabilitymeasureandanumberp2(0;1)on(X;X)suchthatP(x;A)p(A)forallx2XandallA2X.ThemeasureiscalledaminorizingmeasureforkernelP(
;
).UndertheDoeblincondition,wecanintroduceakernelQ(x;
)=(
)thatisidenticallyequaltoanddeneP(x;
)=P(x;
)�(
)=P(x;
)�pQ(x;
):ThenPisasubprobabilitykernel.Definition8.1.AfunctionR:XX![0;1]iscalledasubprobabilitykernelorsub-Markovkernelifi.e.,(i)foreveryx2X,R(x;
)isameasureon(X;X)satisfyingR(x;X)1and(ii)foreveryA2X,R(
;A)isX-measurable.Theorem8.1.IfDoeblin'sconditionholdstrue,thenthereisaninvari-antmeasure.Proof:WeuseQandPintroducedabovetodene(8.1)=p1Xn=0Pn=p1Xn=0(P�pQ)n:Thisseriesconvergesanddenesaprobabilitymeasuresince(X)=p1Xn=0(1�p)n=1:Letusprovethatisinvariant.WerepresentP=pQ+(P�pQ),notethatQ=foranymeasureon(X;X),andwriteP=pQ+(P�pQ)=p+p1Xn=0(P�pQ)n+1=p+p1Xn=1(P�pQ)n=p1Xn=0(P�pQ)n=:81 1.THEDOEBLINCONDITION83Theorem7.5,wecantrytoexecutethefollowingprogram:(i)ndasubsetAofaMarkovprocessispositiverecurrent,(ii)provetheDoeblinconditionfortheMarkovprocessobservedonlyattimesitvisitsA,(iii)constructaninvariantdistributionforthis\embedded"Markovprocess,and(iv)usethismeasuretoconstructaninvariantditributionfortheoriginalprocess.HowcanoneensurethatasetAispositiverecurrent?Theorem8.3.LetP(
;
)beatransitionkernelonaspace(X;X).Sup-posethereareafuctionV:X!R+andasetAXsuchthatforeveryx2A,LV(x)�1.ThenExAV(x)1forallx2Ac.TheproofofthistheoremliterallyrepeatstherstpartoftheproofofTheorem7.5.ThesetAcanoftenbechosentobemuchsmallerthanX.LetusconsideranARMA(1)examplewhereP(x;
)isgivenbyN(ax;1)witha2(0;1).LetustakeV(x)=x2andcomputePV.LetxN(ax;1).ThenPV(x)=E2x=a2x2+1=a2V(x)+1;OnecaneasilycheckthatPV(x)�V(x)�1ifx&#x]TJ/;༷ ;.9;‘ ;&#xTf 1;
.51; 0 ;&#xTd [;p 2=(1�a2),sowecanconcludethatthesetA=fx2R:jxjp 2=(1�a2)gispositiverecurrent.Letusintroducethefollowing(subprobability)kerneloftheembeddedMarkovprocess(observedonlywheninA):Q(x;B)=PxfA1;XA2Bg;x2A;B2XjA:ThisisaprobabilitykernelifPxfA1g=1forallx2A.ThiskerneloftensatisestheDoeblinconditions.Forexample,inourARMAexample,Q(x;B)PxfX12BgcjBj;x2A;B2XjAwherec=minjxj;jyj2A1 p 2e�(x�y)2=2�0:LetusnowunderstandhowtoconstructinvariantdistributionsusingembeddedMarkovprocesses.Theorem8.4.LetP(
;
)beaprobabilitykernelon(X;X).LetA2Xandameasureon(A;XjA)satisfy(1)EA1.(2)isQ-invariant,whereQisaprobabilitykernelon(A;XjA)denedbyQ(x;B)=PxfA1;XA2Bg:Thenthemeasuredenedby(B)=E1Xk=01fXk2B;kAg=ZX(dx)1Xk=0PxfXk2B;kAgB2X;isniteandP-invariant. 2.AHARRISPOSITIVERECURRENCECONDITION85SowearegoingtousetheDoeblinconditiontoprovethattheMarkovkernelQ(
;
)on(A;XjA)denedbyQ(x;B)=PxfmA1;XmA2Bg:hasaninvariantdistributionandthenapplytheapproachofTheorem8.4.Firstofall,Q(x;B)isindeedaprobabilitykerneldueto(8.3).Letusdene(B)=ZX(dy)PyfA1;XA2Bg;B2XjA:Thenisaprobabilitymeasureon(A;XjA)dueto(8.3)andCondi-tion(B).SinceQ(x;B)=ZXPm(x;dy)PyfA1;XA2BgZXp(dy)PyfA1;XA2Bg=p(B):thekernelQsatisestheDoeblinconditionandthereisa(unique)Q-invariantmeasure.Nowtoprovetheexistencepart,itremainstoprovethatthefollowingmeasureisniteandP-invariant:(B)=E1Xk=01fXk2B;kmAg=ZX(dx)1Xk=0PxfXk2B;kmAgB2X;andtheproofofthisfollowstheproofofTheorem8.4,althoughsomechangesareneeded.Problem8.1.ProvenitenessandP-invarianceof.Letusproveuniqueness.IftherearetwodistinctP-invariantdistri-butionsgivingpositiveweighttoA,thentherearetwodierentergodicdistributionsgivingpositiveweighttoA.IfisP-invariant,thenforanyB2X,(B)=Pm(B)ZA(dx)P(x;A)=(A)p(B);sothesemeasurescannotbemutuallysingular,acontradiction.Definition8.2.ThesetAthatappearsintheHarrisconditionisoftencalledasmallsetDefinition8.3.WesaythatA2Xisaccessiblefromx2XifPm(x;A)&#x-278;0forsomem.Letusdenoteallpointsx2XsuchthatAisaccessiblefromxby(A).IfadistributionisP-invariant,then((A))&#x-278;0implies(A)&#x-278;0.So,onecanstrengthenTheorem8.5inthefollowingway: 3.COUPLINGANDCONVERGENCEINTOTALVARIATIONUNDERTHEDOEBLINCONDITION87Letusprovethatisinvariant.LetusrstnoticethatP(B)=ZX(dx)P(x;B)=ZAc(dx)P(x;B)+(1�p)ZA(dx)R(x;B)+pZA(dx)(B)=1Xr=0~PfXr2Ac;Xr+12B;r+1g+1Xr=0~PfXr2A;Xr+12B;r+1g+1Xr=0~PfXr2A;Xr+12B;=r+1g:Nowwe(i)combinethersttwosumsintooneand(ii)usetheinvarianceofundertheembeddedMarkovchaintoderive1Xr=0~PfXr2A;Xr+12B;=r+1g=(B):TheresultisP(B)=1Xr=0~PfXr+12B;r+1g+(B)=1Xr=1~PfXr2B;rg+~PfX02B;0g=1Xr=0~PfXr2B;rg=(B);whereweused1inthesecondindentity.3.CouplingandconvergenceintotalvariationundertheDoeblinconditionInthissectionweintroduceaveryimportanttoolcalledcoupling.TheideaistobenetfromrealizingseveralMarkovprocesseswiththesametransitionmechanismononeprobabilityspace.WewillgiveadierentproofofuniquenessofproofofuniquenessofaninvariantdistributionundertheDoe-blincondition:Let1and2betwodierentinvariantdistributions.Letusprovethattheyareequaltoeachotherusingthecouplingmethod.LetusorganizeaMarkovprocesson(XX;XX)withinitialdistribution12andspecialtransitionprobabilitiesP((x1;x2);
)thatweproceedtodescribe.So,wearegoingtodescribethedistributionof(X1n+1;X2n+1)conditionedon(X1n;X2n)=(x1;x2). 3.COUPLINGANDCONVERGENCEINTOTALVARIATIONUNDERTHEDOEBLINCONDITION89Theorem8.7.LetDoeblin'sconditionbesatised.Ifistheinvariantdistribution(theuniquenessofwhichisguaranteedbyTheorem8.2),thenforanyotherinitialdistribution,kPn�kTV(1�p)n: CHAPTER9Takingthetopologyintoaccount1.FellerpropertyandexistenceofinvariantmeasurestheviaKrylov{BogolyubovapproachSomeregularitypropertiesoftransitionkernelsareconvenienttoformu-lateintermsoftheoperatorPonboundedfunctionsorthesemigroup(Pn)n2Z+generatedbyP.Let(X;d)beametricspacewithBorel-algebraX.Definition9.1.AtransitionprobabilityP(
;
)on(X;d)isFellerifforeveryboundedcontinuousf:X!R,Pfisalsocontinuous.FellerpropertymeansthatP(x;
)iscontinuousinxinthetopologyofweakconvergence.LetusnowstateananalogueoftheKrylov{BogolyubovtheoremforMarkovprocesses.Theorem9.1.Letbeaprobabilitymeasureon(X;X)suchthatthesequence(n)n2Ndenedby(9.1)n=1 nn�1Xj=0Pj=1 nn�1Xj=0Pj;n2Nistight.ThenthereisaP-invariantmeasure.Also,condition(9.1)isguaranteedbytightnessofthesequence(Pn)n2N.Proof:ByProkhorov'stheorem,tightnessimpliesrelativecompactness,sothereisasequencenk"1andaprobabilitymeasuresuchthatnkconvergestoweakly.ToprovethatP=,itissucienttoprove(9.2)Pf=fforallboundedcontinuousfunctionsf.Forsuchafunctionf,theFellerpropertyguaranteesthatPfiscontinuous,soduetotheweakconvergence,91 3.STRONGFELLERPROPERTYANDUNIQUENESS.932.ApplicationstoSDEs,Stochasticheatequation,stochasticNavier{StokesequationWillllinlater.3.StrongFellerpropertyanduniqueness.Definition9.2.TheresolventkernelisdenedbyQ(y;A)=Xn2N2�nPn(y;U);y2X;A2X:Definition9.3.Thesupportofaprobabilitymeasureon(X;X)de-notedbysuppconsistsofallpointsx2XsuchthatforeveryopensetUcontainingx,(U)�0Definition9.4.Apointx2XiscalledP-acccessibleifforally2X,x2suppQ(y;
).ThesetofallP-accessiblepointswillbedenotedbyAcc(P).Lemma9.1.ForeveryP-invariantmeasure,Acc(P)supp().Proof:Since=P,wecomputeQ=Xn2N2�nPn=Xn2N2�n=;i.e.,isalsoinvariantunderQ.Therefore,foreveryx2Acc(P)andeveryopensetUcontainingx,weobtain(U)=ZX(dy)Q(y;U)�0:Definition9.5.ThekernelPiscalledstrongFellerifforeveryboundedmeasurablefunctionf:X!R,Pfisacontinuousfunction.AnimmediatecorollaryofthisdenitionisthatforastrongFellerkernelPandanysetA2X,P(x;A)=P1A(x)isacontinuousfunction.WhiletheusualFellerpropertymeansthatP(x;
)convergestoP(y;
)weaklyasx!y,thestrongFellerpropertymeansconvergencealmostasstrongastotalvariationconvergence.Anycontinuousdeterministicmap:X!XgeneratesaMarkovkernelP(x;
)=(x)thatisFellerbutnotstrongFeller.OnecanalsosaythatstrongFellertransitionprobabilitiesimprovetheregularitybysmootheningwhereastheusualFellerkernelsonlypreservetheregularity.OneobviousconsequenceofthestrongFellerpropertyisthatifP(x;A)�0forsomexandA,thenP(y;A)�0forallythataresucientlyclosetox.InparticularP(x;
)andP(y;
)cannotbemutuallysingularifxandyarecloseenough. CHAPTER10Randomdynamics1.StabilityinstochasticdynamicsLetusbeginwithaverysimpleexample,adeterministiclineardynam-icalsystemwithonestablexedpoint.WetakethephasespaceXtobethereallineRanddenethetransfor-mationf:R!Rbyf(x)=ax;x2R,whereaisarealnumberbetween0and1.Toanypointx2Xonecanassociateitsforwardorbit(xn)1n=0,wherexn=fn(x),n0.Inthissimpleexample,theanalysisoftheasymptoticbehavioroftheforwardorbit(xn)1n=0asn!1,isstraightforwardanddoesnotrequireergodictheory.Namely,zeroisauniquexedpointofthetransformation:f(0)=0,andsincexn=anx,n2Nanda2(0;1)weconcludethatasn!1,xnconvergestothatxedpointexponentiallyfast.Therefore,0isastablexedpoint,oraone-pointglobalattractorforthedynamicalsystem(R;f).Inotherwords,thedomainofattractioncoincideswithR.So,duetothecontractionthatispresentinthemapf,thereisafastlossofmemoryinthesystem,andnomatterwhattheinitialconditionis,itgetsforgotteninthelongrunandthepointsxn=fn(x)approachthestableequilibrium0asn!1.Sofarwehavebeenassumingthattheevolutionbeginsattime0,butthepicturewouldnotchangeifweassumethattheevolutionbeginsatanyotherstartingtimen02Z.Infact,sincethemapfisinvertibleinourexample,thefull(two-sided)orbit(xn)n2Z=(fnx)n2ZindexedbyZiswell-denedforanyx2R.Letusnowmodifythedynamicalsystemofourrstexamplealittlebyaddingnoise,i.e.,arandomperturbationthatwillkickthesystemoutofequilibrium.TheresultingmodelisoftencalledARMA(1)(Autoregressivemovingaveragemodeloforder1).Letusconsidersomeprobabilityspace( ;F;P)richenoughtosupportasequence(n)n2ZofindependentGaussianrandomvariableswithmean0andvariance2.Foreveryn2Zwewillnowdenearandommapfn;!:R!Rbyfn;!(x)=ax+n(!):Anaturalanalogueofaforwardorbitfromourrstexamplewouldbeastochasticprocess(Xn)nn0emittedattimen0frompointx,i.e.,satisfying95 1.STABILITYINSTOCHASTICDYNAMICS972=D,i.e.,D=2=(1�a2).Therefore,thecenteredGaussiandistributionwithvariance2=(1�a2)isinvariantandgivesrisetoastationaryprocess.Thereareseveralwaystoestablishuniquenessofthisinvariantdistribution,e.g.,thecelebratedcouplingmethodintroducedbyDoeblinin1930'swhichalsoallowstoprovethatforanydeterministicinitialdata,thedistributionofXnconvergesexponentiallyfasttotheuniqueinvariantdistributionintotalvariationasn!1.Anotherwaytoapproachstabilityisstudyingrandomattractors.Letusconvinceourselvesthatinourexample,therandomattractorcontainsonlyonepointandthatpointisaglobalsolution(Xn)n2ZdenedbyXn=X(n;n�1;n�2;:::)=n+an�1+a2n�2+::::Clearly,thisseriesconvergeswithprobability1.Moreover,foranyn2Z,aXn�1+n=n+a(n�1+an�2+a2n�3+:::)=n+an�1+a2n�2+a3n�3+:::=Xn;and(Xn)n2Zisindeedatwo-sidedorbit,i.e.,aglobal(intime)solutionofequation(10.1)denedontheentireZ.NoticethatXnisafunctionalofthehistoryoftheprocessuptotimen.Sincetheprocessisstationary,thusconstructed(Xn)n2Zisalsoastationaryprocess.Infact,XniscenteredGaussianwithvariance2+a22+a42+:::=2=(1�a2);whichconrmsourpreviouscomputationoftheinvariantdistribution.Letusnowinterprettheglobalsolution(Xn)n2Zasaglobalattractor.Weknowfromthecontractionestimate(10.2)thatforanyx2R,(10.3)jm;n(x)�Xnj=jm;n(x)�m;n(Xm)j=an�mjx�Xmj:UsingthestationarityofXmandapplyingintegrationbyparts,weseethatXm0PfjXmj&#x]TJ/;༴ ;.97;
T; 14;&#x.077;&#x 0 T; [0;jmjgXm0PfjX0j&#x]TJ/;༴ ;.97;
T; 14;&#x.077;&#x 0 T; [0;jmjgEjX0j1:TheBorel{CantellilemmaimpliesnowthatXmgrowsatmostlinearlyinm(infact,onecanprovemuchbetterestimatesonthegrowthrateofXmsinceitisaGaussianstationaryprocess),and(10.3)impliesthatforanyx2R,limm!�1jm;n(x)�Xnj=0:Inwords,ifwexaninititalconditionxandruntherandomdynamicsfromtimemtotimen,thentheresultofthisevolutionconvergestothespecialglobalsolutionXnaswepullthestartingtimembackto�1.Thisallowsustocall(Xn)n2Zaone-pointpullbackattractorforourrandomdynamicalsystem.Pullbackattractorswereintroducedin[CF94],[CDF97],[Sch94]. 2.MARKOVPROCESSESANDRANDOMTRANSFORMATIONS99Xk=fk(Xk�1)=F(Xk�1;fk).Foranyk,thusconstructedfunctionsXkarerandomvariableson(X ;XF)whichfollowsfromaninductiveargument.Infact,Xn(x;!)=xisarandomvariable,andifXk�1(x;!)isarandomvariable,thenXk(x;!)=fk;!(Xk�1(x;!))=F(fk;!;Xk�1(x;!));andXkismeasurableasacompositionofmeasurablemaps.Itisalsoconvenienttowrite(10.4)Xk(x;!)=n;k!(x);!2 ;x2X;kn;where(10.5)n;k!(x)=fk;!:::fn+2;!fn+1;!(x);x2X;kn:Letustakeanyk2Zanddenote(10.6)P(x;A)=Pf!:fk;!(x)=Ag:Duetoi.i.d.property,thisdenitiondoesnotdependonthechoiceofk.Lemma10.1.ThusdenedP(
;
)isaprobabilitykernel.Proof:Inthisproofwechoosek=1andworkwithf1.ThefactthatP(x;
)isaprobabilitymeasurefollowssincePisaprobabilitymeasure.IfA2X,thenF�1A=f(x;f):f(x)2Ag2XF.DenotingbyPf�11thepushforwardofPontoFbyf1,weseethat,forx2X,P(x;A)isthemeasureassignedbyPf�11tothex-sectionofF�1A,i.e.,ff:f(x)2Ag.(RelatedtoFubini'stheorem,butisthattherightreference?Proofin[Kif86],startingwithrectangles)Suchfunctionsaremeasurable.Lemma10.2.Foranyboundedmeasurablefunctiong:X!R,ZXP(x;dy)g(y)=Eg(f1;!(x));x2X:Proof:Thelemmaholdsforindicatorfunctionsg=1A,A2X,dueto(10.6).Itholdsbylinearityforsimplefunctions,andthenforgeneralfunctionsbyuniformapproximationwithsimplefunctions.Theorem10.1.Theprocess(Xk)kndenedin(10.4)isahomogeneousMarkovprocesswithinitialdistributionandtransitionkernelP.Proof:Letusproveformula(5.1)byinductionassumingn=0withoutlossofgenerality.Form=0,theformulaistriviallytrue.Letusnowassumethatthelemmaholdsformreplacedbym�10.ThenPfX02A0;:::;Xm2Amg=E1X02A0:::1Xm�12Am�11fm;!(Xm�1)2Am:DenotingG=(X0;:::;Xm�1),weobtainPfX02A0;:::;Xm2Amg=E1X02A0:::1Xm�12Am�1E[1fm;!(Xm�1)2AmjG]: 10210.RANDOMDYNAMICSLetusnowsummarizetheconstructionintroducedinthissection.Atdierenttimes,theevolutioninXisgovernedbydierenttransformationschosenfroma\commonpool",atransformationspaceF.Thesetransforma-tionsformanF-valuedstationaryprocessand,whenstudyingtheevolutioninX,itisconvenientalsotokeeptrackofthatstationaryprocess.WehaveastatespaceXequippedwitha-algebraXandasetFofX-measurabletransformationsequippedwitha-algebraFandamea-sureQonF.Themaincanonicalprobabilityspace( ;F;P)willbethen(FZ;FZ;QZ)or(FN;FN;QN).Theshiftoperator: ! givenby(!)i=!i+1,i2Z,or,respectively,i2N,preservesP.Itisconvenienttoidentifyrandomtransformationswithcoordinatesof!,sowecanwritef!=!1.Wealsointroduce(F;F)-valuedrandomvariables(fn)n2Zdenedbyfn;!=!n=fn�1!.Theyarei.i.d.withcommondistributionQ.ThejointevolutioninXand isgivenbythefollowingskewproducttransformation:X !X denedby(10.7)(x;!)=(f!x;!)=(!1x;!):Thusthespace(X ;XF)equippedwithtransformationisausualdynamicalsystem.Iterationsofaregivenbyn(x;!)=(0;n!x;n!)=(!0;nx;n!);wheretransformationsn;m!havebeendenedin(10.5)and!m;n=!n:::!m+1:Skewproductssatisfythefollowingcocycleproperty:(10.8)0;m1+m2!(x)=0;m2m1!0;m1!;or(10.9)!0;m1+m2(x)=(m1!)0;m2!0;m1:Ingeneral,onecanstudyrandomdynamicalsystemsasskew-productssatisfyingthecocyclepropertywithoutintroducingproductmeasuresasaboveoranykindofmeasures.Forexample,linearizationofdierentiablemaps.Supposef:Rd!Rdisadierentiablemap.LetusdenotebyAx=Df(x)theJacobimatrixofpartialderivativesoffatapointx2Rd.Then,foranyn2N,onecanusethechainruletowritethedierentialoffnatapointxappliedtoavectorv2RdasDfn(x)v=Afn�1xAfn�2x:::AfxAxv:3.InvariantmeasuresSupposewehavean(X;X)-valuedMarkovprocess(Xk)k0withone-steptransitionprobabilityfunctionP(
;
).IfthedistributionofX0isgivenbyameasure,thenPfX12Ag=P(A);A2X; 10410.RANDOMDYNAMICSand(ii)projectsontoPunderthecanonicalprojectionX ! ,i.e.(10.13)(XA)=P(A);A2F:Theorem10.4([Ohn83]).AprobabilitymeasureisinvariantforP(
;
)ifandonlyifPisinvariantfor.(Also,theprojectionofPonto isP.)Proof:SupposePis-invariant.Letustakeanyboundedmeasurablefunctiong:X!Randcheck(10.11).InthefollowingcalculationweintroduceafunctionG(x;!)=g(x):ZX(dx)ZXP(x;dy)g(y)=ZX(dx)Eg(!1x)=ZX (P)(dx;d!)g(!1x)=ZX (P)(dx;d!)G(!1x;!)=ZX (P)(dx;d!)G((x;!))=ZX (P)(dx;d!)G(x;!)=ZX (P)(dx;d!)g(x)=ZX(dx)g(x):SupposenowthatisP(
;
)-invariant.Itissucienttocheck(10.12)forindicatorsofrectangles,i.e.,g(x;!)=1x2A1!2B.Theleft-handsideof(10.12)equals(A)P(B).Letuscomputetheright-handside:ZX (P)(dx;d!)g((x;!))=ZX (P)(dx;d!)g(!1x;!)=ZX(dx)Z P(d!)1!1x2A1!2BSince!1and!areindependentunderP=QN,ther.h.s.equals=ZX(dx)ZXQ(d!1)1!1(x)2AZXf2;3;:::gQf2;3;:::g(d!2;d!3:::)1(!2;!3:::)2B=ZX(dx)P(x;A)Pf!2Bg=(A)P(B);andthelemmaisproved.Inthecaseof( ;F;P)=(FZ;FZ;QZ),relation(10.12)canbederivedinthesamewayfor=Pfrominvarianceof,butonlyforfunctionsgthatdonotdependonthepast,i.e.,g(x;(:::!�1;!0;!1;!2:::))=g(x;(!1;!2;:::)):However,Pdoesnothavetobeinvariantinthiscase,anditisactuallynaturaltoexpectinvariantdistributionsnottohavetheproductstructuresincethestateofthesysteminxislargelydeterminedbythedynamicsinthepastdenedby(:::;!�2;!�1;!0)whereasproductstructurewouldmeanindependenceofthestateandthenoiseinput. 10610.RANDOMDYNAMICSthereisaninvariantmeasureoftherandomdynamicalsystemwithnon-anticipatingsamplemeasuressuchthat(B)=(B )=Z !(B)P(d!):Proof:Letustakeanyboundedmeasurablefunctiong:X!RanddenePn;!(g)=ZX(dx)g(!�n;0x):Inparticular,(10.17)Pn;!(B)=Pn;!(1B)denesaprobabilitymeasurePn;!on(X;X).Thesequenceofthesemea-suresistight,soitalwayshaslimitpoints.Letusidentifythem.LetusprovethatPn;!(g)isamartingalewithrespectto(Gn)n2N,whereGn=(!�n+1;:::;!0).WeneedtocheckthatforeveryFn-measurablefunctionh,EPn;!(g)h(!�n+1;:::;!0)=EPn+1;!(g)h(!�n+1;:::;!0):Theright-handsideequals(10.18)ZX(dx)Eg(!�n�1;0x)h(!�n+1;:::;!0)=ZX(dx)EF((!�n+1;:::;!0);!�nx);whereF((!�n+1;:::;!0);y)=g(!0:::!�n+1y)h(!�n+1;:::;!0):Since!�nxisindependentof(!�n+1;:::;!0),EF((!�n+1;:::;!0);!�nx)=Ea(!�nx);wherea(y)=EF((!�n+1;:::;!0);y):So,theright-handsideof(10.18)equalsEZX(dx)a(!�nx)=ZX(dx)a(x)=ZX(dx)EF((!�n+1;:::;!0);x)=ZX(dx)Eg(!0:::!�n+1x)h(!�n+1;:::;!0)=EPn;!(g)h(!�n+1;:::;!0);andourclaimisestablished.NoticethatPn;!(g)isaboundedmartingale.Therefore,ita.s.convergestoarandomvariableG[g],measurablew.r.t.F�.Letusnowtakeadensesetfgngn2NinC(X).ThealgebraRgeneratedbythissetoverQisalsoacountableset.ForeveryY2R,G[Y]isana.s.-limitofPn;!(Y).Letusnowconsiderthefollowingcountablesetofconditions: 10810.RANDOMDYNAMICSDefinition10.4.LetP(
;
)beaprobabilitykernelon(X;X)andletbeaprobabilitymeasureon(X;X).AboundedX-measurablefunctiongoniscalled(P;)-invariantifPg=ga.s.w.r.t..Definition10.5.LetP(
;
)beaprobabilitykernelon(X;X).AP-invariantprobabilitymeasureon(X;X)iscalledergodicifany(P;)-invariantfunctionis-a.s.constant.Ergodicityofmeasureson(X ;XF)underdeterministictransfor-mationhasbeendenedintherstpartofthesenotes.Infactitwasdenedintermsofnonexistenceofnontrivialinvariantsets.Definition10.6.LetP(
;
)beaprobabilitykernelon(X;X)andletbeaprobabilitymeasureon(X;X).AsetAiscalled(P;)-invariantif1Aisa(P;)-invariantfunction.Lemma10.5.AsetA2Xis(;P)-invariantifandonlyifAcis(;P)-invariant.Theorem10.8.LetP(
;
)beaprobabilitykernelon(X;X).AP-invariantprobabilitymeasureon(X;X)isergodiciany(P;)-invariantsetAsatises(A)2f0;1g.Proof:Oneoftheimplicationsisobvious.Letusprovetheotherone.Supposethatgisa(P;)-invariantfunction.Letusprovethatforanyathesetfx:g(x)�agis(P;)-invariant.Tothatendrstweprovethatifgis(P;)-invariantthenjgjis(P;)-invariant.Toseethat,wecanwritejgj=jPgjPjgj.Since,inadditionP-invarianceofimpliesR(Pjgj�jgj)d=0,weconcludethatPjgj=jgj-a.s.Theng+=(g+jgj)=2isalso(P;)-invariant.Also,ifg1andg2are(P;)-invariant,theng1�g2is(P;)-invariantandsoareg1_g2=(g1�g2)_0+g2andg1^g2=�((�g1)_(�g2)).Since1is(P;)-invariant,weconcludefromalloftheabovepropertiesthat(n(g�a)+)^1,n2Nisasequenceof(P;)-invariantfunctions.Therefore,theindicatoroffx:g(x)�agbeingthepointwiselimitofthissequenceisalso(P;)-invariant.Sinceindicatorsofsetsfx:g(x)�agforalla2Qareinvariant,allthesesetshave-measure0or1.Therefore,gis-a.s.constantTheorem10.9.AP-invariantmeasureisergodiciPisergodicon(X ;XF),where( ;F;P)=(FN;FN;QN).Proof:LetusrstassumethatPisergodic.SupposethenthatPg=g-a.s.andisnotaconstant-a.s.Thenthereisanumbercsuchthat(B)=2f0;1gwhereB=fx:g(x)cg.Thissetis(P;)-invariantasfollowsfromtheproofofTheorem10.8.So,1B(x)=P1B(x)=ZF1B(f(x))Q(df): 11010.RANDOMDYNAMICS(A)=ZX(dx)x(A);A2X;forsomekernel
(
)on(X;X).For-almostallx,themeasurex(
)isergodic.Similarstatementsholdtrueforrandomdynamicalsystems5.AuxiliarystatementsLetP(
;
)beakernelon(X;X).Foraprobabilitymeasureon(X;X)wedenotebyPthemeasureconstructedin(5.1).Forapointx2XwedenotePx=Px.WeknowthatifthemeasureisinvariantunderP,thenPdenesastationaryprocesson(XZ+;XZ+).Inotherwords,Pisinvariantunderthestandardshifton(XZ+;XZ+).Inthefollowinglemmawedescribeaone-to-onecorrespondencebetweeninvariantsetsfortwosystems.Lemma10.6.SupposeisaP-invariantmeasure.If(A)2(0;1),thenAis(;P)-invariant.Proof:If(A)=0,thenthestatementisobvious.If(A)=1,then0=(Ac)=ZX(dx)P(x;Ac);andweobtainthatP(x;Ac)=0for-almostallx.Theorem10.11.SupposeisaninvariantmeasureforakernelP(
;
)on(X;X).(1)SupposeA2Xis(;P)-invariant.ThenthesetAZ+=AA:::is-almostinvariantw.r.t.P.(2)IfasetB2(XZ+;XZ+)is-almostinvariantw.r.t.P,thenthereisasetA2Xsuchthat(10.19)BP=AZ+:5.1.Structureofinvariantsetsonthecanonicalspace.SupposeP(
;
)isaprobabilitykernelon(X;X)andsupposeisaP-invariantmea-sureon.(X;X).Thenonecanconstructameasureon(XZ;XZ)accordingto(5.1).Namely,onerstconstructsnite-dimensionaldistributionsindexedbyfn;:::;mgwheren;m2Zandnm,andthenonehastonotethatthesenite-dimensionaldistributionsareconsistentduetotheinvarianceof.Kolmogorov'sconsistencytheoremguaranteestheexistenceofameasureQon(XZ;XZ)withtheprescribedfamilyofnite-deimensionaldistribu-tions.Thecoordinateprocess(Xk)k2Zisastationaryprocessthen,soQisinvariantundertheshift:XZ!XZdenedby(x)k=xk+1;x=(:::;x�1;x0;x1;:::);k2Z:Infactisinvertibleanditgeneratesagroupoftransformationsdenedby(nx)k=xk+1;x=(:::;x�1;x0;x1;:::);k2Z;n2Z: 11210.RANDOMDYNAMICSWewilldenotebyFm;nthe-algebrageneratedbyr.v.'sXm;:::;Xnforanyn;m2Zsatisfyingnm.WedenoteI=I(XZ;XZ;Q;)andwedenotebyI(;P)the-algebraof(;P)-invariantsets.Theorem10.12.IconsistsofallsetsBsuchthatthereisa(;P)-invariantsetA2Xsuchthat(10.20)BP=fx2XZ:x02Ag:Thelatterconditionona(;P)-invariantsetA2Xisequivalentto(10.21)BP=AZ:Proof:Weadaptthefollowingprooffrom[Ros71,SectionIV.2].First,supposeA2I(;P).Thenforanyn2N,QfX�n2A;:::;Xn2Ag=ZA(dx�n)ZAP(x�n;dx�n+1):::ZAP(xn�1;dxn)=(A):LetusdenoteB=AZ.Thissetisclearly-invariant.SinceBisthecountableintersectionofcylindersofthesamemeasure(A),weobtainQ(AZ)=(A).Also,Qfx2XZ:x02Ag=(A)byconstructionofQ,so(10.20)holds.SupposeBisinvariantunder.Thenitisautomaticallyinvariantun-dernforalln2Z.Letusxk2NandapproximateBbyacylinderwithanerroratmost1=k.Namely,letusndnumbersm;n2Zsatisfyingm0n,andasetBk2Fm;nsuchthatQ(Bk4B)1=k.FromtheinvarianceofBandinvarianceofQunder,weobtainalsoQ(mBk4B)1=kandQ(nBk4B)1=kThiscanberewrittenas(10.22)EQ(1mBk�1B)21 k:(10.23)EQ(1nBk�1B)21 k:Fromthesetwoinequalitiesweobtain(10.24)EQ(1nBk�1mB)24 k:TheminimumoftheexpressionEP(1mBk�)2amongF�1;0-measurablerandomvariablesisgivenbyk=EQ(1mBkjF�1;0)=EQ(1mBkjX0):Since1nBkisF�1;0measurable,weobtainfrom(10.22):(10.25)EQ(1mBk�k)24 k: 11410.RANDOMDYNAMICSProof:Letbethenaturalprojection(resatrictionoperator)fromXZtoXZ+.ThenP=Q�1.SupposethereisanalmostinvariantsetB2XZ.LetususeTheo-rem(10.12)tondA2Xsuchthattherepresentations(10.20)and(10.21)hold.ThenBBQ=fx2XZ:xk2Aforallk2Z+g:Therefore,B=AZ+andB=�1AZ+.Clearly,A 11611.INVARIANTDISTRIBUTIONSINDISCRETESPACESwherePk+(respectively,Pk�)denotesthesumovernonnegative(respec-tively,negative)valuesofPn0(a;k)�Pn0(b;k).SincePkPn0ak=PkPn0bk=1,wehaveXk+(Pn0ak�Pn0bk)+Xk�(Pn0ak�Pn0bk)=0;so,Xk�(Pn0ak�Pn0bk)=�Xk+(Pn0ak�Pn0bk);and(11.3)Mn(j)�mn(j)(Mn�n0(j)�mn�n0(j))supa;bXk+(Pn0ak�Pn0bk):WeclaimthatPk+(Pn0ak�Pn0bk)1�foralla;b.Infact,ifj0isnotincludedinthesummationPk+,thenXk+(Pn0ak�Pn0bk)XkPn0ak�Pn0aj01�:Ifj0isincludedinthesummationPk+,thenXk+(Pn0ak�Pn0bk)XkPn0ak�Pn0bj01�;soourclaimisestablishedandfrom(11.3)andmonotonicityofMandm,weconcludebyinductionthatMn(j)�mn(j)(1�)[n=n0].This,alongwithmonotonicityandboundednessofMn(j)andmn(j),impliesnowthatthereisacommonlimit(j)forMn(j)andmn(j),and,moreover,jMn(j)�(j)j(1�)[n=n0]andjmn(j)�(j)j(1�)[n=n0].InvarianceofundertheactionofPfollowsbytakingn!1onbothsidesofPn+1ij=PnikPkj.2.EntropyapproachThisproofisbasedonthefactthatrelativeentropyofaMarkovchainismonotoneintime.Wefollow[Lig05]closely.LetandbetwoprobabilitydistributionsonX=f1;:::;Ng.Wedenetherelativeentropy(alsoknownasKullback{Leiblerdivergence)ofwithrespecttobyH(j)=NXi=1(i)ln(i) (i):Introducing'(x)=(xlnx;x�0;0;x=0; 11811.INVARIANTDISTRIBUTIONSINDISCRETESPACESisaninvariantdistribution.Letbeoneoftheselimitpoints.Thenforeveryi2X,(i)�0since(i)P(j)Pm(j;i)�0.Sowecanap-plyLemma11.1andconcludethatforanydistributiononX,H(Pnj)isanonincreasingfunction.ContinuityofHimpliesthatifthereisase-quencen0!1suchthatPn0convergestoadistribution,H(j)=limn!1H(Pnj).Ontheotherhand,foranyk2N,Pk=limPn0+k,soH(Pkj)=limn!1H(Pnj)=H(j).ApplyingLemma11.1andtheconditionofthetheoremweconcludethat=.SowejustprovedthateverypartiallimitofPnequals,andthetheoremfollows.3.CouplingLetusproveaslightlyweakerversionofTheorem11.1usingamethodcalledcouplingthatwasintroducedbyW.Doeblin[Doe38].LetusintroduceaMarkovprocess(Xn;Yn)onXXwiththefollowingproperties:(i)XnandYnevolveindependentlyandaccordingtotransitionprobabilitiesPijuntilthemomentwhenXn=Yn;(ii)fromthatpointon,theprocessesXnandYnevolvetogether,stayingequaltoeachotherforallfuturetimes,andtheevolutionstillhappensaccordingtotransitionprobabilitiesPij.Todescribethismoreformally,weneedtointroduceatransitionprob-abilityQ(i1;i2);(j1;j2)onXXbyQ(i1;i2);(j1;j2)=8�&#x]TJ ;� -1; .63; Td;&#x [00;:Pi1j1Pi2j2;i16=i2;Pi1j1;i1=i2;j1=j2;0;i1=i2;j16=j2:NoticethatthecomponentsXnandYnoftheresultingMarkovprocessareMarkovprocesseswithtransitionprobabilitiesP:foranyi1;j12X,PfXn+1=j1;Xn=i1g=Xi2;j2PfXn+1=j1;Yn+1=j2;Xn=i1;Yn=i2g=Xi2;j2PfXn=i1;Yn=i2gQ(i1;i2);(j1;j2)=Xi26=i1Xj2PfXn=i1;Yn=i2gPi1j1Pi2j2+PfXn=i1;Yn=i1gPi1j1=Xi26=i1PfXn=i1;Yn=i2gPi1j1+PfXn=i1;Yn=i1gPi1j1=PfXn=i1gPi1j1;andasimilarcomputationisvalidforP(Yn+1=j2;jYn=i2)=Pi2;j2foralli2;j22X. 124BIBLIOGRAPHY[Hal60]PaulR.Halmos.Lecturesonergodictheory.ChelseaPublishingCo.,NewYork,1960.[Har56]T.E.Harris.TheexistenceofstationarymeasuresforcertainMarkovprocesses.InProceedingsoftheThirdBerkeleySymposiumonMathematicalStatisticsandProbability,1954{1955,vol.II,pages113{124.UniversityofCaliforniaPress,BerkeleyandLosAngeles,1956.[Kam82]TeturoKamae.Asimpleproofoftheergodictheoremusingnonstandardanal-ysis.IsraelJ.Math.,42(4):284{290,1982.[KH95]AnatoleKatokandBorisHasselblatt.Introductiontothemoderntheoryofdy-namicalsystems,volume54ofEncyclopediaofMathematicsanditsApplications.CambridgeUniversityPress,Cambridge,1995.WithasupplementarychapterbyKatokandLeonardoMendoza.[Kif86]YuriKifer.Ergodictheoryofrandomtransformations,volume10ofProgressinProbabilityandStatistics.BirkhauserBostonInc.,Boston,MA,1986.[Kin73]J.F.C.Kingman.Subadditiveergodictheory.Ann.Probability,1:883{909,1973.WithdiscussionbyD.L.Burkholder,DarylDaley,H.Kesten,P.Ney,FrankSpitzerandJ.M.Hammersley,andareplybytheauthor.[Kol50]A.N.Kolmogorov.FoundationsoftheTheoryofProbability(TranslationofGrundbegriederWahrscheinlichkeitsrechnung,Springer,Berlin,1933.).ChelseaPublishingCompany,NewYork,N.Y.,1950.[Koo31]B.O.Koopman.HamiltonianSystemsandTransformationsinHilbertSpace.ProceedingsoftheNationalAcademyofScience,17:315{318,May1931.[KS07]LeonidB.KoralovandYakovG.Sinai.Theoryofprobabilityandrandompro-cesses.Universitext.Springer,Berlin,secondedition,2007.[KT09]K.KhaninandA.Teplinsky.Herman'stheoryrevisited.Invent.Math.,178(2):333{344,2009.[Kur66]K.Kuratowski.Topology.Vol.I.Newedition,revisedandaugmented.TranslatedfromtheFrenchbyJ.Jaworowski.AcademicPress,NewYork,1966.[KW82]YitzhakKatznelsonandBenjaminWeiss.Asimpleproofofsomeergodictheo-rems.IsraelJ.Math.,42(4):291{296,1982.[Lal10]StevenP.Lalley.Kingman'ssubadditiveergodictheorem.http://galton.uchicago.edu/~lalley/Courses/Graz/Kingman.pdf,2010.[Lig05]ThomasM.Liggett.Interactingparticlesystems.ClassicsinMathematics.Springer-Verlag,Berlin,2005.Reprintofthe1985original.[Mol68]S.A.Molcanov.ThestrongFellerpropertyofdiusionprocessesonsmoothmanifolds.Teor.Verojatnost.iPrimenen.,13:493{498,1968.[MT09]SeanMeynandRichardL.Tweedie.Markovchainsandstochasticstability.Cam-bridgeUniversityPress,Cambridge,secondedition,2009.WithaprologuebyPeterW.Glynn.[Num78]E.Nummelin.AsplittingtechniqueforHarrisrecurrentMarkovchains.Z.Wahrsch.Verw.Gebiete,43(4):309{318,1978.[Num84]EsaNummelin.GeneralirreducibleMarkovchainsandnonnegativeoperators,volume83ofCambridgeTractsinMathematics.CambridgeUniversityPress,Cambridge,1984.[Ohn83]TaijiroOhno.Asymptoticbehaviorsofdynamicalsystemswithrandomparam-eters.Publ.Res.Inst.Math.Sci.,19(1):83{98,1983.[Poi90]HenriPoincare.Surleproblemedestroiscorpsetlesequationsdeladynamique.ActaMathematica,13:1{270,1890.[Poi99]HenriPoincare.Lesmethodesnouvellesdelamecaniqueceleste,Volume3.Gauthier-Villars,Paris,1899.[Ros71]MurrayRosenblatt.Markovprocesses.Structureandasymptoticbehavior.Springer-Verlag,NewYork-Heidelberg,1971.DieGrundlehrendermathematis-chenWissenschaften,Band184.