SIAMR1999SocietyforIndustrialandAppliedMathematicsVol.41,No.1,pp.45{76 - PDF document

SIAMR1999SocietyforIndustrialandAppliedMathematicsVol.41,No.1,pp.45{76IteratedRandomFunctions PersiDiaconisDavidFreedmanIteratedrandomfunctionsareusedtodrawpicturesorsimulatelargeIsingmodels,among 1.Introduction. ReceivedbytheeditorsMarch11,1998;acceptedforpublication(inrevisedform)July7,1998;publishedelectronicallyJanuary22,1999.http://www.siam.org/journals/sirev/41-1/33844.htmlDepartmentofMathematicsandStatistics,StanfordUniversity,Stanford,CA94305.DepartmentofStatistics,UniversityofCalifornia,Berkeley,CA94720(freedman@stat.berkeley.edu). PERSIDIACONISANDDAVIDFREEDMANisgivenand01.Inpresentnotation,=;suppose(+)=2.Theprocessmoveslinearly,1withprobability12.Thestationarydistributionhasanexplicitrepresentation,asthelawofTherandomseriesontherightconvergestoanitelimitbecause01.Plainly,thedistributionofisunchangedifismultipliedbyandthenanewisadded:thatisstationarity.Theseriesrepresentation(1.3)canthereforebeusedtostudythestationarydistribution;however,manymysteriesremain,evenforthissimplecase(section2.5).Therearemanyexamplesbasedonanemapsin-dimensionalEuclideanspace.Thebasicchainiswherethe()areindependentandidenticallydistributed;isaisa1vector.Section2surveysthisarea.Section2.3presentsaninterestingapplicationfor=2:withanappropriatelychosennitedistributionfor(theMarkovchaincanbeusedtodrawpicturesoffractalobjectslikeferns,clouds,orre.Section3describesnitestatespaceswherethebackwarditerationscanbeexplicitlytestedtoseeiftheyhaveconverged.Theleadexampleisthe\couplingfromthepast"algorithmofProppandWilson(1996,1998),whichallowssimulationforpreviouslyintractabledistributions,suchastheIsingmodelonalargegrid.Section4givesexamplesfromqueuingtheory.Section5introducessomerigorandexplainsaunifyingtheme.Supposethatisacompleteseparablemetricspace.Writeforthemetric.SupposethateachisLipschitz:forsomeandallx;yyf(x);f(y)]K(x;yFor,denethe\forwarditeration"startingfrom


beingindependentdrawsfromaprobabilityon;thisisjustarewriteofequation(1.1).Denethe\backwarditeration"as


Ofcourse,hasthesamedistributionasforeach.However,theforwardprocesshasverydierentbehaviorfromthebackwardprocess:theforwardprocessmovesergodicallythrough,whilethebackwardprocessconvergestoalimit.(Naturally,thereareassumptions.)Thenexttheorem,provedinsection5.2,showsthatifiscontractingonaverage,thenhasauniquestationarydistribution.The\inducedMarkovchain"inthetheoremistheforwardprocess.Thekernelx;dy)isthelawofgiventhatandtheProkhorovmetricisusedforthedistancebetweentwoprobabilitiesonThismetricwillbedenedinsection5.1;itisdenoted\,"likethemetricon(Section5.1alsotakescareofthemeasure-theoreticdetails.) ITERATEDRANDOMFUNCTIONSLetS;beacompleteseparablemetricspace.LetbeafamilyofLipschitzfunctionson,andletbeaprobabilitydistributiononSupposethatthatf(x0);x0](d)1forsome,andTheinducedMarkovchainhasauniquestationarydistributiondistributionPn(x;
);]AxrnforconstantsthisboundholdsforalltimesandallstartingstatesTheconstantdoesnotdependon;theconstantdoesnotdepend,andx;xwhere;bTheconditionthat0makes1fortypical,andformalizesthenotionof\contractingonaverage."ThekeystepinprovingTheorem1.1isprovingconvergenceofthebackwarditerations(1.4).ROPOSITIONUndertheregularityconditionsofTheorem,thebackwarditerationsconvergealmostsurelytoalimit,atanexponentialrate.Thelimithastheuniquestationarydistribution(Asequenceofrandomvariablesconverges\almostsurely"iftheexceptional 21 x1(0;x)(y 21 Asusual,1)=1or0,accordingas.Therstterminthesumcorrespondstoaleftwardmovefrom;thesecond,toarightwardmove. PERSIDIACONISANDDAVIDFREEDMAN 0.000.25 0255075100 0255075100 Fig.1Theleft-handpanelshowsconvergenceofthebackwardprocess;theright-handpanelshowsergodicbehaviorbytheforwardprocess.Didthischainhaveastationarydistribution?Ifso,couldthedistributionbeidentied?Thosewereourtwobasicquestions.Aftersomeinitial oundering,wesawthatthechaincouldberepresentedastheiterationofrandomfunctionsux; chosenuniformlyon(01)and; chosenwithprobability12each.Theorem1.1showsthereisauniquestationarydistribution.Weidentiedthisdistributionbyguesswork,butthereisasystematicmethod.Beginbyassumingthatthestationarydistributionhasadensity).From(2.1),x;y 2Z1yf(x) xdx+1 2Zy0f(x) Dierentiationgives 2f(y) y+1 2f(y) 1�yorf0(y) f(y)=1 2 �1 y+1 1�y!;sof(y p Thisargumentisheuristic,butitiseasytocheckthatthe\arcsinedensity"displayedin(2.3)satisesequation(2.2)|andmustthereforebestationary.Theconstantmakes=1;thenamecomesaboutbecause arcsinp Figure1illustratesthedierencebetweenthebackwardprocess(left-handpanel,convergence)andtheforwardprocess(right-handpanel,ergodicbehavior).Positionattimeisplottedagainst100,withlinearinterpolation.Bothprocessesstartfrom3andusethesamerandomfunctionstomove.Theorderinwhichthefunctionsarecomposedistheonlydierence.Intheleft-handpanel,thelimitisrandombecauseitdependsonthefunctionsbeingiterated;butthelimitdoesnotdependonthestartingpoint ITERATEDRANDOMFUNCTIONS 01 .00.25.50.751.00 0 .00.25.50.751.00 Fig.2TheBetadistribution.Theleft-handpanelplotstheBeta-density(heavyline)andthe-density(lightline).Theright-handpanelplotstheBeta 2;1 -density(heavyline)andtheBetadensity(lightline).Remarks.Suppose01and.Thesameargumentshowsthatchoosing(0)withprobabilityand(1)withprobabilityleadstoaBeta(q;pstationarydistribution,withdensityon(01).Thenormalizingconstantis=�((�(q)�(p)],where�isEuler'sgammafunction.Inour=1,so�()=�(1)=1.Althoughwewillnotpursuethisidea,theprobabilityofmovingto(0)fromcanevenbeallowedtodependon.Forexample,if,thestationarydistributionisuniform.However,Theorem1.1isnotinforcewhen)dependson.Forinstance,if,theprocessconvergesto0or1almostsurely:ifthestartingstateis,thechanceofconvergingto1is.(Theprocessisamartingale,andconvergencefollowsfromstandardtheorems.)Theorem1.1canbeextendedtocoverthatdependon,butfurtherconditionsareneeded.ManyoftheconstructionsinthispaperinvolvetheBetadistribution.Figure2plotssomeofthedensities.Thestationarydensity(2.3)inourleadexampleis 2;1 )|thebowl-shapedcurveintheright-handpanel;wereturntothisexampleinsection6.3.2.2.MatrixRecursions.Matrixrecursionshavebeenusedinahostofmodelingeorts;see,forinstance,Priestley(1988).Todenethingsin,letwith()beingi.i.d.;isamatrixandisa1vector(i.i.d.istheusualshort-handfor\independentandidenticallydistributed").Autoregressiveprocesseslike(2.4)willbediscussedagaininsection6.1.Undersuitableregularityconditions,thestationarydistributioncanberepresentedasthelawofIndeed,supposethissumconvergesa.s.toanitelimit.Thedistributionisunchangedifafresh(A;B)pairischosen,thesumismultipliedby,andthenisadded:thatisstationarity.Thenotationmaybeabitperplexing:;A;Bareallrandomratherthandeterministic,and\a.s."isshort-handfor\almostsurely":thesumconvergesexceptforaneventofprobability0.Conditionsforconvergencehavebeensharpenedover PERSIDIACONISANDDAVIDFREEDMANtheyears;roughly,mustbeacontraction\onaverage."FollowingworkbyVervaat(1979)andBrandt(1986),denitiveresultswereachievedbyBougerolandPicard(1992).Tostatetheresult,letbeamatrixnormon.Supposethat(arei.i.d.for,with0and=0when0.Asubspaceis\invariant"=1forallTheorem2.1.anddenetheMarkovchain.Supposethattheonlyinvariantsubspaceofitself.Theinniterandomseriesconvergesa.s.toanitelimitifandonlyif holds,thedistributionofistheuniqueinvariantdistributionfortheMarkovchainThemomentassumptionsinTheorem2.1cannotbeessentiallyweakened;seeGoldieandMaller(1997).Ofcourse,theMarkovchain(2.4)canbedenedwhenisexpandingratherthancontracting,butdierentnormingsarerequiredforconvergence.Anderson(1959)andRachevandSamorodnitzky(1995)provecentrallimittheoremsinthenoncontractivecase.Inthecontractivecase,seeBenda(1998).Onalighternote,EmbreeandTrefethen(1998)usethismachinerywithtostudyFibonaccisequenceswithrandomsignsandadampingparameter,so2.3.FractalImages.Thissectionshowshowiteratedrandomanemapscanbeusedtodrawpicturesintwodimensions.Fix().Eachisa2contraction,whileisa21vector:istheassociatedanemapoftheplaneintoitself,whichisLipschitzbecauseisacontraction.Fixpositiveweights,with=1.TheseingredientsspecifyaMarkovchainmovingthrough.Startingat,thechainproceedsbychoosingatrandomwithprobabilityandmovingtoRemarkablyenough,givenatargetimage,onecanoftensolveforthatthecollectionofpoints;:::;Xformsareasonablelikenessofthetarget,atleastwithhighprobability.ThetechniqueisbasedonworkofDubinsandFreed-man(1966),Hutchinson(1981),andDiaconisandShahshahani(1986).IthasbeendevelopedfurtherbyBarnsleyandElton(1988)aswellasBarnsley(1993),andisnowwidelyused.Weoutlinetheprocedure.Theorem1.1applies,sothereisauniquestationarydistribution,callit.Letstandforpointmassat:thatis,)=1if)=0if.Accordingtostandardtheorems,theempiricaldistribution;:::;Xconvergesto NNXiXi! ITERATEDRANDOMFUNCTIONS Fig.3AferndrawnbyaMarkovchain.Convergenceisalmostsure,intheweak-startopology.Foranyboundedcontinuous fdwithprobability1.See,forinstance,Breiman(1960).Inshort,thepatterngeneratedbythepoints;:::;Xlookslikeislarge.Theparametersmustbechosensothatrepresentsthetargetimage.Hereisoneoftheearlyalgorithms.Supposeapictureisgivenasblackandwhitepointsonangrid.Correspondingtothispicturethereisadiscreteprobabilityontheplane,whichassignsmass1toeachblackpointandmass0toeachwhitepoint,beingthenumberofblackpoints.Wewantthestationaryapproximate.StationarityimpliesthatforanyboundedcontinuousfunctionThenextideaistoreplacefdontherightsideof(2.9)byfdForappropriate's,wegetasystemofequationsthatcanbesolved|atleastapproxi-.Forinstance,taketobelinearoralow-orderpolyno-mial(andignorecomplicationsduetounboundedness).In(2.10),theunknownsare.Theequationsarelinearinthe'sbutnonlinearintheotherunknowns.Exactsolutionscannotbeexpectedingeneral,becausewillbediscretewhilebecontinuous.Still,theprogramiscarriedoutbyDiaconisandShahshahani(1986)andbymanylaterauthors;seeBarnsley(1993)forarecentbibliography.AlsoseeFisher(1994).Figure3showsapictureofafern.TheparametersweresuggestedbyCrown-over(1995):=10000,=2,7007,and0600+ PERSIDIACONISANDDAVIDFREEDMAN1371+2.4.TailBehavior.Weturnnowtothetailbehaviorofthestationarydistribu-tion.Someinformationcanbegleanedfromthemoments,andinvariancegivesarecursion.Wediscuss(abitinformally)thecase=1.Let()bei.i.d.pairsofreal-valuedrandomvariables.DenetheMarkovchainby(2.4),andsupposethechainstartsfromitsstationarydistribution.Write)forthelawof.Then),whichimplies);soso�E(A1)].Similarexpressionscanbederivedforhighermoments1.See,forinstance,Vervaat(1979)orDiaconisandShashahani(1986);alsosee(6.4)below.Momentsmaynotexist,ormaynotcapturerelevantaspectsoftailbehavior.Undersuitableregularityconditions,Kesten(1973)obtainedestimatesforthetailprobabilitiesofthestationary.Forinstance,when=1,heshowsthereisapositiverealnumbersuchthatGoldie(1991)givesadierentproofofKesten'stheoremandcomputes;alsoseeBabillot,Bougerol,andElie(1997).Ofcourse,thereisstillmoretounderstand.Forexample,ifisuniformon[0isindependentCauchy,and=(1thestationarydistributionforisCauchy.Thus,theconclusionsofKesten'stheoremhold|althoughtheassumptionsdonot.Section7.3containsotherexamplesofthissort.Itwouldbenicetohaveatheorythathandlestailbehaviorinsuch2.5.FineStructure.Evenwithanexplicitrepresentationforthestationarydis-tribution,therearestillmanyquestions.Considerthechaindescribedbyequa-tion(1.2).Asin(1.3),thestationarydistributionisthelawofbeingi.i.d.with1)=12.Wemayaskaboutthe\type"ofisthismeasurediscrete,continuousbutsingular,orabsolutelycontinuous?(Theterminologyisreviewedbelow.)Bythe\lawofpuretypes,"mixturescannotarise,anddiscretemeasurescanberuledouttoo.SeeJessenandWintner(1935).2,thenisuniformon[2].If02,thenissingular.takesonatmost2distinctvalues.Fortheremainderterm, concentratesonasetofintervalsoftotallength2),whichtendsto0asgetslarge|becauseItisnaturaltoguessthatisabsolutelycontinuousfor2.However,thisisfalse.Forexample,if ,thenissingular:seeErdos(1939,1940).Whichvaluesofgivesingular's?Thisproblemhasbeenactivelystudiedfor50years,withnoendinsight.SeeGarsia(1962)forareviewoftheclassicalwork.TherewasarealbreakthroughwhenSolomyak(1995)provedthatisabsolutelycontinuousforalmostallvaluesofin[11];alsoseePeresandSolomyak(1996, ITERATEDRANDOMFUNCTIONS2.6.Terminology.A\discrete"probabilityassignsmeasure1toacountablesetofpoints,whilea\continuous"probabilityassignsmeasure0toeverypoint.A\sin-gular"probabilityassignsmeasure1toasetofLebesguemeasure0.Bycontrast,an\absolutelycontinuous"probabilityhasadensitywithrespecttoLebesguemea-sure.TextbookexampleslikethebinomialandPoissondistributionsarediscrete;thenormal,Cauchy,andBetadistributionsareabsolutelycontinuous.Orderingtherationalsin[01]andputtingmass1onthethrationalgivesyouaninterestingdiscreteprobability.TheuniformdistributionontheCantorsetin[01]iscontinuousbutsingular.3.ThePropp{WilsonAlgorithm.ThisremarkablealgorithmdoesexactMonteCarlosamplingfromdistributionsonhugenitestatespaces.Letbethestatespaceandletbeaprobabilityon.Theobjectiveistomakearandompickfrom,onthecomputer.Whenislargeandiscomplicated,theprojectcanbequitedicultandthebackwarditerationisavaluabletool.Tobeginwith,thereisafamilyoffunctionsandaprobabilityon,sothatisthestationarydistributionoftheforwardchainonInotherwords,foreachThesefunctionswillbeconstructedbelow.Insomecases,theMetropolisalgorithmisuseful(Metropolisetal.,1953).Inthepresentcase,aswillbeseen,theGibbssampleristheconstructiontouse.Theprobabilityonwillbecalledthe\movemeasure":thechainmovesbypickingandgoingfrom).Iftheconstructionissuccessful,thebackwarditerations


willconvergea.s.toalimitingrandomvariablewhosedistributionis.(Asequenceconvergesifitiseventuallyconstant,andareindependentdrawsfromthemovemeasureon.)Convergenceiseasiertocheckifthereismonotonicity.Supposeisapartiallyorderedset;write.Supposetoothereisasmallestelement0andalargestelement1.Withpartialorderings,theexistenceofalargestelementisanadditionalassumption,evenforaniteset;likewiseforsmallest.Finally,supposethateachismonotone:).Nowconvergenceisforcedif,forsome


)(0)=(


Thistakesamomenttoverify.Amongotherthings,convergencewouldnotbeforcedifwehadequalityontheforwarditeration.ProppandWilson(1996,1998)turntheseobservationsintoapracticalalgo-rithmforchoosingapointatrandomfrom.Theymakeasequenceindependentpicksfromthemovemeasurein(3.1),andcomputethebackwarditera-tions(3.2).Ateachstage,theychecktoseeif(3.3)holds.Ifso,thecommonvalue|oftheleftsideandtherightside|isapickfromtheexactstationarydistributionThealgorithmgeneratesarandomelementofwhosedistributionisthesought-foritself,ratherthananapproximationto;thereisanexplicittestforconvergence; PERSIDIACONISANDDAVIDFREEDMANandinmanysituations,convergencetakesplacequiterapidly.Thesethreefeaturesarewhatmakethealgorithmsoremarkable.Bywayofexample,taketheIsingmodelonangrid;areferenceisKindermanandSnell(1980).Thestatespaceconsistsofallfunctions.Thestandard(barbaric)notationhashasn][n].Inthepartialorder,forallpositions(i;j)inthegrid,andboundaryconditionmaybeimposed,forinstance,that=+1ontheperimeterofthegrid.Theminimalstateis1atalltheunconstrainedpositions;themaximalstateis+1atalltheunconstrainedpositions.Theprobabilitydistributiontobesimulatedisisapositiverealnumberandisanormalizingconstant|whichisquitehardtocomputeifislarge.Intheexponent,)countssignchanges.Algebraically,ij;k`Theindicesi;j;k;`runfrom1to,andtheposition(i;j)mustbeadjacentto(k;`forinstance,theposition(22)isadjacentto(23)butnotto(3A\singlesiteheatbath"(aspecializedversionoftheGibbssampler)isusedtoconstructachainwithlimitingdistribution.Fromstate,thechainmovesbypickingasite(i;j)onthegridandrerandomizingthevalueati;j).Morespecically,letagreewithatallsitesotherthan(i;j);let=+1at(i;j).Likewise,agreeswithatallsitesotherthan(i;j),but1ati;j).Let(+)= H(sij+)]+exp[(+).Thechanceofmovingto(+);thechanceofmovingto).Inotherwords,thechanceofrerandomizingto+1at(i;j(+).Thischanceiscomputablebecausetheuglyconstanthascanceledout.Inprinciple,(+)and)dependonthesite(i;j)andonvaluesofatsitesotherthan(i;j);wewriteijs)whenthismatters.Ofcourse,(+)isjustthe-probabilitythat=+,giventhevaluesofatallothersites.Asitturnsout,onlythesitesadjacentto(i;j)aect(+),becausethevaluesofatmoreremotesitesjustcancel:ijs +expThesumisoverthesites(k;`)adjacentto(i;j).Equation(3.6)isinessencethe\Markovrandomeld"propertyfortheIsingmodel.Thesinglesiteheatbathcanbecycledthroughsites(i;j)onthegrid,orthesitecanbechosenatrandom.Wefollowthelattercourse,althoughtheformeriscomputationallymoreecient.Thealgorithmisimplementedusingthebackwarditeration.Therandomfunctionsare).Here,isastateintheIsingmodeli;j;u)consistsofaposition(i;j)inthegridandarealnumber1.Thepositionisrandomlychoseninthegrid,andisrandomover(0 ITERATEDRANDOMFUNCTIONSThefunctionisdenedasfollows:iju)agreeswithexceptatpositioni;j).There,=+1if(+),and1otherwise.Twothingsmustbeveried:isstationary,andismonotone.Stationarityisobvious.Formonotonicity,xasite(i;j),twostatess;t1).Clearly,ijuiju)exceptperhapsat(i;j).Atthisspecialsite,wemustproveijsijtButthetwoconditionalprobabilitiesin(3.7)canbeevaluatedby(3.6),andThecondition0makesmonotoneincreasingratherthanmonotonedecreasing.Thebackwarditerationcompletesafteranite,randomnumberofsteps,essentiallybyTheorem1.1.Completioncanbetestedexplicitlyusing(3.3).Andthealgorithmmakesarandompickfromitself,ratherthananapproximationtoTherearemanyvariationsonthePropp{Wilsonalgorithm,includingsomeforpointprocesses:seeMo/ller(1998)orHom,vanLieshout,andMo/ller(1998).AnovelalternativeisproposedbyFill(1998),whoincludesasurveyofrecentliteratureandawarningaboutbiasesduetoabortedruns.Therearenogeneralboundsonthetimeto\coupling,"whichoccurswhen(3.3)issatised:chainsstartingfrom0andfrom1,butusingthesame's,wouldhavetoagreefromthattimeonwards.ExperimentsshowthatcouplinggenerallytakesplacequiterapidlyfortheIsingmodelbelowacriticalvalue,butquiteslowlyforlarger's.ProppandWilson(1996)havealgorithmsthatworkreasonablywellforallvaluesof|evenabovethecriticalvalue|andforgridsuptosize21002100.Formorediscussion,andacomparisonoftheMetropolisalgorithmwiththeGibbssampler,seeHomandNelander(1998).BrownandDiaconis(1997)showthatahostofMarkovchainsforshuingandrandomtilingsaremonotone.ThesechainsarisefromhyperplanewalksofBidigare,Hanlon,andRockmore(1997).Theanalysisgivesreasonablysharpboundsontimetocoupling.Monotonicitytechniquescanbeusedforinnitestatespacestoo.Forinstance,suchtechniqueshavebeendevelopedbyBorovkov(1984)andBorovkovandFoss(1992)toanalyzecomplexqueuingnetworks|ournexttopic.4.QueuingTheory.Theexistenceofstationarydistributionsinqueuingtheorycanoftenbeprovedusingiteratedrandomfunctions.Thereisaninterestingtwist,becausethefunctionsaregenerallynotstrictcontractions,evenonaverage.Wegiveanexampleandpointerstoavoluminousliterature.Inonerelativelysimplemodel,theG/G/1queue,customersarriveataqueuewithi.i.d.interarrivaltimesThearrivaltimesarethepartialsums0thcustomerhasservicetime;thesetooarei.i.d.,andindependentofthearrivaltimes.Letbethewaitingtimeofthethcustomer|thetimebeforeservicestarts.Bydenition,=0.For0,thesatisfytherecursionIndeed,thethcustomerarrivesattimeandwaitstimenishingserviceattime.The+1stcustomerarrivesattime,then=0;otherwise, PERSIDIACONISANDDAVIDFREEDMANThewaiting-timeprocesscanthereforebegeneratedbyiteratingtherandomfunctionsTheparametershouldbechosenatrandomfrom),whichisaprobabilityonthereallineThefunctionisaweakcontractionbutnotastrictcontraction:theLipschitzconstantis1.AlthoughTheorem1.1doesnotapply,thebackwarditerationstillgivesthestationarydistribution.Indeed,thebackwarditerationstartingfrom0canbewrittenas


)(0)=Nowthereisamagicalidentity:=maxThisidentityholdsforanyrealnumbers.Feller(1971,p.272)asksthereadertoprove(4.4)byinduction,and=1istrivial.Separatingthecases0,onechecksthat(=max;x;x.Thatdoes=2.Nowput+max=maxThatdoes=3.Andsoforth.Ifthestartingpointisratherthan0,youjustneedtoreplacein(4.4)byInthequeuingmodel,arei.i.d.byassumption,asare;andthe'sareindependentofthe's.Set.Sothearei.i.d.too.Itiseasilyseen|given(4.3){(4.4)|thattheMarkovchainforitsstationarydistributionthelawofprovidedthelimitisnitea.s.Manyauthorsnowusethecondition0toinsureconvergence,viathestronglawoflargenumbers:!�1a.s.,sothemaximumofthepartialsumsisnitea.s.Inaremarkablepaper,Spitzer(1956)showedthatnomomentassumptionsareneeded.Theorem4.1.Supposetherandomvariablesarei.i.d.Thelimitinisnitea.s.ifandonlyif Underthiscondition,thelimitinhasaninnitelydivisibledistributionwithcharacteristicfunction where ITERATEDRANDOMFUNCTIONSThe\G/G/1"intheG/G/1queuestandsforgeneralarrivaltimes,generalservicetimes,andoneserver:\general"meansthat)and)arenotrestrictedtoparametricfamilies.Therecentqueuingliteraturecontainsmanyelaborations,including,forinstance,queueswithmultipleserversanddierentdisciplines;seeBaccelli(1992)amongothers.TherearesurveysbyBorovkov(1984)orBaccelliandBremaud(1994).Oneremarkableachievementisthedevelopmentofasortoflinearalgebrafortherealnumbersundertheoperation(x;yx;yThebookbyBaccellietal.(1992)givesmanyapplications;queuesarediscussedinChapter7.Therandom-iterationsideahelpstounifythearguments.5.Rigor.Thissectiongivesamoreformalaccountofthebasicsetup;thenThe-orem1.1isprovedinsection5.2.Thetheoremandthemainintermediateresultsareknown:seeArnoldandCrauel(1992),BarnsleyandElton(1988),DubinsandFreedman(1966),Du o(1997),Elton(1990),orHutchinson(1981).Evenso,theself-containedproofsgivenheremaybeofsomeinterest.5.1.Background.Let(S;)beacomplete,separablemetricspace.Thenisamappingofintoitself,withwithf(x);f(y)]K(x;y).Theleastsuch.Ifisconstant,then=0.Ifforsome,then\Lipschitz";otherwise,.Ofcourse,thesedenitionsarerelativeto.Wepauseforthemeasuretheory.Letbeacountabledensesubsetof,andlet thesetofallmappingsfrominto.Weendow withtheproducttopologyandproduct-eld.Plainly, isacompleteseparablemetricspace.LetbethespaceofLipschitzfunctionson.ThefollowinglemmaputsameasurablestructureonLemma5.1.isaBorelsubsetof isaBorelfunctionon(iii)(f;sisaBorelmapfromProof.For ,let=supsupf(x);f(y)]=(x;yPlainly,isaBorelfunctionon .IfcanbeextendedasaLipschitzfunctiontoallof.Conversely,ifisLipschitzon,itsretractionto.Thus,theLipschitzfunctionscanbeidentiedasthefunctions,and.Thisproves(i)and(ii).For(iii),enumerate.Fixapositiveinteger.Letbethesetofpointsthatarewithin1.Letbethesetofpointsthatarewithin,butatadistanceof1ormorefrom.(Inotherwords,taketheballsofradius1aroundtheandmakethemdisjoint.)Foreach,thearepairwisedisjointandandj=1Bn;j=S:Givenamappingintoitself,let)for.Thatis,approximates)inthevicinityof.Themap(f;s)isBorelfrom .AndonthesetofLipschitz,thissequenceofmapsconvergespointwisetotheevaluationmap. Remark.Tomaketheconnectionwiththesetupofsection1,ifisafamilyofLipschitzfunctionsindexedby,werequirethatthemap)be PERSIDIACONISANDDAVIDFREEDMANmeasurableforeach.Thenisameasurablemapfromto,andameasureoninducesameasureon.ThissectionworksdirectlywithmeasuresonThemetricinducesa\Prokhorovmetric"onprobabilities,alsodenotedbyasfollows.Definition5.1.areprobabilitieson,thenP;Qistheinmumofsuchthatforallcompact,whereisthesetofallpointswhosedistancefromislessRemarks(i)Plainly,P;Q(ii)LetbeasinDenition5.1,withrangingoverallBorelsets.Plainly,entails.Thatis,.Conversely,suppose.FixaBorelsetandasmallpositive.Findacompactset.Thenandsimilarlyfor).Thus,andhence.Inshort,(iii)Dudley(1989)isastandardreferenceforresultsontheProkhorovmetric.Weneedthedenitionofarandomvariablewithan\algebraictail."Basically,hasanalgebraictailiflog(1+)hasaLaplacetransforminaneighborhoodof0,where=maxisthepositivepartof.Ofcourse,itisamatteroftastewhetheroneuseslog(1+)orlogDefinition5.2.Arandomvariablehasanalgebraictailiftherearepositive,niteconstantssuchthat�Uu=uforall.Thisconditionhasforceonlyforlargepositive;andweallow5.2.TheMainTheorem.Fixaprobabilitymeasure.AssumethathasanalgebraictailrelativetoFixareferencepoint;assumetoothatthatf(x0);x0]hasanalgebraictailrelativetoIf,forinstance,isthelineandthe'sarelinear,condition(5.1)constrainstheslopesandthen(5.2)constrainstheintercepts.Aswillbeseenlater,anyreferencepointinmaybeused.ConsideraMarkovchainmovingaroundinaccordingtothefollowingrule:startingfrom,thechainchoosesatrandomfromandgoesto).Wesaythatthechain\movesaccordingto,"or\isthemovemeasure";insection1,thisMarkovchainwascalled\theforwarditeration."Theorem5.1.SupposeisaprobabilityontheLipschitzfunctions.Supposeconditionshold.Supposefurtherthat ITERATEDRANDOMFUNCTIONStheintegralmaybe.ConsideraMarkovchainonthatmovesaccordingtoLetx;dybethelawofthechainaftermovesstartingfromThereisauniqueinvariantprobabilityThereisapositive,niteconstantandansuchthatthatPn(x;
);]AxrnforallTheconstantdoesnotdependon;theconstantdoesnotdepend,andx;x,where;bIn(ii)and(iii),istheProkhorovmetric(Denition5.1).TheargumentforTheorem5.1canbesketchedasfollows.Althoughtheforwardprocess


doesnotconvergeas,thebackwardprocess|withthecompositioninreverseorder|doesconverge.Thus,weconsider


Themainstepwillbethefollowing.Proposition5.1..Denethebackwardprocess.ThenconvergesatageometricrateastoarandomlimitthatdoesnotdependonthestartingpointTorealizethestationaryprocess,letbeindependentwithcommondistribution,andlet


wherethecomposition\goesalltheway."Rigorwillcomeaftersomepreliminarylem-mas,anditwillbeseenthattheprocessisstationarywiththerighttransitionlaw.Lemma5.2.Letbei.i.d.randomvariables;isallowed.Supposetherearepositive,niteconstantssuchthateforall.Letbedistributedas.Then�1isaniterealnumberwith,therearepositive,niteconstantssuchthatʁ.;耀ncrforallconstantsdependonandthelawof,notonProof1.Supposeisboundedbelow.Then(i)isimmediate,with;(ii)isnearlystandard,butwegivetheargumentanyway.First,.Next,let.Weclaimthat=1+)asIndeed,xwith0 ;let1and.Then,so Theright-handsideof(5.8)hasniteexpectedvalue,proving(5.7).Asaresult,therearepositiveconstantsforwhich PERSIDIACONISANDDAVIDFREEDMANprovided0.LetByMarkov'sinequality,�ncIf0,wehaveaboundon.SettocompletetheproofinCase1,with=exp[].Thisislegitimateprovided.Largervaluesofmaybereplacedby2.Lettruncatedbelowataconstantthatdoesnotdependon.Case1appliestothetruncatedvariables,whosemeanwillbelessthanifthetruncationpointissucientlynegative.Ourideaoftruncationcanbedenedbyexample:truncatedbelowat17equals17,and17if beani.i.d.sequenceofpicksfrom.Fix.Considertheforwardprocessstartingfromx;X;::::Lemma5.3.5.3.Xn(x);Xn(y)]Qnj=1Kfj(x;yProof.Thisisobviousfor=0and=1.Now ThenexttwolemmaswillprovetheuniquenesspartofTheorem5.1.Lemma5.4.Suppose.Ifissucientlysmall,therearepositive,niteconstantsrforallTheconstantsdependonbutnotonProof.ApplyLemma5.2totherandomvariables=log Lemma5.5.Suppose.Forsucientlysmallpositive:exceptforasetofofprobabilitylessthanthanXn(x);Xn(y)]exp(�n")(x;yforallx;y.Again,dependonbutnotonProof.UseLemmas5.3and5.4. Corollary5.1.Thereisatmostoneinvariantprobability.Proof.Supposewereinvariant.Choose,inde-pendently.Let)and).Nowexceptforasetofexponentiallysmallprobability.So,thelawsofbuttheformerisandthelatteris Thenextlemmagivessomeresultsonvariableswithalgebraictails,leadingtoaproofthatif(5.1)holds,and(5.2)holdsforsomeparticular,then(5.2)holdsfor.Thelemmaanditscorollaryareonlytoassisttheinterpretation.Lemma5.6.isnonnegativeandboundedabove,thenhasanalgebraictail.hasanalgebraictailand,thenhasanalgebraictail. ITERATEDRANDOMFUNCTIONShavealgebraictails,sodoes;theserandomvariablesmaybedependent.(Inprinciple,therearetwo'sandtwo's;itisconvenienttousethelargerandthesmaller,ifbothofthelatterarepositive.)Proof.Claims(i)and(ii)areobvious.Forclaim(iii),�Vt�Ut=�Vt= Corollary5.2.Supposeconditionholds.Ifholdsforanyparticular,thenholdsforany.Inotherwords,therearenitepositiveconstants;f(x0);x0]�ug=uforall.Theconstantmaydependon,buttheshapeparameterdoesnot.Proof.UseLemma5.6andthetriangleinequality. Lemma5.7.Letbemappingsofintoitself;let.ThenThen(fg)(x);x][f(x);x]+Kf[g(x);x]:Proof.Bythetriangleinequality,,(fg)(x);x][f(x);x]+[(fg)x;fNowusethedenitionof Corollary5.3.Letbemappingsofintoitself;let.ThenThen(g1g2


g1(x);x]+Kg1[g2(x);x]+Kg1Kg2[g3(x);x]+


+Kg1Kg2


Kgm�1[gm(x);x]:ProofofProposition5.1.Weassumeconditions(5.1){(5.3)andconsiderthebehaviorwhenofthebackwarditerations


Convergenceof)aswillfollowfromtheCauchycriterion.InviewofLemma5.5,itisenoughtoconsider.AsinLemma5.3,5.3,Yn+m(x);Yn(x)]Kf1


Kfn[(fn+1fn+2


WeuseCorollary5.3withtoboundtheright-handsideof(5.10),concludingconcludingYn+m(x);Yn(x)]1Xi=0 n+iYj=1Kfj![fn+i+1(x);x]:(5.11)ByLemma5.4,exceptforasetofprobabilityforallandallNext,condition(5.2)comesintoplay.Writeritefj(x);x].BytheDenition5.2ofalgebraictails,therearepositiveniteconstantssuchthat PERSIDIACONISANDDAVIDFREEDMAN 0.000.25 0510152025 0.00 0510152025 Fig.4Thebackwarditerationsconvergerapidlytoalimitthatisrandombutdoesnotdependonthestartingstate..Choose1butsocloseto1that1.Exceptforanothersetofexponentiallysmallprobability,forallandallNowtherearenitepositiveconstants1and1,suchthatforall,forall,andallallYn+m(x);Yn(x)]rn1;(5.14)exceptforasetofprobability.Thus,)isCauchy,andhenceconvergestoalimitin.Wehavealreadyestablishedthatthelimitdoesnotdependon;callthe.Anexponentialratefortheconvergenceof)tofollowsbylettingin(5.14). Lemma5.8.Letberandommappingsinto,withdistributionsSupposecanberealizedsothatX;X.Then;.(Intherstinstance,isthemetricon;inthesecond,istheinducedProkhorovmetriconprobabilities:seeDenitionProof.Letbeacompactsubsetof.Thenentails,exceptforprobability.Likewise,entails,exceptforprobability Remark.TheconversetoLemma5.8istruetoo:oneproofgoesbydiscretizationandthe\marriagelemma."SeeStrassen(1965)orDudley(1989,Chapter11).ProofofTheorem5.1.Thereareonlyafewdetailstocleanup.Recallthedoublyinnitesequencefrom(5.5).ByProposition5.1,wecandeneasfollows:=lim


Thelimitdoesnotdependon.Proposition5.1applies,because|asbefore|Itiseasytoverifythat;:::isstationarywiththerighttransitionprobabilities.Andisdistributedlikeanyof.Thus,theconvergenceassertion(ii)inTheorem5.1followsfromLemma5.8andProposition5.1.Theargumentiscomplete. ITERATEDRANDOMFUNCTIONS Ð5Ð10 510152025 Ð10 510152025 Fig.5Logarithmtobaseoftheabsolutedierencebetweenpathsinthebackwarditeration.ProofofTheoremandProposition1.1.TheseresultsareimmediatefromProposition5.1andTheorem5.1.Indeed,themomentconditionsinTheorem1.1im-plyconditions(5.1){(5.3);westatedTheorem1.1usingthemorerestrictiveconditionsinordertopostponetechnicalities. Theessenceofthethingisthatthebackwarditerationsconvergeatageometricratetoalimitthatdependsonthefunctionsbeingcomposed|butnotonthestartingpoint.Figure4illustratestheideafortheMarkovchaindiscussedinsection2.1.Theleft-handpanelshowsthebackwarditerationstartingfrom3orExactlythesamefunctionsareusedtogeneratethetwopaths;theonlydierenceisthestartingpoint.(Positionattimeisplottedagainst25,withlinearinterpolation.)Thepathsmergeforallpracticalpurposesaround=7.Theright-handpanelshowsthesamething,withanewlotofrandomfunctions.Convergenceisevenfaster,butthelimitisdierent|randomnessinaction.(Bycontrast,theforwarditerationdoesnotconverge,butwandersaroundergodicallyinthestatespace;seeFigure1.)Figure5plotsthelogarithm(base10)oftheabsolutedierencebetweenthepathsinthecorrespondingpanelsofFigure4.Thelineardecayonthelogscalecorrespondstoexponentialdecayontheoriginalscale.Thedierenceinslopesbetweenthetwopanelsisduetotherandomnessinchoiceoffunctions;thisdierencewearsoasthenumberofiterationsgoesup.Remarks(i)Thenotationin(5.15){(5.16)maybeabitconfusing:isnotthebackwardprocess,anddoesnotconverge.(ii)Weusethealgebraictailconditiontoboundtheprobabilitiesoftheexcep-tionalsetsinProposition5.1,thatis,thesetswhere(5.12)and(5.13)fail.TheseprobabilityboundsgivetheexponentialrateofconvergenceinTheorem5.1.Withalittlemoreeort,theoptimalcanbecomputedexplicitly,intermsofthemeanandvarianceoflog,andtheshapeparameterin(5.2).Ifanexponentialrateisnotneeded,itisenoughtoassumethatlog(1+)andloglogf(x0);x0]
areL1.(iii)Furstenberg(1963)usesthebackwarditerationtostudyproductsofrandommatrices.Heconsiderstheactionofamatrixgrouponprojectivespaceandshowsthatthereisauniquestationarydistribution,whichcanberepresentedasaconver-gentbackwarditeration.Convergenceisprovedbymartingalearguments.Itseemsworthwhiletostudythedomainofthismethod.(iv)Let()beameasurablespaceandletx;dy)beaMarkovkernelon).WhenisthereafamilyandaprobabilityonsuchthattheMarkovchaininducedbytheseiteratedrandommappingshastransitionsx;dy PERSIDIACONISANDDAVIDFREEDMANThisconstructionisalwayspossibleif()is\Polish,"thatis,aBorelsubsetofacompleteseparablemetricspace.See,forinstance,Kifer(1986).Theleadingspecialcasehas=[01].Thencanalsobetakenastheunitinterval,andastheLebesguemeasure;x;dy)canbedescribedbyitsdistributionfunctionx;yy;y]).Let)betheinverseof).Ifisuniform,x;Uisdistributedasx;dy).Finally,letx;).Vericationisroutine,andthegeneralcasefollowsfromthespecialcasebystandardtricks.Thequestionismoresubtle|andtheregularityconditionsmuchmoretechni-cal|ifitisrequiredthatthe)becontinuous.BlumenthalandCorson(1970)showthatifisaconnected,locallyconnected,compactspace,andiscontinuous(weakstar),andthesupportof)isforall,thenthereisaprobabilitymeasureontheBorelsetsofthecontinuousfunctionsfromwhichinducesthekernel.Quas(1991)givessucientconditionsforrepresentationbysmoothfunctionswhenisasmoothmanifold.AsurveyoftheseandrelatedresultsappearsinDubischar(1997);alsoseeBlumenthalandCorson(1971).6.MoreExamples.Autoregressiveprocessesareanimportantfeatureofmanystatisticalmodels,andcanusefullybeviewedasiteratedrandomfunctions;thecon-structionwillbesketchedhere.WelearnedthetrickfromAnderson(1959),butheattributesittoYule.Furtherexamplesandcounterexamplestoillustratethetheoryaregiveninsection6.2;section6.3revisitstheexamplediscussedinsection2.1.6.1.AutoregressiveProcesses.,therealline.Letbearealnumberwith01andletbeaprobabilitymeasureon.Forpresentpurposes,anautoregressionisaMarkovprocessonwiththefollowinglawofmotion:starting,thechainpicksaccordingtoandmovesto.Conditions(5.1)and(5.3)areobvious:if,then.Forcondition(5.2),weneedtoassumeforinstancethatifhasdistribution,therearepositive,niteconstants;=uforall0.Ifareindependentwithcommondistribution,theforwardprocessstartingfromandsoforth.Thisprocessconvergesinlaw,butdoesnotconvergealmostsurely:at,newrandomnessisintroducedby.Thebackwardprocessstartingfromlooksatrstglancemuchthesame:andsoforth.Butthisprocessconvergesa.s.,becausethenewrandomnessintroducedisdampedby.ThestationaryautoregressiveprocessmayberealizedasEachisobtainedbydoingthebackwarditerationon.Equa-tion(5.6)isthegeneralization.WiththeusualEuclideandistance,theconstantinTheorem5.1mustdependonthestartingstate.Foraparticularlybrutalillus-tration,take6.2.WithoutRegularityConditions.Thissectiongivessomeexamplestoindi-catewhatcanhappenwithoutourregularityconditions. ITERATEDRANDOMFUNCTIONS 0.000.25 0255075100 Fig.6Iteratedrandomfunctionsontheunitinterval.Withprobability,thechainstandspat;withprobability,thechainmovesfrommodulo.Theforwardandbackwardprocessarethesame,anddonotconverge.6.1.ThisexampleshowsthatsomesortofcontractingpropertyisneededtogetaresultlikeTheorem5.1.Let=[01].Arithmeticistobedonemodulo1;forinstance,271=42.Letx;gmod12.Theforwardandthebackwardprocessescanbothberepresentedasmod1beingindependentandtakingvalues0or1withprobability12each;isthestartingpoint.Clearly,thebackwardprocessconvergesonlyifthestartingpointisabinaryrational.Furthermore,thereareinnitelymanydistinctstationaryprobabilities:ifisastationary0{1valuedprocess,thenthelawofisstationaryforourchain.Since=1and=2,condition(5.3)fails.Figure6100,withlinearinterpolation.Remark.Figure6involvesontheorderof50doublings,sonumericalaccuracyisneededto50binarydigits,or16decimalplacesin.Thatisaboutthelimitofdouble-precisioncomputerpackageslikeMATLABonaPC.If,say,1,000iterationsarewanted,accuracyto150decimalplaceswouldbeneeded.Thework-aroundiseasy.Codethestatesaslongstringsof0'sor1's,anddobinaryarithmetic.Forplotting,converttodecimals:onlytherst10bitsinwillmatter.6.2.Thisexamplehasauniquestationarydistributionbutthebackwardprocessdoesnotconverge.Letbetheintegersmod.Letj;g+1mod2.Theforwardandthebackwardprocessescanbothberepresentedasmodbeingindependentandtakingvalues0or1withprobability12each.Clearly,thebackwardprocessdoesnotconverge.Ontheotherhand,thechainisaperiodicandirreducible,sothereisauniquestationarydistribution(theuniform),andthere PERSIDIACONISANDDAVIDFREEDMANisanexponentialrateofconvergence.Leti;j)betheleastsuchthat.Thenisametric:thedistancebetweentwopointsistheminimalnumberofstepsittakestogetfromonetotheother,wherestepscanbetakenineitherdirection.Relativetothismetric,areLipschitz,with=1;condition(5.3)isviolated.Thenextexampleshowsanothersortofpathologywhencondition(5.3)holdsbut(5.1){(5.2)fail.6.3.Thestatespaceis[0).Lettherandomvariablehaveasymmetricstabledistributionwithindex1;seeSamorodnitskyandTaqqu(1994)orZolotarev(1986).Letbethelawof.ConsideraMarkovchainthatmovess;1)bychoosingatrandomfromandgoingto.Then0isaxedpointandtheuniquestationarydistributionconcentratesat0.Ifi.i.d.symmetricstablewithindex,theforwardandthebackwardprocessescanbothberepresentedas0a.s.as,bythestronglawoflargenumbers.Ontheotherhand,theProkhorovdistancebetween)andisoforder1,byLemmas6.1and6.2below.Inparticular,exponentialratesofconvergencedonotobtain.Condition(5.3)holds:Kd1.However,(5.1)fails,andsodoes(5.2)for=0.Lemma6.1.Letbepointmassat,andletbeacontinuousprobabilitymeasureonThereisaunique(ii)()=Proof.Claims(i)and(ii)areeasytoverify.For(iii),weneedtocomputetheinmumofsuchthatforallcompactIf0,then(6.1)isvacuous.If0,then(6.1)isequivalentto1Furthermore,0entails[0.And=[0)when.Thus,(6.1)forallcompactisequivalenttoLikewise,if0,then(6.2)isvacuous.If0then(6.2)isequivalentto.But00";1).Thus,(6.2)forallcompactisalsoequivalentto(6.3).Now(iii)followsfrom(ii). Lemma6.2.LetbeasymmetricstablerandomvariblewithindexLetbealargepositiveinteger.TheProkhorovdistancebetweenandthelawofisoforderProof.ThisfollowsfromLemma6.1,since�Uu Remark.SomethingcanbedoneevenwhenalltheLipschitzconstantsare1,providedthefunctionsaregenuinelycontractingonarecurrentset.Forinstance, ITERATEDRANDOMFUNCTIONSSteinsaltz(1997,1998)considersaMarkovchainonthatmovesbychoosingoneofthefollowingtwofunctionsatrandom:+1if +1if+2if1if 1if02ifThesefunctionshaveLipschitzconstant1.But,asateam,theyaregenuinelycon-tractingontheinterval[2].Thisintervalisrecurrent.Indeed,fromlargenegative,thechainmoves2unitstotherightand1unittotheleftwithequalprobabilities;thereverseholdsforlargepositive.Thus,whenthechainisnear,itdriftsbacktoward0.Steinsaltzhassomegeneraltheoryandotherexamples.6.3.TheBetaWalk.Thestatespaceistheclosedunitinterval[0,1].Letbeaprobabilitymeasureon,andlet01.Considerachainwiththefollowingtransitionprobabilities.Startingfromfrom;1],thechaingoesleftwithprobabilityandrightwithprobability1.Tomove,itpicksfrom.Ifthemoveistotheleft,thechaingoesto;iftotheright,itgoesto.Callthe\movingmeasure."IfisBeta(;),callthechaina\Betawalk."Theexampleinsection2.1wasaBetawalkwith2and2.Weextendtheterminologyalittle:Beta(00)putsmass12at0and1;Beta()putsmass1at1TheseexamplestintotheframeworkofTheorem5.1:andprobabilizethesetoflinearmapsthatshrinktheunitintervaleithertoward0,whenthemapsendsortoward1,whenthemapsendsAlltheLipschitzconstantsare1orsmaller.Conditions(5.1){(5.3)areobvious,andthereisexponentialconvergencetotheuniquestationarydistribution.Inthebalanceofthissection,weprovethefollowingtheorem.Theorem6.1.Suppose=[0,andthemovemeasureisBeta;Let+1);when,let.If,or,thenthestationarydistributionoftheBetawalkisBeta.Foranyothervalueof,thestationarydistributionissymmetricandhasthesamerstthreemomentsasBetabutadierentfourthmoment:inparticular,thestationarydistributionisnotBeta.Remarks.ThesecondmomentofBeta(a;a)is(+1)+2),whichdetermines;thatiswhyagreementonthreemomentsanddiscrepancyonthefourthshowsthestationarymeasurenottobeBeta.Aswillbeseen,thediscrepancyisremarkablysmall|ontheorderof103,andthatisaboutasbigasitgets.Theproofofthenextlemmaisomitted.Thersttermintheintegralcorrespondstoaleftwardmove,takenwithprobability;thesecond,toarightwardmove;compareLemma6.3.Ifthemovemeasurehasdensity,andthestartingstateischosenfromadensity,thedensityofthepositionafteronemoveis xy x (x)dx+ pZy01 1�xy�x Thenextresulttooisstandard.SupposeisBeta(a;b).Then �(a)�(a+b) +1) +1)( PERSIDIACONISANDDAVIDFREEDMANThesecondequalityfollowsfromtherecursion�(+1)=):thereareinthenumeratorandinthedenominator.Corollary6.1.isBetaa;a,then 2X2 aX3 aX4 a TheproofofTheorem1.Suppose=0,sothemovemeasureputsmass12eachat0and1;thisisthestationarymeasuretoo,withstationaritybeingachievedinonemove.=0,thetheoremholds.2.Suppose,sothemovemeasureconcentrateson12.Starting,thechainmovesto 2xorx+1 2�1 2x=1 2+1 )witha50{50chance.Clearly,theuniformdistributionisinvariant,itsimageunderthemotionhavingmass uniformlydistributedover[0 ],andmass uniformon[ 1].Since=1and1)isuniform,thetheoremholds.3.Thiswasdiscussedinsection2.1.4.SupposethemovemeasureisBeta(;)with01or1Recallthat+1),andlet).ByCorollary6.1andsometediousalgebra, 2U02 U03 U04  Wemustnowcomputetherstfourmomentsofthestationarydistribution;thelatterexistsbyTheorem5.1.Lethavethestationarydistributionandlet;);makethesetworandomvariablesindependent.Asbefore,write)forthelawof.Then 2L( 2L(U+V� 2L( because)andaresymmetric.Inparticular, 2E(Un)E(Vn (�UV)n]:(6.5)E(Vn)isgivenbyCorollary6.1,soequation(6.5)canbesolvedrecursivelyforthemomentsof,and)for3.However, +3)(9+10+2) +1)(5+2)Consequently, +1)(5+3)(5+2)(Again,unpleasantalgebraicdetailsaresuppressed.)Figure7showsthegraphoftherightsideof(6.6),plottedagainst.Aswillbeseen,thediscrepancyisrathersmall.Remark.Theorem6.1isconnectedtoresultsinDubinsandFreedman(1967).Considergeneratingarandomdistributionfunctionbyconstructingitsgraphintheunitsquare.Drawahorizontallinethroughthesquare,cuttingtheverticalaxisinto ITERATEDRANDOMFUNCTIONS 1.500.75Ð0.75 Fig.7Dierencebetweenfourthmomentofstationarydistributionandfourthmomentofapproxi-matingBeta,scaledbyandplottedagainst;symmetricchain,Beta;movedistri-alowersegmentandanuppersegmentwhoselengthsstandintheratioto1Pickapointatrandomonthisline.Thatdividesthesquareintofourrectangles.Nowrepeattheconstructioninthelowerleftandupperrightrectangles.(Thedescriptionmaybecumbersome,buttheinductivestepiseasy.)Thelimitingmonotonecurveconnectingallthechosenpointsisthegraphofarandomdistributionfunction.Theaverageofthesedistributionfunctionsturnsouttobeabsolutelycontinuous:letitsdensity.Thisdensityis,byconstruction,invariantunderthefollowingoperation.Chooseatrandomuniformlyon[01];distributemassaccordingtorescaledoverr;x]andmass1accordingtorescaledover[1].Ifisuniformand)+(1Inshort,isthestationarydensityforourMarkovprocess.TheequationinLemma6.3isdiscussedinsection9ofDubinsandFreedman(1967).7.DirichletDistributions.TheDirichletdistributionisthemultidimensionalanalogofthemorefamiliarBeta,andisoftenusedinBayesiannonparametricstatis-tics.AnearlypaperisFreedman(1963);alsoseeFabius(1964),Ferguson(1973)orFerguson,Phadia,andWari(1992).Sections7.1and7.2sketchaconstructionoftheDirichlet.Thesettingisaninnite-dimensionalspace,namely,thespaceofallprobabilitymeasuresonanunderlyingcompleteseparablemetricspace.Section7.3discussesthelawofthemeanofpickedatrandomfromaDirichletdistribution,whichcansometimesbecomputedinclosedform.Thesettingistherealline.7.1.RandomMeasures.Let()beacompleteseparablemetricspace,forinstance,therealline.Letbethesetofallprobabilitymeasuresonbetypicalelementsof,thatis,typicalprobabilitieson.Wewillbeconsideringrandomprobabilities:thesearerandomobjectswithvaluesin.The\law"ofsuchanobjectisaprobabilityon.Letbeanitemeasureon.The\Dirichletwithbasemeasure,"usuallyabbreviatedas,isthelawofacertainrandomprobabilityon.Thus,isaprobabilityonHere,weshowhowtoconstructbymodifyingtheargumentforTheorem5.1.ThestatespacefortheMarkovchainis.Thevariationdistancebetweenisdenedas=sup PERSIDIACONISANDDAVIDFREEDMANrunsoveralltheBorelsubsetsof.The\parameterspace"fortheLipschitzfunctionswillbe=[0.If01and,letintobythe+(1Itiseasytoseethatisananemapofintoitself.Furthermore,thisfunctionisLipschitz,withLipschitzconstantisanyprobabilitymeasureontheparameterspace,theMarkovchainondrivenbyhasauniquestationarydistribution.TheDirichletwillbeobtainedbyspecializing.Caution:thestationarydistributionisaprobabilityon,thatis,aprobabilityontheprobabilitieson;andthereisaregularitycondition,namely,u;pRecallthatstandsforlaw.Thenhasthestationarydistributionif+(1U;Pindependentof.Thestationarydistributionmayberepresentedbythebackwarditeration,asthelawoftherandomprobability=(1In(7.3),the()areindependent,withcommondistribution;aswillbeseeninamoment,thesumconvergesa.s.Thelimitisarandomprobabilityonbecauseeachisarandomprobabilityon,andthearerandomelementsof[0Furthermore,telescopesto1.Invariationdistance,iscompletebutnotseparable.Thus,Theorem5.1doesnotapply.Ratherthandealwiththemeasure-theoretictechnicalitiescreatedbyaninseparablespace,wesketchadirectargumentforconvergence.First,wehavetoprovethatthesumin(7.4)convergesa.s.Indeed,writeforthethterm.Then=(1,whereby(7.1).Thus n�1gp ,and .Animmediateconsequence:withprobability1,thesumontherightin(7.3)isCauchyandhenceconvergesinvariationnorm(completeness).Thelawofiseasilyseentobestationary,usingthecriterion(7.2).Togetageometricrateofconvergence,supposethechainstartsfrom.Letbethesumofthersttermsin(7.3).Aftermovesstartingfrom,thebackwardprocesswillbeat,where.Bypreviousarguments,exceptforasetofgeometricallysmallprobability,aregeometricallysmall.Wehaveprovedthefollowingresult.Theorem7.1.Supposeholds.ConsidertheMarkovchainondrivenby.Letq;dpbethelawofthechainaftermovesstartingfromThereisauniqueinvariantprobabilityThereisapositive,niteconstantandansuchthatthatPn(q;
);]Arnforallandall ITERATEDRANDOMFUNCTIONSInthistheorem,istheProkhorovmetriconprobabilitieson,constructedfromthevariationdistanceon,asinDenition5.1.Theconstantisuniversal,becausevariationdistanceisuniformlybounded.Ifcondition(7.1)fails,thechainstagnatesatthestartingpositionWenowspecializetogettheDirichlet.Recallthatisanitemeasureon)bethetotalmassofandlet,whichisaprobability.Let~betheimageofunderthemap,withbeingpointmassat.Thus,~isaprobabilityon,namely,thelawofchosenatrandomfrom.(Caution:seesection7.2formeasurability.)Finally,weset=Beta(.Inotherwords,isthelawof(u;),whereischosenfromtheBeta(1)distributionandisindependentlychosenfrom.Forthisthelawoftherandomprobabilitydenedby(7.3)isDirichlet,withbasemeasureWhydoestheconstructiongive?Wesketchtheargumentforaleadingspecialcase,when;fordetails,seeSethuramanandTiwari(1982).Let2.Then.Allweneedtocheckisstationarity.Letbearandompickfrom.Condition(7.2)forstationarityis+(1isBeta(withprobability,andQ;U;Wareindependent.Ofcourse,|themassesassignedbyto0and1|shouldbeDirichletwithby(7.7a).ThedensityofaDirichletdistributionwiththeseparametersisx;yfor(x;y)with;y�1.Thenormalizingconstantmakes=1;itsnumericalvaluewillnotmatterhere.Conditiononin(7.6)anduse(7.7c),(7.7d).Stationarityboilsdowntox;y kkZ1 u2fx� u;y isthedensityoftherandomvariablein(7.6).By(7.7b),Wedealwith,below.Thenexttaskistodeterminetherangeoftheintegralin(7.9).Thereareseveralconstraintson.Firstisthat=u�Second,(=u1,whichfollowsfrom1.Third,,whichfollowsfrom(7.10),because1.Fourth, u+y u1; PERSIDIACONISANDDAVIDFREEDMANwhichfollowsfrom1.Finally,1.Thus,theintegralin(7.9)goesfromto1;thereisquitealotofcancellationof's,andandu�(1�x)]0�1du=Cx0y1�1(1�x�y)2�1:Thetermsin(7.8)canbeevaluatedthesameway: kkZ11�y1 u2fx u;y� ug(u)du=0�1y1(1�x�y)2�1;T2=2 kkZ1x+y1 u2fx u;y because+(1)=1.Thiscompletestheproofof(7.6).Thesameargumentgoesthroughforanynite.Thencompactcanbehandledbytakinglimits.Alongtheway,ithelpstocheckthatAcompleteseparablecanbeembeddedintoacompactset,sothegeneralcasefollowsfromthecompactcase;(7.11)showsthatsitson,asdesired,ratherthanspillingoverontopointsaddedbycompactication.7.2.Measure-TheoreticIssues.Puttheweak-star-eldon:thisisgeneratedbythefunctionsfdprangesovertheboundedcontinuousfunctionsonThevariationnormisweak-starmeasurable,because=supfdpfdqrangesoverthecontinuousfunctionsonwith01.Withabitofeort,wecanrestricttoacountable,densesetofcontinuousfunctions.Measurabilityofthenormisthenclear.Forexample,ifis[01],wecanrestricttothepolynomialswithrationalcoecients.PuttheusualBorel-eldon[01].Then(u;p;q)isjointlymeasurable,from[0.Likewise,(u;pismeasurable.Foreach,the;:::;


isjointlymeasurablefrom.Finally,themapismeasurablefrom ITERATEDRANDOMFUNCTIONSThe\Borel"-eldinisgeneratedbytheopensetsinthenormtopology,andseemstotbetterwithvariationdistance.Butthereisarealproblem:theisnotmeasurableifweputtheBorel-eldson.AreferenceisDubinsandFreedman(1964).WeneedthevariationnormtogettheLipschitzpropertyandtheweak-star-eldtohandlemeasurability.Inacompleteseparablemetricspace,allreasonable-eldscoincide|rangingfromtheBorel-eldto(forinstance)the-eldgeneratedbythebounded,uniformlycontinuousfunctions.Thespaceofprobabilitymeasuresiscompleteinthevariationdistancebutnotseparable.Thatisthesourceofthemeasure-theoreticcomplications.7.3.RandomMeans.bearandompickfrom,asdenedinsection7.1above.Letbeameasurablefunctionon.ConsidertherandomvariablefdP(Ofcourse,therandomvariableisdenedonlywhentheintegralconverges.)FeigenandTweedie(1989)provethefollowingresult.Proposition7.1.-almostallifandonlyiflog(1+Wenowspecializetotherealline(),and)to.Supposelog(1+xdP;Pisarandomvariable|beingthemeanofapickedatrandomfromFormula(7.14)mustbedistinguishedfrom(7.11).In(7.11),youpickatrandom,andtakethemeanoverall'srelativeto:ForanymeasurableIn(7.14),youpickatrandomfrom,andtakethemeanoverall'srelativeto.ThatgivesarandomvariablexdPInanumberofcases,thedistributionofrelativetocanbecomputedexplicitly,usingtheideaofiteratedrandomfunctions.Forinstance,CifarelliandRegazzini(1990)showthatunlessisapointmass,xdPhasanabsolutelycontinuousdistribution,andtheygiveformulasforthedensity.AdditionalresultsareobtainedbyDiaconisandKemperman(1996).7.1.Supposeconcentratesontwopoints,0and1.Relativeto)hastheBeta()distribution.Thisisimmediatefromthediscussioninsection7.1above:afterall,)isthemassassignsto1.7.2.Ifisuniformon[01],thenhasthedensity )for07.3.IfisCauchythenalsohastheCauchydistribution.SeeYamamoto(1984).Ofcourse,)doesnotconverge.Ontheotherhand,(7.13) PERSIDIACONISANDDAVIDFREEDMANholds,sothatforalmostalldrawnfrom,theintegralin(7.14)doesconverge.Picksfromhaveashortertailthan7.4.LetbeCauchy.Ifisthelawof(1+),thenisuniformon[0Forthemathematicsbehindexamples(7.2){(7.4),werefertoDiaconisandKem-perman(1996),whereconnectionstotheMarkovmomentproblemandrecentworkofKerov(1993)areexplained.Weconcludebyshowinghowthelawofin(7.14)canbeobtainedasthestationarydistributionunderiteratedrandomfunctions.Thisisfairlyimmediateonthebasisofsection7.1.Thestatespaceistherealline.From,thechainmovesto+(1,whereisBeta(1),andisaninde-pendentpickfrom.Thelimitingstationarydistribution,whichis),isthedistributionofwhere()arei.i.d.copiesof(U;W):see(7.3).Acknowledgments.WewouldliketothankPhilippeBougerol,OlleHandYuvalPeresformanyusefulsuggestions.Wealsothankourveryhelpfuleditors,DanRockmoreandNickTrefethen.T.W.AndersonOnasymptoticdistributionsofestimatesofparametersofstochasticdif-ferenceequations,Ann.Math.Statist.,30,pp.676{87.L.ArnoldandH.CrauelIteratedfunctionsystemsandmultiplicativeergodictheoryinDiusionTheoryandRelatedProblemsinAnalysisII,M.PinskyandV.Wihstatz,eds.,Birkhauser,Boston,pp.283{305.M.Babillot,P.Bougerol,andL.ElieTherandomdierenceequationinthecriticalcase,Ann.Probab.,25,pp.478{493.F.BaccelliErgodictheoryofstochasticPetrinetworks,Ann.Probab.,20,pp.375{396.F.BaccelliandP.BremaudElementsofQueuingTheory,Springer-Verlag,NewYork.F.Baccelli,G.Cohen,G.J.Olsder,andJ.P.QuadratSynchronizationandLinearityWiley,NewYork.M.BarnsleyFractalsEverywhere,2nded.,AcademicPress,NewYork.M.BarnsleyandJ.EltonAnewclassofMarkovprocessesforimageencoding,Adv.Appl.Probab.,20,pp.14{32.M.BendaAcentrallimittheoremforcontractivestochasticdynamicalsystems,J.Appl.Probab.,35,pp.200{205.P.Bidigare,P.Hanlon,andD.RockmoreAcombinatorialdescriptionofthespectrumfortheTsetlinlibraryanditsgeneralizationtohyperplanearrangements,DukeMathJ.,toappear.R.BlumenthalandH.CorsonOncontinuouscollectionsofmeasures,Ann.Inst.FourierGrenoble,20,pp.193{199.R.BlumenthalandH.CorsonOncontinuouscollectionsofmeasures,inProc.SixthBerkeleySymp.onMathematicalStatisticsandProbability,Vol.2,L.M.LeCam,J.Neyman,andE.L.Scott,eds.,UniversityofCaliforniaPress,Berkeley,pp.33{40A.BorovkovAsymptoticMethodsinQueuingTheory,Wiley,NewYork.A.BorovkovandS.FossStochasticallyrecursivesequencesandtheirgeneralizationsSiberianAdv.Math.,2,pp.16{81.P.BougerolandN.PicardStrictstationarityongeneralizedautoregressiveprocessesAnn.Probab.,20,pp.1714{1730.A.BrandtThestochasticequationwithstationarycoecients,Adv.Appl.Probab.,18,pp.211{220.A.Brandt,P.Franken,andB.LisekStationaryStochasticModels,Wiley,NewYork.L.BreimanThestronglawoflargenumbersforaclassofMarkovchains,Ann.Math.Statist.,31,pp.801{803. ITERATEDRANDOMFUNCTIONSK.BrownandP.DiaconisRandomwalkandhyperplanearrangements,Ann.Probab.,toappear.J.F.ChamayouandG.LetacExplicitstationarydistributionsforcompositionsofrandomfunctionsandproductsofrandommatrices,J.Theoret.Probab.,4,pp.3{36.D.CifarelliandE.RegazziniDistributionfunctionsofmeansofaDirichletprocess,Ann.Statist.,18,pp.429{442.R.CrownoverIntroductiontoFractalsandChaos,JonesandBartlett,Boston.P.DiaconisandJ.KempermanSomenewtoolsforDirichletpriors,inBayesianStatistics5,J.M.Bernardo,J.O.Berger,A.P.Dawid,andA.F.M.Smith,eds.,OxfordUniversityPress,Oxford,UK,pp.97{106.P.DiaconisandM.ShahshahaniProductsofrandommatricesandcomputerimagegen-eration,Contemp.Math.,50,pp.173{182.L.DubinsandD.FreedmanInvariantprobabilitiesforcertainMarkovprocesses,Ann.Math.Statist.,37,pp.837{844.L.DubinsandD.FreedmanMeasurablesetsofmeasures,PacicJ.Math.,14,pp.1211{22.L.DubinsandD.FreedmanRandomdistributionfunctions,FifthBerkeleySymp.Math.Statist.Probab.,II,part1,pp.183{214D.DubischarTheRepresentationofMarkovProcessesbyRandomDynamicalSystemsTechnicalReport393,InstitutfurDynamischeSysteme,UniversityofBremen,Germany.R.DudleyRealAnalysisandProbability,Wadsworth,PacicGrove,CA.M.DufloRandomIterativeModels,Springer-Verlag,NewYork.J.EltonAmultiplicativeergodictheoremforLipschitzmaps,StochasticProc.Appl.,34,pp.39{47.M.EmbreeandL.N.TrefethenSurprisingBehaviorofRandomFibonacciSequencesTechnicalreport,OxfordUniversityComputingLaboratory,Oxford,UK.P.ErdOnafamilyofsymmetricBernoulliconvolutions,Amer.J.Math.,61,pp.974{975.P.ErdOnthesmoothnesspropertiesofBernoulliconvolutions,Amer.J.Math.,62,pp.J.FabiusAsymptoticbehaviorofBayesestimates,Ann.Math.Statist.,35,pp.846{856.P.FeigenandE.TweedieLinearfunctionalsandMarkovchainsassociatedwiththeDirich-letprocess,Math.Proc.CambridgePhilos.Soc.,105,pp.579{585.W.FellerAnIntroductiontoProbabilityTheoryandItsApplications,II,2nded.,Wiley,NewYork.T.FergusonABayesiananalysisofsomenonparametricproblems,Ann.Statist.,1,pp.T.Ferguson,E.Phadia,andR.WariBayesiannonparametricinference,inIssuesinStatisticalInference:EssaysinHonorofD.Basu,M.GhoshandP.K.Pathak,eds.,IMSLectureNotes17,Inst.Math.Statist.,Hayward,CA,pp.127-150.J.FillAninterruptiblealgorithmforperfectsamplingvianiteMarkovchains,Ann.Appl.Probab.,8,pp.131{162.Y.Fisher,ed.FractalImageGeneration,Springer-Verlag,NewYork.D.FreedmanOntheasymptoticbehaviorofBayesestimatesinthediscretecase,Ann.Math.Statist.,34,pp.1386{1403.H.FurstenbergNon-commutingrandomproducts,Trans.Amer.Math.Soc.,108,pp.377{A.GarsiaArithmeticpropertiesofBernoulliconvolutions,Trans.Amer.Math.Soc.,102,pp.409{432.C.GoldieImplicitrenewaltheoryandtailsofsolutionsofrandomequations,Ann.Appl.Probab.,1,pp.126{166.C.GoldieandR.MallerStabilityofPerpetuities,Tech.Report,Dept.ofMathematics,UniversityofWesternAustralia,Nedlands.O.HomandK.NelanderOnExactSimulationofMarkovRandomFieldsUsingCouplingsfromthePast,TechnicalReport,ChalmersTechnicalUniversity,Goteberg,Sweden.O.Hom,M.-C.vanLieshout,andJ.MoCharacterizationresultsandMarkovchainMonteCarloalgorithmsincludingexactsimulationforsomespatialpointprocessesBernoulli,toappear.J.HammersleyandD.HandscombMonteCarloMethods,ChapmanandHall,London.J.HutchinsonFractalsandself{similarity,IndianaUniv.Math.J.,30,pp.713{747.A.JessenandA.WintnerDistributionfunctionsandtheRiemannzetafunction,Trans.Amer.Math.Soc.,38,pp.48{88.S.KerovTransitionProbabilitiesforContinualYoungDiagramsandtheMarkovMomentProblem,TechnicalReport,InstituteforElectricityandCommunications,St.Petersburg,Russia. PERSIDIACONISANDDAVIDFREEDMANH.KestenRandomdierenceequationsandrenewaltheoryforproductsofrandommatricesActaMath.,131,pp.207{248.Y.KiferErgodicTheoryofRandomTransformations,Birkhauser,Boston.R.KindermanandJ.L.SnellMarkovRandomFields,Contemp.Math.1,AMS,Provi-dence,RI.G.LetacAcontractionprincipleforcertainMarkovchainsanditsapplications,Contemp.Math.50,AMS,Providence,RI,pp.263{273.N.Metropolis,A.Rosenbluth,M.Rosenbluth,A.Teller,andE.Tellerofstatecalculationsbyfastcomputingmachines,J.Chem.Phys.,21,pp.1087{1092.J.MoMarkovChainMonteCarloandspatialpointprocesses,inStochasticGeometry,Likelihood,andComputation,O.Barndor-Nielsen,W.S.Kendall,andM.-C.vanLieshout,eds.,SeminaireEuropeendeStatistique,ChapmanandHall,London.S.P.MeynandR.L.TweedieMarkovChainsandStochasticStability,Springer-Verlag,Y.PeresandB.SolomyakAbsolutecontinuityofBernoulliconvolutions,asimpleproofMath.Res.Lett.,3,pp.231{239.Y.PeresandB.SolomyakSelf-similarmeasuresandintersectionsofCantorsets,Trans.Amer.Math.Soc.,toappear.M.PriestleyNon-LinearandNon-StationaryTimeSeriesAnalysis,AcademicPress,NewYork.J.ProppandD.WilsonExactsamplingwithcoupledMarkovchains,RandomStructuresAlgorithms,9,pp.223{252.J.ProppandD.WilsonHowtogetaperfectlyrandomsamplefromagenericMarkovchainandgeneratearandomspanningtreeofadirectedgraph,J.Algorithms,toappear.A.QuasOnrepresentationofMarkovchainsbyrandomsmoothmaps,Bull.LondonMath.Soc.,23,pp.487{492.S.RachevandG.SamorodnitskyLimitlawsforastochasticprocessandrandomrecursionarisinginprobabilisticmodelling,Adv.Appl.Probab.,27,pp.185{202.G.SamorodnitskyandM.TaqquStableNon-GaussianRandomProcesses:StochasticModelswithInniteVariance,Chapman&Hall,NewYork.J.SethuramanandR.TiwariConvergenceofDirichletmeasuresandtheinterpretationoftheirparameters,inStatisticalDecisionTheoryandRelatedTopicsIII,J.BergerandS.Gupta,eds.,AcademicPress,NewYork,pp.305{315.B.SolomyakOntherandomseries(anErdosproblem),Ann.Math.,242,pp.611{625.F.SpitzerAcombinatoriallemmaanditsapplicationtoprobabilitytheory,Trans.Amer.Math.Soc.,82,pp.323{339.D.SteinsaltzZeno'swalk:Arandomwalkwithrenements,Probab.TheoryRelatedFields,107,pp.99{121.D.SteinsaltzLocallyContractiveIteratedFunctionSystems,TechnicalReport,BerlinUni-versity,Germany.V.StrassenTheexistenceofprobabilitymeasureswithgivenmarginals,Ann.Math.Statist.,36,pp.423{438.W.VervaatOnastochasticdierenceequationandarepresentationofnon-negativein-nitelydivisiblerandomvariables,Adv.Appl.Probab.,11,pp.750{783.H.YamatoCharacteristicfunctionsofmeansofdistributionschosenfromaDirichletpro-cesses,Ann.Probab.,12,pp.262{267.V.ZolotarevOne-DimensionalStableDistributions,Transl.Math.Monographs65,AMS,Providence,RI.