page40110SOR2012002iiBernoullirvifp1pp01pqpk0k60or1thenGXsEsXqps34iiiGeometricrvifpkpqk1k12q1pthenGXsps1qsifjsjq1seeHWSheet435ivBinomialrv ID: 94786
Download Pdf The PPT/PDF document "Probability Generating Functions" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
page39110SOR201(2002)Chapter3ProbabilityGeneratingFunctions3.1Preamble:GeneratingFunctionsGeneratingfunctionsarewidelyusedinmathematics,andplayanimportantroleinprobabilitytheory.Considerasequencefai:i=0;1;2;:::gofrealnumbers:thenumberscanbe`parcelledup'inseveralkindsof`generatingfunctions'.The`ordinary'generatingfunctionofthesequenceisde nedasG(s)=1Xi=0aisiforthosevaluesoftheparametersforwhichthesumconverges.Foragivensequence,thereexistsaradiusofconvergenceR(0)suchthatthesumconvergesabsolutelyifjsjRanddivergesifjsj�R.G(s)maybedi erentiatedorintegratedtermbytermanynumberoftimeswhenjsjR.Formanywell-de nedsequences,G(s)canbewritteninclosedform,andtheindividualnumbersinthesequencecanberecoveredeitherbyseriesexpansionorbytakingderivatives.3.2De nitionsandPropertiesConsideracountr.v.X,i.e.adiscreter.v.takingnon-negativevalues.Writepk=P(X=k);k=0;1;2;:::(3:1)(ifXtakesa nitenumberofvalues,wesimplyattachzeroprobabilitiestothosevalueswhichcannotoccur).Theprobabilitygeneratingfunction(PGF)ofXisde nedasGX(s)=1Xk=0pksk=E(sX):(3:2)NotethatGX(1)=1,sotheseriesconvergesabsolutelyforjsj1.AlsoGX(0)=p0.Forsomeofthemorecommondistributions,thePGFsareasfollows:(i)Constantr.v.{ifpc=1;pk=0;k6=c,thenGX(s)=E(sX)=sc:(3:3) page40110SOR201(2002)(ii)Bernoullir.v.{ifp1=p;p0=1 p=q;pk=0;k6=0or1,thenGX(s)=E(sX)=q+ps:(3:4)(iii)Geometricr.v.{ifpk=pqk 1;k=1;2;:::;q=1 p;thenGX(s)=ps1 qsifjsjq 1(seeHWSheet4.):(3:5)(iv)Binomialr.v.{ifXBin(n;p);thenGX(s)=(q+ps)n;(q=1 p)(seeHWSheet4.):(3:6)(v)Poissonr.v.{ifXPoisson();thenGX(s)=1Xk=01k!ke sk=e(s 1):(3:7)(vi)Negativebinomialr.v.{ifXNegBin(n;p),thenGX(s)=1Xk=0 k 1n 1!pnqk nsk=ps1 qsnifjsjq 1andp+q=1:(3:8)UniquenessTheoremIfXandYhavePGFsGXandGYrespectively,thenGX(s)=GY(s)foralls(a)i P(X=k)=P(Y=k)fork=0;1;:::(b)i.e.ifandonlyifXandYhavethesameprobabilitydistribution.ProofWeneedonlyprovethat(a)implies(b).TheradiiofconvergenceofGXandGYare1,sotheyhaveuniquepowerseriesexpansionsabouttheorigin:GX(s)=1Xk=0skP(X=k)GY(s)=1Xk=0skP(Y=k):IfGX=GY,thesetwopowerserieshaveidenticalcoecients.}Example:IfXhasPGFps(1 qs)withq=1 p,thenwecanconcludethatXGeometric(p):GivenafunctionA(s)whichisknowntobeaPGFofacountr.v.X,wecanobtainpk=P(X=k)eitherbyexpandingA(s)inapowerseriesinsandsettingpk=coecientofsk;orbydi erentiatingA(s)ktimeswithrespecttosandsettings=0. page41110SOR201(2002)Wecanextendthede nitionofPGFtofunctionsofX.ThePGFofY=H(X)isGY(s)=GH(X)(s)=EsH(X)=XkP(X=k)sH(k):(3:9)IfHisfairlysimple,itmaybepossibletoexpressGY(s)intermsofGX(s).Example:LetY=a+bX.ThenGY(s)=E(sY)=E(sa+bX)=saE[(sb)X]=saGX(sb):(3:10)3.3MomentsTheoremXbeacountr.v.andG(r)X(1)therthderivativeofitsPGFGX(s)ats=1.ThenG(r)X(1)=E[X(X 1):::(X r+1)](3:11)Proof(informal)G(r)X(s)=drdsr[GX(s)]=drdsr"Xkpksk#=Xkpkk(k 1):::(k r+1)sk r(assumingtheinterchangeofdrdsrandXkisjusti ed).Thisseriesisconvergentforjsj1,soG(r)X(1)=E[X(X 1):::(X r+1)];r1:tuInparticular:G(1)(1)(orG0(1))=E(X)(3:12)andG(2)(1)(orG00(1))=E[X(X 1)]=E(X2) E(X)=Var(X)+[E(X)]2 E(X)soVar(X)=G(2)(1) hG(1)X(1)i2+G(1)X(1):(3:13)Example:IfXPoisson(),thenGX(s)=e(s 1);G(1)X(s)=e(s 1)soE(X)=G(1)(1)=e0=:G(2)X(s)=2e(s 1)soVar(X)=2 2+=: page42110SOR201(2002)3.4SumsofindependentrandomvariablesTheoremXandYbeindependentcountr.v.swithPGFsGX(s)andGY(s)respectively,andletZ=X+Y.ThenGZ(s)=GX+Y(s)=GX(s)GY(s):(3:14)ProofGZ(s)=E(sZ)=E(sX+Y)=E(sX)E(sY)(independence)=GX(s)GY(s):}CorollaryX1;:::;Xnareindependentcountr.v.swithPGFsGX1(s);:::;GXn(s)respectively(andnisaknowninteger),thenGX1+:::+Xn(s)=GX1(s)::::GXn(s):(3:15)Example3.1Findthedistributionofthesumofnindependentr.v.sXi;i=1;:::n;whereXiPoisson(i).SolutionGXi(s)=ei(s 1):SoGX1+X2++Xn(s)=nYi=1ei(s 1)=e(1++n)(s 1):ThisisthePGFofaPoissonr.v.,i.e.nXi=1XiPoisson(nXi=1i):(3:16)Example3.2InasequenceofnindependentBernoullitrials.LetIi=1;iftheithtrialyieldsasuccess(probabilityp)0;iftheithtrialyieldsafailure(probabilityq=1 p).LetX=nXi=1Ii=numberofsuccessesinntrials.WhatistheprobabilitydistributionofX?SolutionSincethetrialsareindependent,ther.v.sI1;:::Inareindependent.SoGX(s)=GI1GI2:::GIn(s):ButGIi(s)=q+ps;i=1;:::n:SoGX(s)=(q+ps)n=nXx=0 nx!(ps)xqn x:ThenP(X=x)=coecientofsxinGX(s)= nxpxqn x;x=0;:::;ni.e.XBin(n;p):} page43110SOR201(2002)3.5Sumofarandomnumberofindependentr.v.sTheoremN;X1;X2;:::beindependentcountr.v.s.IfthefXigareidenticallydistributed,eachwithPGFGX,thenSN=X1++XN(3:17a)hasPGFGSN=GN(GX(s)):(3:17b)(Note:WeadopttheconventionthatX1++XN=0forN=0.)Thisisanexampleoftheprocessknownascompoundingwithrespecttoaparameter.ProofWehaveGSN(s)=EsSN=1Xn=0EsSNjN=nP(N=n)(conditioningonN)=1Xn=0EsSnP(N=n)=1Xn=0GX1++Xn(s)P(N=n)=1Xn=0[GX(s)]nP(N=n)(usingCorollaryinprevioussection)=GN(GX(s))byde nitionofGN:}CorollariesE(SN)=E(N):E(X):(3.18)Proofdds[GSN(s)]=dds[GN(GX(s))]=dGN(u)du:dudswhereu=GX(s):Settings=1;(sothatu=GX(1)=1),wegetE(SN)=hG(1)(1)i:hG(1)X(1)i=E(N):E(X):tuSimilarlyitcanbededucedthat2.Var(SN)=E(N)Var(X)+Var(N)[E(X)]2:(3.19)(prove!). page44110SOR201(2002)Example3.3ThePoissonHenAhenlaysNeggs,whereNPoisson():Eachegghatcheswithprobabilityp,independentlyoftheothereggs.Findtheprobabilitydistributionofthenumberofchicks,Z.SolutionWehaveZ=X1++XN;whereX1;:::areindependentBernoullir.v.swithparameterp.ThenGN(s)=e(s 1);GX(s)=q+ps:SoGZ(s)=GN(GX(s))=ep(s 1){thePGFofaPoissonr.v.,i.e.ZPoisson(p):}3.6*UsingGFstosolverecurrencerelations[NOTE:Notrequiredforexaminationpurposes.]Insteadofappealingtothetheoryofdi erencerelationswhenattemptingtosolvetherecurrencerelationswhichariseinthecourseofsolvingproblemsbyconditioning,onecanoftenconverttherecurrencerelationintoalineardi erentialequationforageneratingfunction,tobesolvedsubjecttoappropriateboundaryconditions.Example3.4TheMatchingProblem(yetagain).AttheendofChapter1[Example(1.10)],therecurrencerelationpn=n 1npn 1+1npn 2;n3wasderivedfortheprobabilitypnthatnomatchesoccurinann-cardmatchingproblem.Solvethisusinganappropriategeneratingfunction.SolutionMultiplythroughbynsn 1andsumoverallsuitablevaluesofntoobtain1Xn=3nsn 1pn=s1Xn=3(n 1)sn 2pn 1+s1Xn=3sn 2pn 2:IntroducingtheGFG(s)=1Xn=1pnsn(N.B.:thisisnotaPGF,sincefpn:n1gisnotaprobabilitydistribution),wecanrewritethisasG0(s) p1 2p2s=s[G0(s) p1]+sG(s)i.e.(1 s)G0(s)=sG(s)+s(sincep1=0;p2=12):This rstorderdi erentialequationnowhastobesolvedsubjecttotheboundaryconditionG(0)=0:TheresultisG(s)=(1 s) 1e s 1:NowexpandG(s)asapowerseriesins,andextractthecoecientofsn:thisyieldspn=1+( 1)1!+( 1)n 1(n 1)!+( 1)nn!;n1{theresultobtainedpreviously.} page45110SOR201(2002)3.7BranchingProcesses3.7.1De nitionsConsiderahypotheticalorganismwhichlivesforexactlyonetimeunitandthendiesintheprocessofgivingbirthtoafamilyofsimilarorganisms.Weassumethat:(i)thefamilysizesareindependentr.v.s,eachtakingthevalues0,1,2,...;(ii)thefamilysizesareidenticallydistributedr.v.s,thenumberof`children'inafamily,C,havingthedistributionP(C=k)=pk;k=0;1;2;:::(3:20)Theevolutionofthetotalpopulationastimeproceedsistermedabranchingprocess:thisprovidesasimplemodelforbacterialgrowth,spreadoffamilynames,etc.LetXn=numberoforganismsbornattimen(i.e.,thesizeofthenthgeneration):(3:21)Theevolutionofthepopulationisdescribedbythesequenceofr.v.sX0;X1;X2;:::(anexampleofastochasticprocess{ofwhichmorelater).WeassumethatX0=1,i.e.westartwithpreciselyoneorganism.Atypicalexamplecanbeshownasafamilytree:Generation0X0=1123:::X1=3:::X2=6:::X3=9WecanusePGFstoinvestigatethisprocess.3.7.2EvolutionofthepopulationLetG(s)bethePGFofC:G(s)=1Xk=0P(C=k)sk;(3:22)andGn(s)thePGFofXn:Gn(s)=1Xx=0P(Xn=x)sx:(3:23)NowG0(s)=s;sinceP(X0=1)=1;P(X0=x)=0forx6=1;andG1(s)=G(s).AlsoXn=C1+C2++CXn 1;(3:24)whereCiisthesizeofthefamilyproducedbytheithmemberofthe(n 1)thgeneration. page46110SOR201(2002)So,sinceXnisthesumofarandomnumberofindependentandidenticallydistributed(i.i.d.)r.v.s,wehaveGn(s)=Gn 1(G(s))forn=2;3;:::(3:25)andthisisalsotrueforn=1.Iteratingthisresult,wehaveGn(s)=Gn 1(G(s))=Gn 2(G(G(s)))=::::=G1(G(G(::::(s)::::)))=G(G(G(::::(s)::::)));n=0;1;2;:::i.e.,GnisthenthiterateofG.NowE(Xn)=G0(1)=G0n 1(G(1))G0(1)=G0n 1(1)G0(1)[sinceG(1)=1]i.e.E(Xn)=E(Xn 1)(3:26)where=E(C)isthemeanfamilysize.HenceE(Xn)=E(Xn 1)=2E(Xn 2)=::::=nE(X0)=n:ItfollowsthatE(Xn)!80;if11;if=1(criticalvalue)1;if�1:(3:27)ItcanbeshownsimilarlythatVar(Xn)=8n2;if=12n 1n 1 1;if6=1:(3:28)Example3.5InvestigateabranchingprocessinwhichChasthemodi edgeometricdistributionpk=pqk;k=0;1;2;:::;0p=1 q1;withp6=q[N.B.]SolutionThePGFofCisG(s)=1Xk=0pqksk=p1 qsifjsjq 1:WenowneedtosolvethefunctionalequationsGn(s)=Gn 1(G(s)): page47110SOR201(2002)First,ifjsj1,G1(s)=G(s)=p1 qs=p(q p)(q2 p2) qs(q p)(rememberwehaveassumedthatp6=q).ThenG2(s)=G(G(s))=p1 qp1 qs=p(1 qs)1 qp qs=p(q2 p2) qs(q p)(q3 p3) qs(q2 p2):WethereforeconjecturethatGn(s)=p(qn pn) qs(qn 1 pn 1)(qn+1 pn+1) qs(qn pn)forn=1;2;:::andjsj1;andthisresultcanbeprovedbyinductiononn.WecouldderivetheentireprobabilitydistributionofXnbyexpandingther.h.s.asapowerseriesins,buttheresultisrathercomplicated.Inparticular,however,P(Xn=0)=Gn(0)=pqn pnqn+1 pn+1=n 1n+1 1;where=q=pisthemeanfamilysize.Itfollowsthatultimateextinctioniscertainif1andlessthancertainif�1.Separateconsiderationofthecasep=q=12(seehomework)showsthatultimateextinctionisalsocertainwhen=1.}Wenowproceedtoshowthattheseconclusionsarevalidforallfamilysizedistributions.3.7.3TheProbabilityofExtinctionTheprobabilitythattheprocessisextinctbythenthgenerationisen=P(Xn=0):(3:29)Nowen1,andenen+1(sinceXn=0impliesthatXn+1=0)-i.e.fengisaboundedmonotonicsequence.Soe=limn!1en(3:30)existsandiscalledtheprobabilityofultimateextinction.Theorem1eisthesmallestnon-negativerootoftheequationx=G(x):ProofNotethaten=P(Xn=0)=Gn(0).NowGn(s)=Gn 1(G(s))=::::=G(G::::(s)::::))=G(Gn 1(s)): page48110SOR201(2002)Sets=0.Thenen=Gn(0)=G(en 1);n=1;2;:::withboundaryconditione0=0.Inthelimitn!1wehavee=G(e):Nowletbeanynon-negativerootoftheequationx=G(x).Gisnon-decreasingontheinterval[0;1](sinceithasnon-negativecoecients),andsoe1=G(e0)=G(0)G()=;e2=G(e1)G()=[usingpreviousline]andbyinductionen;n=1;2;:::Soe=limn!1en:Itfollowsthateisthesmallestnon-negativeroot.}Theorem2e=1ifandonlyif1.[Note:weruleoutthespecialcasep1=1;pk=0fork6=1,when=1bute=0]ProofWecansupposethatp0�0(sinceotherwisee=0and�1).Nowontheinterval[0;1],Gis(i)continuous(sinceradiusofconvergence1);(ii)non-decreasing(sinceG0(x)=Pkkpkxk 10);(iii)convex(sinceG00(x)=Pkk(k 1)pkxk 20).Itfollowsthatin[0;1]theliney=xhaseither1or2intersectionswiththecurvey=G(x),asshownbelow:11yxeey=G(x)y=x(a)11yxy=xy=G(x)(b)Thecurvey=G(x)andtheliney=x,inthetwocaseswhen(a)G0(1)�1and(b)G0(1)ยท1.SinceG0(1)=,itfollowsthate=1ifandonlyif1:tu page49110SOR201(2002)Example3.6FindtheprobabilityofultimateextinctionwhenChasthemodi edgeometricdistributionpk=pqk;k=0;1;2;:::;0p=1 q1(thecasep=qisnowpermitted-cf.Example3.5).SolutionThePGFofCisG(s)=p1 qs;jsjq 1;andeisthesmallestnon-negativerootofG(x)=p1 qx=x:Therootsare1(2p 1)2(1 p):Soifp12(i.e.=q=p�1);e=p=q(= 1);ifp12(i.e.1);e=1:{inagreementwithourearlierdiscussioninExample3.5.}[Note:intheabovediscussion,thecontinuityofprobabilitymeasures(seeNotesbeloweqn.(1.10))hasbeeninvokedmorethanoncewithoutcomment.]