In tro duction ha observ ed indep enden data oin ts x from the same densit restrict to the class of Gamma densities ie a a Ga a 57344 exp 10 12 14 16 18 20 002 004 006 008 01 012 014 px Figure 1 The Ga3 2 densit function Figure plots ypical Gamma d ID: 23852
Download Pdf The PPT/PDF document "Estimating Gamma distribution Thomas Min..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
EstimatingaGammadistributionThomasP.Minka2002AbstractThisnotederivesafastalgorithmformaximum-likelihoodestimationofbothparametersofaGammadistributionornegative-binomialdistribution.1IntroductionWehaveobservednindependentdatapointsX=[x1::xn]fromthesamedensity.WerestricttotheclassofGammadensities,i.e.=(a;b):p(xja;b)=Ga(x;a;b)=xa 1 (a)baexp( xb)0246810121416182000.020.040.060.080.10.120.14xp(x)Figure1:TheGa(3;2)densityfunction.Figure1plotsatypicalGammadensity.Ingeneral,themeanisabandthemodeis(a 1)b.2MaximumlikelihoodThelog-likelihoodislogp(Dja;b)=(a 1)Xilogxi nlog (a) nalogb 1bXixi(1)=n(a 1)logx nlog (a) nalogb nx=b(2)Themaximumforbiseasilyfoundtobe^b=x=a(3)1 05101520-6-5.5-5-4.5-4ExactFigure2:Thelog-likelihood(4)versustheGamma-typeapproximation(9)andthebound(6)atconver-gence.Theapproximationisnearlyidenticaltothetruelikelihood.Thedatasetwas100pointssampledfromGa(7:3;4:5).Substitutingthisinto(1)giveslogp(Dja;^b)=n(a 1)logx nlog (a) nalogx+naloga na(4)Wewilldescribetwoalgorithmsformaximizingthisfunction.The rstmethodwilliterativelymaximizealowerbound.Becausealogaisconvex,wecanusealinearlowerbound:aloga(1+loga0)(a a0)+a0loga0(5)logp(Dja;^b)n(a 1)logx nlog (a) nalogx+n(1+loga0)(a a0)+na0loga0 na(6)Themaximumisat0=nlogx n (a) nlogx+n(1+loga0) n(7) (^a)=logx logx+loga0(8)where isthedigammafunction.Theiterationproceedsbysettinga0tothecurrent^a,theninvertingthe functiontogetanew^a.Becausethelog-likelihoodisconcave,thisiterationmustconvergetothe(unique)globalmaximum.Unfortunately,itcanbequiteslow,requiringaround250iterationsifa=10,lessforsmallera,andmoreforlargera.Thesecondalgorithmismuchfaster,andisobtainedvia`generalizedNewton'[1].Usinganapproximationoftheform,logp(Dja;^b)c0+c1a+c2log(a)(9)theupdateis1anew=1a+logx logx+loga (a)a2(1=a 0(a))(10)Thisconvergesinaboutfouriterations.Figure2showsthatthisapproximationisveryclosetothetruelog-likelihood,whichexplainsthegoodperformance.2 Agoodstartingpointfortheiterationisobtainedviatheapproximationlog (a)alog(a) a 12loga+const.(Stirling)(11) (a)log(a) 12a(12)^a0:5logx logx(13)(NotethatlogxlogxbyJensen'sinequality.)2.1NegativebinomialThemaximum-likelihoodproblemforthenegativebinomialdistributionisquitesimilartothatfortheGamma.ThisisbecausethenegativebinomialisamixtureofPoissons,withGammamixingdistribution:p(xja;b)=ZPo(x;)Ga(;a;b)d=Zxx!e a 1 (a)bae =bd(14)=a+x 1xbb+1x1 bb+1a(15)Let'sconsideraslightlygeneralizednegativebinomial,wherethe`waitingtime'forxisgivenbyt:p(xjt;a;b)=ZPo(x;t)Ga(;a;b)d=Z(t)xx!e ta 1 (a)bae =bd(16)=a+x 1xbtbt+1x1 btbt+1a(17)GivenadatasetD=f(xi;ti)g,wewanttoestimate(a;b).OneapproachistouseEM,wheretheE-stepinfersthehiddenvariablei:E[i]=(xi+a)bbti+1(18)E[logi]= (xi+a)+logbbti+1(19)TheM-stepthenmaximizes(a 1)XiE[logi] nlog (a) nalogb 1bXiE[i](20)whichisaGammamaximum-likelihoodproblem.ReferencesThomasP.Minka.Beyondnewton'smethod.research.microsoft.com/~minka/papers/newton.html,2000.3