MassiveMetricDataStreamsAirTrafcExamplesTengKuhnandSJnlAerospaceCompInfCommunacc2012SyntheticExamplesTengHarlowLeeandSACMTransModCompSimr22012TheoryofRegularPavingsRPs ID: 393840
Download Pdf The PPT/PDF document "StatisticalRegularPavingsinBayesianNonpa..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
StatisticalRegularPavingsinBayesianNonparametricDensityEstimation MassiveMetricDataStreamsAirTrafcExamples(Teng,KuhnandS.,Jnl.AerospaceComp.,Inf.&Commun.,[acc.]2012)SyntheticExamples(Teng,Harlow,LeeandS.,ACMTrans.Mod.&Comp.Sim.,[r.2]2012)TheoryofRegularPavings(RPs)TheoryofRealMappedRegularPavings(R-MRPs)StatisticalRegularPavings(SRPs)AdaptiveHistogramsS.E.B.PriorityQueue–L1ConsistentInitializationSmoothingbyAveragingPosteriorExpectationoverHistogramsinS0:1Examples-good,badanduglyConclusionsandReferences 2/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation MassiveMetricDataStreams AirTrafcExamples(Teng,KuhnandS.,Jnl.AerospaceComp.,Inf.&Commun.,[acc.]2012) MassiveMetricDataStreams–AirTrafcExampleOnaSunnyDay 4/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation MassiveMetricDataStreams SyntheticExamples(Teng,Harlow,LeeandS.,ACMTrans.Mod.&Comp.Sim.,[r.2]2012) MassiveMetricDataStreams–SyntheticExamplesandproduceaconsistentestimateofthedensity 7/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation TheoryofRegularPavings(RPs) IntervalsandBoxesinRdIntervalsandBoxesasintervalvectors:x=[x 1; x1][x 2; x2]:::[x d; xd];x i xi: 1-dim. 2-dim. 3-dim. Figure:Boxesin1D,2D,and3D.8/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation TheoryofRegularPavings(RPs) AnRPtreearootintervalx2IRd Theregularlypavedboxesofxcanberepresentedbynodesofniterootedbinary(frb-trees)ofgeometricgrouptheoryAnoperationofbisectiononaboxisequivalenttoperformingtheoperationonitscorrespondingnodeinthetree: LeafboxesofRPtreepartitiontherootintervalx2IR1Bisectatthemidpointofthechosenleafinterval ~ x~~L~R������@@@@@@xLxR ~������~������~LL@@@@@@~LR@@@@@@~RxLRxLLxR LeafboxesofRPtreepartitiontherootintervalx2IR2Bisectatthemidpointoftherstwidestsideofthechosenleafbox z xzzLzR���@@@ xLxRz���z���zLL@@@zLR@@@zR xLRxLLxRz���z���zLL@@@zLR@@@zzRL@@@zRR xLRxLLxRLxRRz���@@@zAAAzAAAzLLzzRLzRRAAAzLRLzLRR xLRL xLRR xLLxRLxRR Bythis“RPPeano'scurve”frb-treesencodeparitionsofx2IRd 10/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation TheoryofRegularPavings(RPs) AnRPtreearootintervalx2IRd Theregularlypavedboxesofxcanberepresentedbynodesofniterootedbinary(frb-trees)ofgeometricgrouptheoryAnoperationofbisectiononaboxisequivalenttoperformingtheoperationonitscorrespondingnodeinthetree: LeafboxesofRPtreepartitiontherootintervalx2IR2Bisectatthemidpointoftherstwidestsideofthechosenleafbox z xzzLzR���@@@ xLxRz���z���zLL@@@zLR@@@zR xLRxLLxRz���z���zLL@@@zLR@@@zzRL@@@zRR xLRxLLxRLxRRz���@@@zAAAzAAAzLLzzRLzRRAAAzLRLzLRR xLRL xLRR xLLxRLxRR Bythis“RPPeano'scurve”frb-treesencodeparitionsofx2IRd 12/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation TheoryofRegularPavings(RPs) RPsareclosedunderunionoperations Lemma1:Thealgebraicstructureoffrb-trees(underlyingThompson'sgroup)isclosedunderunionoperations. Proof:bya“transparencyoverlayprocess”argument(cf.Meier2008).s(1)[s(2)=sisunionoftwoRPss(1)ands(2)ofx2R2. 18/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation TheoryofRealMappedRegularPavings(R-MRPs) Dfn:RealMappedRegularPaving(R-MRP) ILets2S0:1beanRPwithrootnodeandrootboxx2IRd ILetV(s)andL(s)denotethesetsallnodesandleafnodesofs,respectively. ILetf:V(s)!RmapeachnodeofstoanelementinRasfollows:fv7!fv:v2V(s);fv2Rg: ISuchamapfiscalledaR-mappedregularpaving(R-MRP). IThus,aR-MRPfisobtainedbyaugmentingeachnodevoftheRPtreeswithanadditionaldatamemberfv. 20/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation TheoryofRealMappedRegularPavings(R-MRPs) Dfn:RealMappedRegularPaving(R-MRP) ILets2S0:1beanRPwithrootnodeandrootboxx2IRd ILetV(s)andL(s)denotethesetsallnodesandleafnodesofs,respectively. ILetf:V(s)!RmapeachnodeofstoanelementinRasfollows:fv7!fv:v2V(s);fv2Rg: ISuchamapfiscalledaR-mappedregularpaving(R-MRP). IThus,aR-MRPfisobtainedbyaugmentingeachnodevoftheRPtreeswithanadditionaldatamemberfv. 21/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation TheoryofRealMappedRegularPavings(R-MRPs) Dfn:RealMappedRegularPaving(R-MRP) ILets2S0:1beanRPwithrootnodeandrootboxx2IRd ILetV(s)andL(s)denotethesetsallnodesandleafnodesofs,respectively. ILetf:V(s)!RmapeachnodeofstoanelementinRasfollows:fv7!fv:v2V(s);fv2Rg: ISuchamapfiscalledaR-mappedregularpaving(R-MRP). IThus,aR-MRPfisobtainedbyaugmentingeachnodevoftheRPtreeswithanadditionaldatamemberfv. 23/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation TheoryofRealMappedRegularPavings(R-MRPs) ExampleofanR-MRPSimplefunctionsoveranRPtreepartitionR-MRPovers221withx=[0;8] 25/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation TheoryofRealMappedRegularPavings(R-MRPs) R-MRPArithmeticIf?:RR!Rthenwecanextend?point-wisetotwoR-MRPsfandgwithrootnodes(1)and(2)viaMRPOperate((1);(2);?).ThisisdoneusingMRPOperate((1);(2);+)fgf+g 26/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation TheoryofRealMappedRegularPavings(R-MRPs) R-MRPAdditionbyMRPOperate((1);(2);+)addingtwopiece-wiseconstantfunctionsorR-MRPs27/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation TheoryofRealMappedRegularPavings(R-MRPs) UnarytransformationsareeasytooLetMRPTransform(; )applytheunarytransformation :R!RtoagivenR-MRPfwithrootnodeasfollows:IcopyftogIrecursivelysetfv= (fv)foreachnodevingIreturngas (f)29/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation TheoryofRealMappedRegularPavings(R-MRPs) ArithmeticandAlgebraofR-MRPsThus,wecanobtainanyR-MRParithmeticalexpressionthatisspeciedbynitelymanysub-expressionsinvolving: 1.constantR-MRP, 2.binaryarithmeticoperation?2f+;�;;=govertwoR-MRPs, 3.standardtransformationsofR-MRPsbyelementsofS:=fexp;sin;cos;tan;:::gand 4.theircompositions. 33/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation TheoryofRealMappedRegularPavings(R-MRPs) Stone-WierstrassTheorem:R-MRPsDenseinC(x;R) TheoremLetFbetheclassofR-MRPswiththesamerootboxx.ThenFisdenseinC(x;R),thealgebraofreal-valuedcontinuousfunctionsonx. Proof:Sincex2IRdisacompactHausdorffspace,bytheStone-WeierstrasstheoremwecanestablishthatFisdenseinC(x;R)withthetopologyofuniformconvergence,providedthatFisasub-algebraofC(x;R)thatseparatespointsinxandwhichcontainsanon-zeroconstantfunction. WewillshowalltheseconditionsaresatisedbyF 35/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation TheoryofRealMappedRegularPavings(R-MRPs) Stone-WierstrassTheorem:R-MRPsDenseinC(x;R) TheoremLetFbetheclassofR-MRPswiththesamerootboxx.ThenFisdenseinC(x;R),thealgebraofreal-valuedcontinuousfunctionsonx. Proof:Sincex2IRdisacompactHausdorffspace,bytheStone-WeierstrasstheoremwecanestablishthatFisdenseinC(x;R)withthetopologyofuniformconvergence,providedthatFisasub-algebraofC(x;R)thatseparatespointsinxandwhichcontainsanon-zeroconstantfunction. WewillshowalltheseconditionsaresatisedbyF 37/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation TheoryofRealMappedRegularPavings(R-MRPs) Stone-WierstrassTheoremContd.:R-MRPsDenseinC(x;R) IFisasub-algebraofC(x;R)sinceitisclosedunderadditionandscalarmultiplication. IFcontainsnon-zeroconstantfunctions IFinally,RPscanclearlyseparatedistinctpointsx;x02xintodistinctleafboxesbysplittingdeeplyenough. IThus,F,theclassofR-MRPswiththesamerootboxx,isdenseinC(x;R),thealgebraofreal-valuedcontinuousfunctionsonx. IQ.E.D. 39/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation TheoryofRealMappedRegularPavings(R-MRPs) Stone-WierstrassTheoremContd.:R-MRPsDenseinC(x;R) IFisasub-algebraofC(x;R)sinceitisclosedunderadditionandscalarmultiplication. IFcontainsnon-zeroconstantfunctions IFinally,RPscanclearlyseparatedistinctpointsx;x02xintodistinctleafboxesbysplittingdeeplyenough. IThus,F,theclassofR-MRPswiththesamerootboxx,isdenseinC(x;R),thealgebraofreal-valuedcontinuousfunctionsonx. IQ.E.D. 41/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation TheoryofRealMappedRegularPavings(R-MRPs) KernelDensityEstimate(visualizationofaprocedure) 43/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation TheoryofRealMappedRegularPavings(R-MRPs) ApproximatingKernelDensityEstimatesbyR-MRPs 44/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation TheoryofRealMappedRegularPavings(R-MRPs) ApproximatingKernelDensityEstimatesbyR-MRPs 45/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation TheoryofRealMappedRegularPavings(R-MRPs) FindingimageofR-MRPisbyfastlook-ups Algorithm4:PointWiseImage(;x) input:withboxx,therootnodeofR-MRPfwithRPs,andapointx2x.output:Returnf(x)attheleafnode(x)thatisassociatedwiththeboxx(x)containingx.ifIsLeaf()then returnfendelse ifx2xRthen PointWiseImage(R;x)endelse PointWiseImage(L;x)endend ICostofKDEimageO(n)KFLOPs(FLOPsforkernelevaluationprocedure) I10-foldCVcost10O�1 10n9 10n=O(n2)KFLOPs IButusingR-MRPapproximationtoKDErequires10O�1 10nlg�9 10n=O(nlg(n))tree-look-ups 46/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation TheoryofRealMappedRegularPavings(R-MRPs) FindingimageofR-MRPisbyfastlook-ups Algorithm7:PointWiseImage(;x) input:withboxx,therootnodeofR-MRPfwithRPs,andapointx2x.output:Returnf(x)attheleafnode(x)thatisassociatedwiththeboxx(x)containingx.ifIsLeaf()then returnfendelse ifx2xRthen PointWiseImage(R;x)endelse PointWiseImage(L;x)endend ICostofKDEimageO(n)KFLOPs(FLOPsforkernelevaluationprocedure) I10-foldCVcost10O�1 10n9 10n=O(n2)KFLOPs IButusingR-MRPapproximationtoKDErequires10O�1 10nlg�9 10n=O(nlg(n))tree-look-ups 49/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation TheoryofRealMappedRegularPavings(R-MRPs) Coverage,Marginal&SliceOperatorsofR-MRP Marginaldensitiesff1g(x1)andff2g(x2)alongeachcoordinateofR-MRPapproximation51/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation StatisticalRegularPavings(SRPs) StatisticalRegularPavings(SRPs) IExtendedfromtheRP;ICachesrecursivelycomputablestatisticsateachboxornodeasdatafallsthrough;IThesestatisticsinclude:Ithesamplecount;Ithesamplemeanvector;Ithesamplevariance-covariancematrix;Iandthevolumeofthebox. Cachingthesamplecountineachnode(orbox). z10 rrrrrrrrrr rrrrrrrrrr���z@@@zR55 ���zLL@@@zLR32 xLRxLLxR 54/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation AdaptiveHistograms SRPsasAdaptiveHistogramsSRPestimateofffromrandomvectorsX1;X2;:::;Xniidfisfn;_s(x)=1 nnXi=111(xi2x(x)) vol(x(x));x(x)2`(_s)istheleafboxcontainingxwithvolumevol(x(x)) Figure:ASRPasahistogramestimate. z10 rrrrrrrrrr���z@@@zR55 ���zLL@@@zLR23 xLRxLLxR 56/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation AdaptiveHistograms SRPsasAdaptiveHistogramsSRPestimateofffromrandomvectorsX1;X2;:::;Xniidfisfn;_s(x)=1 nnXi=111(xi2x(x)) vol(x(x));x(x)2`(_s)istheleafboxcontainingxwithvolumevol(x(x)) Figure:ASRPasahistogramestimate. z10 rrrrrrrrrr���z@@@zR55 ���zLL@@@zLR23 xLRxLLxR 57/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation AdaptiveHistograms NonparametricDensityEstimation–RecapApproach:UsestatisticalregularpavingtogetR-MRPdata-adaptivehistogram 59/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation AdaptiveHistograms NonparametricDensityEstimation–RecapSolution:R-MRPhistogramaveragingallowsustoproduceaconsistentBayesianestimateofthedensity(upto10dimensions)(Teng,Harlow,LeeandS.,ACMTrans.Mod.&Comp.Sim.,2013) 60/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation AdaptiveHistograms S.E.B.PriorityQueue–L1ConsistentInitialization APrioritizedQueuebasedAlgorithm(forL1ConsistentInitialization) AlgorithmSplitMostCountsAsdataarrives,ordertheleafboxesoftheSRPsothattheleafboxwiththemostnumberofpointswillbechosenforthenextbisection. Splittherootbox. z10 xrrrrrrrrrr ���z@@@zRL55 xLxR L ���zLL@@@zLRLR32 xLRxLL xRRxRLzz@@@RRLRRzz32 62/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation AdaptiveHistograms S.E.B.PriorityQueue–L1ConsistentInitialization APrioritizedQueuebasedAlgorithm(forL1ConsistentInitialization) AlgorithmSplitMostCountsAsdataarrives,ordertheleafboxesoftheSRPsothattheleafboxwiththemostnumberofpointswillbechosenforthenextbisection. Bisectuntileachboxhas knpoints(let kn=3here). z10 xrrrrrrrrrr ���z@@@zRL55 xLxR L ���zLL@@@zLRLR32 xLRxLL xRRxRLzz@@@RRLRRzz32 65/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation AdaptiveHistograms S.E.B.PriorityQueue–L1ConsistentInitialization TheSplitMostCountsAlgorithmInput:(i)data:x1;:::;xnRd;(ii)rootbox:x//optional;(iii)paddingtohandlepulseddata: 0//optional;(iv)S.E.B.max: kn;(v)maximumpartitionsize: mn.Output:histogramestimatefn;s.initialize i 1;s x+ ;repeatuntil #xv knforeachxv2`(s)andi mn//`(s)=fleafboxesgxv Uniform(^`(s))//randomizedPQonleafboxess bisect(s;xv)//bisectleafboxxvofsrecursivelyupdatecountsins;i i+1;return fn;s.67/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation AdaptiveHistograms S.E.B.PriorityQueue–L1ConsistentInitialization ProofSketchWewillassumethat kn!1,n�1 kn!0, mnn= kn,and mn=n!0,asn!1,andshowthatthethreeconditions:(a)n�1m(Ln)!0;(b)n�1logn(Ln)!0;and(c)(x:diam(x(x))� )!0withprobability1forevery �0;aresatised.ThenbyTheorem1ofLugosiandNobel,1996ourdensityestimatefn;_sisstronglyconsistentinL1.Theseconditionsmean:(a)sub-lineargrowthofthenumberofleafboxes(b)sub-exponentialgrowthofacombinatorialcomplexitymeasureofthegrowthofthepartition(c)shrinkingleafboxesinthepartition70/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation SmoothingbyAveraging AddingandAveragingSRPs AddingmSRPhistogramdensityestimatesmXi=1fn;s(i)=fn;s(1)+fn;s(2)+fn;s(3)+:::+fn;s(m)=fn;s(1)+fn;s(2)+fn;s(3)+:::+fn;s(m): AveragingmSRPhistogramdensityestimatesrecursivelyyieldsthesamplemeanSRPhistogram fn;m=1 mmXi=1fn;s(i): 75/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation PosteriorExpectationoverHistogramsinS0:1 PosteriorDistributionoverHistogramsinS0:1 ILet^fsbeahistogramwithpartition`(s)givenbytheleavesofRPswithksplitsandk+1leavesinSk IThenforthispartition,themostlikelyhistogramestimateis^fs(x;data)=1 n^fs(x;X1:n)=nXi=111(xi2x(x)) vol(x(x)) ILetthepriorprobabilitybeP(s)/1 C2k,s2S0:1 IThentheposteriordensityofhistogram^fswithksplitsisP(^fsjX1:n)/P(X1:njs)P(s)=Yxv2`(s)#xv nvol(xv)nxv1 C2k 77/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation PosteriorExpectationoverHistogramsinS0:1 PosteriorDistributionoverHistogramsinS0:1 ILet^fsbeahistogramwithpartition`(s)givenbytheleavesofRPswithksplitsandk+1leavesinSk IThenforthispartition,themostlikelyhistogramestimateis^fs(x;data)=1 n^fs(x;X1:n)=nXi=111(xi2x(x)) vol(x(x)) ILetthepriorprobabilitybeP(s)/1 C2k,s2S0:1 IThentheposteriordensityofhistogram^fswithksplitsisP(^fsjX1:n)/P(X1:njs)P(s)=Yxv2`(s)#xv nvol(xv)nxv1 C2k 79/113 StatisticalRegularPavingsinBayesianNonparametricDensityEstimation PosteriorExpectationoverHistogramsinS0:1 Metropolis-HastingsAlgorithmIUseaproposaldensityq(s0js(i))whichdependsoncurrentstates(i),togenerateanewproposedstates0IWeproposeuniformlyatrandomtosplitaleaformergeacherryofcurrentSRPstates(i)IRepeatIDrawuU(0;1)IIfuP(^fs0jX1:n) P(^fs(i)jX1:n)q(s(i)js0) q(s0js(i))thens(i+1) s0Ielses(i+1) s(i)IWitha“longenough”burn-intime,thisMarkovchainwillbeatthedesiredstationarydistributionP(^fsjX1:n)overS0:181/113