Package bandit February Type Package Title Functions for simple AB split test and multiarmed bandit analysis Version - PDF document

Download presentation
Package bandit February   Type Package Title Functions for simple AB split test and multiarmed bandit analysis Version
Package bandit February   Type Package Title Functions for simple AB split test and multiarmed bandit analysis Version

Embed / Share - Package bandit February Type Package Title Functions for simple AB split test and multiarmed bandit analysis Version


Presentation on theme: "Package bandit February Type Package Title Functions for simple AB split test and multiarmed bandit analysis Version"— Presentation transcript


2best_binomial_bandit bandit-packageFunctionsforsimpleA/Bsplittestandmulti-armedbanditanalysis DescriptionAsetoffunctionsfordoinganalysisofA/Bsplittestdataandwebmetricsingeneral.DetailsPackage:banditType:PackageTitle:FunctionsforsimpleA/Bsplittestandmulti-armedbanditanalysisVersion:0.5.0Date:2014-05-03Imports:boot,gam�(=1.09)Author:ThomasLotzeandMarkusLoecherMaintainer:ThomasLotze&#xthom; slo;&#xtze@;&#xthom; slo;&#xtze.; om0;License:GPL-3Author(s)ThomasLotzeandMarkusLoecher best_binomial_banditbest_binomial_bandit DescriptionComputetheBayesianprobabilitiesforeacharmbeingthebestbinomialbandit.Usagebest_binomial_bandit(x,n,alpha=1,beta=1)Argumentsxasinprop.test,avectorofthenumberofsuccessesnasinprop.test,avectorofthenumberoftrialsalphashapeparameteralphaforthepriorbetadistribution.betashapeparameterbetaforthepriorbetadistribution. 4best_binomial_bandit_simUsagebest_binomial_bandit_sim(x,n,alpha=1,beta=1,ndraws=5000)Argumentsxasinprop.test,avectorofthenumberofsuccessesnasinprop.test,avectorofthenumberoftrialsalphashapeparameteralphaforthepriorbetadistribution.betashapeparameterbetaforthepriorbetadistribution.ndrawsnumberofrandomdrawsfromtheposteriorValueavectorofprobabilitiesforeacharmbeingthebestbinomialbandit;thiscanbeusedforfuturerandomizedallocationAuthor(s)ThomasLotzeandMarkusLoecherReferencesStevenL.Scott,AmodernBayesianlookatthemulti-armedbandit,Appl.StochasticModelsBus.Ind.2010;26:639-658.(http://www.economics.uci.edu/~ivan/asmb.874.pdf)SeeAlsoprop.testExamplesx=c(10,20,30,33)n=c(100,102,120,130)best_binomial_bandit_sim(x,n,ndraws=1000)round(best_binomial_bandit(x,n),3)best_binomial_bandit_sim(c(2,20),c(100,1000))best_binomial_bandit_sim(c(2,20),c(100,1000),alpha=2,beta=5)#quicklookatthevariousshapesofthebetadistributionaswechangetheshapeparams:AlphaBeta=cbind(alpha=c(0.5,5,1,2,2),beta=c(0.5,1,3,2,5))M=nrow(AlphaBeta)y=matrix(0,100,ncol=M)x=seq(0,1,length=100)for(iin1:M)y[,i]=dbeta(x,AlphaBeta[i,1],AlphaBeta[i,2])matplot(x,y,type="l",ylim=c(0,3.5),lty=1,lwd=2) distribution_estimate7}else{print("Nosignificanttime-basedfactor")} distribution_estimatesummarize_metrics DescriptionAconveniencefunctiontoperformoverallmetricanalysis:mean,median,CI.Usagedistribution_estimate(v,successes=NULL,num_quantiles=101,observed=FALSE)Argumentsvavectorofvaluestobeanalyzed(fornonbinarydata),ornumberoftrials(forbinarydata)successesnumberofsuccesses(forbinarydata)num_quantilesnumberofquantilestosplitintoobservedwhethertogeneratetheobserveddistribution(ratherthantheestimateddistri-butionofthemean);defaultFALSEValueadataframewiththefollowingcolumns:quantilestheestimatedquantiles(0,0.01,0.02,...,1)forthemean,usingaBeta-binomialestimateofpforbinomialdata,abootstrappedquantiledistributionforreal-valuednumbersxxvaluesforplottingalineplotoftheestimateddistributionyyvaluesforplottingalineplotoftheestimateddistributionmidsmidvaluesforplottingabarplotoftheestimateddistributionleftsleftvaluesforplottingabarplotoftheestimateddistributionrightsrightvaluesforplottingabarplotoftheestimateddistributionwidthswidthvaluesforplottingabarplotoftheestimateddistributionheightsheightvaluesforplottingabarplotoftheestimateddistributionprobabilitiesprobabilitiesindicatinghowmuchprobabilityiscontainedineachbarplotAuthor(s)ThomasLotze&#xthom; slo;&#xtze@;&#xthom; slo;&#xtze.; om0; signicance_analysis9 significance_analysissignicance_analysis DescriptionAconveniencefunctiontoperformoverallproportioncomparisonusingprop.test,beforedoingpairwisecomparisons,toseewhatoutcomesseemtobebetterthanothers.Usagesignificance_analysis(x,n)Argumentsxasinprop.test,avectorofthenumberofsuccessesnasinprop.test,avectorofthenumberoftrialsValueadataframewiththefollowingcolumns:successesxtotalsnestimated_proportionx/nlower0.95condenceintervalontheestimatedamountbywhichthisalternativeout-performsthenext-loweralternativeupper0.95condenceintervalontheestimatedamountbywhichthisalternativeout-performsthenext-loweralternativesignificancep-valueforthetestthatthisalternativeoutperformsthenext-loweralternativeorderorder,byhighestsuccessproportionbest1ifitispartofthe'highestperforminggroup'–thosegroupswhichwerenotsignicantlydifferentfromthebestgroupp_bestBayesianposteriorprobabilitythatthisalternativeisthebestbinomialbanditNoteThisisintendedforuseinA/Bsplittesting–sosizesofnshouldberoughlyequal.Also,notethatalternativeswhichhavethesamerankaregroupedtogetherforanalysiswiththe'next-lower'alternative,soyoumaywanttochecktoseeifranksareequal.Author(s)ThomasLotze&#xthom; slo;&#xtze@;&#xthom; slo;&#xtze.; om0; summarize_metrics11Examplesx=c(10,20,30,50)n=c(100,102,120,130)sim_post(x,n) summarize_metricssummarize_metrics DescriptionAconveniencefunctiontoperformoverallmetricanalysis:mean,median,CI.Usagesummarize_metrics(v,successes=NULL)Argumentsvavectorofvaluestobeanalyzed(fornonbinarydata),ornumberoftrials(forbinarydata)successesnumberofsuccesses(forbinarydata)Valuealistwiththefollowingitems:meanmeanmedianmedianlower0.95condenceintervalonthemeanupper0.95condenceintervalonthemeannum_obsnumberofobservationsofthismetrictotalthesumofallvaluesofthismetric(mean*num_obs)Author(s)ThomasLotze&#xthom; slo;&#xtze@;&#xthom; slo;&#xtze.; om0;Examplesmetric_list=list(rbinom(n=100,size=1,prob=0.5),rbinom(n=100,size=1,prob=0.7),rpois(n=100,lambda=5))summarize_metrics(length(metric_list[[1]]),sum(metric_list[[1]]))summarize_metrics(length(metric_list[[2]]),sum(metric_list[[2]]))summarize_metrics(metric_list[[3]]) IndexTopicdesignbest_binomial_bandit,2best_poisson_bandit,5significance_analysis,9Topichtestbest_binomial_bandit,2best_poisson_bandit,5deseasonalized_trend,6distribution_estimate,7significance_analysis,9summarize_metrics,11Topicpackagebandit-package,2bandit(bandit-package),2bandit-package,2bbb(best_binomial_bandit),2best_binomial_bandit,2best_binomial_bandit_sim,3best_poisson_bandit,5bpb(best_poisson_bandit),5deseasonalized_trend,6distribution_estimate,7prob_winner,8prop.test,3–5,10significance_analysis,9sim_post,10summarize_metrics,11value_remaining,1213

By: cheryl-pisano
Views: 184
Type: Public

Package bandit February Type Package Title Functions for simple AB split test and multiarmed bandit analysis Version - Description


50 Date 20140503 Imports boot gam 109 Author Thomas Lotze and Markus Loecher Maintainer Thomas Lotze Description A set of functions for doing analysis of AB split test data and web metrics in general License GPL3 NeedsCompilation no Repository CRAN ID: 37626 Download Pdf

Related Documents