Calculateweightedbalancestatistics - PDF document

2bal.stat bal.stat Calculateweightedbalancestatistics Description bal.statcomparesthetreatmentandcontrolsubjectsbymeans,standarddeviations,effectsize,andKSstatisticsUsage bal.stat(data,vars=NULL,treat.var,w.all,get.means=TRUE,get.ks=TRUE,na.action="level")Arguments data adataframecontainingthedata vars avectorofcharacterstringswiththenamesofthevariablesonwhichthefunc-tionwillassessthebalance treat.var thenameofthetreatmentvariable w.all observationweights(e.g.propensityscoreweights,samplingweights,orboth) get.means logical.IfTRUEthenbal.statwillcomputemeansandvariances get.ks logical.IfTRUEthenbal.statwillcomputeKSstatistics na.action acharacterstringindicatinghowbal.statshouldhandlemissingvalues.Currentoptionsare"level","exclude",or"lowest"Details bal.statcalls ps.summary.f and ps.summary.n foreachvariableandassemblesthere-sultsinatableValue See ps.summary fordetailsonthereturnedobject.get.meansandget.ksmanipulatetheinclusionofcertaincolumnsinthereturnedresult.SeeAlso Theexamplefor ps containsanexampleoftheuseofbal.table bal.table3 bal.table Computebalancetable Description Extractthebalancetablefrom ps and dx.wts objectsUsage bal.table(x)Arguments x a ps or dx.wts objectDetails bal.tableisagenericfunctionforextractingbalancetablesfrom ps and dx.wts objects.Theseobjectsusuallyhaveseveralsetsofcandidateweights,oneforanunweightedanalysisandperhapsseveral stop.methods .bal.tablewillreturnatableforeachsetofweightscom-binedintoalist.Eachlistcomponentwillbenamedasgiveninthex,usuallythenameofthestop.method.Thebalancetablelabeled“unw”indicatestheunweightedanalysis.Value Returnsadataframecontainingthebalanceinformation. tx.mn Themeanofthetreatmentgroup tx.sd Thestandarddeviationofthetreatmentgroup ct.mn Themeanofthecontrolgroup ct.sd Thestandarddeviationofthecontrolgroup std.eff.sz Thestandardizedeffectsize,(tx.mn-ct.mn)/tx.sd.Iftx.sdissmallor0,thestandardizedeffectsizecanbelargeorINF.Thereforestandardizedeffectsizesgreaterthan500aresettoNA stat thet-statisticfornumericvariablesandthechi-squarestatisticforcontinuousvariables p thep-valueforthetestassociatedwithstat ks theKSstatistic ks.pval theKSp-valuecomputedusingtheanalyticapproximation,whichdoesnotnec-essarilyworkwellwithalotoftiesSeeAlso Theexamplefor ps containsanexampleoftheuseofbal.table 4desc.wts check.err Reportsonerrorsandwarnings Description Reportsonerrorsandwarningsencounteredwhilerunning ps and desc.wts .ThisfunctionisnotintendedfortheusertocalldirectlyUsage check.err(cov.table,stage,alerts.stack)Arguments cov.table abalancetable,intendedtobetheresultscomponentofthelistthat bal.stat returns stage atitleforthemethod“type"usedtocreatetheweights,usedtolabeltheresults alerts.stack anobjectforcollectingwarningsissuedduringtheanalysesDetails Checksfortreatmentstandarddeviationsthatareexceedinglysmallorzeroandforeffectsizesthatareunusuallylarge,bothindicativeofnumericalproblemsorextremesampleimbalanceValue check.errreturnsnoobjectsbutdoesalterthealerts.stackobject desc.wts Diagnosisofweights Description desc.wtsassessesthequalityofasetofweightsonbalancingatreatmentandcontrolgroup.Usage desc.wts(data,w,vars=NULL,treat.var,tp,na.action="level",perm.test.iters=0,verbose=TRUE,alerts.stack) diag.plot5 Arguments data adataframecontainingthedataset w avectorofweightsequaltonrow(data) vars avectorofvariablenamescorrespondingtodata treat.var thenameofthetreatmentvariable tp atitleforthemethod“type"usedtocreatetheweights,usedtolabeltheresults na.action astringindicatingthemethodforhandlingmissingdata perm.test.iters annon-negativeintegergivingthenumberofiterationsofthepermutationtestfortheKSstatistic.Ifperm.test.iters=0thenthefunctionreturnsananalyticapproximationtothep-value.Thisargumentisignoredisxisapsobject.Settingperm.test.iters=200willyieldprecisiontowithin3%ifthetruep-valueis0.05.Useperm.test.iters=500tobewithin2% verbose ifTRUE,lotsofinformationwillbeprintedtomonitorthetheprogressofthetting alerts.stack anobjectforcollectingwarningsissuedduringtheanalysesDetails desc.wtscalls bal.stat toassesscovariatebalance.If�perm.test.iters0itwillcall bal.stat multipletimestocomputeMonteCarlop-valuesfortheKSstatisticsandthemaximumKSstatistic.Itassemblestheresultsintoalistobject,whichusuallybecomesthedesccomponentofpsobjectsthat ps returns.Value Seethedescriptionofthedesccomponentofthepsobjectthat ps returnsSeeAlso ps diag.plot Creatediagnosticplots Description Createsdiagnosticplotsofpropensityscoresincluding,aside-by-sideboxplotofpropensityscores,ahistogramofpropensityscoreweights,QQplotsofKSandt-statisticp-values,andaplotofabsoluteeffectsizesUsage diag.plot(title=NULL,treat=NULL,p.s=NULL,w.ctrl=NULL,desc.unw=NULL,desc.w=NULL, dx.wts7 dx.wts Propensityscorediagnostics Description dx.wtstakesapsobjectorasetofpropensityscoresandcomputesdiagnosticsassessingcovari-atesbalance.Usage dx.wts(x,data,vars=NULL,treat.var,x.as.weights=TRUE,sampw=NULL,perm.test.iters=0,plots=TRUE,title)Arguments x adataframe,matrix,orvectorofpropensityscoreweightsorapsobject.xcanalsobeadataframe,matrix,orvectorofpropensityscoresifx.as.weights=FALSE data adataframe vars avectorofcharacterstringsnamingvariablesindataonwhichtoassessbal-ance treat.var acharacterstringindicatingwhichvariableindatacontainsthe0/1treatmentgroupindicator x.as.weights TRUEorFALSEindicatingwhetherxspeciespropensityscoreweightsorpropensityscores.Ignoredifxisapsobject sampw optionalsamplingweights.Ifxisapsobjectthenthesamplingweightsshouldhavebeenpassedto ps andnotspeciedhere.dx.wtswillissueawarningifxisapsobjectandsampwisalsospecied perm.test.iters annon-negativeintegergivingthenumberofiterationsofthepermutationtestfortheKSstatistic.Ifperm.test.iters=0thenthefunctionreturnsananalyticapproximationtothep-value.Thisargumentisignoredisxisapsobject.Settingperm.test.iters=200willyieldprecisiontowithin3%ifthetruep-valueis0.05.Useperm.test.iters=500tobewithin2% plots ifplots=TRUEthendx.wtswillcall diag.plot generatingdiagnosticplots title ashorttexttitle,itwillbeusedinplotsandsavedles.BydefaultthisissettothecurrentdateandtimeDetails Createsabalancetablethatcomparesunweightedandweightedmeansandstandarddeviations,computeseffectsizes,andKSstatisticstoassesstheabilityofthepropensityscorestobalancethetreatmentandcontrolgroups. 8egsingle Value Returnsalistcontaining treat thevectorof0/1treatmentassignmentindicators desc anestedlistcontainingdetaileddiagnosticinformationontheweights.Thisin-cludesthenumberoftreatmentandcontrolsubjects,theeffectivesamplesize,thelargestKSstatistic,theaverageabsoluteeffectsize,andthecompletebal-ancetable summary.tab adataframeshowingbalanceinformation ps thegivenpropensityscores w thegivenweights datestamp thedateandtimeofthecalltodx.wts parameters theparametersusedwhencallingdx.wts alerts textcontaininganywarningsaccumulatedduringtheestimationSeeAlso Theexamplefor ps containsanexampleoftheuseofdx.wts, diag.plot egsingle USSustainingEffectsstudy Description AsubsetofthemathematicsscoresfromtheU.S.SustainingEffectsStudy.Thesubsetconsistsofinformationon1721studentsfrom60schools.ThisdatasetisavailableinthemlmRevpackage.Usage data(egsingle)Format Adataframewith7230observationsonthefollowing12variables. schoolid afactorofschoolidentiers childid afactorofstudentidentiers year anumericvectorindicatingtheyearofthetest grade anumericvectorindicatingthestudent'sgrade math anumericvectoroftestscoresontheIRTscalescoremetric retained afactorwithlevels01indicatingifthestudenthasbeenretainedinagrade. female afactorwithlevelsFemaleMale black afactorwithlevels01indicatingifthestudentisBlack hispanic afactorwithlevels01indicatingifthestudentisHispanic size anumericvectorindicatingthenumberofstudentsenrolledintheschool lowinc anumericvectorgivingthepercentageoflow-incomestudentsintheschool mobility anumericvector 10ks.stat ks.stat Functionsforevaluatingbalance Description Theseareacollectionoffunctionsthatcanbeusedascomponentsof stop.methods forevalu-atingthebalanceoftwogroupsUsage ks.stat(logw=NULL,w.ctrl=NULL,gbm1=NULL,i=1,data,sampw=rep(1,nrow(data)),rule.summary=mean,na.action="level",vars,treat.var,collapse.by.var=FALSE,verbose=FALSE)es.stat(logw=NULL,w.ctrl=NULL,gbm1=NULL,i=1,data,sampw=rep(1,nrow(data)),rule.summary=mean,na.action="level",vars,treat.var,collapse.by.var=FALSE,verbose=FALSE)strata.stat(logw=NULL,w.ctrl=NULL,gbm1=NULL,i=1,data,sampw=rep(1,nrow(data)),rule.summary=mean,na.action="level",vars,treat.var,collapse.by.var=FALSE,verbose=FALSE)Arguments Theweightsbepassedtothesefunctionswithanyoftherstthreearguments logw thelogarithmoftheweights w.ctrl theweightsforthecontrolsubjects gbm1 a gbm.object usedforestimatingthepropensityscores,usuallythegbmcomponentofapsobjectreturnedfrom ps i theiterationof gbm withwhichtocomputetheweights data adataframewiththedata sampw optionalsamplingweights rule.summary afunctionforsummarizingthetotalbalance.Usedtocollapsestatisticsacrossallthecovariates.Examplesincludemeanandmax na.action astringindicatingthemethodforhandlingmissingdata vars avectorofvariablenamescorrespondingtodata treat.var thenameofthetreatmentvariable collapse.by.var ifTRUE,thenstatisticscomputedforfactorsarecollapsedacrossthelevels 12print.dxwts Source http://www.columbia.edu/rd247/nswdata.htmlhttp://cran.r-project.org/src/contrib/Descriptions/MatchIt.htmlReferences Lalonde,R.(1986).Evaluatingtheeconometricevaluationsoftrainingprogramswithexperimentaldata.AmericanEconomicReview76:604-620.Dehejia,R.H.andWahba,S.(1999).CausalEffectsinNonexperimentalStudies:Re-EvaluatingtheEvaluationofTrainingPrograms.JournaloftheAmericanStatisticalAssociation94:1053-1062. metric.i Lossesfor*.statfunctions Description Rearrangestheargumentsofthe*.statfunctionssothattheymaybepassedto optimize Usage metric.i(i,fun=ks.stat,...)Arguments i thenumberof gbm iterations fun avalid*.statfunction ... otherargumentstobepassedtofunValue EvaluatesfunatiSeeAlso ks.stat , es.stat , strata.stat print.dxwts Printadiagnosisoftheweights Description Printsadiagnosisoftheweights.Extractssummary.tabfromthe dx.wts objectUsage print.dxwts(x,...)Arguments x a dx.wts object ... furtherargumentspassedtoorfromothermethods 14ps perm.test.iters anon-negativeintegergivingthenumberofiterationsofthepermutationtestfortheKSstatistic.Ifperm.test.iters=0thenthefunctionreturnsananalyticapproximationtothep-value.Settingperm.test.iters=200willyieldprecisiontowithin3%ifthetruep-valueis0.05.Useperm.test.iters=500tobewithin2% print.level theamountofdetailtoprinttothescreen iterlim maximumnumberofiterationsforthedirectoptimization verbose ifTRUE,lotsofinformationwillbeprintedtomonitorthetheprogressofthettingDetails formulashouldbesomethinglike"treatmentX1+X2+X3".Thetreatmentvariableshouldbea0/1indicator.Thereisnoneedtospecifyinteractiontermsintheformula.interaction.depthcontrolsthelevelofinteractionstoallowinthepropensityscoremodel.Ifpdf.plots=TRUEthenpscausesplotstobesavedasasinglepdflewiththename"[ti-tle].pdf"intheworkingdirectory.See diag.plot fordetailsoftheplots.Value Returnsanobjectofclassps,alistcontaining gbm.obj Thereturned gbm object ps adataframecontainingtheestimatedpropensityscores.Eachcolumnisassoci-atedwithoneofthemethodsselectedinstop.methods w adataframecontainingthepropensityscoreweights.Eachcolumnisassociatedwithoneofthemethodsselectedinstop.methods.Ifsamplingweightsweregiventhentheseareincorporatedintotheseweights plot.info alistcontainingtherawdatausedtogeneratetheplots desc alistcontainingbalancetablesforeachmethodselectedinstop.methods.Includesacomponentfortheunweightedanalysisnames“unw”.Eachdesccomponentincludesalistwiththefollowingcomponents ess Theeffectivesamplesizeofthecontrolgroup n.treat Thenumberofsubjectsinthetreatmentgroup n.ctrl Thenumberofsubjectsinthecontrolgroup max.es Thelargesteffectsizeacrossthecovariates mean.es Themeanabsoluteeffectsize max.ks ThelargestKSstatisticacrossthecovariates mean.ks TheaverageKSstatisticacrossthecovariates bal.tab a(potentiallylarge)tablesummarizingthequalityoftheweightsforequalizingthedistributionoffeaturesacrossthetwogroups.Thistableisbestextractedusingthe bal.table method.Seethehelpfor bal.table fordetailsonthetable'scontents n.trees Theestimatedoptimalnumberof gbm iterationstooptimizethelossfunctionfortheassociated stop.methods datestamp Recordsthedateoftheanalysis parameters Savesthepscall alerts Textcontaininganywarningsaccumulatedduringtheestimation ps15 Author(s) GregRidgewayhgregr@rand.orgi,DanMcCaffreyhdanielm@rand.orgi,AndrewMorralhmorral@rand.orgiReferences DanMcCaffrey,G.Ridgeway,AndrewMorral(2004).“PropensityScoreEstimationwithBoostedRegressionforEvaluatingAdolescentSubstanceAbuseTreatment,”PsychologicalMethods9(4):403-425.SeeAlso gbm Examples data(lalonde)print(nrow(lalonde))ps.lalondeps(treat~age+educ+black+hispan+nodegree+married+re74+re75,data=lalonde,title="Lalondeexample",stop.method=stop.methods[c("ks.stat.mean","ks.stat.max")],#generateplots?plots="all",pdf.plots=FALSE,#gbmoptionsn.trees=2000,interaction.depth=3,shrinkage=0.005,perm.test.iters=0,verbose=TRUE)#getthebalancetablesbal.table(ps.lalonde)#diagnosetheweightsusingapsobjectadx.wts(ps.lalonde,data=lalonde,treat.var="treat")print(a)bal.table(a)#diagnosetheweightsaspropensityscoreweights#willbethesameasbefore,exceptforMCvariationintheKSp-values#whenperm.test.itersisgreaterthan0wwith(ps.lalonde,ps/(1-ps))w[lalonde$treat==1,]1dx.wts(w,data=lalonde,treat.var="treat",perm.test.iters=0)#diagnosetheweightsaspropensityscorespps.lalonde$psdx.wts(p,data=lalonde,treat.var="treat",x.as.weights=FALSE)#lookatpropensityscoresnames(ps.lalonde$ps)hist(ps.lalonde$ps$ks.stat.max) ps.summary17 get.means ifTRUE,meancomparisonsarecomputed get.ks ifTRUE,theKSstatisticsarecomputed na.action astringindicatingthemethodforhandlingmissingdata collapse.by.var ifTRUE,thenstatisticscomputedforfactorsarecollapsedacrossthelevelsDetails ps.summarydispatchesps.summary.norps.summary.fdependingonwhetherxisanu-mericvectororafactor.Value Returnsadataframecontainingthebalanceinformation. tx.mn Themeanofthetreatmentgroup tx.sd Thestandarddeviationofthetreatmentgroup ct.mn Themeanofthecontrolgroup ct.sd Thestandarddeviationofthecontrolgroup std.eff.sz Thestandardizedeffectsize,(tx.mn-ct.mn)/tx.sd stat thet-statisticfornumericvariablesandthechi-squarestatisticforcontinuousvariables p thep-valueforthetestassociatedwithstat ks theKSstatistic ks.pval theKSp-valuecomputedusingtheanalyticapproximation,whichdoesnotnec-essarilyworkwellwithalotoftiesget.meansandget.ksmanipulatetheinclusionofcertaincolumnsinthereturnedresult.SeeAlso bal.stat , ks.stat , es.stat Examples treatrbinom(100,1,0.5)wrexp(100)#categoricaldatax.catfactor(sample(letters[1:3],size=100,replace=TRUE))ps.summary.f(x.cat,treat,w)#numericdatax.numrnorm(100)ps.summary.n(x.num,treat,w)#orletps.summaryfigureoutwhichtocallps.summary(x.num,treat,w) sensitivity19 sensitivity Sensitivityanalysis Description Producesatabletohelptheuserassesstheextenttowhichahiddenbiasmightremoveanydiffer-encesobservedinthepropensityscoreanalysis.Usage sensitivity(ps1,data,outcome,order.by.importance=TRUE,verbose=TRUE)Arguments ps1 apsobjectasreturnedfrom ps data thedataframeusedtotps1 outcome acharacterstringindicatingthenameofthevariableindatatouseastheoutcome order.by.importance ifTRUEthenthevariablesaresortedbytheirrelativeinuenceinthe gbm.object usedtocreateps1 verbose ifTRUE,lotsofinformationwillbeprintedtomonitorthetheprogressofthettingDetails ThisfunctionimplementsthesensitivityanalysisdescribedinRidgeway(2006),Section5.5.Thisanalysishelpstheuserassesstheextenttowhichahiddenbiasmightremoveanydifferencesob-servedinthepropensityscoreanalysis.Ifthereisanimportantunobservedfactortheoddsthanthecorrectpropensityscoreweightisnotw(xi),asthepropensityscoremodelpredicts,butactuallyw(xi;zi)wherezrepresentstheunobservedfactor.Letai=w(xi;zi)=w(xi).Theseai'sgiveanestimateofg(a),thedistributionofthemultiplicativeerrorsthatweobserveintheweightswhenexcludingzi.Changingthevaluesoftheai'swillaffectthetreatmenteffectestimateifaiscorrelatedwithy,theoutcome.Thestrongerthecorrelationthemoresensitivetheresultswillbetothehiddenbias.sensitivitycomputesovercontrolgroupsubjectsamodiedestimateofE(Y0jt=1).PCaiwiyi PCaiwisubjecttotheconstraintthataig(a)andcor(ai;yi)=.Severalg(a)'sareconsideredbyremovingeachvariablefromthepropensityscoremodelinturnandcomputingtheratiooftheoriginalweightstotheweightswiththevariableremoved.Severalchoicesforarealsoconsidered,makingaslargeaspossible,assmallaspossible,andsolvingforthe“breakeven”,thethateliminatesanytreatmenteffect. 20stop.methods Value Returnsalistwhereeachcomponentcontainsthesensitivityanalysisforeachstop.methodusedinttingps1.Eachcomponentcontainsadataframewitharowforeachvariableintheoriginalpropensityscoremodel.Thecolumnsare var thenameofthevariableexcludedfromthemodel E0 theestimatedE(Y0jt=1)withvarexcludedfromthepropensityscoremodel a.min,a.max thesmallestandlargestvaluesofaobserved a.cor theobservedcorrelationbetweenaandy a.mincor,a.maxcor thesmallestandlargestvaluesofpossible minE0,maxE0 thesmallestandlargestvaluesofestimatedE(Y0jt=1)possible breakeven.cor thebreakevencorrelation(seeDetailssection)Author(s) GregRidgewayhgregr@rand.orgiReferences G.Ridgeway(2006).“Assessingtheeffectofracebiasinpost-trafcstopoutcomesusingpropen-sityscores,”JournalofQuantitativeCriminology22(1):1-29.SeeAlso See ps foranexample stop.methods Rulesforselectingthepropensityscores Description Alistofstop.methodobjectsbuiltintothetwangpackagethatencoderulesforselectingpropensityscoreweightsDetails The ps functionusesastop.methodobjectforinstructionsonhowtoselectthepropensityscoreweights.twanghassomestop.methodobjectsbuiltinbuttheusermayimplementtheirowniftheywishandpassthemto ps forittooptimize.Avalidstop.methodobjectisalistthatdenesthefollowing metric afunctionthatevaluatesthesimilarityofthedistributionofavariableacrossthetreatmentandcontrolgroups.Currently,thetwangpackagehasfunctions es.stat , ks.stat ,and strata.stat .Theusermayimplementtheirown. rule.summary afunctionthattakesthevectorofresultsfromthemetricfunctionandsumma-rizesthemintoasinglenumber.twangcurrentlyutilizes mean and max forrule.summary direct logical.IfTRUEthen ps willtrytooptimizetheweightsdirectlyratherthanutilizing gbm twang-package21 na.action acharacterstringindicatinghowbal.statshouldhandlemissingvalues.Currentoptionsare"level","exclude",or"lowest" name acharacterstring,preferablyuniquefromotherstop.methodsforlabelingtheresultingweightsInaddition,theobjectmusthaveclass(mystopmethod)=="stop.method" summary.ps Summarizeapsobject Description ComputessummaryinformationaboutastoredpsobjectUsage ##S3methodforclassps:summary(object,...)Arguments object a ps object ... additionalargumentsaffectingthesummaryproducedDetails Compressestheinformationinthedesccomponentofthepsobjectintoashortsummarytabledescribingthesizeofthedatasetandthequalityofthepropensityscoreweights.Value See ps fordetailsonthereturnedtableSeeAlso ps twang-package ToolkitforWeightingandAnalysisofNonequivalentGroups Description Thispackageoffersfunctionsforpropensityscoreestimatingandweighting,nonresponseweight-ing,anddiagnosisoftheweightsDetailsPackage:twangVersion:0.6-7Date:2006-5-17Depends:R�(=2.2),gbm�(=1.5-3),surveyLicense:GPL(version2ornewer)Built:R2.2.1;;2006-02-1510:35:20;windows Index Topicdatasetsegsingle, 7 lalonde, 10 raceprofiling, 17 Topicerrorcheck.err, 3 Topichplotdiag.plot, 5 Topicmodelsbal.table, 2 desc.wts, 4 dx.wts, 6 ks.stat, 9 metric.i, 11 ps, 12 ps.summary, 15 sensitivity, 18 summary.ps, 20 Topicmultivariatebal.stat, 1 ps, 12 stop.methods, 20 Topicpackagetwang-package, 21 Topicprintprint.dxwts, 12 Topicutilitiesget.weights, 8 bal.stat, 1 ,3,4,17bal.table, 2 ,14check.err, 3 desc.wts,3, 4 ,5diag.plot, 5 ,5,7,13dx.wts,2, 6 ,12egsingle, 7 es.stat,11,17,20es.stat(ks.stat), 9 gbm,10–14,20,22gbm.object,10,18get.weights, 8 ks.stat, 9 ,11,17,20lalonde, 10 max,20mean,20metric.i, 11 optimize,11plot.dxwts(diag.plot), 5 plot.ps(diag.plot), 5 print.dxwts, 12 ps,2–7,9,10, 12 ,12,18–21ps.summary,2, 15 ps.summary.f,2ps.summary.n,2raceprofiling, 17 sensitivity, 18 stop.methods,2,9,10,13,14, 20 strata.stat,11,20strata.stat(ks.stat), 9 summary.ps, 20 twang(twang-package), 21 twang-package, 21 23