Perceptual Guidelines for Creating Rectangular Treemaps Nicholas Kong Jeffrey Heer and - PDF document

aslength[12,34].Thus,abarchartthatuseslengthencodingsmaybeamoreeffectivedisplayforquantitativedatathanatreemap.However,barchartsareinherentlylessspace-efcientthantreemapsbecausetheymustincludewhitespacearoundeachbar;incontrast,treemapsmaximizethenumberofdata-representativepixels.Treemapsalsodi-rectlyconveythehierarchicalstructureofthetreeusingcontainment.Despitethepopularityoftreemaps,thereremainsinsufcientguid-anceforchoosingdesignparametersorformanagingthetradeoffbe-tweenspace-efciencyandreadability.Priorworkontreemapshasprimarilyfocusedondevelopingnewlayoutalgorithmsthatprovidecontroloveraspectratio[6,9,26,28],shading[37],andborderde-sign[23].Beyondrectangles,researchershavealsodevelopedrecur-siveareasubdivisionsforVoronoiregions[4],circles[42]andjigsawshapes[40].Priorstudiesoftreemapeffectivenessexaminedtreemapsystemsthatenableinteractiveanalysisofdata[3,21,31,36]anddidnotexplicitlyexaminedesignparameters.Choosingthemosteffectivedesignparametersislargelylefttotheintuitionofthevisualizationde-signerorconstrainedbytheavailablelayoutalgorithms.Inthispaperweconductaseriesofcontrolledexperimentsaimedatproducingasetofdesignguidelinesforcreatingeffectiverectan-gulartreemaps.Werststudyhowtheluminanceandaspectratioofrectanglesaffecttheperceptionofarea.Ourstudiessuggestthatareacomparisonsareperceptuallyindependentofluminance,aswendnoevidencethatrectangleluminanceaffectsareajudgments.However,aspectratiodoesappeartoaffectareajudgments.Wendthattheaccuracyofareacomparisonssufferswhenthecomparedrectangleshaveextremeaspectratiosorwhentheyarebothsquares.Contrarytocommonassumptions,theoptimaldistributionofrectangleaspectratioswithinatreemapshouldincludenon-squaresbutshouldavoidextremeratios.Wethencomparetreemapswithhierarchicalbarchartdisplaystoidentifythedatadensitiesatwhichlength-encodedbarchartsbecomelesseffectivethanarea-encodedtreemaps.Weidentifythetransitionpointsatwhichtreemapsexhibitcomparisonjudgmentswithaccuracyonparwithbarchartsforbothleafandnon-leaftreenodes.Wealsondthatevenatrelativelylowdatadensities,treemapssupportfastercomparisonsthanbarcharts.Basedontheseresults,weconcludewithasetofguidelinesfortheeffectiveuseoftreemapsandsuggestalternateapproachesfortreemaplayout.2RELATEDWORKWerstreviewrelatedworkinthreeoverlappingareas:generalstud-iesofgraphicalperception,experimentsassessingtheeffectsofdata-density,andevaluationsoftreemapvisualizations.2.1GraphicalPerceptionAwealthofpriorresearchhasinvestigatedhowvisualencodingssuchaslength,area,color,andtextureaffectgraphcomprehension.Bertin[8]wasamongthersttoconsiderthisissue,claimingthat“re-semblance,order,andproportionarethethreesigniedsingraphics.”Atabasiclevel,visualizationshelpusidentifylikeandnon-likeel-ements,perceiverank-orderrelations,andcomparequantities.Boththeoreticalandempiricalevaluationsassesshowdifferentvisualen-codingtechniquesfacilitatethesebasicjudgments.Basedonhisexperienceasacartographer,Bertinproposedanor-deringofvisualencodingsforthreecommontypesofdata:nominal,ordinal,andquantitative.Hewrotethatspatialencodingsaresuperiortootherencodings,andthathueeffectivelyencodesnominal(categor-ical)databutnotquantitativedata.Cleveland&McGill[12]extendedBertin'sworkbyapplyingresultsfrompsychologytoprovideempiri-calgroundingfortheorderofvisualencodings.Theirhuman-subjectsexperimentsestablishedasignicantaccuracyadvantageforpositionjudgmentsoverbothlengthandanglejudgments.S.S.Stevens[33]modeledthemappingbetweenthephysicalin-tensityofastimulus(e.g.,ashape'slengthorarea)anditsperceivedintensityasapowerlaw:P=kIa,wherePistheperceivedintensity,Iisthephysicalintensity,kisanempiricallydeterminedscaleconstant,andaisthepowerlawexponent.Ifa�1,perceptiontendstowardoverestimation:e.g.,doublingtheweightofanobjectmakesitfeelmorethattwiceasheavy.Ifa1,perceptiontendstowardsunderes-timation:e.g.,anobjecttwiceaslargemaynotlookso.Ifa=1,thereisnosystematicbiastotheperceivedintensity.Stevensfoundthattheexponentsforlengthandareajudgmentswereroughlya1anda0:7,respectively.Thussubjectsper-ceivedlengthwithminimalbias,butunderestimateddifferencesinarea.Basedonthisnding,Cleveland&McGill[12]arguedthatlengthshouldbepreferredtoareawhenencodingquantitativevari-ables.Clevelandetal.[11]replicatedthendingthat,onaverage,peopleunderestimatearea;however,theyalsofoundwidevariationinper-subjectexponents,suggestingthatperceptualrescalingofareaen-codingsisunlikelytoprovidebenets.Whileoftenreportingdifferentexponentvalues,additionalresearchonthissubjectcontinuestondthatpeopleingeneralunderestimatearea[30,34].Morerecently,Heer&Bostock[17]investigatedhowtheshapeofanareaaffectsjudgmentaccuracy.Theyfoundthatareacomparisonsamongcirclesandrectangleshavesimilarjudgmentaccuracy,andthatbotharelessaccuratethanlengthjudgments,whichinturnarelessac-curatethanpositionjudgments.Theyalsofoundthatwhencomparingrectangleswithaspectratiosdrawnfromthesetf2 3,1,3 2g,acompari-sonoftwosquaresistheleastaccurate.Inthispaper,weextendthisanalysistoagreaterdiversityofaspectratios.2.2DataDensityResearchers(e.g.,[19,26,32])oftenpromotevisualizationtech-niquessuchastreemapsfortheir“space-lling”properties.Similarly,Tufte[35]advisesdesignerstomaximizedatadensity:thenumberofdatamarksperchartarea.However,onlyafewstudieshavechar-acterizedtheeffectivenessofvariousdata-densedisplays.Clevelandetal.[10]investigatedscaleeffectsoncorrelationperceptioninscat-terplotsbyvaryingaxisrangeswhileholdingdisplaysizeconstant.Woodruffetal.[44]presentedmethodsforpromotingconstantdatadensityinsemanticzoomingapplications,butdidnotpresentanem-piricalevaluation.Lametal.[22]studiedtheeffectsoflowandhighresolutiontime-seriesdisplaysonvisualcomparisonandsearchtasks.Theirlow-andhigh-resdisplaysuseddifferentvisualencodingvari-ables(colorvs.position),confoundingdatadensityandvisualen-codingtype.Finally,Heeretal.[18]evaluatedaformofdata-densetime-seriesdisplayknownashorizongraphs[14,25].Theysteadilydecreasedchartsizesuntiltheyidentiedthepointsatwhichhorizongraphsoutperformedstandardlinecharts.Inthispaper,wecharacter-izeasimilartrade-offbetweentreemapandbarchartdisplaysunderconditionsofincreasingdatadensity.2.3TreemapEvaluationShneiderman[26]introducedtreemapsasawaytovisualizelargehi-erarchicaldatasets(orcolloquially,“trees”).Sincethen,anumberofresearchershavecomparedtreemapstoothertreevisualizationtech-niques[3,21,31,36].Thesestudiesfoundthatwhentreemapswereusedinahierarchyexplorationtool,theyperformedaswellorbetterthanothercommontools,suchasdirectorybrowsers.However,theseexperimentsconsideredfullyinteractivesystemsthatincludedsearchandlteringcapabilities,makingitdifculttoseparatetheeffectoftheinteractionwidgetsfromthegraphicalperceptionofthecharts.Otherstudieshaveinvestigatedstatictreemapdisplays.Barlow&Neville[5]comparedrepresentationsofdecisiontrees,includingtreemaps,tooneanother.Theyfoundthattreemapsperformedpoorlywhenusedtocomparetwovaluesorexaminethestructureofthetree.Onefactorthatmayhavecontributedtothesedifcultiesisthattheyusedtreemapswithlargebordersbetweenlevels.Whileuserscouldclicktheborderstoselectbothleafandnon-leafnodes,thethicknessofthebordersfurtherreducedtheeffectivenessoftheareaencoding.Changingadesignparameter(borderthickness)thusentailedatrade-offbetweenenablinginteractionandimprovinggraphicalperception.Bedersonetal.[6]proposedorderedtreemaps,introducingacollec-tionoflayoutalgorithmsthatpreservetheorderingofdata.Theythencomparedorderedtreemapstoothertreemapalgorithmsusingmetricssuchastheaverageaspectratio.Forthreedifferenttreeswithvaluesdrawnfromalog-normaldistribution,theyfoundsquariedtreemaps Fig.4.Examplestimulifromtheaspectratiostudy.Rectanglesvariedintermsofbothpercentagedifferenceandaspectratio.5EXPERIMENT1:THEEFFECTSOFASPECTRATIOOurrstexperimentassessedtheeffectsofaspectratioonrectangularareajudgments.Acommoncriticismoftheoriginal“slice-and-dice”treemapalgorithm[26]isthatitproducesrectangleswithawidedis-tributionofaspectratios(e.g.,Figure1);intreeswithhighbranchingfactorsitcanproduceaspectratioswithmagnitudesof4andhigher.Suchextremeratiosmaycomplicateareacomparisons.Inresponse,researchersdeveloped“squaried”treemapalgorithms[9,39]thatat-tempttooptimizerectangleaspectratiostosquares.Brulsetal.[9]positedthreebenetsforsquariedlayouts:Squaresminimizerectangularperimeter,reducingborderink.Squaresareeasiertoselectwithamousecursor.Rectangleswithsimilaraspectratiosareeasiertocompare.Whiletherstassertionismathematicallytrueandthesecondas-sertionissupportedbyboththeoryandempiricalevidence(e.g.,Fitts'Law[24]),thevalidityofthethirdassertionislessclear.Theassump-tionthatsquareaspectratiosareoptimalisnotrootedinempiricalperceptiondata.ArecentexperimentbyHeer&Bostock[17]foundthatcomparingtwosquaresleadstosignicantlyhighererrorwhenvaryingaspectratiosamongthesetf2 3,1,3 2g,thoughthecauseofthiseffectremainsunclear.Inthisexperiment,wefurtherexaminedtheeffectsofaspectratioonproportionaljudgments.WereplicatedHeer&Bostock'sstudyde-sign,butincorporatedmoreextremeaspectratiosandalsoassessedtheeffectsoforientation(rotation)onjudgmentaccuracy.Basedonpriorresults,wehypothesizedthatbothextremeaspectratiosandsquareswouldhamperjudgmentaccuracy.Wealsohypothesizedthatjudg-mentsofrectangleswithdifferentprimaryorientations(horizontalorvertical)wouldresultinincreasederror.Thishypothesisisbasedinpartonpriorperceptualresearch[7]whichsuggeststhatmentalrota-tionismorecognitivelydemandingthaneithertranslationorscaling.5.1MethodWeaskedsubjectstocomparerectangularareasofvaryingsizeandas-pectratio.Weshowedsubjectsa600400pixelimagecontainingtwocenter-alignedrectanglesandinstructedthemtoidentifywhichoftherectangles(AorB)wasthesmallerandthenestimatethepercentagethesmallerwasofthelargerbymakinga“quickvisualjudgment.”Thestimulusimages(Fig.4)consistedoftworectangles,notafulltreemapdisplay.Heer&Bostock[17]testedbothstand-alonerectanglesandrectangleswithinatreemapwhilevaryingaspectratio.Astheyfoundnosignicantaccuracydifferencesbetweenthesetwostimulustypes,webelieveourresultsareapplicabletotreemapdisplays.Wecontrolledboththetruepercentagebetweenrectanglesandtheiraspectratios.Truepercentagesvariedover32%,48%,58%,72%.Toreduceaccuracydifferencesduetoresponsebias,thesevalueswereexplicitlyplacedatequal,symmetricdistancesfromthenearestmul-tipleof5.Wechosetheaspectratiosforeachpairofrectanglesfromthecrossproductofthesetf2 9,2 3,1,3 2,9 2gwithitself.TheseaspectratiosextendthesetusedbyHeer&Bostock[17]withmoreextremevalues.Sincenon-squareaspectratioshaveamatchingrotatedvariant(e.g.,arectanglewithratio2 3isa90rotationofonewithratio3 2),weincludedanadditionalreplicationofthe11conditiontoimprovestatisticalpower.Ourexperimentdesignthusconsistedof104uniquetrials(HITs):4(difference)26(aspectratiopairswithreplication). Fig.5.Experiment1areajudgmenterrorbytruepercentage.Errorbarsindicate95%condenceintervals.Asaqualicationtaskweusedmultiple-choiceversionsoftwoex-ampletrialstimuli.Foreachtrial,subjectsrstspeciedwhichrect-anglewassmallerandthenenteredtheirjudgmentofthenumericalpercentagethesmallerrectanglewasofthelarger.Werequested104HITswithN=25assignmentsandpaidarewardof$0.03perHIT.5.2ResultsWecollected10425=2,600responses,fromwhichweremoved18outliers(0.7%)withabsoluteerrorsabove35%.Toanalyzethedata,weusedlogabsoluteerror:log2(jjudgedpercent-truepercentj+1).WethenconductedanANOVAwitha462factorialdesign:(4)Truepercentage:oneof32%,48%,58%,or72%.(6)aspectratiopairs:anyrotatedvariantsaretreatedasthesameratio(e.g.,2 99 2,2 33 2),anddenotedbythegreatervalue.(2)relativeorientation:indicatesifthecomparedrectangleshaveidentical(9 29 2)ordifferent(9 22 9)orientations.5.2.1TruePercentageDominatesComparisonAccuracyWeagainfoundastrongeffectduetothetruepercentage(F(3,2173)=94.56,p0.001).AsshowninFigure5,theresultsexhibitasimilarproleasinourpilotstudy.Moreover,truedifferenceproducedthestrongesteffectinourmodel.Thisresultagainarguesfortheimpor-tanceofincludingtruedifferenceasacontrolledfactorinproportionaljudgmentstudies.ApplyingBonferroni-correctedpost-hoct-tests,wefoundthatalltruepercentageconditionsweresignicantlydifferent(p0.05)exceptfor58%and72%.Wefoundnosignicantinterac-tionsoftruepercentagewitheitherorientationoraspectratio.5.2.2OrientationAffectsExtremeAspectRatiosExaminingtheeffectsoforientationonjudgmentaccuracy,wefoundnomaineffect(F(1,1490)=0.669,p=0.414).Thisresultimpliesthat,onaverage,90rotationofrectangleshadlittletonoeffect.How-ever,wedidndasignicantinteractioneffectbetweenorientationandaspectratio(F(2,1490)=7.23,p0.001).Figure6showser-rorratebybothorientationandaspectratio.Whenorientationsdiffer,errorappearstoincreaseforcomparisonsinvolvingthemostextremeratiosinourstudy(9 29 2).Thisresultsuggeststhatrotationmaycon-tributetohigherjudgmenterrorsasaspectratiosdeviatefurtherfromsquares(e.g.,asoccursinslice-and-dicetreemaps[26]).5.2.3DiverseAspectRatiosImproveAccuracyFinally,weanalyzedtheimpactofaspectratioonjudgmentaccuracy,ndingasignicanteffect(F(5,2173)=13.85,p0.001).Applyingpost-hoct-testswithBonferronicorrection,wefoundthataspectratiopairsof9 29 2and11exhibitedsignicantlyhighererrorthanthepairs13 2,3 23 2and3 29 2.Similarly19 2wassignicantlymoreerrorpronethan3 29 2.Nootherdifferencesweresignicant.Fig-ure7showstheresultingrankorderingbyerrorofaspectratiopairsandtheircorrespondingcondenceintervals.Theresultsindicatethataveragejudgmentaccuracyimproveswhencomparingrectangleswithdiverseaspectratios,evenwhenoneoftheratiosislarge.Thehighesterroroccurredwhencomparingtwoextremeaspectratiosorcompar-ingsquares.ThelatterresultreplicatesHeer&Bostock's[17]ndingthatcomparingsquaresleadstoincreasederror. Fig.6.Judgmenterrorbyorientationandaspectratio.Squaresomittedduetoinvarianceunderrotation.Errorbarsshow95%conf.intervals. Fig.7.Areajudgmenterrorbyaspectratio.Squaresandextremeratioshavethehighesterror.Errorbarsindicate95%condenceintervals.5.3DiscussionOurexperimentfoundthatgraphicalperceptionsufferswhencompar-ingextremeaspectratios,particularlywhentherectangleshavedif-ferentorientations.Theseresultssupportthegeneralintuitionagainstusingtreemaplayoutalgorithmsthatproducerectangleswithextremeaspectratios(e.g.,slice-and-dice[26]).Ontheotherhand,subjectsexhibitedequallypooraccuracywhencomparingsquares.Asaresult,theperceptualjusticationforsquariedtreemaplayoutalgorithms[9,39]—thatsquarespromotemoreaccuratecomparisons—appearstobefaulty.Itinsteadseemsthatsquariedalgorithmsareeffectiveinpartbecause(a)theyavoidextremeaspectratiosand(b)inmostcasestheyareunabletoperfectlyachievetheir“squarication”objective,insteadproducingadistributionofaspectratios.6EXPERIMENT2:THEEFFECTSOFDATADENSITYAsthedatadensityofavisualizationincreases,themarksthatencodethedatamusteitheroverlapordecreaseinsize.Pastacertainpoint,suchoverlaporreductioninsizemakesitdifculttodistinguishin-dividualmarksandreadthevaluestheyencode.Thepointatwhichsuchdifcultiesoccurdependsonthespaceefciencyofthevisualen-coding.Forexample,anareaencoding,asinatreemap,makesmoreefcientuseofspacethanalengthencoding,asinabarchart.There-fore,wehypothesizedthatathighdensitiesatreemapwouldprovideamoreeffectivedisplaythanabarchart.Yet,areaislessperceptu-allyeffectivethanlengthforencodingquantitativevalues,aspeopletypicallyunderestimatearea[12,17].Thus,wehypothesizedthatatlowerdensitiesthebarchartwouldbemoreperceptuallyeffectivethanatreemap.Wedesignedoursecondexperimenttodeterminethedatadensityatwhichtreemapsbecomemoreeffectivethanbarchartsforcomparingquantitativevalues.Thisexperimentdidnotexaminehowwellpeoplecouldextractthestructureofthetree.Insteadwechosetofocusonvaluecomparisontasks,whichwebelievetobethemostcommonperceptualtaskperformedwithtreemaps.Nevertheless,oneadvantageoftreemapsisthat,unlikebarcharts,theydirectlyencodehierarchicalstructureviacontainment.Icicleplotsareatypeofvisualizationthatfallbetweentreemapsandbarchartsastheydirectlyencodehierarchybutusealengthencodingforvalue.However,Figure8showsthatasdatadensityincreasesandthe Fig.8.Datavisualizedasbotha(a)treemap,and(b)icicleplot.Notethatitisdifculttodistinguishleafvalueswithintheicicleplot.numberofnodesexceedsthehorizontalextentofthedisplay,theici-cleplotbecomesverydifculttoread.Therefore,toassesstrade-offsbetweenareaandlengthencodings,wedesignedahierarchicallayoutalgorithmwhichusesbarchartstoencodethevaluesofleafnodes.Figure9showsanexampleofourhierarchicalbarchartandatreemap,eachencodingthesamedata.Eachbarofthehierarchicalbarchartrepresentsaleafnodeinthetreeandthebarsaregroupedtogetherintoacellbasedontheirsiblingrelationships.Thevalueofaparentnodeisgivenbythesumofallbarswithinthecorrespond-ingparentcell.Thehierarchicalbarchartmakesmoreefcientuseofspacethananicicleplot,butatthecostofincreasingthecognitiveloadwhencomparingnon-leafvalues.Wehypothesizedthat,regardlessofdatadensity,treemapswouldbemoreeffectivethanhierarchicalbarchartsforvaluecomparisontasksthatinvolvenon-leafnodescontain-ingmultiplechildrenbecauseoftheadditionalcognitiveoverheadre-quiredtosumthebars.Ourhierarchicalbarchartsaredesignedtodepicttwo-leveltreeswhereallleavesareatthesecondlevel.Manyreal-worlddatasetstthisstructure,includingstockvaluesorganizedbysectororproductsorganizedbycategory.Infact,manytreemapvisualizationsofsuchtwo-leveldatamaybefoundontheweb[1,2,39].Sincethegoalofoursecondexperimentwastoexaminehowdatadensityaffectsvaluejudgmentsratherthanhowwellpeoplecanextractstructuralpropertiesofthetree,weworkedwithsuchtwo-levelbarcharts.Weleaveittofutureworktodesignhierarchicalbarchartsthatcanencodetreescontainingmorethantwolevels.Ourexperimentincludedthreetypesofcomparisons:leaftoleaf(LL),leaftonon-leaf(LN),andnon-leaftonon-leaf(NN).Inallthreecases,wepresentedparticipantswitheitheratreemaporahierarchicalbarchartandaskedthemtocomparetwoelements.Withatreemap,participantswereaskedtocomparetworectangularareas.Withahier-archicalbarchart,participantswereaskedeithertocomparetwobars(LL),ortocomparegroupsofbarstooneanother(LNorNN).Toex-tractthevalueofnon-leafnodessubjectshadtoaggregatethelengthsofallofthebarswithinacell.Asaresult,comparisonsinvolvingnon-leafnodesinthebarchartrequiredamorecomplicatedcognitiveprocessthansimplyjudgingthelengthofasinglebar.6.1MethodsForeachtrial,weshowedparticipantsachartwithtwohighlightednodes.Weaskedparticipantstoindicatewhichofthetwonodeswassmaller,andthenestimatethepercentagethesmallerwasofthelarger.Wetested2charttypes(treemapandhierarchicalbarchart),5datadensities(256,512,1024,2048,4096leaves),and3comparisontypes(LL,LN,NN).Basedontheresultsofourrstpilotstudywealsocontrolledfortruepercentagedifferencebetweenrectangles,using4levelsforeachcomparisoncondition.FortheLLandNNcomparisonsweusedthesamedifferencesasinExperiment1:32%,48%,58%,and72%.Aswewillexplaininthenextsubsection,thevalueassociatedwitharst-levelparentnodewasthesumofthevaluesassociatedwithitschildrennodes.Asaresult,rst-levelnodeswereusuallymuchlargerinvaluethanleaf-nodes.Thus,fortheLNcomparisonweusedthefollowingdifferences:3%,8%,13%,17%.Analysisofourresultsrevealedbarchartstobeasaccurateastreemapsathigherdensities,soweaddedthreemoredensitycondi- Fig.9.ExampleExp.2stimuliwith256leaves.(a)Squariedtreemap.(b)Hierarchicalbarchart;eachbarrepresentsaleafnodeandsiblingbarsaregroupedtogetherinacell.Thechartsdepictthesamedata.tions(6000,7000,and8000leaves)fortheLLcase.Ourexperimentdesignconsistedof432uniquetrials(HITs):fortheLNandNNcases,2(chart)5(density)2(LNorLNcomparisons)4(truepercent-ages)3(repetitions)=240HITs,andfortheLLcase,2(chart)8(density)4(truepercentages)3(repetitions)=192HITs.SimilartoExperiment1,ourqualicationtaskcontainedmultiple-choiceversionsofthetrials,oneforeachcharttype.Eachchartwassizedat600400pixels.WeimplementedtheexperimentusingMe-chanicalTurk's“externalquestion”option,allowingustohostthetrialsonourownserveranduseJavaScripttotrackresponsetimes(c.f.,[17]).Werequestedatotalof432HITswithN=20assignmentsandpaidarewardof$0.03perHIT.Figure9showsexampletrialstimuliforeachcharttype.Ourdataconsistedoftwo-leveltreeswherealltheleavesoccuratthesecondlevel;datagenerationdetailsaregivenintheappendix.Werenderedatreemapandahierarchicalbarchartoutofeachtree,ensuringthatsubjectssawexactlythesamedatainbothchartconditions.WecreatedtreemapsusingBrulsetal.'s[9]squariedtreemaplay-outalgorithm.Weusedborderthicknesstoencodetreedepth:theborder2pixelswidefornodesjustbelowtherootand1pixelwidefornodestwolevelsbelowtheroot.Ourdesigngoalsforthehierarchicalbarchartlayoutwereto:1)usespaceasefcientlyaspossibleand2)revealsomeofthehierarchi-calstructureofthedata,butto3)encodeallleafnodesusinglength.Figure10showsanexampletreevisualizedasahierarchicalbarchart.Eachcellofthebarchartrepresentsarst-levelnode.Eachbarinsideacellrepresentsasecondlevel,leafnode.Inthisexample,theroothasfourchildren,whichproducesfourcells.Weusedaregulargridtolayoutthecellsofthehierarchicalbarchartinordertoaidvisualcomparisonofbarsacrosscells.Inchoos-ingthenumberofrowsandcolumnsinthegrid,wesoughttomini-mizetheaspectratioofeachcellwhilestillallowingforenoughspacetolayoutallthebars.Werstcomputedthemaximumnumberofcolumnsbycomputingtheminimumcellwidth,whichisequaltothewidthofthewidestbarchart(i.e.,therst-levelnodewiththelargestnumberofchildren),assumingbarswere1-pixelwideandspacedby1-pixelgaps.Wethendividedthetotalwidthofthedisplayareabytheminimumcellwidthtoobtainthemaximumnumberofpossiblecolumns.Nextwecalculatedthenumberofrowsthatminimizedtheaspectratioofacell(makingeachcellassquareaspossible)giventhemaximumnumberofcolumns.Althoughthisalgorithmcouldresultinunusedcells,webelievethatminimizingcellaspectratiosinthiswaybetterfacilitatesperceptionthanpurelymaximizingspaceefciency.6.2ResultsWecollected43220=8,640responses,fromwhichweremoved389outliers(4:5%)withabsoluteerrorsabove70%orestimationtimesgreaterthan60seconds.Weweremoreconservativeineliminatingoutliersinthisexperimentthanintherstbecausethetaskwasmoredifcultandproducedgreatervariability.AsinExperiment1,wean-alyzedlogabsoluteerrors.Wefoundthatthedifferentcomparisontypes(LL,LNandNN)exhibiteddifferentdistributions,andsoweanalyzedeachseparately.Foreachcomparisontype(LL,NN,LN),weconductedaMANOVAofbothresponsetimeandlogabsoluteer-rorusinga2f5,8g4factorialdesign:(2)Charttype:Treemaporbarchart. Fig.10.Asampletreeshownas(a)anode-linktree,and(b)ahierar-chicalbarchart.Eachgroupofsiblingleavesformacellofbarcharts.(5or8)Datadensity:oneof256,512,1024,2048,or4096leafnodes(LN,NN),oroneof256,512,1024,2048,4096,6000,7000,or8000leafnodes(LL).(4)Truepercentage:oneof32%,48%,58%,72%(LL,NN),oroneof3%,8%,13%,or17%(LN).Figure11summarizestheestimationerrorandestimationtimedatawecollectedacrosstheentireexperiment.Notethatinthefollowingsubsectionsweomitdiscussionoftheeffectoftruepercentages,astheseresultsmatchthoseofourpriorexperiments.6.2.1Leaf-LeafComparison:TreemapsExcelatHighDensityTherstrowofFigure11showsestimationaccuracyandtimeforleaf-leaf(LL)comparisonsbycharttype.Wefoundasignicanteffectduetocharttypeonaccuracy(F(1,3586)=13.998,p0.001).Barchartsweremoreaccuratethantreemapsonaverage(mbar=3:38,mtreemap=3:65).Thisresultislargelyduetoperformancedifferencesatlowdatadensities,asbarchartsweresignicantlymoreaccurateatthe512,1024,and2046leafconditions(Fig.11).However,athigherdatadensities,errorsequalizedbetweenbarchartsandtreemaps.Wealsofoundasignicantmaineffectduetodensityonaccuracy(F(7,3586)=14.233,p0.001),asresponsesbecamelessaccurateasdensityincreased.Finally,wefoundasignicantinteractionbetweencharttypeanddensity(F(7,3586)=2.159,p=0.035);barchartaccuracydegradedmorerapidlythantreemapaccuracyasdensityincreased.Atdensitiesof4096leavesandhigher,wefoundnosignicantdifferenceinaccuracybetweenbarchartsandtreemaps.Wefoundamaineffectofcharttypeonestimationtimes(F(1,3586)=17.949,p0:001),andaninteractionbetweendensityandcharttype(F(7,3586)=7.323,p0:001).Treemapsandbarchartsledtocomparableestimationtimesupto2048leaves,butresponseswithtreemapsbecamesignicantlyfasteratthehigherdensities.Thedifferencewas5secondsinthe8000leafnodecondition.6.2.2TreemapsMoreAccurateForNon-LeafNodesThesecondandthirdrowsofFigure11showerrorratesbycharttypeanddatadensityfortheLNandNNcomparisons,respectively.Inbothcases,wefoundastrongmaineffectofcharttypeonaccuracy(F(1,2229)=21.189,p0.001forNN,F(1,2281)=68.535,p0.001forLN).IntheNNcondition,themeanlogerrorfortreemapswas3.24,comparedto3.62forthehierarchicalbarcharts.IntheLNcon-dition,themeanlogerroroftreemapswas2.14,comparedto2.61forthebarcharts.(NotethatthelowererrorsintheLNconditionaredueprimarilytotheunavoidablysmallertruepercentagedifferences.)WealsofoundasignicantinteractionbetweencharttypeanddensityintheLNcomparisontask(F(4,2281)=10.837,p0.001),butnottheNNcomparison.AsshowninthesecondandthirdrowsofFigure11,treemapsmaintainedtheiraccuracyasdatadensityincreased,whilebarchartstrendedtowardshighererrorrates.Treemapsweremoreac-curateatalldensitiesinNNcomparisons,andoutperformedbarchartsbeyond1024leavesinLNcomparisons.Lookingatestimationtimes,wedidnotndasignicanteffectofcharttypeforeitherNNorLNcomparisons,nordidwendaninterac-tioneffectbetweencharttypeanddensityineithernon-leafcondition. Fig.11.Estimationtimeanderrorforeachnodecomparisontype.Errorbarsindicate95%condenceintervals.(a)Leaf/leaf(LL)comparisons(rstrow).Barchartsaremoreaccuratethantreemapsuptoadensityof2,048leaves,afterwhichtreemapsbecomeequallyaccurate.At4,096leaves,treemapsbecomefasterthanbarcharts—upto5secondsfasterat8,000leaves.(b)Leaf/non-leaf(LN)comparisons(secondrow).Treemapsaremoreaccuratethanbarchartsatalldensities,butnofaster.(c)Non-leaf/non-leaf(NN)comparisons(thirdrow).AsinLNcomparisons,treemapsaremoreaccurate,butexhibitsimilarestimationtimes. 6.3DiscussionTheresultssupportourhypothesisthattreemapsaremoreaccurateforcomparisonsofnon-leafnodes.Ofcourse,thisndingisunsurpris-ing,asestimatingthevalueofnon-leafnodesinthebarchartdisplayinvolvesthecognitiveoverheadofcombiningbars.Moresurprisingly,treemapswerenotsignicantlyfasterthanbarchartsineitherNNorLNcomparisons.Weexpectedtreemapstobefasterinthesecasesastheydonothavethecognitiveoverheadofthebarcharts.Itispossiblethatparticipantsmayhavemadeaquickguessratherthantryingtoaddupthebars,tradingaccuracyforspeedgiventhedifculttask.Ourresultsdemonstratethatwhencomparingleafnodestheeffec-tivenessofbarchartsversustreemapsismodulatedbythedatadensity.Atlowdatadensities,barchartsresultinsignicantlymoreaccurateestimationswithoutasignicantdifferenceinresponsetime.Asdatadensityincreases,theaccuracydifferenceequalizes,withtreemapsmatchingbarchartsatdatadensitiespast4096leaves.AlthoughFig-ure11(upper-left)mayseemtoshowanaccuracyincreasebetween4096and6000leaves,notethatthedifferencesinerrorfor4096leavesandhigherarenotstatisticallysignicant.Treemapsresultinsignicantlybetterestimationtimes,particularlyathigherdensities:at8000leavestreemapsarealmost5secondsfaster.Onereasonforthisdifferencemaybethatathighdensities,individualbarsinthebarchartdisplayaresmallanddifculttond.Athigherdensities,ndingthehighlightednodesmaytakelongerinthebarchartdisplaythanintherelativelyspace-efcienttreemap.7DESIGNGUIDELINESBasedonourexperimentalresults,weofferthefollowingguidelinesforcreatingperceptuallyeffectiverectangulartreemaps.UseTreemapLayoutsthatAvoidExtremeAspectRatiosOurresultsshowthatgraphicalperceptionsufferswhencompar-ingsquaresorrectangleswithextremeaspectratios(9 29 2).Inaddition,wefoundthatdiverseorientationsadverselyaffectedtheperceptionofrectangleswiththelargestaspectratios.Basedonthesetwondingsweadvise—inkeepingwithcommonwisdom—thatsquariedtreemaplayoutsshouldbepreferredtoslice-and-dicelayouts.Althoughwefoundthatcomparisonofsquaresalsoreducesjudgmentaccuracy,theinabilityofsquariedlayoutstoachievetheiroptimizationobjectiveappearstoworkintheirfavor.However,thesendingssuggestalternativeapproachesthatmayimproveaccuracy.Forexample,futureworkmightassessjudgmentaccuracyinatreemaplayoutthatoptimizestowardsa3 2aspectratio.UseBarChartsatLowDensity,TreemapsatHighDensityBarchartsresultedinsignicantlylowererrorwhencomparingleafnodesatlowdensities.Ifadatasethasonlyafewhundredelements,barchartsaremoreeffectivethantreemaps.Asdatadensityincreases,treemapsbecomefasterthanbarchartswhileexhibitingequivalentaccuracy.Thetransitionpointwefoundwasat4096leaves,wheretreemapswerealmost3secondsfaster.At8000leaves,treemapswerealmost5secondsfaster.Ifadatasethasthousandsofelements,treemapsaremoreeffective.UseTreemapsWhenComparingNon-LeafNodesWefoundthattreemapsweremoreaccuratethanbarchartswhencomparingleafnodestonon-leafnodes(Figure11b)andcomparingtwonon-leafnodes(Figure11c).Therefore,weadviseusingtreemapsincasesrequiringcomparisonsamongnon-leafnodes.UseLuminanceToEncodeSecondaryValuesInTreemapsWefoundnoevidencethatareaperceptionwasaffectedbyrectangleluminance.Thisresultsuggeststhatdesignerscanuseluminancetoencodeanadditionalvariablewithoutaffectingjudgmentaccuracyorestimationtime.Weleavetestingtheconverse—whetherdifferingar-easbiasluminancejudgments—tofuturework.8LIMITATIONSANDFUTUREWORKWhilewedesignedourstudiestoprovidegeneralizableinsightsintotreemapdesign,inevitablyourexperimentshavelimitations.InExperiment1(theaspectratioexperiment),wetestedveaspectratios,rangingfrom2 9to9 2.However,sometreemaplayoutalgo-rithms,suchastheslice-and-dicealgorithm,canproducerectangleswithevenmoreextremeaspectratios.Furthermore,Experiment1wasperformedwithstand-alonerectangles,outofthecontextoftreemaps.WhileHeer&Bostock[17]foundthatplacingrectanglesinthecon-textofatreemapdidnotproduceasignicanteffectontheirresults,itmaybeworthinvestigatinghowevenmoreextremeaspectratiosaffectperformanceandwhethercontextaffectstheaccuracyofcomparisons.Ourresultssuggestthatsquariedtreemapsworkwellbecausetheyaimtominimizeaspectratiobutareunabletosquarifyallmarks.Moreover,theresultssuggestthatatreemapwhichoptimizesfor3 2(orothernon-extreme,non-unity)aspectratiosmayperformbetterthansquariedtreemaps.Wewouldliketoimplementsuchatreemaplay-outandcompareittosquariedtreemaps.InExperiment2(thedata-densityexperiment),ourdatagenerationalgorithmcreatedxed-depthtreeswithleavesoccurringonlyatthesecondlevel.Althoughthisstructureiscommoninmanyreal-worlddatasets,futurestudiescouldconsidermoregeneraltreestructureswithdeeperhierarchiesandleavesatdifferentlevelsofthetree.De-signingahierarchicalbarchartforsuchgeneraltreestructuresremainsanopenproblem.Moreover,ourstudiesdidnotaskusersquestionsaboutthestructureofthedata.Insteadwefocusedonvaluecompar-isontasksbecausewewereprimarilyinterestedinhowdatadensityaffectedtheperceptionofareaorlength.Extendingthestudytoin-cludestructuralquestionsisanopendirectionforfuturework.Wefoundtreemapsweremuchfasterthanbarchartswhencompar-ingleafnodesatdatadensitiesbeyond4096leavesinour600400stimulusimages.Onepossiblereasonisthatitcanbeverydifculttondthestimuliwhentheyaresmall,particularlyinthebarchartdis-play.Futurestudiesusingeye-trackingmaybeabletoseparatevisualsearchtimefromestimationtime.Ourguidelinesarebasedonstudiesthatusedxed-size600400pixelstimulusimages.BecauseweusedMTurkasourexperimentalplatformandcannotcontrolthephysicalscreensizeofoursubjects,wearemeasuringdatadensityasthenumberofmarksperpixelarea.Futureworkmightcharacterizeareajudgmentsintermsofphysicalmeasuressuchasnumberofmarkspercm2orperopticalsteradian.Replicatingthesestudiesinthelaboratorywillhelpensureourcrowd-sourcedresultsareexternallyvalid[17].Anopenquestioniswhethervaryingphysicaldisplaysizewhilekeepingthenumberofelementsxedwouldproduceresultssimilartothoseweobtainedthroughon-linecrowdsourcing.Wehaveonlyscratchedthesurfaceofinvestigatingtheconse-quencesofdesignchoicesoftreemaps.Otherchoicesincludetheborderthicknessbetweennodes(sometimesusedtoplacelabels),andalternatelayoutalgorithmsthatenforceorderingofthedata[6],ensurevisibilityofthehierarchy[23],orusenon-rectangularshapes[4,40].Futurestudiesmightfurtherextendourunderstandingoftheseandothervisualizationdesignparameters.APPENDIX:DATAGENERATIONWegeneratedtwo-leveltreesforourexperimentusingarandomizedprocess.Werstaddedaxednumberofchildrentoanimplicitrootnode,dependingonthedatadensity(i.e.,totalnumberofleaves).Forexample,therootnodehad12childreninthe256leafcondition(3rowsx4columns),56childreninthe512leafcondition(7x8),and30childreninthe1024leafcondition(10x3).Eachrst-levelchildoftherootdenesacellinourhierarchicalbarchartandwexedthenumberofrst-levelnodesinordertocontrolthenumberofcells.Toeachrst-levelnodewethenrandomlyaddedbetween2and16childrenrepresentingleafnodes.Finally,weaddedadditionalleafnodestorandomlyselectedrst-levelnodesuntilweachievedtherequireddatadensity.Treescreatedwiththismethodhavetwolevelsbelowtheroot