Thewithinsubjectsdesignhasseveraladvantagesitrequiresfewerparticipantsvarianceduetoparticipantspredispositionswillbeapproximatelythesameacrosstestconditionsitisnotnecessarytobalancegroupsofpartici ID: 248781
Download Pdf The PPT/PDF document "Week5Lecture1:Ordere ects,Counterbalanci..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Week5Lecture1:Ordere ects,CounterbalancingandLatinSquaresInstructor:EaktaJainCIS6930,ResearchMethodsforHuman-centeredComputingScribe:Chris(Yunhao)Wan,1677-3116UniversityofFlorida,Spring2015 Thewithin-subjectsdesignhasseveraladvantages:itrequiresfewerparticipants;varianceduetoparticipantspredispositionswillbeapproximatelythesameacrosstestconditions;itisnotnecessarytobalancegroupsofparticipants.Becauseoftheseadvantages,experimentsinHCItendtofavorwithin-subjectsdesignsoverbetween-subjectsdesigns[Mac12].1OrderE ectsWhenthetestconditionsareassignedwithin-subjects,participantsaretestedwithonecondition,thenanothercondition,andsoon.Insuchadesign,participants'performancewillchangeduetoseveralreasons:Practice.Inmostwithin-subjectsdesigns,itispossiblethatparticipants'performancewillimproveastheyprogressfromonetestconditiontothenext.Throughpracticefromoneconditiontoanothercondition,theybecomefamiliarwiththeapparatusandprocedure,andtheyarelearningtodothetaskmoree ectively.Fatigue.Insomecases,itisalsopossiblethattheparticipants'performancewillbecomeworseonconditionsthatfollowotherconditions.Thismaybecauseofmentalorphysicalfatigue.OrderE ect.Theparticipants'performancemayeitherimprovedorworsenduetotheorderofthetestconditions.Thegoaloftheexperimentistoevaluatetheperformancethroughthetestconditions.Theseconfoundingin uencewillgreatlya ecttheaccuracyoftheexperimentsresults.Whenthere'sproblems,therealwaysbeasolution.Themostcommonsolutionfortheordere ectistodivideallparticipantsintogroupsandarrangethetestconditionsinadi erentorderforeachgroup.Thisiscalledcounterbalancing,whichisalsothetopicfornextsection.2CounterbalancingandLatinSquareLet'sstartwithanexample:Inadesigntherearetwolevels(testconditions)AandB.ThetotalnumberofparticipantsisN.Ifwedivideallparticipantsintotwogroups,eachgrouphasN 2participants,andthedesignwilllookslikethefollowingtable:1 Group TestLevelSequence 1 AB2 BAFurthermore,ifweincreasebothtestlevelandgroupnumber,e.g.howtodealwiththedesignof3groups?2.1LatinSquareLet'scontinuewiththeexamplefor3testlevels.Ifweequallydividethetheparticipantsintothreegroups,theneachgroupwillhaveN 3participants.Andwewilldistributethetestconditionsequenceasfollowingtable:Group TestLevelSequence 1 ABC2 BCA3 CABThroughfurtherinspectionfromtheprevioustwocases,wecanndoutthateachtestconditionappearedonceineverypossiblepositioninthesequence.e.g.intheexampleofthreetestconditions,conditionAisintherstplaceingroupone;inthirdplaceofgroup2andinsecondplaceofgroup3.Furthermore,matrixlikethistypecanbeexpandandgenaralizedtoannmatrix,andinexperimentaldesign,ithasaformalname:LatinSquares,whichwasinspiredbymathematicalpapersbyLeonhardEuler,whousedLatincharactersassymbols.[WG11]nn 12...n-1n23...n1...............n-1n...n-3n-2n1...n-2n-1AsimplealgorithmtogeneratetheLatinSquaretalkedpreviouslyistousecircular.ThedetailsonhowtoconstructsuchanLatinSquareandtheproofwastalkedin[Bra58].2.2BalancedLatinSquareTheLatinSquareprovidesanequallyappearanceforeachconditionineachpossibleposition.however,thereisstillanordere ectexists.Recalltheexampleofthe33matrix:BfollowsAtwice,butAfollowsBonlyonce.That'snotacoincidence.ActuallyadeciencyinLatinsquaresoforder3andhigheristhatconditionsprecedeandfollowotherconditionsanunequalnumberoftimes.SoanA-Bsequencee ectisnotfullycompensatedfor.Weintroduceonesolutiontosuchdeciency,whichiscalled:BalancedLatinSquares.Ifwereconstructthe44squareasthefollowingmatrix:2 Group TestConditionSequence 1 ABDC2 BCAD3 CDBA4 DACBIntheabovebalancedLatinSquare,theA-Bsequencee ectasbeensolved.IfwewanttoconstructannbalancedLatinSquare,wecanfollowthematrixbelow12n3n-14...2314n5...342516........................n-1nn-21n-32...n1n-12n-23...Beforeusetheabovesquaretoconstructthebalancedlatinsquares,onethingmustbenoticed:balancedlatinsquarescanonlybegeneratedforevennumberconditions.Andforanconditiondesign,thenumberofparticipantsneededisakn,whichkisapositiveinteger.3ExampleAttheend,let'sgothroughadetailexampleforwhatwe'vetalkedaboutpreviously.Inanexperiment,thereare3algorithmstobetested:A.Videoframecodedwithaverageoptical ow;B.Videoframecodedwithaveragesoundlevel;C.don'tcodethevideo.Andtherearealso3tasks:1.BirthdayParty;2.GroupSkiing;3.BasketballGame.Theexperimentwasdesignedandaveragedataforeachcondition(averagevaluefor3tasks/condition)isshownasfollowing: Participant TestCondition Group Mean A B C 1 12.5 16.9 12.2 1 1A 2 14.4 16.3 14.1 3 ... ... ... 2 2A 4 5 ... ... ... 3 3A 6 mean A B C Ifwepresenttheresultsinachart:Inthisexample,ifthevaluesof1A,2Aand3Aareallcloseenough.Thuswecaninferthereisnoordere ectinthisdesignonconditionA.ThisconclusioncanalsoapplytoconditionBandC.3 References[Bra58]JamesVBradley.Completecounterbalancingofimmediatesequentiale ectsinalatinsquaredesign.JournaloftheAmericanStatisticalAssociation,53(282):525{528,1958.[Mac12]IScottMacKenzie.Human-computerinteraction:Anempiricalresearchperspective.Newnes,2012.[WG11]WalterDWallisandJohnGeorge.Introductiontocombinatorics.CRCPress,2011.4