AddresscorrespondencetoMathiasFleckDukeUniversityLSRCBldgBox90999DurhamNC27708emailmathias ID: 148642
Download Pdf The PPT/PDF document "ResearchReportRareTargetsAreRarelyMissed..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
ResearchReportRareTargetsAreRarelyMissedinCorrectableSearchMathiasS.FleckandStephenR.MitroffDepartmentofPsychologyandNeuroscience,CenterforCognitiveNeuroscience,DukeUniversityABSTRACTFailingtoÞndatumorinanx-rayscanoraguninanairportbaggagescreeningcanhavedireconse-quences,makingitfundamentallyimportanttoelucidatethemechanismsthathinderperformanceinsuchvisualsearches.Recentlaboratoryworkhasindicatedthatlow AddresscorrespondencetoMathiasFleck,DukeUniversity,LSRCBldg.,Box90999,Durham,NC27708,e-mail:mathias.ßeck@duke. PSYCHOLOGICALSCIENCEVolume18ÑNumber11Copyright2007AssociationforPsychologicalScience Collections(HemeraPhotoObjects,Gatineau,Quebec,Cana-da).TheybelongedtoÞvecategories:toys,fruitsandvegetables,clothing,birds,andtools.Eachobjectwasconvertedtograyscaleandpartiallyblurred,thenpresentedwitharandomro-tationinanonoverlappingarrayonawhitebackground.ThearrayofpossiblelocationswasspeciÞedbyaninvisible5grid(subtending19.1atanapproximateviewingdis-tanceof60cm),andeachitem(subtending3.23.2,onaverage)wasplacedwithslightspatialjitterwithinarandomlyselectedcell,withthecentercellexcluded.Ontarget-presenttrials,oneoftheitemswasrandomlyselectedfromthetoolcategory(e.g.,hammer,wrench,clamp,saw,drill,axe),andtheremainingitemsweredrawnrandomly,withoutreplacement,fromtheotherfourcategories.Ontarget-absenttrials,allitemsweredrawnfromthenontoolcategories.Observerssearchedthedisplayforatoolforaslongastheydesired,terminatingthetrialbypressingeithertheÔÔ/ÕÕkeytoindicatetargetpresenceortheÔÔZÕÕkeytoindicatetargetab-sence.Observerswereencouragedtotreattheexperimentastheymightanairportsecuritytask:Itwasimportanttokeepthetrialsprogressing,butalsoimperativethatnoÔÔdangerousÕÕitems(i.e.,tools)weremissed.Uponresponse,thedisplaydisap-peared,andthenexttrialappearedautomaticallyaftera0.5-sdelay.Halfoftheobservers(correctioncondition)weregiventheopportunitytocorrecttheirresponsetotheprevioustrial;theywereinstructedtopresstheÔÔEscÕÕkeyduringatrialiftheirresponseontheprevioustrialshouldbereversed.Theotherhalfoftheobservershadnosuchoption(no-correctioncondition).UnlikeinthestudybyWolfeetal.(2005),nofeedbackwasgivenforaresponse,norwasthereanyfeedbackprovidedafterthecorrectionkeywaspressed.Observersweretoldinadvancethatcorrectionswouldberecordedandthatafteracorrection,theyshouldrespondtothenexttrialnormally.Theexperimentconsistedof1,400trialsdividedintothreeblocksdeÞnedbythefrequencywithwhichtargetswerepresent.Thehigh-prevalenceblockconsistedof200trials,50%ofwhichweretarget-presenttrials.Themedium-prevalenceblockalsoconsistedof200trials,butinthiscaseatargetwaspresenton10%oftrials.Thelow-prevalenceblockconsistedof1,000trials,2%ofwhichcontainedatarget.Observerswerewarnedthattargetfrequencywouldgenerallybeverylow,andthattheyshouldresistanytendencytofallintoanautomaticÔÔtarget-absentÕÕresponsemode.Halfoftheobserversineachconditionviewedtheblocksinthefollowingorder:highprevalence,me-diumprevalence,lowprevalence;theotherhalfviewedtheblocksinthereverseorder.Therewasnosystematiceffectoforderineithercondition,andallanalysesreportedherearecollapsedoverorder.Afterevery200trials,theprogrampromptedobserverstotakeabreak,andtheexperimentcon-tinuedwhenabuttonwaspressed.Eachsetof200trialswasprecededbyanon-screenindicationofthetargetprevalence(high,medium,orlow)intheupcomingset.Observerswerestronglyencouragedtotakeadvantageofthebreaks,particu-larlyiftheywerefeelingtiredorbored.Theentireexperimentranapproximately80mininlength,dependingonthespeedoftheobserver.RESULTSANDDISCUSSIONResultsfortheno-correctionconditionreplicatedthepreva-lenceeffectsofWolfeetal.(2005).Missrateswere10%,19%,and31%forthehigh-,medium-,andlow-prevalenceblocks,respectively,(2,18)632(seeFig.2).Incontrast,thecorrectionconditionshowednoeffectofprevalence,withmissratesof4%,10%,and10%forthehigh-,medium-,andlow-prevalenceblocks,(2,18)1.618,152.Amixed-effectsanalysisofvariancerevealedasigniÞcantinteractionbetweenprevalenceandcondition,208,indicatingthatthepreva-lence-linkedincreaseinmissesoccurredspeciÞcallywhenobserverscouldnotcorrecttheirmistakes.Further,themissratesinthecorrectionconditioncalculatedincorporatingthecorrectionresponses(8%,19%,and27%,respectively)werestatisticallyequivalenttothemissratesintheno-correc-tioncondition:Amixed-effectsanalysisofvarianceusingthesemissratesrevealednointeractionbetweenprevalenceand(2,36)0.186,010,againhighlightingthespeciÞcimpactofallowingobserverstheop-portunitytocatchtheirownmistakes.Averagefalsealarmratesontarget-absenttrialswereverylowforallblocksinbothconditions;thehigh-,medium-,andlow-prevalenceblocksintheno-correctionconditionyieldedfalsealarmratesof0.70%,0.22%,and0.06%,respectively,andthecorrectionconditionproducedratesof0.80%,0.00%,and Fig.1.Samplesearcharray.Observerssearchedforatoolamidran-domlyrotateditemsfromothercategories.Volume18ÑNumber11 CorrectableSearch 0.03%,respectively.Thecorrectionkeywasusedalmostex-clusivelytocorrectmisses(94.4%ofallcorrections).Observerswerefreetorespondattheirownpace,andtheresponsetimedataarehighlyinformative.Figure3showstheaverageresponsetimepatternsintheno-correctionconditionfortrialsbeforeandaftertarget-presenttrialsinthelow-prev-alenceblock,plottedseparatelyforhitsandmisses.Responsetimesfortrialsleadinguptomisseswereonaverage231msfasterthanresponsetimesfortrialsprecedinghits,agreeingwiththepatternobservedbyWolfeetal.(2005)andsupportingthenotionthatincreasedspeedleadstomisses(Chun&Wolfe,1996;Rabbitt,1966).Theresponsetimedatathusseemtoshowadirectrelationbetweenaccuracyandspeedforthisvisualsearchtask.Discussingthispatternofresults,Wolfeetal.(2005)proposedthatmissesseemtooccurbecauseÔÔobserversabandontheirsearchinlessthantheaveragetimerequiredtoÞndatargetÕÕ(p.439).Weproposeinsteadthatasearchmaybeabandonedinlessthantheaveragetimerequiredtorespondtoatarget,buteitherway,bothsetsofdatastronglysuggestthatfasterspeedsmayberesponsiblefortheincreaseinmissesatlowtargetprevalence.However,thenotionofaspeed-accuracytrade-offinvisualsearchforraretargetshasrecentlybeenchallenged.Newdata(Wolfeetal.,inpress)suggestthatwhenobserversaregivenÔÔspeedingticketsÕÕonfasttrialstoinduceslowerrespondingoverall,missratesremainrelativelyhigh.Ifthereisadirectlinkbetweenresponsetimeanderrors,onewouldexpectimprovedaccuracyinthiscondition.However,thereareafewpossibleexplanationsforthisdiscrepancy.First,trialdurationinthespeeding-ticketexperimentwasstillyokedtoresponsespeed,ratherthanbeingÞxed,andthismayhavelimitedanydelay-drivenbeneÞtsbyaddingasecondtask.Becauseobservershadtomonitortheirresponsespeedtoavoidpenalty,theymighthavebeenjudgingdurationwhilesimultaneouslytryingtocompletethesearch.Second,providingdifferentialfeedbackforspeciÞcdurations(punishmentafterveryfastresponsesandnothingafterslowerresponses)couldencourageobserverstoadoptthestrategyofdelayinginitiationoftheirsearch(andthustheirresponse)soastoavoidpenalty.Suchaprocessofrescalingaresponseruleinreferencetotemporalregularitiesissimilartomechanismsformalizedininformationprocessingmodelsofintervaltiming(seeMacDonald&Meck,2004,forareview).Finally,itisentirelypossiblethattheinducedslowdownsimplydidnotprovideenoughtimetoovercometheprepotentÔÔtarget-absentÕÕresponse.Althoughthesenewresultsareintriguing,futureworkwillbeneededtoreconcilethemwiththecurrentresultsandanaccumulatingbodyofdata(e.g.,Chun&Wolfe,1996;Wolfeetal.,2005,inpress)thathasconsistentlyrevealedarelationbetweenfasterresponsesandlowerdetectionrates.AnotherinterestingpatternisevidentintheresponsetimedatainFigure3:Observersintheno-correctionconditionwereonaverage161msslowertorespondontrialsimmediately Fig.2.Missrateasafunctionoftargetprevalence.Thegraybarsshowresultsfortheno-correctionandcorrectionconditionsinthepresentexperiment;resultsfromWolfe,Horowitz,andKennerÕs(2005)study,inwhichobserversdidnothavetheopportunitytocorrecterrors,arereproducedhereforcomparison.Notethatlow-prevalencetargetsappearedon1%(ratherthan2%)oftrialsinthestudybyWolfeetal.ThedashedlinesindicatethemissratesinthecorrectionconditionbeforeobserversÕcorrectionswereincorporated.ErrorbarsrepresentVolume18ÑNumber11 MathiasS.FleckandStephenR.Mitroff followingamissedraretargetthanontrialsimmediatelypre-cedingthetargetÑaneffectsimilartothatfoundbyWolfeetal.(2005)eventhoughinthepresentcase,critically,therewasnofeedbackprovided.Thisslowdown,likethedataontheaccuracyofcorrectionsinthecorrectioncondition,stronglysuggeststhatobserverswerecognizantoftheirmistakeonsomemisstrials(Rabbitt,1966)andthatprocessingmaybesimilarformissesandcorrectresponses(Egeth&Smith,1967).Thus,thesemissesappeartobeactionerrorsratherthanperceptualerrors.ItshouldbenotedthatourmissrateandresponsetimedatafullyreplicatethoseofWolfeetal.(2005),whichshouldalleviateconcernsaboutmethodologicaldifferences(e.g.,thepresenceorabsenceoffeedback).Whengiventheopportunity,observersreadilycorrecttheirmisses,eliminatingtheeffectoftargetprevalenceinvisualsearch.TheseÞndingsindicatethatthehighmissratesintheno-correctionconditionarosefromexecutionerrors;thatis,ob-serversinfactnoticedthesetargetsbutrespondedtooquickly.WhethersuchlaterecognitionisdrivenbyalingeringsensoryrepresentationofthedisplayorreßectsaninabilitytoinhibittherepetitiveandprepotentÔÔtarget-absentÕÕresponse,itisclearthattheriseinerrorsassociatedwithlowprevalenceislargelydrivenbyadeÞcitinresponseexecution,ratherthanbyamoregeneralperceptualfailureoftargetidentiÞcationorsearch.Becauseaprimaryaimofthepresentworkwastorelatevisualsearchre-sultstosociallyimportantsituations,thisredeÞnitionofthelow-prevalenceeffectiscriticalanddemandsacomparisonbetweentheresponseparametersoflaboratorytasksandthoseofradio-logicalandairportscreenings.Inradiology,imagereaderstypicallyspend30to90sonanx-rayscanandassessfewerthan100imagesinaday,asharpcontrasttotheconditionsofthepresentstudy,whichhad1,400trialsandaverageresponsetimesoflessthan3s.Giventhesedifferences,missesinthismedicalcontextarenotlikelytobeduetotheincreaseinmissescausedbyrapidrespondingintaskswithlowtargetfrequency,and,indeed,arecentcompre-hensiveradiologicalstudy(Gur,Rockette,ArmÞeld,etal.,2003)reportednosigniÞcanteffectsoftargetprevalenceonimagereadersÕaccuracyindetectingdisease.Althoughtargetfrequencyislowinthiscontext,thehighincidenceoferrorislikelyexplainedbyothermechanisms,includinginterpretationdeÞcits(Manning,Ethell,&Donovan,2004),ÔÔsatisfactionofsearchÕÕissues(Samuel,Kundel,Nodine,&Toto,1995;Wolfeetal.,2005),andincompletevisualscanpatterns(Kundel,No-dine,&Carmody,1978).Incontrasttoradiologicalscreening,airportbaggagescreeninghasrelativelyfastresponsetimes(averageinspectiontimesare3Ð5s;Schwaninger,Hardmeier,&Hofer,2005),andthenumberofbagsscreenedinasinglesessioncanbequiteextensive.However,adirectlinkbetweenbaggagescreeningandourtaskistenuousgiventhedifferencesinresponsepa-rameters,stimuli,andmotivation.Nevertheless,ourresultsunderscorethenecessityofbeingabletoimmediatelycorrecterrors(e.g.,rewindthebaggageconveyorbeltormoreclosely Fig.3.Responsetimeontarget-absenttrialsintheno-correctionconditionasafunctionofordinalrelationtoatarget-presenttrial.Negativenumbersrepresentsuccessivetrialsprecedingthetarget(T),andpositivenumbersrepresentsubsequenttrials.Resultsareshownseparatelyfortrialssurroundinghitsandtrialssurroundingmisses.Thedashedlinesindicatethechangebetweentrialsimmediatelybeforeandaftertarget-presenttrials.Errorbarsindicatestandarderrorsofthemeans.Volume18ÑNumber11 CorrectableSearch examineindividualimages)inanyfastsearchforlow-preva-lencetargets.Moregenerally,ourresultssuggesttheneedtofocuslessontheeffectofprevalenceandmoreonotherissuesthathavebeenshowntodrivehigherrorratesinairportsearches,includingbagcomplexity,nonprototypicalviewsofprohibiteditems,andoverlappingx-rayimages(Schwaningeretal.,2005),aswellasobserver-speciÞcfactorssuchastheabilitytogeneralizerecognitiontrainingtoadiversesetofpossiblethreatitems(McCarley,Kramer,Wickens,Vidoni,&Boot,2004).Thereremainsthepossibilitythatlowprevalencemayinter-actwithotherfactorstoincreaseerrorratesinamanneryetunrevealed.Althoughourdatademonstratenosuchinteraction,theydohighlighttheneedtominimizeoreliminatemotorerrorswhenlookingforanyinßuenceofprevalence.Moreover,es-tablishingthatthereisnoprevalenceeffectincorrectablesearchescouldinfactfacilitatethestudyofmissesinsearchesthatmaytypicallyinvolveraretargets(Gur,Rockette,Warfel,Lacomis,&Fuhrman,2003;Obuchowski,2005):Thelargenumberoftrialsneededtoimplementrare-targetsearchesinthelaboratorycanbeextremelycumbersome,andthelackofaprevalenceeffectindicatesthatitmightbesafetoinßatethenumberoftarget-presenttrialstobetterexplorethemechanismsunderlyinghighmissrates.Insum,prevalencedoesnotinßuencetheerrorrateincor-rectablesearches.Theoptiontocorrectmistakesparsesoutresponse-executionerrors,thuseliminatingtheriseinmissratespreviouslyfoundinsearchforraretargets.Ultimately,improvingreal-worldsearchperformancewillbeservedbestbyseparatelyaddressingerrorsofactionanderrorsofperception.AcknowledgmentsÑWethankGeorgeAlvarez,GregAppel-baum,JasonArita,JessicaCantlon,IanDobbins,ChrisMac-Donald,GregStabler,NickTurk-Browne,JeremyWolfe,andtwoanonymousreviewersforassistanceandcomments.Berlin,L.(1994).ReportingtheÔÔmissedÕÕradiologicdiagnosis:Med-icolegalandethicalconsiderations.,183Ð187.Brainard,D.H.(1997).ThePsychophysicsToolbox.SpatialVisionChun,M.M.,&Wolfe,J.M.(1996).Justsayno:Howarevisualsearchesterminatedwhenthereisnotargetpresent?,39Ð78.Egeth,H.,&Smith,E.E.(1967).Onthenatureoferrorsinachoicereactiontask.PsychonomicScience,345Ð346.Gur,D.,Rockette,H.E.,ArmÞeld,D.R.,Blachar,A.,Bogan,J.K.,Brancatelli,G.,etal.(2003).Theprevalenceeffectinalabora-toryenvironment.,10Ð14.Gur,D.,Rockette,H.E.,Warfel,T.,Lacomis,J.M.,&Fuhrman,C.R.(2003).Fromthelaboratorytotheclinic:TheÔÔprevalenceeffect.ÕÕAcademicRadiology,1324Ð1326.Kundel,H.L.,Nodine,C.F.,&Carmody,D.(1978).Visualscanning,patternrecognitionanddecision-makinginpulmonarynoduleInvestigativeRadiology,175Ð181.MacDonald,C.J.,&Meck,W.H.(2004).Systems-levelintegrationofintervaltimingandreactiontime.NeuroscienceandBiobehavioral,747Ð769.Manning,D.J.,Ethell,S.C.,&Donovan,T.(2004).Detectionorde-cisionerrors?MissedlungcancerfromposteroanteriorchestBritishJournalofRadiology,231Ð235.McCarley,J.S.,Kramer,A.F.,Wickens,C.D.,Vidoni,E.D.,&Boot,W.R.(2004).Visualskillsinairportsecurityscreening.logicalScience,302Ð306.Obuchowski,N.A.(2005).Onelessbiastoworryabout.,302.Rabbitt,P.M.A.(1966).Errorsanderrorcorrectioninchoice-responseJournalofExperimentalPsychology,264Ð272.Renfrew,D.L.,Franken,E.A.,Jr.,Berbaum,K.S.,Weigelt,F.H.,&Abu-Yousef,M.M.(1992).Errorinradiology:ClassiÞcationandlessonsin182casespresentedataproblemcaseconference.,145Ð150.Samuel,S.,Kundel,H.L.,Nodine,C.F.,&Toto,L.C.(1995).Mecha-nismsofsatisfactionofsearch:Eyepositionrecordingsinthereadingofchestradiographs.,895Ð902.Schwaninger,A.,Hardmeier,D.,&Hofer,F.(2005).Aviationsecurityscreenersvisualabilities&visualknowledgemeasurement.IEEEAerospaceandElectronicSystems,29Ð35.Wolfe,J.M.,Horowitz,T.S.,&Kenner,N.M.(2005).Rareitemsoftenmissedinvisualsearches.,439Ð440.Wolfe,J.M.,Horowitz,T.S.,VanWert,M.J.,Kenner,N.M.,Place,S.S.,&Kibbi,N.(inpress).Lowtargetprevalenceisastubbornsourceoferrorsinvisualsearchtasks.JournalofExperimentalPsy-chology:General1/8/07;REVISIONACCEPTEDINALMATERIALSRECEIVEDVolume18ÑNumber11 MathiasS.FleckandStephenR.Mitroff