This article was downloaded by: [James Madison University]On: 22 Febru - PDF document

This article was downloaded by: [James Madison University]On: 22 February 2014, At: 08:04Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH,UK Thinking & ReasoningPublication details, including instructions for authorsand subscription information:http://www.tandfonline.com/loi/ptar20 Assessing miserly informationprocessing: An expansion of theCognitive Reflection TestMaggie E. Toplaka, Richard F. Westb & Keith E.Stanovichca Department of Psychology, York University, Toronto,Canadab Department of Graduate Psychology, James MadisonUniversity, Harrisonburg, VA, USAc Department of Applied Psychology and HumanDevelopment, University of Toronto, Toronto, CanadaPublished online: 28 Oct 2013. To cite this article: Maggie E. Toplak, Richard F. West & Keith E. Stanovich (2014)Assessing miserly information processing: An expansion of the Cognitive Reflection Test,Thinking & Reasoning, 20:2, 147-168, DOI: 10.1080/13546783.2013.844729 To link to this article: http://dx.doi.org/10.1080/13546783.2013.844729 PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all theinformation (the “Content”) contained in the publications on our platform.However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, orsuitability for any purpose of the Content. Any opinions and views expressedin this publication are the opinions and views of the authors, and are not theviews of or endorsed by Taylor & Francis. The accuracy of the Content shouldnot be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions,claims, proceedings, demands, costs, expenses, damages, and other liabilitieswhatsoever or howsoever caused arising directly or indirectly in connectionwith, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes.Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expresslyforbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions Assessingmiserlyinformationprocessing:AnexpansionoftheCognitiveReßectionTestMaggieE.Toplak,RichardF.West,andKeithE.StanovichDepartmentofPsychology,YorkUniversity,Toronto,CanadaDepartmentofGraduatePsychology,JamesMadisonUniversity,Harrisonburg,VA,USADepartmentofAppliedPsychologyandHumanDevelopment,UniversityofToronto,Toronto,CanadaTheCognitiveReßectionTest(CRT;Frederick,2005)isdesignedtomeasurethetendencytooverrideaprepotentresponsealternativethatisincorrectandtoengageinfurtherreßectionthatleadstothecorrectresponse.Itisaprimemeasureofthemiserlyinformationprocessingpositedbymostdualprocesstheories.Theoriginalthree-itemtestmaybebecomingknowntopotentialpar-ticipants,however.Weexaminedafour-itemversionthatcouldserveasasub-stitutefortheoriginal.Ourdatashowthatitdisplaysa.58correlationwiththeoriginalversionandthatithasverysimilarrelationshipswithcognitiveability,variousthinkingdispositions,andwithseveralotherrationalthinkingtasks.Combiningthetwoversionsintoaseven-itemtestresultedinameasureofmiserlyprocessingwithsubstantialreliability(.72).Theseven-itemversionwasastrongindependentpredictorofperformanceonrationalthinkingtasksafterthevarianceaccountedforbycognitiveabilityandthinkingdispositionshadbeenpartialledout.CognitiveReßectionTest;Rationalthinking;Cognitiveability;Thinkingdispositions;Dualprocesstheory.Onebackgroundassumptionofmostdualprocesstheoriesisthatpeopletendtobecognitivemisersintheirthinking.Thisiswhatmakestheoverridefunction CorrespondenceshouldbeaddressedtoMaggieE.Toplak,DepartmentofPsychology,YorkUniversity,126BSB,4700KeeleSt.Toronto,M3J1P3,OntarioCanada.E-mail:mtoplak@yorku.caPreparationofthismanuscriptwassupportedbygrantsfromtheSocialSciencesandHumanitiesResearchCouncilofCanadatoMaggieE.ToplakandfromtheJohnTempletonFoundationtoKeithE.StanovichandRichardF.West.Theopinionsexpressedinthispublica-tionarethoseoftheauthorsanddonotnecessarilyreßecttheviewsoftheJohnTempletonFoundation.TheauthorswishtothankGeoffSorge,MohamedAl-Haj,andAmandaEdwardsforassistancewithdatacollectionandscoringtheWASI. 2013Taylor&FrancisThinking&Reasoning,2014Vol.20,No.2,147Ð168,http://dx.doi.org/10.1080/13546783.2013.844729 inmostdualprocesstheoriessoimportant.Thecognitivemiserassumptionthatisretainedinmostmoderndualprocesstheories(Evans,;Evans&Stanovich,;Stanovich,)hasbeenamajorthemethroughoutthepast50yearsofresearchinpsychologyandcognitivescience(Dawes,;Evans,;Johnson-Laird,;Kahneman,;Simon,;Stano-;Taylor,;Tversky&Kahneman,Whenapproachinganyproblem,ourbrainshaveavailablevariouscomputationalmechanismsfordealingwiththesituation.Thesemecha-nismsembodyatrade-off,however,welldescribedincontemporarydualprocesstheory(Evans&Stanovich,2013).ThedeÞningfeatureofType1processingisitsautonomyÑtheirexecutionismandatorywhenthetrigger-ingstimuliareencounteredandtheycanoperateinparallelwithoutinterfer-ingwiththemselvesorwithType2processing.Type2processingisrelativelyslowandcomputationallyexpensive,andoneofitsmostcriticalfunctionsistooverrideType1processing.Thetrade-offbetweenType1andType2processingisonebetweenpowerandexpense.Type2processingenablesustosolveawiderangeofnovelproblems,andsolvethemwithgreataccuracy.However,thispowercomeswithacost.Type2processingtakesupagreatdealofattention,tendstobeslow,tendstointerferewithotherthoughtsandactionsthatwearecar-ryingout,andrequiresgreatconcentrationthatisoftenexperiencedasaver-sive.Incontrast,Type1processesarelowincomputationalpowerbuthavetheadvantagethattheyarelowincost.ThesemechanismscannotsolveawiderangeofproblemsanddonotpermitÞne-grainedaccuracy,buttheyarefastacting,donotinterferewithotherongoingcognition,requirelittleconcentration,andarenotexperiencedasaversive.HumansarecognitivemisersbecausetheirbasictendencyistodefaulttoType1processingmechanismsoflowcomputationalexpense.Usinglesscomputationalcapacityforonetaskmeansthatthereismoreleftoverforanothertaskiftheybothmustbecompletedsimultaneously.Thiswouldseemtobeadaptive.Nevertheless,thisstrongbiastodefaulttothesimplestcognitivemechanismÑtobeacognitivemiserÑmeansthathumansareoftenlessthanrational.Type1processesoftenprovideaquicksolutionthatisaÞrstapproximationtoanoptimalresponse.Butmodernlifeoftenrequiresmoreprecisethoughtthanthis.Moderntechnologicalsocietiesareinfacthostileenvironmentsforpeoplereliantononlythemosteasilycom-putedautomaticresponse(seeKahneman,;Stanovich,;Sunstein,;Thaler&Sunstein,Becausebeingacognitivemiserwillseriouslyimpedepeoplefromachievingtheirgoals,psychologistshavebeeninterestedinstudyingindivid-ualdifferencesinthemiserlytendency(Stanovich,;Stanovich&West,2000).PeoplevaryinhowlikelytheyaretooverrideaprepotentresponseTOPLAK,WEST,STANOVICH alternativethatisincorrectandtoengageinfurtherreßectionthatleadstothecorrectresponse.ByfarthemostpopularmeasureofmiserlyprocessinghasbeenFrederickÕsCognitiveReßectionTest(CRT).TheproblemsontheCRTseematÞrstglancetobesimilartothewell-knowninsightprob-lemsintheproblem-solvingliterature,butinfacttheydisplayacriticaldif-ference.Classicinsightproblems(seeGilhooly&Fioratou,donotusuallytriggeranattractivealternativeresponse.Insteadtheparticipantsitslostinthoughttryingtoreframetheproblemcorrectlyasin,forexample,theclassicninedotproblem.ThethreeproblemsontheCRTareofinteresttoresearchersworkinginthedual-processtraditionbecauseastrongalter-nativeresponseisinitiallyprimedandthenmustbeoverridden.Shockingly,sinceitisbasedonjustthreeitems,theCRThasproventobeapotentpredictorofperformanceonrationalthinkingtasks.FrederickobservedthathisCRTcouldpredictthetendencytochoosehighexpected-valuegamblesandthatCRTscoreswereassociatedwithtemporaldiscountingandframing.Likewise,CokelyandKelleyfoundacorre-lationbetweenperformanceontheCRTandtheproportionofchoicescon-sistentwithexpectedvalue.OthershavefoundtheCRTtobesigniÞcantlyassociatedwithavoidingtheconjunctionfallacy;expectedvaluechoices;maximisingstrategiesonprobabilisticpredictiontasks;theendorsementofproÞtmaximisingstrategies;theavoidanceoftheillusionofexplanatorydepth;non-superstitiousthinking;performancecalibration,andgeneralnumeracy(Fernbachetal.,;Koehler&James,2010;Liberalietal.,;Mataetal.,;Moritzetal.,;Oechssleretal.,;Penny-cooketal.,;Shenhavetal.,Inthemostcomprehensivestudyyet,Toplak,West,andStanovichformedacompositevariableof15separaterationalthinkingtasksfrommanydifferentdomainsintheheuristicsandbiasesliterature.TheyfoundthattheCRTwasabetterpredictorofrationalthinkingthaneithermeasuresofintelligenceormeasuresofexecutivefunctioning.SeveraloftheregressionanalysesconductedindicatedthattheCRTcouldpredictrationalthinkingperformanceindependentofnotonlyintelligencebutalsoexecutivefunctioningandthinkingdispositions.Infact,inalloftheanalyses,theCRTbyitselfaccountedformoreuniquevarianceexplainedthantheblockofcognitiveabilitymeasures(intelligence).Thisisastoundingpredictiveper-formanceforathree-itemmeasure!Nevertheless,thereareproblemsonthehorizonfortheCRTgoingintothefuture.TheitemsarebecomingextremelywellknownÑespeciallythefamousbat-and-ballitem.Thelatterisusedincountlessclassroomdemon-strationsnow,andithasappearedinmanymagazinesandfamousbooksÑmostnotablyDanielKahnemanÕsrightlylaudedandextensivelyreviewedThinking,FastandSlow.Fromthestandpointofreliability,threeCOGNITIVEREFLECTIONTEST itemsisobviouslytoofew.Finally,insomepopulations,theoverallscoreonthethree-itemversionmightbeßoored.Frederickreportedthemeanperformanceonthethreeitemsacrossavarietyofacademicinstitutionsandfoundthat,forexample,studentsatMichiganStateUniversityandBowlingGreenStateUniversitygotlessthanoneitemoutofthreecorrect.ThemeanfortheUniversityofToledowasjust0.57.Clearly,usingthethree-itemver-sioninhighschoolsandcommunitycollegeswillbeproblematicintermsofßooreffects.ThustheCRTisbadlyinneedofsupplementandextension.Herewereporttheresultsofusingaseven-itemCRT,onethatincludestheoriginalthreeitemsreportedbyFrederickandfourotherswithouttheexten-siveresearchtrack-recordoftheoriginalproblems.Weexamineditsabilitytopredictperformanceonsevenrationalthinkingtasksfromtheheuristicsandbiasesliteratureandwhetherthefournewitemsaddtothevarianceexplained.Inordertosituatetheseven-itemversionwithintheoverallspaceofindividualdifferences,wealsoassessedcognitiveability(intelligenceandexecutivefunctioning)andfourdifferentthinkingdispositions(NeedforCognition,ActivelyOpenmindedThinking,SuperstitiousThinking,andConsiderationofFutureConsequences).ParticipantsandprocedureAtotalof160participants(20.7years,3.7;63malesand97females)tookpartinthestudy.Theparticipantswererecruitedatalargeuni-versityandwereeitherpartofaparticipantpoolwhoreceivedcoursecredit123)orpaid(37)fortheirparticipation.Thepaidparticipantswereolderthantheunpaidparticipants(difference3.4years);.001,butdidnotdiffersigniÞcantlyfromtheunpaidparticipantsinsex,cognitiveability(WASI)scores,highschoolGPA,orcollegeGPA.Partici-pantsprovidedestimationsoftheircurrentuniversitygrade-pointaveragesusingtheuniversityÕspercentagescale(7.8;Onthisuni-versityÕsgradingscale,70Ð74%correspondstoaBlettergrade).Participantscompletedthebatteryoftasksdescribedbelowplussomeothermeasuresduringasingle,2-hoursession.Thetaskswerepresentedinthefollowingorder:WASI,demographicspart1,othersidethinkingtasks,framingproblems,denominatorneglect,beliefbiassyllogisticreasoning,selectiontasks,cognitivereßectiontest,biasblindspot,temporaldiscount-ing,thinkingdispositionmeasures,demographicspart2.AllofthetaskswerepresentedonacomputerusingMediaLabv2008software,withtheexceptionofthecognitiveabilitytesting(WASI),whichwasadministeredindividuallybyanexaminer.TOPLAK,WEST,STANOVICH TasksandvariablesCognitiveReßectionTest(CRT)TakenfromFrederick,theoriginaltestwascomposedofthreequestions,asfollows:(1)Abatandaballcost$1.10intotal.Thebatcostsadollarmorethantheball.Howmuchdoestheballcost?____cents[Correctanswer5cents;intuitiveanswer10cents](2)Ifittakes5machines5minutestomake5widgets,howlongwouldittake100machinestomake100widgets?____minutes[Correctanswer5minutes;intuitiveanswer100minutes](3)Inalake,thereisapatchoflilypads.Everyday,thepatchdoublesinsize.Ifittakes48daysforthepatchtocovertheentirelake,howlongwouldittakeforthepatchtocoverhalfofthelake?____days[Correctanswer47days;intuitiveanswer24days]ThescoreontheseoriginalthreeitemswillbedesignatedCRT3inourstudy.Weaddedthefollowfour-itemstotheCRT:(4)IfJohncandrinkonebarrelofwaterin6days,andMarycandrinkonebarrelofwaterin12days,howlongwouldittakethemtodrinkonebarrelofwatertogether?_____days[correctanswer4days;intuitiveanswer(5)Jerryreceivedboththe15thhighestandthe15thlowestmarkintheclass.Howmanystudentsareintheclass?______students[correctanswer29students;intuitiveanswer(6)Amanbuysapigfor$60,sellsitfor$70,buysitbackfor$80,andsellsitÞnallyfor$90.Howmuchhashemade?_____dollars[correctanswer$20;intuitiveanswer(7)Simondecidedtoinvest$8,000inthestockmarketonedayearlyin2008.Sixmonthsafterheinvested,onJuly17,thestockshehadpur-chasedweredown50%.FortunatelyforSimon,fromJuly17toOctober17,thestockshehadpurchasedwentup75%.Atthispoint,Simonhas:a.brokeneveninthestockmarket,b.isaheadofwherehebegan,c.haslostmoney[correctanswerc,becausethevalueatthispointis$7,000;intuitiveresponseItems#4and#5werekindlysuppliedtousbyShaneFrederickinper-sonalcorrespondence(13June2011);item#6wasadaptedfromDomi-;seeGilhooly&Murphy,;andtheseventhitemwascreatedbytheauthors.Thescoreonthesefournewitemswillbedesig-natedCRT4inourstudy.ThescoreonallsevenitemswillbelabelledCRT7.COGNITIVEREFLECTIONTEST Cognitiveability:WechslerabbreviatedscalesofintelligenceTheVocabularyandMatrixReasoningsubtestsfromtheWechslerAbbreviatedScalesofIntelligence(WASI;Wechsler,)wereusedasindicesofverbalandnonverbalability.Thesesubtestswereadministeredindividuallybyanexaminer.ThemeanrawscoreontheVocabularysubtestwas56.3(6.9),andthemeanrawscoreoftheMatrixReasoningsub-testwas26.0(4.2).TherawscoresfortheVocabularyandMatrixReasoningsubtestswereconvertedinto-scoresandsummedtocreateacompositemeasureofcognitiveability.ThinkingdispositionmeasuresParticipantscompletedaself-reportquestionnaireinwhichtheywereaskedtoratetheiragreementwitheachquestionusingthefollowingsix-pointscale:StronglyDisagree(1),DisagreeModerately(2),DisagreeSlightly(3),AgreeSlightly(4),AgreeModerately(5),StronglyAgree(6).Questionswerepresentedinmixedordersothatthetargetscalesofinterestwouldbelesstransparenttoparticipants.Severalscaleswereintermixedinthequestionnaire.Needforcognition(NFC).This18-itemscaleassessesthemotivetoengageineffortfulcognitiveactivities(Cacioppoetal.,).Sampleitemsinclude:ÒThenotionofthinkingabstractlyisappealingtomeÓ,andÒIwouldpreferataskthatisintellectual,difÞcult,andimportanttoonethatissomewhatimportantbutdoesnotrequiremuchthoughtÓ.Themeanscore68.6(10.4).Thesplit-halfreliability(Spearman-Browncor-rected)oftheneedforcognitionscalewas.78andCronbachÕsalphawas.79.Activelyopenmindedthinking(AOT).This41-itemmeasureisscoredinthedirectionthathigherscoresrepresentedagreatertendencytowardsopenmindedthinking(Stanovich&West,).ExamplesofitemsareÒPeopleshouldalwaystakeintoconsiderationevidencethatgoesagainsttheirbeliefsÓ,ÒCertainbeliefsarejusttooimportanttoabandonnomatterhowgoodacasecanbemadeagainstthemÓ(reversescored),andÒNoonecantalkmeoutofsomethingIknowisrightÓ(reversescored).Thescoreonthescalewasobtainedbysummingtheresponsestothe41items(21.3).Thesplit-halfreliability(Spearman-Browncorrected)ofthescalewas.85andCronbachÕsalphawas.86.Superstitiousthinking(ST).This13-itemscalewascomposedoftwoitemsfromaparanormalscaleusedbyJones,Russell,andNickelfouritemsfromaluckscaleusedbyStanovichandWest(1998c),fouritemsfromanESPscaleusedbyStanovich,andthreeitemsfromasupersti-tiousthinkingscalepublishedbyEpsteinandMeier.Examplesofitemsinclude:ÒAstrologycanbeusefulinmakingpersonalityjudgmentsÓ,ÒThenumber13isunluckyÓ,andÒIdonotbelieveinanysuperstitionsÓ(reversescored).ThescoreonthescalewasobtainedbysummingtheTOPLAK,WEST,STANOVICH responsestothe13items(10.9).Thatscorewasturnedinto-scoreandthe-scorereßected(multipliedbyÐ1)sothathigherscoresindicategreatresistancetosuperstitiousthinking.Thusthesuperstitiousthinkingscale(reßected)wouldbeexpected,basedonpreviousresearch,tocorrelatepositivelywiththeotherthinkingdispositionsandwithcognitiveability.Thesplit-halfreliability(Spearman-Browncorrected)ofthescalewas.86andCronbachÕsalphawas.84.Considerationoffutureconsequences(CFC).This12-itemscaleassessestheextenttowhichindividualsconsiderdistantoutcomeswhenchoosingtheirpresentbehaviour(Strathman,Gleicher,Boninger,&Edwards,Asampleitemfromthescalewas:ÒIonlyacttosatisfyimmediateconcerns,ÞguringthefuturewilltakecareofitselfÓ(reversescored).Thescoreonthescalewasobtainedbysummingtheresponsestothe12items(7.4).Thesplit-halfreliability(Spearman-Browncorrected)ofthescalewas.76andCronbachÕsalphawas.74.RationalthinkingtasksBeliefbiasinsyllogisticreasoning.Eightsyllogisticreasoningproblems,largelydrawnfromMarkovitsandNantel,werecompletedbytheparticipants.Eachproblemwaswordedsuchthatthevalidityjudgementwasinconßictwiththebelievabilityoftheconclusion.Thereweretwotypesoftheseso-calledinconsistentsyllogisms.Onetypeofinconsistentsyllogismhadabelievableconclusionbutaninvalidformat(e.g.,ÒPremises:Alllivingthingsneedwater;Rosesneedwater;Conclusion:RosesarelivingthingsÓÑwhichisinvalid).Theothertypehadanunbelievableconclusionsinalogi-callyvalidformat(e.g.,ÒPremises:Allthingsthataresmokedaregoodforthehealth;Cigarettesaresmoked;Conclusion:CigarettesaregoodforthehealthÓÑwhichisvalid).Thereforethebelievabilityofthecontentwasinconsistentwiththelogicalformatofthesyllogisminbothtypes.ProblemsofthistypehavetypicallybeenthoughttomirrorthecriticalthinkingskillofbeingabletoputasideoneÕspriorknowledgeandreasonfromnewprem-ises.Aftereachitem,theparticipantsindicatedtheirresponsesbyselectingoneofthetwoalternatives:(1)Conclusionfollowslogicallyfrompremises,or(2)Conclusiondoesnotfollowlogicallyfrompremises.Theeightsyllo-gismswerepresentedtogetherinthebattery.Acompositescoreofperfor-manceontheeightitemswasformedbysummingthenumberofcorrectresponses(1.5);thesplit-halfreliability(Spearman-Browncorrected)ofthescalewas.51andCronbachÕsalphaSelectiontask.Threeversionsoftheselectiontaskwereutilised,twowithnon-deonticcontent(AbstractandDestinationProblem),andonewithdeonticcontent(SearsProblem).Onenon-deontic,abstractversionwasoriginallyusedbyWason,andhasbeenstudiedextensivelyintherea-soningliterature.ThesecondnondeonticproblemwastheDestinationCOGNITIVEREFLECTIONTEST ProblemstudiedbyStanovichandWest(1998a).ThedeonticversionwastheSearsProblem(Dominowski,;Stanovich&West,1998a).Eachver-sionoftheselectiontaskwasseparatedinthebatterybyotherrationalthinkingtasks.BecausetheruleisintheformofanifP,thenQrule,thepar-ticipantmustturnoverthecardsthatcouldpotentiallyfalsifytheruleÑthePandnot-Qcards,whichwasscoredasthecorrectresponses(andscoredas1).Becausecorrectrespondingonnondeonticversionsistypicallysolow,wealsoscoredP-onlychoosersascorrect(seeToplak&Stanovich,analternativetaskconstrualchampionedbyMargolis.HehasarguedthatturningthePcardonlyisanappropriateresponseiftheparticipanthasadoptedaso-calledÒopenÓreadingoftheruleÑonewherethecardsrepre-sentclassesratherthanindividualexemplars.Allotherselectionswerescoredas0.Thescoresonthethreeselectiontasksweresummedtoformaselectiontaskcompositescore(Denominatorneglect.TheÞve-problemsonthistaskweremodelledonKirkpatrickandEpstein;seealsoDenes-Raj&Epstein,.Anexampleofatrialreadasfollows:Assumethatyouarepresentedwithtwotraysofblackandwhitemarbles(picturedbelowandright):Thelargetraycontains100marbles.Thesmalltraycontains10marbles.Themarblesarespreadinasinglelayerineachtray.Youmustdrawoutonemarble(withoutpeeking,ofcourse)fromeithertray.Ifyoudrawablackmarbleyouwin$5.Consideraconditioninwhich:Thesmalltraycontains1blackand9whitemarbles.Thelargetraycontains8blackand92whitemarbles.[Adrawingoftwotrayswiththeircorrespondingnumbersofmarblesarrangedneatlyin10-marbles-rowswaspictured.Thecorrespond-ingnumberofblackandwhitemarbleswasprintedinparenthesesdirectlyunderneatheachtray.]Fromwhichtraywouldyouprefertoselectamarbleinarealsituation?Thefollowingscalewasusedtoindicatepreferences:(1)IwoulddeÞ-nitelypickfromthesmalltray;(2)Iwouldpickfromthesmalltray;(3)Iwouldprobablypickfromthesmalltray;(4)Iwouldprobablypickfromthelargetray;(5)Iwouldpickfromthelargetray;(6)IwoulddeÞnitelypickfromthelargetray.Intheremainingfourtrials,theratioofblack:whitenumberswereasfol-lows:1:4versus19:81,1:19versus4:96,2:3versus19:31,and3:12versus18:82.Eachproblemwasseparatedinthebatterybyotherrationalthinkingtasks.Inallcasesthecorrectresponsewastoselectthesmalltray,asthechancesofpullingablackmarblewashigher(10%inthecurrentexample)thaninthelargetray(8%).ThesumoftheratingsoftheÞveproblemswasreßectedtoformacompositescorewherehighervaluesindicatedmoreresis-tancetodenominatorneglect.ThemeancompositescoreacrosstheÞveproblemswas20.91(TOPLAK,WEST,STANOVICH Temporaldiscounting.ThisÞve-itemmeasurewasadaptedfromFrederick(2005).Foreachitem,participantsindicatedthestrengthoftheirpreferenceforeitherasmalleramountofmoneynoworalargeramountofmoneylater.Ineachcasethedelayedlargeramountcorrespondedtoasubstantialpercent-ageincreaseinvalue,whichonasimpleinterestbasiswouldhaveresultedinvalueincreasesofbetween40%to240%ifearnedannually.TheÞrstitemofthismeasure,forexample,askedparticipantstoindicatewhethertheywouldÒprefer$3400thismonthor$3800nextmonthÓ.Inthisexample,awilling-nesstowaitwasworthanextra$400Ñtheequivalentofaboutan11.8%gaininvalueinonemonth,whichonasimpleinterestbasiswouldhaveresultedinavalueincreaseofabout141%ifearnedannually.Participantsindicatedtheirpreferencesusingthefollowingresponsescale:(1)Istronglyprefer$3400thismonth;(2)Islightlyprefer$3400thismonth;(3)Iprefer$3400thismonth;(4)Iprefer$3800nextmonth;(5)Islightlyprefer$3800nextmonth;(6)Istronglyprefer$3800nextmonth.Theremainingfouritemswerethefollowing:Ò$100nowor$140nextyearÓ;Ò$100nowor$1100in10yearsÓ;Ò$9nowor$100in10yearsÓ;andÒ$40immediatelyor$1000in10yearsÓ.AcompositescorewascreatedbysummingtheseÞveitems.Ahigherscoreonthiscompositeindicatedapreferencetowaitforthelargeramountofmoney.Themeancompositescoreonthistaskwas17.826.27).Othersidethinking.Twoissuesthatthecollege-studentparticipantswerelikelytohavestrongopinionsaboutwereusedforthistask.Oneissueper-tainedtoraisingthecostofthetuition(Toplak&Stanovich,,andthesecondissuepertainedtobanningcarsoncampus.IntheÞrstandopinion-gatheringpartofthetask,participantswereaskedtogivetheiropinionsoneachoftheissuesusingasix-pointscalerangingfromdisagreestronglytoagreestrongly.Theissuesreadasfollows:(1)Considerthefollowingissue:Therealcostofauniversityeducationis$15000/year.Studentsarecurrentlypayingapproximately$6000/yearintuition.Thedifferenceispaidforbythetaxpayer.Indicatetowhatextentyouagreeordisagreewiththefollowingstatement:Universitystudentsshouldpayforthefullcostoftheiruniversityeducation.(2)Considerthefollowingissue:ThereisconsiderabledebateaboutwhetherstudentsshoulddrivecarsortakepublictransittogettotheYorkUniversitycampus.Indicatetowhatextentyouagreeordisagreewiththefollowingstatement:Carsshouldbebannedfromcampus.Thereason-generationpartofthetaskwasadministeredatalaterpointinthebattery.Thistimethedescriptionofeachissuewasrepeated,followinginstructionstoÒThinkthroughthefollowingissuecarefullyandfeelfreetotakeyourtime.PleasetypeyourargumentsbothforandagainstthisCOGNITIVEREFLECTIONTEST position.Trytowriteasmuchasyoucan,andremembertotryandgiverea-sonsbothforreasonsagainstyourposition.Pleasetypeyourresponsesandlabeleachargument(i.e.,1.,2.,3.,etc.).ÓAscoringschemewasdevelopedfordistinguishingbetweentheconcep-tuallyuniquereasonsparticipantslistedasbeingforandagainsttheraisingtuitionandbanningcarsissues.Thelistedreasonswereindependentlyscoredbytwotrainedcoderswhousedapreviouslydevelopedcodingschemeforclassifyingthestatements.Theinter-rateragreementbetweenthetwocoderswasmoderatetohigh.UsingPearsonÕstheinter-rateragree-mentbetweenthetwocoderswas.90and.78,respectively,forthenumberofreasonstoraiseandnotraisetuition.Theinter-rateragreementwas.96and.92,respectively,forthenumberofreasonstobanandnotbancarsfromcampus.IncaseswherethetwocoderÕsscoreswerenotinidenticalagreement,athirdcoderindependentlyresolvedthediscrepancyanddeter-minedthescore.EachparticipantÕsnumberofothersidereasonswasderivedbasedonhis/herprioropinion.Ifaparticipantendorsednotraisingtuitioninhis/herprioropinion,thenumberofreasonsgeneratedinfavourofraisingtuitionscoredasthenumberofothersidereasons,andvice-versa.ThetotalnumberofeachparticipantÕsuniqueothersidereasonsfortheraisingtuition(1.03)andbanningcars(1.34)issueswerethenderivedbasedontheirresponsesduringtheopinion-gatheringpartoftheFraming.Inthreeproblems,participantschosebetweenrisklessandriskyalternativesthathadidenticalexpectedvalues,underbothagain-framingandaloss-framingcondition.OneproblemwasanadaptationofTverskyandKahnemanÕsfamousdiseaseproblem,andthetworemainingproblemswerebasedonitemsusedbyBruinedeBruin,Parker,andFischh-;highschooldropoutandinvestmentproblems).Eachproblemwasseparatedinthebatterybyotherrationalthinkingtasks.Thefollowingscalewasusedtoindicatepreferences:(1)stronglyfavouroption1;(2)favouroption1;(3)slightlyfavouroption1;(4)slightlyfavouroption2;(5)favouroption2;(6)stronglyfavouroption2.Problemswerescoredbysubtractingtheirpositiveframeratingsfromtheircorrespondingnegativeframeratings.Negativedifferencescoresindi-catedaframingeffectintheexpecteddirectionandrepresentedaviolationoftheprincipleofdescriptiveinvariance.Differencescoresof0indicatedtheabsenceofaframingeffect.SigniÞcantframingeffectsintheexpecteddirectionwerefoundforeachofthethreeframingproblems(differencesÐ.44toÐ.56);Ð4.25toÐ4.32,.001.Acompositeframingscorewasformedbysummingthethreedifferencescoresandreßectingtheresult-ingvalueswiththeresultthathigherscoreindicatedmoreresistancetofram-ing.Themeancompositescoreacrossthethreeitemswas2.28(2.2),TOPLAK,WEST,STANOVICH .001.Forthepurposesofthecompositescore,differencescoresofframingeffectsintheunexpecteddirectionweresetto0.Biasblindspot.ParticipantsreadshortdescriptionsoffourspeciÞccogni-tivebiases:framing,baserateneglect,mysidebias,andcellphonehazard,andratedthelikelihoodthattheyorafellowstudentswouldhavethebias(seetheAppendixofWest,Meserve,&Stanovich,,forthefullproto-cols).Theframingeffectitemprovidesanexample:Psychologistshaveshownthatpeopletendtoevaluatestatements,arguments,orpoliciesdifferentlydependingonthechoiceofwords.Thismeansthatpeo-pleÕsopinionsoftheverysamepolicyordecisionorproductcanbemanipu-latedbyslightchangesinwordingthatdonÕtchangethemeaning.Forexample,afooditemlabelledÒ98%fatfreeÓisjudgedmoreattractivethanonelabelledÒcontains2%fatÓ.WhenpeopleÕsopinionsaremanipulatedbasedonarewordingthatdoesnotchangethemeaning,thisistermedaframingTowhatextentdoyoubelievethatyouarelikelytobesusceptibletoframingTowhatextentdoyoubelievethattheaverageYorkUniversitystudentislikelytobesusceptibletoframingeffects?Responsestothelikelihoodquestionsweregivenonasix-pointLikert-typescaleanchoredat1(Notatalllikely)and6(Veryhighlylikely).Partici-pantsratedotherstudentsasmorelikelytocommitthebiasthanthemselves(all.01).Acompositebiasblindspotscorewasderivedbysummingtheself/otherdifferencescoresforthefouritems(1.74,2.18),(159)10.07,.001;CronbachÕsalphawas.57.Table1displaystheintercorrelationsamongthesevenCRTitemsaswellasCRT3,CRT4,andCRT7;44ofthe45correlationsarestatisticallysigniÞ-cant.Atthebottomofthetablearethemeansandstandarddeviationsofthevariables.PerformanceonCRT3wasquitelow,averaginglessthanahalfanitemcorrect.PerformanceonCRT4averagedalmostoneitemcor-rect.Moreimportantly,thefournewitemsdisplayedamoderatelyhighcor-relationof.58withthethreeoriginalitems.Therewereindicationsthatthefournewitems,althoughvalidCRTitems,weresomewhatlessperfectindi-catorsthantheoriginalthreeitems.ThemediancorrelationamongthethreeCRT3itemswas.40,whereasthemediancorrelationamongtheitemsofCRT4was.21.AllthreeCRT3itemshadtheintuitive(wrong)answerasthemodalresponse.Theintuitiveresponsewasgiven85.6%,75.2%,and60.0%ofthetimeforthebat/ball,widgits,andlilypadproblems,respectively.TheCOGNITIVEREFLECTIONTEST intuitiveresponsewas,infact,themodalresponseforeachoftheCRT4problemsaswell,butitwasaless-dominantresponse,beinggiven31.3%,51.9%,41.9%,and53.1%ofthetimeforthebarrel,marks,pig,andstocksproblems,respectively.Nevertheless,mostindicationswerethattheCRT4itemscouldbecom-binedwiththeCRT3items.Theinternalconsistentlyofthesevenitems,con-sideredtogether,wasquitesubstantial.All21inter-itemcorrelationswerepositiveandallbutonewerestatisticallysigniÞcant.Themedianinter-itemcorrelationwas.27,themeanwas.29,andCronbachÕsalphawas0.72.Inter-estingly,asix-itemCRTscalewithoutthemostclassicandprototypicalitemofallÑthebatandballitemÑstillhadaCronbachÕsalphaof0.67.Inshort,thefournewitemshadasubstantialcorrelationwiththeclassicthreeandwhenamalgamatedtoformaseven-itemscale,thecompositescaledisplayedsubstantialinternalconsistency.FrederickreportedthattherewasahighlysigniÞcantsexdiffer-enceonCRT3Ñadifferenceofalmostahalfanitemcorrect.WereplicatedhisÞndingofsigniÞcantlybetterperformancebymalesonCRT3(versus.29),.001.Likewise,wefoundagenderdifferenceofaboutthesamemagnitudeandinthesamedirectiononCRT4(versus.71),.001.Theeffectsizeof.652forCRT4(CohenÕs)wassimilartotheeffectsizeof.637forCRT3. TABLE1CorrelationsamongtheCRTvariablesCRTcompositesIndividualCRTitemsVariable123456789101.CRT312.CRT4.5813.CRT7(.86)(.92)14.Bat/ball(.79).46(.68)15.Widgits(.79).38(.63).5916.Lilypads(.81).52(.73).38.4017.Barrel.44(.67)(.64).36.32.3718.Marks.46(.63)(.62).39.27.41.3219.Pig.33(.60)(.54).27.19.31.16.24110.Stocks.28(.64)(.54).18.20.26.24.18.121Mean.49.981.47.12.10.28.23.15.26.34SD.851.071.71.32.30.45.42.36.44.48160.CRT33originalCRTitems;CRT44newCRTitems;CRT7all7CRTitems;CRTitem1;WidgitsCRTitem2;LilypadsCRTitem3;BarrelCRTitem4;CRTitem5;PigCRTitem6;StocksCRTitem7..001(two-tailed).CorrelationsinparenthesesreßectpartÐwholerelationshipsTOPLAK,WEST,STANOVICH Table2displaysthecorrelationsofCRT7,CRT3,andCRT4withtheremainingvariablesinthestudyÑthemeasureofcognitiveability(WASI),thefourthinkingdispositions,thesevenrationalthinkingtasks,andcollegeaverage(aswellastwocompositescorestobedescribed).Ofthe48correla-tionsinthetable,41werestatisticallysigniÞcant.FromthetablewecanseethatCRT3isabitmorestronglyrelatedtointelligencethanisCRT4(.48vs.41),althoughthedifferencewasnotstatisticallysigniÞcantusingatestfordifferencebetweendependantcorrelations(Cohen&Cohen,1983,p.57).TheCRT4displayedlargercorrelationsthanCRT3witheachofthefourthinkingdispositions,butonlythedifferencebetweentheAOTcorrelations(.45vs.29)reachedstatisticalsigniÞcanceusingatestfordifferencebetweendependantcorrelations,Regardingtherationalthinkingtasks,twoofthesevenrationalthinkingmeasures(framingandthebiasblindspot)failedtodisplaycorrelationswithanyoftheCRTmeasures(anddidnotcorrelatewithcognitiveabilityeither),buttheotherÞvedid.CRT4waspositivelycorrelatedwithfourofthetasks(beliefbias,selectiontask,othersidethinking,andtemporaldiscounting).Table3presentsaseriesofsimultaneousregressionsthatexaminewhetherCRT3andCRT4explainuniqueoroverlappingvariance. TABLE2CorrelationsofCRT7,CRT3,andCRT4withtheothervariablesinthestudyCRT7CRT3CRT4WASI.50.48.41NFC.31.25.30AOT.42.29.45ST(reßected).19.15.19CFC.30.21.32BeliefBiasinSyllogisticReasoning.57.55.48SelectionTask.20.22.15DenominatorNeglect.42.37.38TemporalDiscounting.16.15.14OthersideThinkingÐTuition.29.21.29OthersideThinkingÐBanCars.26.24.23Framing.05.06.03BiasBlindSpot.03Ð.01.05CollegeAverage.25.23.21RationalThinkingComposite.56.52.48ThinkingDispositionsComposite.41.30.42160.CRT7all7CRTitems;CRT33originalCRTitems;CRT44newCRTitems;WechslerAbbreviatedScaleofIntelligencecomposite;NFCNeedforCognition;ActivelyOpenmindedThinking;STSuperstitiousThinking:CFCConsiderationofFutureConsequences..001(two-tailed).COGNITIVEREFLECTIONTEST TABLE3SimultaneousregressionsexaminingwhetherCRT3andCRT4explainuniqueoroverlap-pingvarianceStandardisedBetat(157)UniqueVarianceCriterionvariableNeedforCognitionCRT30.1231.320.01CRT40.2232.39OverallRegression:(2,157)CriterionvariableCRT30.0440.510.001CRT40.4224.81OverallRegression:(2,157)CriterionvariableResistancetoSuperstitiousThinkingCRT30.0540.560.002CRT40.1591.650.017OverallRegression:(2,157)CriterionvariableConsiderationofFutureConsequencesCRT30.4150.490.001CRT40.293.12OverallRegression:(2,157)CriterionvariableBeliefBiasCRT30.4155.20CRT40.2352.94OverallRegression:(2,157)CriterionvariableSelectionTaskCRT30.2042.13CRT40.0310.320.001OverallRegression:(2,157)CriterionvariableDenominatorNeglectCRT30.2312.60CRT40.2432.74OverallRegression:(2,157)TOPLAK,WEST,STANOVICH Regardingthethinkingdispositions,CRT4wasthemostpotentpredictorinallfourcases,andexplainedsigniÞcantuniquevarianceinthreeoffourcases.RegardingtheÞverationalthinkingtasksthatdidshowarelationshipwiththeCRT,theresultsweremoremixedbutfavouredCRT3asthemorepotentpredictor.OnthebeliefbiastaskbothCRT3andCRT4predicteduniquevarianceoncetheotherwasaccountedfor,butCRT3hadalargerbetaweight(and,redundantlyofcourse,moreuniquevarianceexplained).OntheselectiontaskCRT3explainedsigniÞcantuniquevarianceonceCRT4wasintheregressionequation,buttheconversewasnottrue.OnthedenominatorneglecttaskbothCRT3andCRT4predicteduniquevarianceoncetheotherwasaccountedfor(3.5%and3.9%,respectively),andbothwereaboutequallystronguniquepredictors.NeithervariablewasaverygoodpredictoroftemporaldiscountingÑCRT3andCRT4explainedsimi-larlysmallamountsofuniquevarianceinthattask.CRT4wasamore TABLE3StandardisedBetat(157)UniqueVarianceCriterionvariableTemporalDiscountingCRT30.0981.010.006CRT40.0820.850.005OverallRegression:(2,157)CriterionvariableOthersideÐTuitionCRT30.060.640.002CRT40.2572.74OverallRegression:(2,157)CriterionvariableOthersideÐBanCarsCRT30.1551.640.016CRT40.1391.470.013OverallRegression:(2,157)CriterionvariableCollegeAverageCRT30.1681.770.019CRT40.1131.190.008OverallRegression:(2,157)160.CRT7allsevenCRTitems;CRT3threeoriginalCRTitems;CRT4fournewCOGNITIVEREFLECTIONTEST potentuniquepredictorofothersideargumentsinthetuitiontask,butbothCRT3andCRT4wereroughlyequalmodestpredictorsofothersideargu-mentsinthebancarstaskandofthecollegeaverage.Table4presentssomeregressionanalysesrunoncompositevariables.ArationalthinkingcompositescorewasformedfromtheÞverationalthinkingtasksthatcorrelatedwithCRT.The-scoresofperformanceoneachoftheÞveweresummedtoformtherationalthinkingcomposite(thetuitionandbancarsotherside-scoresweresummedÞrst,sothattheothersidetaskwas TABLE4AdditionalsimultaneousregressionanalysesStandardisedBetat(156or157)UniqueVarianceCriterionvariableRationalThinkingCompositeCRT30.3594.42CRT40.2693.31OverallRegression:(2,157)CriterionvariableRationalThinkingCompositeCRT70.4075.26CognitiveAbility0.2273.02ThinkingDispositionsComposite0.0841.180.006OverallRegression:(3,156)CriterionvariableRationalThinkingCompositeCognitiveAbility0.3925.31ThinkingDispositionsComposite0.1952.64OverallRegression:(2,157)CriterionvariableCognitiveAbility0.4055.75ThinkingDispositionsComposite0.2723.87OverallRegression:(2,157)CriterionvariableCollegeAverageCRT7Ð0.035Ð0.420.001CognitiveAbility0.2723.35ThinkingDispositionsComposite0.3584.63OverallRegression:(3,156)160.CRT7allsevenCRTitems;TOPLAK,WEST,STANOVICH representedbyjustone-score,asweretheothervariables).Thefourthink-ingdispositionswerestandardisedandthefour-scoresaddedtogethertoformathinkingdispositionscompositescore.TheÞrstregressioninTable4indicatesthatCRT3andCRT4werebothindependentpredictorsoftherationalthinkingcompositescore(8.5%and4.8%ofthevariance,respectively).ConsistentwiththeanalysesinTable3thefournewitems,CRT4,contributetoincreasedpredictiveaccuracyfortheseÞverationalthinkingtasks,takenasaset.Thesecondregressionanal-ysisinTable4regressestherationalthinkingcompositeonthethreemainpredictorvariables:CRT7,cognitiveability,andthethinkingdispositionscomposite.ThisanalysisaddressesthequestioninvestigatedinToplaketal.ÑwhethertheCRTpredictsrationalthinkingmerelybecauseofitsassociationwithcognitiveabilityandthinkingdispositions.Inthissimulta-neousregressionCRT7isthedominantuniquepredictor,explaining11.4%uniquevariance,followedbythecognitiveability(WASI)whichexplainedastatisticallysigniÞcant3.8%uniquevariance.Thethinkingdispositionscom-positedidnotpredictuniquevariance.ThethirdregressioninTable4removesCRT7fromtheequationanddemonstratesthatthinkingdisposi-tionsareasigniÞcantindependentpredictor(3.4%uniquevariance)whenonlycognitiveabilityispartialledout.InthefourthregressioninTable4weexaminedwhichindividualdiffer-encevariablesarepredictorsofCRT7byregressingthelatteroncognitiveabilityandthethinkingdispositions.BothvariablesweresigniÞcantinde-pendentpredictorsofCRT7,withcognitiveabilityexplaining14.5%uniquevarianceandthethinkingdispositionscompositeexplaining6.5%uniquevariance.TheÞnalanalysisinTable4regressescollegeaverageonCRT7,cognitiveability,andthethinkingdispositionscomposite.Inthesimulta-neousregressiononlythinkingdispositionsandcognitiveabilitypredicteduniquevariance,withtheformerbeingthestrongerindependentpredictor(10.3%uniquevarianceversus5.4%uniquevariance).DISCUSSIONInthisstudywewereremarkablysuccessfulinestablishingaparallelfour-itemversionofFrederickÕsCognitiveReßectionTest.Thefour-itemmeasure,CRT4,couldbeusedinresearchasanalternativetotheclassicthree-itemmeasure.TheneedforanalternativeisclearbecauseoftheincreasingexposurethatthethreeclassicitemsaregettingÑparticularlythebat-and-ballproblem,whichhasappearedinbooksandmagazinesandisacommonclassroomdemonstration.Alternatively,aseven-itemversioncouldbeused,providingamorecomprehensivetestandonethatwouldsub-merge(thatis,attenuate)theeffectofanycontaminationfromthebat-and-ballitem.COGNITIVEREFLECTIONTEST ThereasonforouroptimisticconclusionregardingCRT4andCRT7derivesfrommanydifferentÞndingsofourstudy.Firstofall,regardingCRT7,Table1indicatesthatthereissubstantialcommonalityamongthesevenitems.Themediancorrelationamongthesevenitemswas.27andCronbachÕsalphawasasubstantial.72.Table2indicatesthat,forseveraloftherationalthinkingtasks,CRT7wasabetterpredictorthaneitherCRT3orCRT4.Thiswasparticularlytrueforthebeliefbiassyllogismsanddenominatorneglect.Table4indicatesthatCRT7isasubstantialindepen-dentpredictorofagroupofrationalthinkingtasksÑspeciÞcally,thetasksintherationalthinkingcompositescore.ThesecondanalysisinthattableindicatesthatCRT7wasamorepotentpredictoroftherationalthinkingcompositethanwaseithercognitiveabilityorthinkingdispositions.Afterthelattertwovariableswereenteredintotheequation,CRT7stillexplainedsubstantialuniquevariance(11.4%).OurresultsdemonstratedthatCRT4does,insomecases,contributeincrementallytothepredictivepowerofCRT7.Theremaybeavarietyofreasonsforthis.First,thelongermeasurewillofcoursebemorereliable.Also,theCRT3isknowntobeadifÞculttestandscoresonitareverylowevenamongelitepopulations(Frederick,.Amongnon-elitesamplesßooreffectsmightbeaproblem.Oursample,andasampleattheUniver-sityofToledo(seeFrederick,2005),answeredonlyone-halfofoneitemcorrect.TheCRT4,ontheotherhand,isatleastsomewhateasierthantheCRT3.ThemeanprobabilityofansweringanitemcorrectontheCRT3is.17,whereasthemeanprobabilityofansweringanitemcorrectontheCRT4is.24.Table3indicates,particularlyregardingthedispositions,CRT4wasamorepotentpredictorthanCRT3.InthreeoffourcasesCRT4accountedforsigniÞcantuniquevarianceinthethinkingdispositiononcethatthevari-anceexplainedbyCRT3hadbeenpartialledout.Likewise,CRT4wasasig-niÞcantuniquepredictorofperformanceonthebeliefbiassyllogismsevenafterthevarianceattributabletoCRT3hadbeenpartialledout.Thesamewastrueforthedenominatorneglecttaskandforoneoftheothersidethink-ingmeasures.OurdatashowthataresearcherwhowishedtosubstituteCRT4forCRT3(perhapsduetothefamiliarityissuesdiscussedabove)wouldbeamplyjustiÞed,givenourresults.Thetwoformshadasubstantial.58corre-lation.CRT3andCRT4hadsimilarcorrelationswiththerationalthinkingcompositescore(.52and.48,respectively,seeTable2).CRT4actuallyhadahighercorrelationwiththethinkingdispositionscompositescorethandidCRT3(.42versus.30).NeitherCRTmeasurepredictedperformanceontheframingtaskorinthebiasblindspottask.However,measuresofcognitiveabilityandthinkingdispositionsalsofailedtocorrelatewiththesetwotasks.ThislackofTOPLAK,WEST,STANOVICH associationofindividualdifferencevariableswithperformanceonthebiasblindspottaskisnotsurprisingbecausewehavepreviouslyfoundittobefairlyindependentofcognitiveability(West,Meserve,&Stanovich,Framingperformancealsotendstobeindependentofcognitiveabilitywhenassessedinabetween-participantscontext(Stanovich&West,),butsometimesshowsrelationshipswithcognitiveabilityinwithin-participantsdesignslikethisone(Stanovich&West,).Perhapsitisbecausethetwoversionswerewidelyseparatedinourbatterythatframingfailedtocor-relatewitheithertheCRTorcognitiveability.Finally,itshouldbenotedthattheCRThaditslargestcorrelationswiththetwotasks(beliefbiasanddenominatorneglect)that,accordingtoourtaxonomy(Stanovich,;Stanovich,West,&Toplak,),arethetwotasksinthecurrentbatterythatmostcloselyrepresenttherationalthinkingcategoryofmiserlyprocessing.ConsistentwiththeresultsofToplaketal.,inthisstudywedem-onstratedthatanexpandedCRTwasasubstantialuniquepredictorofratio-nalthinkingperformanceindependentnotonlyofcognitiveability,butalsoofafairlycomprehensivesetofthinkingdispositions.Thispredictivepowerderives,wespeculate,becausetheCRTisastrongindicatorofthemiserlyprocessingthat,indualprocesstheory,isthesourceofmuchnon-normativeresponding.Onthisview,theCRTcouldbeinterpretedasanactualmeasureofrationalthought,ratherthanasadistalpredictororanunderlyingabilitysupportingrationalthought.Thistypeofinterpretationisconsistentwithitshighcorrelationwiththerationalthinkingcompositescore.Inshort,theCRTisameasureofthetendencytowardstheclassofreasoningerrorthatderivesfrommiserlyprocessing.ThismaybewhythepredictivepoweroftheCRTisinpartseparablefromcognitiveability.Thelattermeasurescomputationalpowerthatistotheindividual,butnotnecessarilythedepthofprocessingthatistypicallyusedinmostsituations.IntelligencetestsdonotassessthetendencytowardsmiserlyprocessinginthewaythattheCRTdoes.Instead,intheCRT,thetendencytoacceptType1responsesismeasuredinarealperformancecontextwherepeoplearesearchingforanaccuratesolution.However,classifyingtheCRTasarationalthoughtindicatordoesnotmeanthatitwillcorrelateequallywitheveryrationalthinkingtaskbecauserationalthinking,inourframework,ismultifarious(Stanovich,Stanovichetal.,.TheCRTcarriesvarianceduetoalgorithmiccogni-tivecapacity(theWASIinthepresentstudy)andthuswillhavevariablecor-relationswithotherrationalthinkingtasksbecausethelatterhaveveryvariablecorrelationswithcognitivecapacity(Stanovich&West,).Cor-relationswiththeCRTwillalsotendbehigherwithtaskswherenon-norma-tiverespondingisduetomiserlyprocessing.However,notallfailuresofrationalthinkingareofthistype(Stanovich&West,).Less-rationalCOGNITIVEREFLECTIONTEST respondingcanalsobeduetothefactthatpeoplehavenotacquiredtheproperdeclarativeknowledgeindomainssuchasscientiÞcthinking,proba-bilisticreasoning,andÞnancialandeconomicliteracy.BruinedeBruin,W.,Parker,A.M.,&Fischhoff,B.(2007).Individualdifferencesinadultdeci-sion-makingcompetence.JournalofPersonalityandSocialPsychology,938Ð956.Cacioppo,J.T.,Petty,R.E.,Feinstein,J.,&Jarvis,W.(1996).Dispositionaldifferencesincog-nitivemotivation:Thelifeandtimesofindividualsvaryinginneedforcognition.icalBulletin,197Ð253.Cokely,E.T.,&Kelley,C.M.(2009).Cognitiveabilitiesandsuperiordecisionmakingunderrisk:Aprotocolanalysisandprocessmodelevaluation.JudgmentandDecisionMaking,20Ð33.Cohen,J.,&Cohen,P.(1983).Appliedmultipleregression/correlationanalysisforthebehavioral(2nded.).Hillsdale,NJ:LawrenceErlbaumAssociates.Dawes,R.M.(1976).Shallowpsychology.InJ.S.Carroll&J.W.Payne(Eds.),Cognitionandsocialbehavior(pp.3Ð11).Hillsdale,NJ:Erlbaum.Denes-Raj,V.,&Epstein,S.(1994).Conßictbetweenintuitiveandrationalprocessing:Whenpeoplebehaveagainsttheirbetterjudgment.JournalofPersonalityandSocialPsychology,819Ð829.Dominowski,R.(1994).Insightandinstructions.AnnualConference,BritishPsychologicalSoci-ety,CognitivePsychologySection(pp.1Ð3).Cambridge,UK:NewHall.Dominowski,R.L.(1995).ContenteffectsinWasonÕsselectiontask.InS.E.Newstead&J.St.B.T.Evans(Eds.),Perspectivesonthinkingandreasoning(pp.41Ð65).Hove,England:Erlbaum.Epstein,S.,&Meier,P.(1989).Constructivethinking:AbroadcopingvariablewithspeciÞcJournalofPersonalityandSocialPsychology,332Ð350.Evans,J.St.B.T.(2008).Dual-processingaccountsofreasoning,judgmentandsocialcogni-AnnualReviewofPsychology,255Ð278.Evans,J.St.B.T.(2010).Thinkingtwice:Twomindsinonebrain.Oxford:OxfordUniversityEvans,J.St.B.T.,&Stanovich,K.E.(2013).Dualprocesstheoriesofhighercognition:Advancingthedebate.PerspectivesonPsychologicalScience,223Ð241.Fernbach,P.M.,Sloman,S.A.,Louis,R.S.,&Shube,J.N.(2013).ExplanationÞendsandfoes:Howmechanisticdetaildeterminesunderstandingandpreference.JournalofConsumer,1115Ð1131.Frederick,S.(2005).Cognitivereßectionanddecisionmaking.JournalofEconomicPerspec-,25Ð42.Gilhooly,K.J.,&Fioratou,E.(2009).Executivefunctionsininsightversusnon-insightprob-lemsolving:Anindividualdifferencesapproach.Thinking&Reasoning,355Ð376.Gilhooly,K.J.,&Murphy,P.(2005).Differentiatinginsightfromnon-insightproblems.ing&Reasoning,279Ð302.Johnson-Laird,P.N.(1983).Mentalmodels.Cambridge,MA:HarvardUniversityPress.Johnson-Laird,P.N.(1999).Deductivereasoning.AnnualReviewofPsychology,109Ð135.Jones,W.,Russell,D.,&Nickel,T.(1977).Beliefintheparanormalscale:Anobjectiveinstru-menttomeasurebeliefinmagicalphenomenaandcauses.JSASCatalogofSelectedDocu-mentsinPsychology(100),Ms.No.1577.Kahneman,D.(2011).Thinking,fastandslow.NewYork:Farrar,Straus&Giroux.Kirkpatrick,L.,&Epstein,S.(1992).Cognitive-experientialself-theoryandsubjectiveprobabil-ity:Evidencefortwoconceptualsystems.JournalofPersonalityandSocialPsychologyTOPLAK,WEST,STANOVICH Koehler,D.J.,&James,G.(2010).Probabilitymatchingandstrategyavailability.Memory&,667Ð676.Liberali,J.M.,Reyna,V.F.,Furlan,S.,Stein,L.M.,&Pardo,S.T.(2012).Individualdifferen-cesinnumeracyandcognitivereßection,withimplicationsforbiasesandfallaciesinproba-bilityjudgment.JournalofBehavioralDecisionMaking,361Ð381.Margolis,H.(1987).Patterns,thinking,andcognition.Chicago:UniversityofChicagoPress.Markovits,H.,&Nantel,G.(1989).Thebelief-biaseffectintheproductionandevaluationoflogicalconclusions.Memory&Cognition,11Ð17.Mata,A.,Ferreira,M.B.,&Sherman,S.J.(2013).Themetacognitiveadvantageofdeliberativethinkers:Adual-processperspectiveonoverconÞdence.JournalofPersonalityandSocial,353Ð373.Moritz,B.B.,Hill,A.V.,&Donohue,K.(2013).Individualdifferencesinthenewsvendorproblem:Behaviorandcognitivereßection.JournalofOperationsManagement,72Ð85.Oechssler,J.,Roider,A.,&Schmitz,P.W.(2009).Cognitiveabilitiesandbehavioralbiases.JournalofEconomicBehavior&Organization,147Ð152.Pennycook,G.,Cheyne,J.A.,Seli,P.,Koehler,D.J.,&Fugelsang,J.A.(2012).Analyticcogni-tivestylepredictsreligiousandparanormalbelief.,335Ð346.Shenhav,A.,Rand,D.G.,&Greene,J.D.(2012).Divineintuition:Cognitivestyleinßuencesbeliefingod.JournalofExperimentalPsychology:General,423Ð428.Simon,H.A.(1955).Abehavioralmodelofrationalchoice.TheQuarterlyJournalofEconom-,99Ð118.Simon,H.A.(1956).Rationalchoiceandthestructureoftheenvironment.Psychological,129Ð138.Stanovich,K.E.(1989).Implicitphilosophiesofmind-thedualismscaleanditsrelationtoreli-giosityandbeliefinextrasensoryperception.JournalofPsychology,5Ð23.Stanovich,K.E.(1999).Whoisrational?Studiesofindividualdifferencesinreasoning.Mahwah,NJ:Erlbaum.Stanovich,K.E.(2004).TherobotÕsrebellion:FindingmeaningintheageofDarwin.Chicago:UniversityofChicagoPress.Stanovich,K.E.(2009).WhatIntelligencetestsmiss:Thepsychologyofrationalthought.NewHaven,CT:YaleUniversityPress.Stanovich,K.E.(2011).Rationalityandthereßectivemind.NewYork:OxfordUniversityStanovich,K.E.,&West,R.F.(1997).Reasoningindependentlyofpriorbeliefandindividualdif-ferencesinactivelyopen-mindedthinking.JournalofEducationalPsychology,342Ð357.Stanovich,K.E.,&West,R.F.(1998a).Cognitiveabilityandvariationinselectiontaskperfor-Thinking&Reasoning,193Ð230.Stanovich,K.E.,&West,R.F.(1998b).IndividualdifferencesinframingandconjunctionThinking&Reasoning,289Ð317.Stanovich,K.E.,&West,R.F.(1998c).Individualdifferencesinrationalthought.JournalofExperimentalPsychology:General,161Ð188.Stanovich,K.E.,&West,R.F.(2000).Individualdifferencesinreasoning:Implicationsfortherationalitydebate?BehavioralandBrainSciences,645Ð726.Stanovich,K.E.,&West,R.F.(2007).Naturalmysidebiasisindependentofcognitiveability.Thinking&Reasoning,225Ð247.Stanovich,K.E.,&West,R.F.(2008).Ontherelativeindependenceofthinkingbiasesandcog-nitiveability.JournalofPersonalityandSocialPsychology,672Ð695.Stanovich,K.E.,West,R.F.,&Toplak,M.E.(2011).Intelligenceandrationality.InR.J.Sternberg&S.B.Kaufman(Eds.),CambridgeHandbookofIntelligence(pp.784Ð826).NewYork:CambridgeUniversityPress.COGNITIVEREFLECTIONTEST Strathman,A.,Gleicher,F.,Boninger,D.S.,&ScottEdwards,C.(1994).Theconsiderationoffutureconsequences:Weighingimmediateanddistantoutcomesofbehavior.JournalofPer-sonalityandSocialPsychology,742Ð752.Sunstein,C.R.(2013).Simpler:Thefutureofgovernment.NewYork:Simon&Schuster.Taylor,S.E.(1981).Theinterfaceofcognitiveandsocialpsychology.InJ.H.Harvey(Ed.),Cognition,socialbehavior,andtheenvironment(pp.189Ð211).Hillsdale,NJ:Erlbaum.Thaler,R.H.,&Sunstein,C.R.(2008).Nudge:Improvingdecisionsabouthealth,wealth,and.NewHaven,CT:YaleUniversityPress.Toplak,M.E.,&Stanovich,K.E.(2002).ThedomainspeciÞcityandgeneralityofdisjunctivereasoning:Searchingforageneralizablecriticalthinkingskill.JournalofEducationalPsy-,197Ð209.Toplak,M.E.&Stanovich,K.E.(2003).Associationsbetweenmysidebiasonaninformalrea-soningtaskandamountofpost-secondaryeducation.AppliedCognitivePsychologyToplak,M.E.,West,R.F.,&Stanovich,K.E.(2011).TheCognitiveReßectionTestasapre-dictorofperformanceonheuristicsandbiasestasks.Memory&Cognition,1275Ð1289.Tversky,A.,&Kahneman,D.(1974).Judgmentunderuncertainty:Heuristicsandbiases.,1124Ð1131.Tversky,A.,&Kahneman,D.(1981).Theframingofdecisionsandthepsychologyofchoice.,453Ð458.Wason,P.C.(1966).Reasoning.InB.Foss(Ed.),Newhorizonsinpsychology(pp.135Ð151).Harmonsworth,England:Penguin.Wechsler,D.(1999).WechslerAbbreviatedScaleofIntelligence(WASI).SanAntonio,TX:ThePsychologicalCorporation.West,R.F.,Meserve,R.J.,&Stanovich,K.E.(2012).Cognitivesophisticationdoesnotatten-uatethebiasblindspot.JournalofPersonalityandSocialPsychology,506Ð519.TOPLAK,WEST,STANOVICH