CHAPTERFIVETheDunning150KrugerEffectOnBeingIgnorantofOne146sOwnIgnoran - PDF document

1 CHAPTERFIVETheDunning–KrugerEffect:
CHAPTERFIVETheDunning–KrugerEffect:OnBeingIgnorantofOne’sOwnIgnoranceDavidDunning1.TwoAssertionsAboutIgnorance1.1.Ignoranceisprevalentineverydaylife1.2.Ignoranceisofteninvisibletothosetosufferfromit1.3.Overviewofchapter2.WhyIgnoranceisInvisible2.1.Ignoranceliesintherealmofunknownunknowns2.2.Ignoranceisdisguisedbydomain-specificmisbeliefs2.3.Ignoranceisdisguisedby“reach-around”knowledge2.4.Athresholdconditionforlackofrecognition3.TheDunning–KrugerEffect3.1.Definition3.2.Thedoubleburdenofincompetence3.3.Expertiseandmetacognitivejudgment3.4.Empiricaldemonstrations4.AlternativeAccounts4.1.Regressiontothemean4.2.Noiseplusbias4.3.Lackofincentives5.TheErrorsofTopandBottomPerformersCompared5.1.Counterfactualcomparisons5.2.Impactofsocialcomparisoninformation5.3.Theparadoxofgainingexpertise6.SourcesofSelf-evaluation6.1.Theissueofindirectindicators6.2.Theproblemof“rationalerrors”6.3.Theimpactofpreconceivednotionsofskill6.4.Preconceivednotions“versus”bottom-upexperience7.OutstandingIssues7.1.Individualdifferencesinmeta-ignorance7.2.Perseveranceinignorance AdvancesinExperimentalSocialPsychology,Volume442011ElsevierInc.ISSN0065-2601,DOI:10.1016/B978-0-12-385522-0.00005-6Allrightsreserved.DepartmentofPsychology,CornellUniversity,Ithaca,NewYork,USA 7.3.Boundaryconditionstotheinvisibilityofincompetence7.4.Canignorancebebliss?8.ConcludingRemarksInthischapter,Iprovideargumentandevidencethatthescopeofpeople’signoranceisofteninvisibletothem.This(orignoranceofignorance)arisesbecauselackofexpertiseandknowledgeoftenhidesintherealmofthe“unknownunknowns”orisdisguisedbyerroneousbeliefsandbackgroundknowledgethatonlyappeartobesufficienttoconcludearightanswer.Asempiricalevidenceofmeta-ignorance,IdescribetheDunning–Krugereffect,inwhichpoorperformersinmanysocialandintellectualdomainsseemlargelyunawareofjusthowdeficienttheirexpertiseis.Theirdeficitsleavethemwithadoubleburden—notonlydoestheirincompleteandmisguidedknowledgeleadthemtomakemistakesbutthoseexactsamedeficitsalsopreventthemfromrecognizingwhentheyaremakingmistakesandotherpeoplechoosingmorewisely.Idiscusstheoreticalcontroversiesovertheinterpretationofthiseffectanddescribehowtheself-e

2 valuationerrorsofpoorandtopperformersdif
valuationerrorsofpoorandtopperformersdiffer.Ialsoaddressavexingquestion:Ifself-percep-tionsofcompetencesooftenvaryfromthetruth,whatcuesarepeopleusingtodeterminewhethertheirconclusionsaresoundorfaulty?AllowmetobeginthischapterwithastipulationthatIhopewillnotbetoocontroversial.Thatstipulationisthatpeopleconducttheirdailyaffairsundertheshadowoftheirowninevitableignorance.Peoplesimplydonotknoweverythingabouteverything.Thereareholesintheirknowl-edge,gapsintheirexpertise.I,forexample,cannamemanyareasinwhichmyknowledgeisincomplete,ifitevenbeginsatall.Iamnotuponthelatestdevelopmentsinhydrostaticsandhydrauliccircuitrydesign.IdonotknowmuchaboutthehighlightsoftwentiethcenturyZimbabweansculpture.Iamnotyour“goto”guywhenitcomestogoodrestaurantsinDuOfcourse,onemightconcedetheinevitabilityofignorance,butarguethatmost—ifnotall—ofpeople’signorancecoversobscuretopicsthatcarrynoimplicationsfortheireverydaylives.Muchlikeantsfailtosufferbecausetheydonotknow,orevenconceiveof,suchtopicsasbebopjazzorquantummechanics,peoplemaynotsufferbecausethetopicstheyfailtoknowfallwellbeyondtheissuesthatactuallyinfluencetheiroutcomesinlife.Economists,forexample,havearguedthatmostignoranceisrational,inthatthereareseveraltopicsforwhichgainingexpertisewouldjustnotprovidethetangiblebenefittomakeitworthwhile(Downs,1957 DavidDunning 1.TwoAssertionsAboutIgnoranceButIbelievethisstancetowardignoranceismistaken.Instead,Iwishtomaketwoassertionsaboutpeople’sinevitableignorancethatmakesitaquiterelevantissuefortheirdailylives.Ofcourse,makingthoseassertionsconvincinglytakessomeargumentationand,moreimportantly,data.1.1.IgnoranceisprevalentineverydaylifeFirst,Iwishtoarguethattheboundarywherepeople’sknowledgeendsandtheirignorancebeginsfrequentlyarrivesfarsoonerthanonewouldexpect.Thatboundaryofteninsinuatesitselfwellwithinthegeographyofeverydaytasksthatdeterminewhetherpeoplelivehappyandeffectivelives—certainlywithinthecircleofchallengesthatpeopletypicallyfaceoverthecourseofalifetime.Forexample,incontemporarysociety,peoplemustfilteragooddealofnewsaboutscientificfactsonsuchimportantissuesastheenvironment,medicaltreatment,andbiotechnology.Inthatregard,theNationalScienceFoundation,initsbiannualsurv

3 eyofscientificknowledge,findslargegapsin
eyofscientificknowledge,findslargegapsinthebasicfactsofwhatpeopleknow.Inits2008surveyofroughly1500UnitedStatesadults,onlyabout53%ofrespondentsknewthatelectronsaresmallerthanatomsandonly51%couldsuccessfullyidentifythatitwastheearththatrevolvedaroundthesun(ratherthantheotherwayaround),takingayearfortheearthtodoit.Whenaskedwhetheritwasbettertotestanewhighbloodpressuredrugbygivingit(a)to1000participantsor(b)to500participants,withanadditional500receivingaplacebo,only38%gavethecorrectanswerwithanappropriaterationaleNationalScienceBoard,2010Butperhapsscienceisnotaday-to-dayactivityfortypicalcitizens,sotheycanbeexcusedfornothavingbasicknowledgeabouttopicstheymakenodirectdecisionsabout.Theydo,however,makedecisionsineveryelection;thus,itisimportantforcitizenstohaveabasicworkingknowledgeoftheirgovernment.Ina2009surveyofroughly2500Americancitizens,onlyhalfofrespondentscouldnameallthreebranchesoftheFederalgovernment,only54%knewthatthepowertodeclarewarrestswithCongressratherthanthePresident,andonly57%couldproperlyidentifytheroleplayedbytheelectoralcollege,withmanythinkingit“trainsthoseaspiringforhigheroffice”or“supervisedthefirsttelevisiondebates”IntercollegiateStudiesInstitute,2008Foraresponsetobecodedasaccurate,therespondentmustprovideanappropriaterationale.Manyrespondentsoptfortheplacebogroup,butdoso,forexample,tokeepthefatalityratedownifthedrugshouldprovedeadly.Thisisnotcodedasaccurate(Miller,1998 TheDunning-KrugerEffect Or,perhaps,peoplehavemoreexpertiseaboutdecisionsthatcarryspecificandconcreteconsequencesforthem,likesavingforretirement.Overrecentdecades,manycompaniesandinstitutionshavemovedfrom“definedbenefit”plans,inwhichthebenefitspeoplereceiveoncetheyretirearefixed,to“definedcontribution”plans,inwhichemployershandoverlumpsumsofmoneyfortheiremployeestoinvestastheyseefit.Fordefinedcontributionplanstobesuccessful,employeesmustbesavvyabouthowtoinvest.Withthisasbackground,studiesoffinancialliteracyprovidecauseforLusardiandMitchell(2009),forexample,presentedrespondentswiththefollowingtwoquestions,thefirstprobingpeople’sunderstandingofinterestratesandthesecondtheirunderstandingofinflation.Supposeyouhad$100inasavingsaccountandtheinterestratewas

4 2%peryear.After5years,howmuchdoyouthinky
2%peryear.After5years,howmuchdoyouthinkyouwouldhaveintheaccountifyouleftthemoneytogrow:morethan$102,exactly$102,lessthan$102.Imaginethattheinterestrateonyoursavingsaccountwas1%peryearandinflationwas2%peryear.After1year,wouldyoubeabletobuymore,exactlythesameas,orlessthantodaywiththemoneyinthisaccount?Only56%answeredbothquestionscorrectly.Inmattersofhealthliteracy,anareacertainlywithineverydayconcern,thepictureremainsthesame.TheInstituteofMedicinereportsthat90millionpeopleintheUnitedStates(withapopulationofjustover300million)havesubstantialdifficultyunderstandingandfollowinghealthinformation,thustakingdrugserraticallyorinwaysthatundercuttheireffectiveness(Nielsen-Bohlman,Panzer,&Kindig,2004).Inonespecificstudy,asthmasuffererswereaskedtodemonstratehowtouseaninhaler,withresearchersnotingwhetherrespondentsfollowedsixstepsessentialforinhalerstobeeffective(e.g.,didtherespondentshaketheinhalerbeforeusing,exhalebeforetakingapuff,waitatleast30sbetweenpuffs).Respondentsdidnotshowahighdegreeofcompetence,with48%ofthosereadingatahighschoolleveland89%ofthosereadingbelowathirdgradelevelfailingtofollowthreeormoreofthecrucialstepsidentifiedWilliams,Baker,Honig,Lee,&Nowlan,19981.2.IgnoranceisofteninvisibletothosetosufferfromitButitisthesecondassertionthatmaybemoreimportant,andtowhichthebulkofthischapterisdevoted.Thatassertionisthatpeoplearedestinednottoknowwherethesolidlandoftheirknowledgeendsandtheslipperyshoresoftheirignorancebegin.Inperhapsthecruelestirony,theonethingpeoplearemostlikelytobeignorantofistheextentoftheirownigno-rance—whereitstarts,whereitends,andallthespaceitfillsin-between.Thisisnotamatteroftrying.ItisreasonabletoassumethatpeoplearealotlikeMarcusTulliusCicero,theeminentRomanorator,whoonceadmitted DavidDunning thathewasnotashamedtoconfesshewasignorantofwhathedidnotknow.Thetrickisifonlyhe,andwe,couldfigureoutwhatthat“what”is.Inthediscussionthatfollows,Iwillarguethatitisnearlyimpossible,lefttoone’sowndevices,foronetosurmisewhatonedoesnotknow.Itisanintrinsicallydifficulttaskandonethatpeoplefailrepeatedly(Carter&Dunning,2008).Assuch,weshouldnotdemandofpeoplethattheyhavesomemagicalawarenessofallthattheydonotknow.Tobesure,peopleoccasionallycanidentifypocketsoftheir

5 ownincompetence,buttheyarefarfromperfect
ownincompetence,buttheyarefarfromperfectinidentifyingallofthem.Instead,theyoftenbelievetheyactwithadequateifnotexcellentexpertise,wheninsteadtheymisstepoutofmisun-derstandingandmiscalculationthattheyfailtorecognizeassuch.Theymaythinkthattheyaredoingjustfinewhentheyare,instead,doinganythingbut.1.3.OverviewofchapterInthischapter,Ibeginbydescribingwhyignorancesooftenslinksaroundinvisiblytothosewhosufferfromit,coveringanumberofissuesthatarisebecausepeopleactoutofaninevitableignorancethattheyarenotinapositiontorecognize.Ithenturntoaninstanceinwhichthispredicamentisitsmostvisibleandflamboyant—namely,theDunning–Krugereffect,inwhichpeoplesufferingthemostamongtheirpeersfromignoranceorincompetencefailtorecognizejusthowmuchtheysufferfromit.Idescribethephenomenon,reporttheempiricalevidenceforit,discussalternativetheoreticalaccountsforit,andlayoutsomeofitsmanyimplications.Ialsodiscussthetypesof“errors”madebytopperformers—thatis,thoseimbuedwithamplecompetenceandexpertise—andshowhowtheydifferfromthoseofpoorperformers.Next,Inotethatalthoughpeoplemayhavelittleinsightintotheirownignorance,theydogoaheadwithafirmsensethattheyareknowledgeableaboutcertaintopicsandtasks.Wheredotheseself-impressionsofskillcomefrom?Idiscussempiricalworkinmylabthathasdocumentedtwosourcesofpeople’sself-impressions—sourcesthatare,regrettably,notnecessarilytiedcloselytoactualskill.Finally,Iendbydiscussingtheopenissuesthatdeservefurtherempiricalstudy.Amongthosequestionsiswhetherpeopleultimatelylearnabouttheirdeficits?Andifnot,whynot? 2.WhyIgnoranceisInvisibleThecentralassertionofthischapteristhatpeople’signoranceisofteninvisibletothem—thattheysuffer,forlackofabetterterm,aremainingignorantofthemultitudeofwaystheydemonstrategapsinknowledge.Tobesure,peopleareoftensuccessfulinidentifyingafew TheDunning-KrugerEffect areaswheretheirexpertiseislacking,ortopicstheywishtheyknewmoreabout—butIwouldassertthatanyindividual’smentalcatalogueoftheirareasofignoranceislikelytobeveryincomplete.People’scatalogsarelikelytobeimperfectbecausemanyoftheirdeficitsarecamouflagedinoneoftwoways.First,manyinstancesofanignorancefallintothecategoryof.Second,manyinstancesofignorancemaybeobsc

6 uredbecausetheyarehiddenbehindmisbeliefs
uredbecausetheyarehiddenbehindmisbeliefsthatpeoplemistakeforvalidknowledgeinthedomaininquestion.Third,peoplemaybeabletoconstructresponsesongeneralworldknowledge,or“reach-around”knowledgeappearstoberelevantandreasonablewhenitreallyisnot.2.1.IgnoranceliesintherealmofunknownunknownsConsideranycomplexproject,whetheritbebuildingafunctionalbuilding,craftingawinninglegalargument,orprotectingone’scountryfromterror-istattack.Informationrelevanttothatprojectcanbebrokendownintothreedifferentcategories.First,thereareknownknowns,informationthatpeoplehaveandknowthattheyhave.Second,thereareknownunknownsinformationthatpeopledonothaveandknowthattheylack.Butmostimportanttoouranalysisofignoranceisathirdcategoryofinformation,unknownunknowns,informationthatisrelevanttotheprojectbutthatpeopledonotknowtheylack.Theseareconsiderationsthatthepersondoesnotevenconceiveof.Questionsthatpeopledonotknowenoughtoask.Thenotioncanrefertoanypieceofinformationthatliesoutsideaperson’sken.Itcanrefertopotentialproblemsorrisksthatthepersondoesnotanticipate,actionsthatareessentialtoattainsuccessthatthepersondoesnotknowabout,possiblemovesorstrategiesthatadecision-makermightmakeifonlythatdecision-makerknewoftheirexistence,contingenciesthatoneshouldprepareforifonewereforewarned,orevensolutionsthatadecision-makermightarriveatifonlytheycouldbeintuited.Itislikelythatpeoplearenotawareoftheirignorancebecausemuchofitisstashedintherealmofunknownunknowns.Ahypotheticalexamplemaymakethisclearer.Supposeacouplewerebringingtheirnewbornbabyhomeandknewthatnowwasthetimetochildprooftheirhouseagainstriskstotheirinfant.Aspartoftheirknownknowns,theymayknowthattheyhavetoplacegatesinfrontofstairwaysandbarriersaroundfireplaces.Aspartoftheirknownunknowns,theymayhavequestionsaboutotherpotentialprecautionsthattheyaresuspecttheymayhavetotake.Shouldthey,forexample,dosomethingabouttheirelectricaloutlets?(Yes,theyThecarefulreaderwillnoticeonelastcategoryofknowledgethathasbeenomitted—unknownknowns.Suchacategorylikelyexists,andonewouldprobablyhavetostartanytreatmentofitwiththephilosophyofZizek(2004),whoalignsitwithideology.Butthenotionofunknownknownsisatopicthatdeservesitsownfocuseddiscussion.Thus,itliesoutsidethescopeoftheprese

7 ntchapter. DavidDunning shouldcoverthemw
ntchapter. DavidDunning shouldcoverthemwithchild-resistantcoversbutnotplasticplugs,whichcanbepriedoff.)Andoncethoseknownunknownsareaddressed,thecouplebreathesasighofreliefandbringstheirinfanthome,confidentthattheyhavedoneanadequatejobandthatthehouseissafe.But,worryingly,beyondthecouple’srealmofawarenessmayliethatextensiveclassofunknownunknowns—precautionsthattheparentsshouldhavetakenbuthavenoconceptionof—suchasraisingthecordsofdrapesandmini-blindsothatthebabydoesnotaccidentallybecomestrangled,ormovingallhouseholdplantstowherethebabycannotreachthem,lesttheyturnintoapoisonoussnack.Thenotionofunknownunknownswasmadenotoriousin2002inapressconferencebytheUnitedStatesSecretaryofDefenseDonaldRums-feldwhenhenotedthathisdepartmentcarriedtheburdenofnotnecessarilyknowingalltheydidnotknowaboutterroristrisksfacingtheUnitedStatesKamen,2002),buttheconcepthasalonghistoryindesignandengineeringKossiakoff&Sweet,2003).Engineersaretaughttobevigilantagainstunknownunknowns,andtotestanysystemtheycreateagainstanycontin-gencytheycanthinkoftobestflushoutasmanyunknownunknownsaspossible.Architectsareaskedtocalculatetheamountofconcreteabuildingneedstoremainstable,andthenuseeighttimesthatamounttoguardagainstunknownunknowndangersthatwouldotherwiseidentifythem-selvesonlyafteritistoolate(Heath,Larrick,&Klayman,1998Thenotionofunknownunknownsliesalsoatthecenterofanemergingyetstillunconventionalstrainineconomicsanddecisiontheory(e.g.,Schipper,2010).Inthisarea,scholarsrecognizethatdecision-makersmaynotliveintheworldportrayedintraditionaleconomicanalysis,whererationalactorshavecompleteinformationofallpossiblecontingenciesandoutcomesthatmaybefallthem.Instead,actorsareleftunawareofpossiblestatesoftheworldthatmightobtain.Forexample,decision-makersmaybeaskedtoplayagameinwhichtheyhavetodiscoverforthemselveswhattheparametersofthegamereallyare(e.g.,Halpern&Rego,2006)ratherthanhavingthegameexplainedcompletelytothem.Giventheexistenceofunknownunknowns,itisnotsurprisingthatanaccumulatingbodyofevidencefromfar-flungcornersofpsychologyshowsthatpeopleseemtoknownothingaboutthegapsintheirknowledge.Forexample,readersoftenclaimtohavereachedadeepcomprehensionofanarrativepassageyetfailtorecognizethedirectcontradictionscon

8 tainedwithin(Epstein,Glenberg,&Bradley,1
tainedwithin(Epstein,Glenberg,&Bradley,1984;Glenberg,Wilkinson,&Epstein,1982).Theycanclaimtheyknowhowhelicopters,flushtoilets,andcylinderlockswork,buthavetobackoffthoseclaimsoncetheytakeastabexplaininghowthosegadgetswork(Rozenblit&Keil,2002).Theysimilarlyclaimtheycanexplaintheirfavoritepoliticalcandidate’spositiononanimportantsocialissuebutoftencannotdosowhenasked(Oppenheimer,&Zemla,2010 TheDunning-KrugerEffect Inourownwork,wehavefoundthatgraduatestudentspursuingdegreesinpsychologyfailtonoticeshortcomingsintheirknowledgeofresearchmethods.Viae-mail,wepresentedanationalsampleofgraduatestudentsataskinwhichtheyhadtocritiquethemethodsoffourseparatestudiesandthenself-evaluatehowwelltheyhaddone.Wevariedthenumberofmethodologicalflawswewoveintothosestudiestoseeifrespondentsgaveweighttothenumberofflawstheymissedintheirself-evaluationsofperformance.Theydidnot.Respondentsappearedtohavenomagicalawarenessof“unknownunknown”methodologicalflawsthatwereinthematerialstospotbutthattheyhadmissed.Indeed,informingthemoftheflaws,theyhadmissedcausedrespondentstosignificantlylowertheirself-ratingsontheirmethodologicalskills—except,interestingly,forskillsrelatedspecificallytotheirownresearch(Caputo&Dunning,2005,Study4).Unknownunknowngapsinknowledgemaygounrecognizedinevery-daylifebecausepeoplefailtohaveoutsideagentshoveringoverthem,pepperingthemwithexamsthatcouldimpolitelyexposeholesintheirknowledge.Studentsinmedicalschools,however,oftendohavesuchagentshoveringaroundthem,eagertoassessskillswithwell-honed,objec-tivelystructuredexercises.Inthesecircumstances,howoftendomedicalandnursingstudentsshowgapsinknowledgethattheyappeartoknownothingabout?Theanswerappearstobeoften.Barnsleyetal.(2004)askedstudentinternstoperformsevencommonclinicalprocedureswhilebeingwatchedbytheirtutors.Thetutorsgradedtheinternsalonganassessmentinstrumentthathadbeencarefullycraftedbyconsensusamongthehospi-tal’sexperienceddoctorsandnursestocontainstandardsindicatingthattheinternstillneededsupervisionontherelevantprocedureorwassocompetentthatheorshecouldnowteachittoothers.Theevaluationsoftheinternsandthetutorsdramaticallydisagreed.Alltheinterns,forexample,felttheyknewvenipuncturewellenoughtoteachothers,butonly10%

9 oftheirtutorsagreed—withnearly50%of
oftheirtutorsagreed—withnearly50%ofinternsjudgedasstillneedingsupervision.Onbladdercatheterizationofamalepatient,80%ofinternsthoughttheyknewtheprocedurewellenoughtoteach—butnoneoftheirtutorsconcurred,judgingthathalfoftheinternswerestillinneedofsupervision.Otherstudieshavediscoveredsimilarunknownunknowngapsinclinicalknowledge.Watts,Rush,andWright(2009)askedfirst-yearnursingstudentstocompleteanexerciseinwhichtheydressedawound.Thenursingstudentthenwatchedavideotapeoftheirperfor-mancealongwithaninstructor.Onaverage,studentssawroughlythreemistakesinwhichtheycouldhavecontaminatedthewound,buttheirinstructorsonaveragesawmorethansix.Studentswereknowledgeableabouthowmisuseofglovescouldhavecontaminatedthewound, DavidDunning catching92%oftheinstancesinwhichtheirinstructorssawanerror,butrecognizedonly15%oftheerrorscomingfrommishandlingofswabsand24%fromthehandlingofcleaningsolutions.Vnuk,Owen,andPlummerasked95first-yearmedicalstudentstocompleteaCPR(cardio-pulmonaryresuscitation)exerciseandthenaskedthemhowwelltheyhaddone.Onlythreefelttheyhad“failed”theexercise(whichtheyknewmeantthattheyhadmissedsteps,puttheminthewrongorder,executedthemincorrectly,ormovedtooslowly),butanexpertexaminerjudgedthatafull36hadfailed.Peoplealsodemonstrateunknownunknowngapsinthepossiblesolu-tionstheycangeneratetoproblems.Forexample,DeannaCaputoandI)presentedparticipantswithapopularwordpuzzlecalledBoggle,inwhichparticipantslookovera44arrayoflettersandtrytofindstringsoflettersthatformEnglishwords.AnexampleofaBogglearrayisgiveninFig.5.1,withthewordindicatedasitisfoundinthepuzzle.WeaskedparticipantstofindasmanywordsastheycouldinthreeBogglepuzzles,spending3minoneach,andthentoratehowwelltheyhadthoughttheyhaddone.Wevariedthespecificpuzzlesparticipantscon-fronted,sosomeparticipantsfacedpuzzleswithmanymoresolutionsthandidothers.Wewereinterestedinwhetherparticipantsdisplayedanyinsightintogapsintheirperformance.Wouldtheyhaveanadequateunderstandingofthesolutionstheyhadmissed?Theanswerwasaclearno,asindicatedinTable5.1,whichshowshowmuchweightparticipantsgavetosolutionsfoundandmissedintheirself-evaluations.However,onceexplicitlyinformedofthenumberofsolutionstheyhadmissed,participantswerequitewillingtogiveweightt

10 othatnumber(seeTable5.1 ABCDEFGHIJKLMNOP
othatnumber(seeTable5.1 ABCDEFGHIJKLMNOP Figure5.1Exampleofa44BogglepuzzlearraylikethoseusedinCaputo&Dunning(2005,Study1).Theletterscomprisingthewordknifearehighlighted.Ifyoucontinuereading,Iwillrevealthenumberofthree-letterplusEnglishwordscontainedintheBogglepuzzleinFig.5.1 TheDunning-KrugerEffect Thelackofweightgiventothenumberofsolutionsmissedismorecomprehensiblewhenotherstatisticsareconsidered.Participants’estimatesofhowmanysolutionstheymissedborenocorrelation(withthetruth.Moreimportantly,participantstended,onaverage,tothinktheyhadmissed18possiblesolutionswhen,infact,theymissed154.Notsurprisingly,participantsloweredtheirevaluationoftheirBoggleacumenafterseeingthelonglistofallthesolutionstheyhadshownnoawarenessofCaputo&Dunning,2005,Study1).Inafollow-upstudy,participantsbetlessmoneythattheycouldbeatanotherstudentinaBogglecompetitiononcetheirerrorsofomissionwerepointedouttothem(Study5).2.2.Ignoranceisdisguisedbydomain-specificmisbeliefsIgnoranceisalsohiddenbecauseitisoftenindisguise.Peoplemaybelievetheypossessaccurateknowledgeinadomainthatis,infact,misguidedandmisinformed.Forexample,despitealifetimeofinteractionswithobjects,peoplepossessanintuitivephysicsthatcontainsmanymistakesandmis-perceptionsabouthoweverydayobjectsmove(Proffitt,1999).Rollaballintothecoiledtubeandmanypeoplebelievethattheballwillrolloutofthetubeinacurvingtrajectory,whenit,infact,willjustgostraight(Jonides,&Alexander,1986).Peopledisplaycommonmisconceptionsabouthowsucheverydayitemsasmirrors(e.g.,Hecht,Bertamini,&Gamer,2005)andbicycles(Lawson,2006)work.Theyalsodisplayerrone-ousbeliefsabouthowemotionsworkinhumans(Wilson&Gilbert,2003;Woodzicka&LaFrance,2001 Table5.1Weightgiven()tosolutionsfoundversusmissedinself-evaluationsbeforeandafterparticipantswereinformedofallpossiblesolutionstotheBogglepuzzles(Caputo&Dunning,2005,Study1)SolutionsfoundSolutionsmissedSolutionsfoundSolutionsmissedThereare15EnglishwordsintheBogglepuzzle.Sothattheyallmayberemovedfromtherealmoftheunknownunknowns,theyareplonk,knife,mink,knop,jink,glop,fink,pol,nim,lop,kop,jin,ink,fin,andfie. DavidDunning Peoplealsodisplayremarkablemisbeliefsaboutsocialconditions,withpeoplewhoaremostwrongsometimesexpressingthegreatestconfidenceintheirbeliefs

11 .Forexample,inasurveyofopinionsaboutwelf
.Forexample,inasurveyofopinionsaboutwelfare,Kuk-linskiandcolleaguesfoundthatthemostconfidentrespondentsthoughtthat25%offamiliesreceivedwelfareintheUnitedStates(thefigureiscloserto7%)andthat80%ofthosereceivingwelfarewereAfrican-American(therealityislessthanhalf).Respondentswhothoughtthat15%ofthefederalbudgetwenttowelfarewerejustasconfidentasthosewhoexpressedthetruth(1%)(Kuklinski,Quirk,Jerit,Schwieder,&Rich,20002.3.Ignoranceisdisguisedby“reach-around”knowledgePeoplealsodisplaytheirignoranceinotherremarkablewaysthatprovideathirdexplanationastowhypeopleoftenfailtoknowwhattheyfailtoknow.Inshort,researchershavecaughtpeopleexpressingknowledgeabouttopicsthatresearchersknowwithcertaintypeoplecannotknowanythingabout.Whyaretheycertain?Theyarecertainbecausethesetopicsdonot2.3.1.Over-claimingIn2003,Paulhusandcolleaguesaskedrespondentstoratetheirknowl-edgein150differenttopics,rangingfromdoubleentendresTheDivineComedyandsoon.Sprinkledwithinthosetopicswere30thatweremerelytheinventionoftheexperimenters,suchasLaNeigeJaune,andesotericdeduction.Oftherealtopics,respondentsclaimedatleastsomeknowledgeof44%ofthem.Ofthenonexistenttopics,respondentsclaimedthesameforroughly25%ofthem.Paulhusandcolleaguesreferredtothistendencyasanddescribeditasaformofself-enhancementthatwasindependentofactualintellectualability.2.3.2.NonattitudesButPaulhus’sworkfollowedalongtraditioninsociologicalresearchshowingthatpeoplefrequentlyexpressopinionsaboutnonexistentsocialgroups(e.g.,theWallonians),politicalfigures,andgovernmentagenciesandpolicies(e.g.,theMetallicMetalsAct)(Bishop,Tuchfarber,&Oldendick,).Thesearealltopicsaboutwhichparticipants,bydefinition,cannothaveanyactualknowledge—butsubstantialnumbersofpeopleclaimenoughbackgroundtohaveformedanopinion.Forexample,11%ofrespondentswillprovideanopinionaboutafictitious“agriculturaltradeact”and14%ofa“monetarycontrolact”evenifgiventheexplicitoptionofsayingtheydonotknowwhattheactis.Ifanexplicit“don’tknow”responseoptioniswithheld,thepercentagesofferinganopinionrisetoaround36%foreach“act”(Bishopetal.,1986).Thistendencydoesnotseemtoarise TheDunning-KrugerEffect entirelyfrommeredeceitonthepartofrespondents(Bishop,Oldend

12 ick,Tuchbarber,&Bennett,1980),butratherf
ick,Tuchbarber,&Bennett,1980),butratherfromsomeotherprocess.2.3.3.Reach-aroundknowledgedefinedThisotherprocessappearstobeanimportantone,foritmayprovideyetanotherexplanationforwhypeopleclaimknowledgefortopicsaboutwhichtheyarereallyuninformedormisinformed.Theprocessisthatpeopletakecuesfromthesocialsituationtheyareinandtheirgeneralworldknowledgetocobbletogetherenoughapparentinformationtoformanimpression.Thatis,peoplereachbackoraroundtoanyknowledgetheyhavethatmightappeartoberelevant,andthenuseittoimposesomemeaningonthequestionstheyareaskedandthentoformajudgment.Thatis,theydonotusedomain-specificinformationtoinformtheirjudgments(howcouldthey,fornodomainexists),butinsteadusemoregeneralreach-aroundknowledge—thatseemslikeitmightberelevanttothetaskathand.Forexample,whenaskedaboutafictitiousAgriculturalTradeAct,surveyrespondentsfrequentlymakecommentsaboutissuesthatwereplausiblyrelevant(e.g.,“ShipmentsfromJapanarekillingourpro-ductshere”),ormaderesponsesconsistentwiththeirmoregeneralattitudestowardthegovernment(Schuman&Presser,1980ThisreachingbacktomoregeneralknowledgemightalsobebehindGraeff(2003)askedrespondentstheirimpressionsofcon-sumerproductsthatdidnotexist,suchasThompsondrillbits,Yamijitsustereoequipment,andBarjoletcheeses.Hefoundthatrespondentsweremorewillingtoclaimknowledgeforbrandsforwhichtherewasbroadknowledgethattheycouldreferto—suchasYamijitsustereos,forwhichrespondentscouldfallbacktotheirgeneralimpressionofJapanesestereoequipment,andBarjoletcheeses,inwhichtheycouldrelyonanygeneralknowledgetheyhadofFrenchcheeses.Suchgeneralknowledgewasnotavailableforotherbrands(suchasThompsondrillbits),andhefoundthatpeopleweremuchlesslikelytoclaimanyknowledgeinthosesituations.Thisreachingaroundbacktowardgeneralknowledgemayalsobebehindotherinstancesinwhichrespondentsadheretobeliefsevenifthosebeliefscomeunderpresumablydefinitivechallenge.Prasadandcol-leaguesidentifiedrespondentswhobelievedthatSaddamHusseinhadplayedaroleintheattacksof9/11andthenconfrontedthemwiththefactthatthefederalcommissionhadconcludedthathehadplayednorolePrasadetal.,2009).Ofthoseconfronted,10%directlyrefutedthecommis-sion’sconclusion,arguingfrommoregeneralknowledgeratherthananyspecificknowledgeaboutt

13 heeventsof9/11,makingassertionssuchas:&#
heeventsof9/11,makingassertionssuchas:“Ibelievehewasdefinitelyinvolvedwithitbecausehewasdefinitelypump-ingmoneyintotheterroristorganizationseverywayhecould.Andhewouldevensend$25,000tosomebodywhocommittedsuicidetokillanotherperson,totheirfamily”(Prasadetal.,p.153). DavidDunning 2.4.AthresholdconditionforlackofrecognitionThisabilitytoreachbacktogeneralknowledge,importantly,mayalsoprovideaboundaryconditionforwhenpeoplewillclaimknowledgetheydonothaveversusprofess(correctly)theirignorance.Ifthereisnodomain-specificbelieforgeneralreach-aroundknowledgetofallbackon,peoplemayrightlyclaimnoknowledgeoropinion.Veryfewpeople,forexample,wouldvolunteertostandinforacardiothoracicsurgeontoperformatriple-bypassonafriend,presumablybecausepeoplehavenobackgroundinfor-mation,nointellectualscaffolding,withwhichtheycanconstructamentalmodelabouthowtoproceed.ThesamepresumablyholdstrueforsuchesoterictopicsasbuildingarocketengineorrecitingtheIcelandicSagas.However,pubstheworldoverarefilledwithfootballfans(whetheritbetheAmerican,Canadian,European,orAustraliangame)whothinktheycandoabetterjobmanagingtheirfavoriteteamthanitsmanager,presum-ablybecauseanumberofinformalconversationsandargumentswithbarmatesovertheyearshasledthemtoconceiveofsomeintellectualscaffold-ingofthoughtsandintuitionsthatmayormaynotconstituteactualexpertise.Likewise,therearelikelymanyDIY(do-it-yourself)homerepairenthusiastswhohappilygoaboutrewiringtheelectricalcircuitsintheirhousebasedonwatchingtheirneighbordoitonceorseeingsomeprogramlastyearontheHomeandGardenchannel.Theymaysucceed,butsafetyexpertssuggesthiringaprofessionaltotakecareofelectrical,plumbing,orroofrepairs.TheissueisnotsomuchtheaddedcostofhavingthatexpertcorrectanymistakestheDIY’ermightmake.Rather,theissueisthenumberoftripstothehospitalemergencyroomthattheseepisodesofhomerepairsmightendin(Leamy&Weber,2009Thatis,thereisathresholdthathastobemetforpeopletomakeinappropriateclaimsofexpertise.Theyhavetohavesomefragmentsofinformation,enoughscaffoldingbasedondomain-specificorgeneralworldknowledge,toallowthemtocobbletogetheraplausibleresponse.Iftheycannotdothat,theywillnotmakeaninappropriateclaim.Thequestion,thus,ishowcommonlycanpeoplegatherenoughinformationorargumenttofe

14 elliketheyhavepassedthatthreshold? 3.The
elliketheyhavepassedthatthreshold? 3.TheDunning–KrugerEffectArguingthatignorancetendstobeinvisibleissomewhatdifficult,inthatpeoplelisteningtothecontentionhaveahardtimeresonatingwithit.Iftheytrytointrospectaboutanyunknownunknownsorinvisiblepocketsofignoranceintheirownlife,theywill,bydefinition,comeupempty—leavingthecontentiontofeelalittlealienordisputable.Butthereisa TheDunning-KrugerEffect manifestationoftheargumentthatisquitevisibleineverydaylifeandthatpeopledoresonatewith.Itisnotthemeta-ignorancetheywitnessinthemselves;rather,itisthemeta-ignorancetheywitnessinothers.3.1.DefinitionSpecifically,foranygivenskill,somepeoplehavemoreexpertiseandsomehaveless,someagooddealless.Whataboutthosepeoplewithlowlevelsofexpertise?Dotheyrecognizeit?Accordingtotheargumentpresentedhere,peoplewithsubstantialdeficitsintheirknowledgeorexpertiseshouldnotbeabletorecognizethosedeficits.Despitepotentiallymakingerroraftererror,theyshouldtendtothinktheyaredoingjustfine.Inshort,thosewhoareincompetent,forlackofabetterterm,shouldhavelittleinsightintotheirincompetence—anassertionthathascometobeknownastheDunning–Krugereffect(Kruger&Dunning,1999).Thisistheformofmeta-ignorancethatisvisibletopeopleineverydaylife.Thus,thecentralquestioniswhetherthepeopletheyspotreallydoremaininnocentoftheirowndeficitsevenwhenthosedeficitsarerelativelysevere.In1999,JustinKrugerandIdecidedtoexaminetheextenttowhichpoorperformersinknowledgedomainsrevealanyinsightaboutthedepthoftheirshortcomingsandlacklusterperformance.Ourstrategywastoaskparticipantstotaketestsassessingintellectualexpertiseinsuchdomainsaslogicalreasoningandgrammar,aswellastasksassessingsocialskill.Wethenaskedparticipantstoratehowwelltheythoughttheyweredoing.Overtheyears,wehavedonesointwodifferentways.First,wehaveaskedpartici-pantstoprovidecomparativeself-evaluations,ratinghowwelltheythinktheyaredoingrelativetotheirpeers.Second,wehaveaskedparticipantstoprovideself-evaluationsalongmore“absolute”scalesinvolvingnosocialcomparison,suchasestimatinghowmanyspecificquestionstheythinktheyaregettingrightonthetestpresentedtothem.Wouldpoorperformersunderstandhowbadlytheydid?Wepredictedthattheywouldnot.Muchliketheirmoreskilledpeers,theseindividualswou

15 ldselecttheanswersthatlookedthemostsensi
ldselecttheanswersthatlookedthemostsensibletothem—andsoattheendofthedaywouldthinkthattheiroverallperformancewasratherreasonable.Ofcourse,operatingfromincompleteandcorruptedknowl-edge,theywouldmakemanymistakesandnotrecognizethosemistakesastheymadethem.3.2.ThedoubleburdenofincompetenceInessence,weproposedthatwhenitcametojudgmentsofperformancebasedonknowledge,poorperformerswouldfaceadoubleburden.First,deficitsintheirexpertisewouldleadthemtomakemanymistakes.Second,thoseexactsamedeficitswouldleadthemtobeunabletorecognizewhen DavidDunning theyweremakingmistakesandwhenotherpeoplewerechoosingmorewisely.Asaconsequence,becausepoorperformerswerechoosingtheresponsesthattheythoughtwerethemostreasonable,thiswouldleadthemtothinktheyweredoingquitewellwhentheyweredoinganythingbut.Thisdouble-cursearisesbecause,inmanylifedomains,theactofthecorrectnessofone’s(oranyoneelse’s)responsedrawsupontheexactsameexpertisethatisnecessaryinthecorrectresponseinthefirstplace.Thatis,intheparlanceofpsychologicalresearch,theskillsneededtoexecutethetaskofjudgingtheaccuracyofaresponse(cf.Everson&Tobias,1998;Maki,Jonas,&Kallod,1994)arepreciselythesameasthosenecessarilyforthetaskofproducinganaccurateresponse.Needtojudgewhetherone(orsomeoneelse)haswrittenagrammaticallycorrectsentence?Thatactofjudgmentreliesonthesamesetofskillsastheactofformingagrammaticallycorrectsentenceinthefirstplace.Wanttoknowifonehasconstructedalogicallysoundargument?Thatactofevaluationdependsontheexactsameknow-howneededtoconstructasoundargument.Thus,ifpoorperformerssufferdeficitsinknowledgethatfailedthemwhenitcametimetoformcorrectresponses,thoseexactsamedeficitswouldsimilarlyfailthemwhenitcametimetojudgetheworthofthoseresponses.Theywouldnotknowwhentheirresponseswereincorrect;theywouldnotknowwhenothersformedbetterones.3.3.ExpertiseandmetacognitivejudgmentWeknewgoingintothisworkthatpreviousresearchsupportedouranalysis.Previousworkhasshownthatstrongandpoorperformersdifferintheirsuccessatthemetacognitivetaskofevaluatingtheirperformance.Whenpeopleareaskedtoevaluateresponsestoindividualtestitems,strongperformersanticipatebetterwhichindividualitemstheyarelikelytogetrightversuswrongthandopoorperformers.Thisdifferenceinmetacogni-tiveachiev

16 ementhasbeendiscoveredinawiderangeoftask
ementhasbeendiscoveredinawiderangeoftasks,suchasstudentstakinganexam(Shaughnessy,1979;Sinkavich,1995),readersindicatinghowwelltheycomprehendanarrativepassage(Maki&Berry,1984;Makietal.,1994),cliniciansmakingmentalillnessdiagnoses(1989;Levenberg,1975),bridgeplayersindicatingtheirbestversusworstmoves(Keren,1987),pharmacyschoolgraduatesseekinglicensure(Gregory,&Galli,2008),physicsexpertsknowingwhichproblemswillbemoredifficult(Chi,Glaser,&Rees,1982),andtennisplayersknowingwhichshotsaremorelikelytobewinners(McPherson&Thomas,1989).Ineachcase,thejudgmentsofstrongperformersaboutwhichindividualresponseswouldmeetwithsuccessversusfailureweremoreaccuratethan TheDunning-KrugerEffect thejudgmentsoftheirlesscompetentpeers(althoughseeGlenberg&Epstein,1987;Wagenaar&Keren,1985,fornullresults).3.4.EmpiricaldemonstrationsButhowwouldthisdifferencebetweenstrongandpoorperformerstrans-latefromjudgmentsofindividualitemstoevaluationsofoverallperfor-Figure5.2showstheresultsofonesuchstudyexaminingwhetherpoorperformersshowanyinsightintotheweaknessoftheirperformance.Inthisparticularstudy,141studentswhohadjustcompletedanexaminoneoftheircollegecourseswereaskedtoevaluatetheir“masteryofcoursematerial”aswellastheirperformanceonthespecificexamtheyhadjustcompleted.Participantsestimatedtheirperformancesalongpercentilescales;thatis,theyestimatedthepercentageofotherstudentsinthecoursetheythoughttheyhadoutperformed.Theyalsogaveuspermissiontoretrievetheiractualexamscore,sothatwecouldcomparetheirperceptionoftheirperformanceagainsttherealityofit(Dunning,Johnson,Ehrlinger,&Kruger,2003Fig.5.2shows,therearemanyobservationsonecanmakeabouthowwellperceivedperformancetracksactualperformance.Inthefigure,weusedparticipants’objectiveperformancetodividethemintofourgroups—frombottomquartileperformersuptotopquartileperformers.Ascanbeseeninfigure,threemainfindingsemerge(actuallyfour,butwewillwithholddiscussionofthefourthuntillater).First,whetheroneistalkingaboutmasteryofcoursematerialorperformanceonthetest,respondentstendedtothinkoftheirperformance,onaverage,asanythingbutaverage.Respon-dentsinallfourperformancegroupstendedtothinktheyscoredabovethe50thpercentile,orrathertheaverageoftheclass.Overall,participantsthoug

17 httheirmasteryofcoursemateriallayinthe70
httheirmasteryofcoursemateriallayinthe70thpercentileandtheirtestperformanceinthe68th—wellabovethatwhichisstatisticallypossible.Whenaskedtoestimatetheirrawscore,theyoverestimatedonaverageby3points—perceivingascoreof37versusarealityofachieving34(Thesefindingsarenotnews.Peopletypicallytendtoholdoverlyinflatedviewsoftheircompetenceandperformance—thinkingonaveragethattheyareoutperformingtheirpeerswhenitisstatisticallyimpossibleforagrouptopost,onaverage,“above-average”performances(seeAlicke&Govorun,2005;Dunning,2005;Dunning,Heath,&Suls,2004;Dunning,Meyerowitz,&Holzberg,1989;Weinstein,1980Zenger(1992),forexample,foundinasurveyofseveralhundredengineersintwocompaniesthat32%inonecompanyand42%intheotherthoughttheirskillputtheminthetop5%ofperformersinthatcompany—astatisticallyabsurdresult.Thatbiasaside,therewasastatisticallyobservablerelationbetweenperceivedandactualperformance(0.47and0.60,0.01,forper-centileandrawscoreestimates).Peoplewhodidpoorlyontheexam DavidDunning intuitedthattheyweredoingworseontheexamthanthosewhodidwell.Thatsaid,althoughitwasstatisticallysignificant,therelationwasquiteshallow—withbottomperformers,onaverage,thinkingtheyperformedroughly15–20percentilepoints(8rawscorepoints)worsethantopperformers’self-judgments.Thisfindingalsoreplicatesaraftofpastwork, PercentileObjective Performance Quartile Actual Perceived Mastery Perceived Performance Raw scoreObjective performance quartile Actual Perceived ThirdSecond Figure5.2Perceivedperformanceasafunctionofobjectiveperformanceonaclassexam.Toppanelpresentspercentileratingsforperceivedmasteryofcoursematerialandperformanceontheexam.Thebottompanelpresentsperceivedrawscoreontheexam(outof45points).FromDunningetal.(2003),pp.83–86,bySAGEPublications.Adaptedwithpermission. TheDunning-KrugerEffect showingthattheretendstobeastatisticallysignificant,albeitonlyweaktomeager,relationbetweenwhatpeoplebelieveabouttheirskillandtherealityasrevealedbyactualperformance(forreviews,seeDunning,2005;Dunningetal.,2004;Mabe&West,1982)—whetherintheclassroomCamerer&Hogarth,1999,1982),workplace(Harris&Schaubroeck,1988;Stajkovic&Luchins,1998),orthedoctor’soffice(Davisetal.,2006Allthisleadstothethi

18 rd—andmostcentral—finding,that
rd—andmostcentral—finding,thatpeopleinthebottom25%ofperformers,whoseactualperformanceliesinthe12thpercentile,thoughtthattheirmasteryofcoursematerialandtestperfor-mancelayclosertothe60thpercentile—amisjudgmentofover45percen-tilepoints.Ifwelookattheirrawscoreestimates,wefindthatpeopleatthebottomoverestimatedtheirrawperformanceonthetestbynearly30%Dunningetal.,2003Wehaveobservedthispatternofdramaticoverestimationbybottomperformersacrossawiderangeoftasksinthelab—fromtestsoflogicalreasoningandgrammarskills(Kruger&Dunning,1999)tomoresocialabilitieslikeemotionalintelligence(Sheldon,Ames,&Dunning,2010)anddiscerningwhichjokesarefunny(Kruger&Dunning,1999).Weandothershavealsoobservedsimilaroverestimationinrealworldsettingsaspeopletackleeverydaytasks,suchashunterstakingaquizonfirearmuseandsafety,basedononecreatedbytheNationalRifleAssociation,ataTrapandSkeetcompetition(Ehrlinger,Johnson,Banner,Dunning,&Kruger,),andlaboratorytechnicianstakinganexamaboutmedicallabproce-duresandknowledge(Haun,Zerinque,Leach,&Foley,2000).Inallcases,toptobottomperformersprovideself-evaluationsalongpercentilescalesthatlargelyreplicate(Fig.5.2Similardatahavebeenobservedinrealworldsettingsonmeasuresotherthanpercentilerankings.Amongstudentstakingpartinaregionalcollegiatedebatetournament,thoseamongthebottomquartileduringpreliminaryroundsdramaticallyoverestimatedthelikelihoodthattheywerewinningtheirmatches.Theythoughtthattheywerewinningnearly60%oftheirmatches,whentheyactuallywononly22%ofthem(Ehrlingeretal.,2008Study2).Ofindividualsenteringchesstournaments,peoplewhopossesslessskill,asindicatedbytheirElorating,mispredictedtheirtournamentperfor-mancemorethanthosewithgreaterskill,irrespectiveofpreviousexperi-encewithtournamentchess(Park&Santos-Pinto,2010).NovicedriversintheNetherlandsandFinlandwhofailedtheirfirstdriver’stestoverestimatedhowtheirexaminerswouldratethemtoagreaterdegreethandidthosewhopassedthetest(Mynttinsenetal.,2009).OfinternationalpharmacyschoolgraduatesseekinglicensureinCanada,thoseperforminginthebottom25%providedoverlyinflatedviewsofhowwelltheyperformedrelativetotheirpeers(Austinetal.,2008).Medicalstudentsreceivingthelowestgrades(D)amongtheirpeersinclinicalclerkshipsinobstetricsa

19 ndgynecologyoverestimatedtheirgradesbytw
ndgynecologyoverestimatedtheirgradesbytwofullgrades,thinkingon DavidDunning averagetheywouldreceiveaB.Theself-underestimatesofstudentsreceivinganAweretrivialbycomparison(Edwards,Kellner,Sistrom,&Magyari,2003).TheworstperformersamongmedicalstudentsconductinganexerciseinwhichtheyinterviewedaparentwhomaybeabusingherchildratedthemselvesmuchmorepositivelythantheirinstructorsdidHodges,Regehr,&Martin,2001).Asurveyacross34countriesofthemathskillsof15-year-oldsdiscoveredthathighermathperformancewasassociatedwithmoreaccurateself-perceptionsofmathskill(Chiu&Klassen,2010 4.AlternativeAccountsMycolleaguesandIhavelaidblameforthislackofself-insightamongpoorperformersonadouble-curse—theirdeficitsinexpertisecausethemnotonlytomakeerrorsbutalsoleavethemunabletorecognizetheflawsintheirreasoning.However,overthepastdecade,afewpsychologistshavesuggestedalternativeaccountsforthedatawehaveobservedinFig.5.2elsewhere.Seeingsuchcritiquesofourworkhassharpenedourthinkingandledustocollectdatatestingouraccountmorediscerningly.But,perhapsunknowntoourcritics,theseresponsestoourworkhavealsofurnishedusmomentsofdeliciousirony,inthateachcritiquemakesthebasicclaimthatouraccountofthedatadisplaysanincompetencethatwesomehowwereignorantof.Thus,attheveryleast,wecantakethepresenceofsomanycritiquestobeprimafacieevidenceforboththephenomenonandtheoreticalaccountwemadeofit,whoeverturnsouttoberight.Thatsaid,themajorcritiquesalldeservediscussion.4.1.RegressiontothemeanThemostcommoncritiqueofourmetacognitiveaccountoflackofself-insightintoignorancecentersonthestatisticalnotionofregressiontothemean.Recallfromelementarystatisticsclassesthatnotwovariablesareeverperfectlycorrelatedwithoneanother.Thismeansthatifoneselectsthepoorestperformersalongonevariable,onewillseethattheirscoresonthesecondvariablewillnotbesoextreme.Similarly,ifoneselectsthebestperformersalongavariable,oneisguaranteedtoseethattheirscoresonthesecondvariablewillbelower.Stirthatobservationwiththewell-knownfactthatpeopletendtoratethemselvesasaboveaverage,andonegetsthegraphdisplayedinFig.5.2Krueger&Mueller,2002Thereareactuallytwodifferentversionsofthis“regressioneffect”accountofourdata.SomescholarsobservethatFig.5.2lookslikearegressioneffect,andthenclaim

20 thatthisconstitutesacompleteexplanation
thatthisconstitutesacompleteexplanation TheDunning-KrugerEffect fortheDunning–Krugerphenomenon.Whatthesecriticsmiss,however,isthatjustdismissingtheDunning–Krugereffectasaregressioneffectisnotsomuchexplainingthephenomenonasitismerelyrelabelingit.Whatonehastodoistogofurthertoelucidatewhyperceptionandrealityofperformanceareassociatedsoimperfectly.Whyistherelationsoregressive?Whatdrivessuchadisconnectfortopandbottomperformersbetweenwhattheythinktheyhaveachievedandwhattheyactuallyhave?Thesecondversionofthisregressionaccountgoesfurthertoexplainwhytherelationbetweenperceivedandactualperformanceissoregressive.Onefactorthatmaypromptsuchanimperfectperception/realitycorrelationismeasurementerror.Wheneverestimatingsomeone’slevelofexpertise,therearealwayserrorsintheestimate;sometimespeoplegetluckyandpostaperformancethatoverstatestheirtruelevelofknow-how;sometimestheygetunluckyandposttoolowaperformance.Thiserror,orratherlackofmeasurementreliability,maycauseperformancetobecomeuntetheredtoperceptionsofexpertise,andthuscauseFig.5.2Krueger&Mueller,2002Fortunately,therearewaystoestimatethedegreeofmeasurementunreliabilityandthencorrectforit.Onecanthenassesswhattherelationisbetweenperceptionandrealityonceunreliabilityinmeasuringactualperformancehasbeeneliminated.SeeFig.5.3,whichdisplaysstudents’estimatesofexamperformance,inbothpercentileandrawterms,foradifferentcollegeclass(Ehrlingeretal.,2008,Study1).Ascanbeseeninthefigure,correctingformeasurementunreliabilityhasonlyanegligibleimpactonthedegreetowhichbottomperformersoverestimatetheirper-formance(seealsoKruger&Dunning,2002).Thephenomenonremainslargelyintact.4.2.NoiseplusbiasBurson,Larrick,andKlayman(2006)extendedtheregressionaccounttoconstructa“noiseplusbias”explanationfortheDunning–Krugereffect.Theyacceptedthepresenceofregressioneffectsandsuggestedthatpeople’spercentileratingsoftheirperformancecouldbepushedupordowndependingonhoweasyordifficulttheyperceivedtheoveralltasktobe.Fortasksperceivedtobeeasy,mostparticipantswouldratetheirperfor-mancehigh—thusproducingthetypicalDunning–Krugereffectoflowperformersgrosslyoverestimatingtheirperformance.However,fortasksperceivedasdifficult,peoplewouldratetheirp

21 erformancesmuchmorenegatively,causinglow
erformancesmuchmorenegatively,causinglowperformerstoratetheirperformancelow—andaccurately—whereashighperformerswouldalsoratetheirperformancemorenegatively—andthusprovideundulyunfavorableratingsoftheirperformance.Inasense,thiswould“flip”thetypicalDunning–Krugereffect,withhighperformersnowbeingtheonesgrosslymisestimatingtheirachievements. DavidDunning Inthreestudies,Bursonetal.(2006)gaveparticipantseasyversusdifficulttasksandobtaineddatathatwerelargelysupportiveoftheiranalysis.However,twomainissuespreventtheiranalysisfrombeingamoreplausi-ble—andaccurate—accountoftheDunning–Krugereffect.First,insteadofusingabroadrangeoftasks,Bursonetal.focusedonperformanceontriviaquestions(andinalaststudyawordprospectorpuzzle).AsBursonetal.themselvesnoted,participantsmaynotcommonlyhavehadenoughintel-lectualscaffoldingtobelievethattheiranswerswerereasonableones—animportantpreconditionfortheDunning–Krugereffecttoemerge(asnotedaboveandinKruger&Dunning,1999Tobesure,Bursonetal.(2006)worriedaboutthisissueandshowedthattheirparticipantsperformedabovechancelevels,butitstillcouldhaveleftparticipantswithmanyexperiencesinwhichtherewerequestionstheyknewtheycouldnotanswer.Consistentwiththisinterpretation,partici-pantstendedtoratethemselvesasbelowaverageacrossalltasksBursonetal.presentedtothem—afindingthatisquiteatypicalrelativetootherresearchinthisarea(forreviews,seeDunning,2005;Dunningetal.,2004).Abettersetoftaskswouldhavebeenthetypesofproblem-solvingtasks(e.g.,logical PercentileActual percentile Raw scoreActual raw score3050709024.52831.535 Original Corrected Actual Original Corrected Actual Figure5.3Relationshipbetweenperceivedandactualperformanceonacourseexambeforeandaftercorrectingformeasurementunreliability.Thetoppaneldisplayspercentileestimatesofperformanceasafunctionofactualpercentile.Thebottompaneldisplaysperceivedrawscoreasafunctionofactualrawscore(outof40points).Ehrlingeretal.(2008),byElsevierPublications.Adaptedwithpermission. TheDunning-KrugerEffect reasoning)thatKrugerandDunning(1999)presentedtotheirpartici-pants—andwhichtheworldtendstopresenttoitsinhabitantsonaday-to-daybasis.Second,thequarrelthatBursonetal.(2006)hadwiththeDunning&

22 #150;Krugereffectcenteredontheuseofperce
#150;Krugereffectcenteredontheuseofpercentilescales,suggestingthatpeopletypicallydidnotknowhowtotranslateadifficultoreasyexperiencetoapercentileevaluation.Thatis,itwasabitproblematicforparticipantstoassesshowwelltheyweredoingrelativetotheirpeerswithoutreallyknowinghowwellthosepeersweredoing.Consequently,participants’percentileestimateswere“biased”byperceptionsofoveralltaskdifficultyorease.Thisanalysissuggeststhatpeoplewouldgettheirrawscoresonanytestright.Wheretheyhaveproblemsiswithtranslatingarawscoreintoa“social”scoreinwhichtheycomparetheirachievementtothoseoftheirpeers.Thus,thisreasoningwouldpredictthatpeoplewouldshowlittleDunning–Krugereffectiftheyratedthemselvesalongobjectivescalesratherthansocialorcomparativeones.However,wehavefoundthattheDun-ning–Krugereffectarisesevenonestimatesalongmoreobjectivescales,asenumeratedabove.Bottomperformersreliability,anddramatically,overes-timatetheirperformancesevenonassessmentsthatrequirenosocialcom-parison.Topperformers,incontrast,tendtoneitherover-orunderestimatehowwelltheyaredoingontheseobjectivemeasures—aspredictedbytheoriginalmetacognitiveanalysisoftheDunning–Krugereffect.Tobesure,theirestimatesarenotperfect,buttheydonotshowthetypeofoverwhelm-ingbiasfoundintheestimatesofpoorperformers(Ehrlingeretal.,20084.3.LackofincentivesAlamentthatoneoftenhearsfromeconomictheoristsisthatpsychologiststypicallyprovidenoincentivestoparticipantstoreachcareful,serious,oraccuratejudgments(e.g.,Ariely&Norton,2007;Camerer&Hogarth,1999;Hertwig&Ortmann,2001).Asaconsequence,participantsmayprovidesloppyestimates,orhavethoseestimatescontaminatedbymotivessuchaslookinggoodintheeyesoftheexperimenter.Thus,theinflatedself-ratingsthatpoorperformingparticipantsprovidemayfailtoreflectwhatparticipantsreallythinkaboutthemselves.Poorperformersmayactuallyhaveampleinsightintotheinferiorqualityoftheirperformance;theyjustdonotwanttoadmitit,eithertothemselvesortotheexperimenter.4.3.1.MoneyWehavefound,however,thatprovidingampleincentivesforaccurateself-judgmentsdoesnothingtoenhancethetruthfulnessofpeople’sassessmentsoftheircompetence.Inoneexamplestudy,webroughtparticipantsintotakeapopquizonlogicalreasoning.Roughlyhalfofthepart

23 icipantsassessedtheirperformanceafterbei
icipantsassessedtheirperformanceafterbeingtoldthattheywouldbepaid$30iftheir DavidDunning estimateoftheirrawscoreonthetestcamewithin5%oftheirtruescore;$100iftheirestimatewasexactlyright.Nosuchincentiveswerementionedfortheremainingparticipants.AsseeninFig.5.4,comparingtheaccuracyofbothgroupsrevealednoenhancementinaccuracyfortheincentivegroupEhrlingeretal.,2008,Study4).(Andnoonewonthe$100.)4.3.2.AccountabilityIncreasingparticipants’accountabilityfortheirself-ratingsisawayinwhichasocialincentivecanbeadded.Specifically,askingparticipantstojustifytheirresponsestoanauthorityhasbeenshowntocausepeopletomakemorecarefulandconsideredjudgments(Lerner&Tetlock,1999)thatthey 0102060709010010Perceived percentileActual percentile 02461014168Perceived raw scoreActual raw score30507090101214 Control Incentive Actual Control Incentive Actual Figure5.4Relationshipbetweenperceivedandactualperformanceonacourseexamwithandwithoutfinancialincentivesforaccuracy.Thetoppaneldisplayspercentileestimatesofperformanceasafunctionofactualpercentileandfinancialincentive.Thebottompaneldisplaysperceivedrawscoreasafunctionofactualrawscore(outof20points)andfinancialincentive.FromEhrlingeretal.(2008),byElsevierPublications.Adaptedwithpermission. TheDunning-KrugerEffect imbuewithlessoverconfidence(Tetlock&Kim,1987).Thus,inastudyinvolvingalogicalreasoningtest,wetoldroughlyhalftheparticipantsthatasupervisingprofessorwouldinterviewthemforupto10minabouttheiranswerstothetest.Thismanipulationdidnothingtoimprovetheaccuracyofparticipants’viewsabouthowwelltheyhaddoneonthetest,withpoorperformersstillgrosslyoverestimatinghowwelltheyhaddone(etal.,2008,Study5).4.3.3.BehavioralchoicesFinally,thechoicesofpoorperformersrevealthattheydobelievetheiroverlyoptimisticassessmentsofachievement.Ferraro(2010)offeredstu-dentssome“insurance”abouttheirfinalexamperformancebeforetheexambegan.Forthepriceof10exampoints,participantscouldpurchaseinsur-ancethatwouldadd20pointstotheirexamscoreiftheirfinalexamscorefellwithinthebottom50%.For4exampoints,participantscouldinsteadbuya8-pointbumpintheirexamscoreshouldtheyfallbetweenthe50thand75thperformancepercentile.Aquickanalysissuggeststhatthefirstinsurancecontractshouldbemorepopularthant

24 hesecond,inthattwiceasmanypeoplewouldbee
hesecond,inthattwiceasmanypeoplewouldbeeligibletoprofitfromit—withtheprofitbeingmuchlarger(10points)thanthealternative(4points).However,Ferrarofoundthattwiceasmanystudentsboughtthesecondcontractthanthefirst.Andofthosestudentsbuyingthesecondcontract,over80%fellbelowthe50thpercentileintheirperfor-mance.Inessence,theyboughtinsurancethinkingthattheywouldplacesomewhatabovethecourseaverageintheirperformance,buttheiractualperformancefailedtoreachthatmark. 5.TheErrorsofTopandBottomPerformersComparedAdiscerningreaderismostlikelyalreadytohavediscoveredthefourthfindinginherentinFig.5.2.Itisnotonlypoorperformerswhomisestimatehowwelltheydo.Topperformers,aswell,tendtounderestimatetheirperformances—afindingwehavereplicatedacrossmanysettings.How-ever,ourdatasuggestthatthesemisjudgmentscomefromadifferentsourcefromthemisjudgmentsofpoorperformers.Essentially,bottomperformersoverestimatetheirproficiencybecausetheirintellectualdeficitsdeprivethemoftheresourcesnecessarytorecog-nizethattheyarechoosingincorrectly.Theymakethemistakeofthinkingthatalltheirchoicesareatleastreasonable,oratleastthemostreasonabletheycandetect.Theproblemfortopperformersisdifferent.Theyhaveampleresourcestoknowwhentheyaremostlikelytoberightorwrongin DavidDunning theirchoices.Theygetthemselvesright.Whattheygetwrongisotherpeople.Becausecorrectanswerscomerelativelyeasytothem,theymistak-enlybelievethatotherpeoplemustbecomingtothesamecorrectchoices.Asaconsequence,theirownperformances,albeitgood,arenotthatspecialrelativetohowwelltheythinkotherpeoplearedoing.Inaphrase,topperformerssufferfromafalseconsensuseffect(Greene,&House,1977),thinkingthatotherpeoplearerespondingsimi-larlytothemselvesmuchmorethanotherpeoplereallyare.Thisassertionisconsistentwithpastworkontheattributionofknowledge,whichhasshownthatpeople,onceprivytoknowledge,tendtooverestimatehowmuchotherpeoplepossessthesameknowledge(Fischhoff,1975,1977;Fussell&Krauss,1991,1992;Nickerson,Baddeley,&Freeman,1987).ItisalsoconsistentwithonepatternofdatawehaveobservedacrossthenumerousstudieswehaveconductedontheDunning–Krugereffect.Topperformersconsistentlyunderestimatehowwelltheyperformonpercentilescales—inessence,underestimatinghowwelltheyaredoingrelativetotheirpe

25 ers.However,onobjectiveorabsolutescales(
ers.However,onobjectiveorabsolutescales(e.g.,howmanytestitemsansweredcorrectly),weseenoconsistentevidenceofunderestimationoroverestimation(Ehrlingeretal.,20085.1.CounterfactualcomparisonsWehavealsoconductedstatisticalanalysesshowingthattheerrorsoftopandbottomperformerscomefromdifferentsources,focusingonthosepercentileestimatesthatparticipantsprovidewhentheyjudgehowwelltheyaredoingrelativetotheirpeers.Whenwegiveparticipantsatest,whatleadsthemtotheirbestguessabouthowmanyoftheirpeerstheyhaveoutperformed?Tobegintheanalysis,weexaminedhowparticipants,acrossseveralofourstudies,combinedtwootherestimateswehadaskedfortogettotheiroverallpercentileestimate.Thosetwounderlyingestimatesweretheirperceptionsoftherawtestscoretheythoughttheyhadachievedandtherawscoretheybelievedtheaverageparticipanthadobtained.Notsurprisingly,wefoundthatparticipantstendedtoprovidehigherpercentileratingstotheextenttheythoughttheirunderlyingrawscorewashigh,andalsototheextentthattheythoughttheaverageparticipanthaddonepoorlyEhrlingeretal.,2008).Conductingregressionanalysesallowedustogaugetheexactweightparticipantsgavetobothtypesofunderlyingestimateswhenevaluatingtheirperformanceinpercentileterms.Wethenaskeda“whatif”question:Whatifparticipantsactuallyknewthetruthofhowwell(orhowpoorly)theyortheaveragepersonhadobjectivelydoneonthetestinthatstudy:howmuchmoreaccuratewouldtheirself-judgmentonthepercentilemeasurehavebeen?Thatis,knowinghowmuchweightparticipantsgavetotheirownversustheaverageperson’sscoresintheirpercentileestimates,wecouldestimatehowmuch TheDunning-KrugerEffect participants’percentileself-estimateswouldchangeifwereplacedtheirsubjectiveguessabouttheirown(ortheaverageperson’sscore)withthetruth.Thisstatisticalapproach,borrowedfromsociology,essentiallyaskswhatparticipants’self-judgmentswouldhavebeeniftheylivedinacoun-terfactualworldinwhichtheyaccuratelykneweithertheirownobjectiveperformanceorthatoftheaveragepeer.Suchatechnique,calledtualregressionanalysis,iscommonlyusedtoaddresssuchquestionsashowmuchaperson’sIQwouldhaveincreasedhadheorshehadhypotheticallystayedinhighschoolforonemoreyear(Winship&Korenman,1997;Winship&Morgan,2000Ourstatisticalexplorationshowedthattheself-ratingerro

26 rsofbottomperformersdifferedintheirsourc
rsofbottomperformersdifferedintheirsourcefromthoseoftopperformers.Weknewatthestartthatbottomperformersgrosslyoverestimatedtheirowntestscoreperformance.Thus,itwasnosurprisethatcounterfactuallycorrectingforthisoverestimationledtosignificantlymoreaccuratepercentileself-ratings,asseeninFig.5.5.Bottom-performingparticipants,whoover-estimatedtheirperformanceby45percentilepointsintheoriginaldata,wouldhaveoverestimatedtheirperformancebyonly15pointshadtheyknowntheirtrueobjectivescore.Interestingly,bottomperformersalsotendedtooverestimatehowwelltheirpeers,onaverage,haddone.Thus,correctingforthissocialerroraloneledtoincreasederrorinhowbottomperformerswouldhaveratedthemselves—fromanoverestimateof45percentilepointstooneof50points(Ehrlingeretal.,2008Asimilaranalysisfortopperformersproducedadifferentsetofconclu-sions,alsoasseeninFig.5.5.Correctingfortopperformers’misestimatesoftheirownobjectiveperformancewouldhaveimprovedtheaccuracytheirpercentileratingsfromanunderestimateof14percentilepointsto9points.However,unlikebottomperformers,correctingtopperformers’beliefsabouttheirpeers(theytendedtooverestimatehowwelltheirpeersdidbyanaverageof26%)alsoimprovedtheirratings,fromanunderestimateof14to8percentilepoints(Ehrlingeretal.,2008).Thatis,theself-evaluationerrorsoftopperformerswereassociatedwithamixofmistakenimpressionsofbothself-andpeer-performance,whereastheerrorsofbottomperfor-merswereentirelyassociatedwithfaultyimpressionsofself-performance.5.2.ImpactofsocialcomparisoninformationInasense,callingthephenomenontheDunning–Krugereffectisamisno-mer,inthatthereisnosinglephenomenonbutratherafamilyofeffectsflowingfromthefactthatpeoplewithsurfeitsofignorancesufferadouble-curse.Oneadditionaleffectinthisfamilyisthatpoorperformerstendtobeworsejudgesofotherpeoples’competencethantopperformers.Indeed,whentopandbottomperformersingrammaticalskillareaskedtojudgethe DavidDunning grammarofothers,topperformersprovidemuchmoreaccuratejudgmentsthandobottomperformers(Kruger&Dunning,1999,Study3).Butfromthisfact,wecansurmiseyetanothereffectofthedouble-curse.Onewaytolearnaboutone’sownincompetenceisbyobservingthebehaviorofotherpeople—thatis,usingsocialcomparisoninformation.Onemerelyhastose

27 ewhenotherpeopleapproachataskdifferently
ewhenotherpeopleapproachataskdifferently,judgewhenthoseotherapproachesaresuperiororinferiortoone’sown,andadjustone’sself-viewofcompetenceaccordingly.Butthereisahitchforthebottomperformer.Whatifyoucannotreliablyintuitwhichapproachesareinferiororsuperior?Ifthatisthecase,thensuchsocialcomparisoninformation,althoughitmaybeabundant,islessusefulforthetaskofgainingself-knowledge. Original estimateestimateperformanceperformancePercentilePercentileCorrection forCorrection forAverage peerAverage peer Figure5.5Impactofcounterfactualregressionanalysisinwhicherrorsinself-andaveragepeerestimatesarecorrected.Toppaneldisplaysimpactofcorrectionsforbottomquartileperformers.Bottompaneldisplaysimpactofcorrectionsfortopquar-tileperformers(Ehrlingeretal.,2008 TheDunning-KrugerEffect Wehavefoundthistobethecase.AsseeninTable5.2,whenbottomperformersareshownhowotherpeoplehaverespondedtoaquizongrammarskill,theyfailtorevisetheiropinionsoftheirownaptitudeongrammar.Theexperienceoftopperformersisquitedifferent.Theyaccu-ratelyseethattheirpeersareperforminglesswellthantheythemselvesare—thatis,theirfalseconsensuserroriscorrected—andthusincreasehowspecialordistinctivetheybelievetheirownperformanceandskillstobe.Bottomperformers,unabletorecognizesuperiorperformance,donotreceivesuchacorrectivebenefit(Kruger&Dunning,1999,Study3).Thisdifferencebetweentopandbottomperformershasbeenreplicatedamongmedicalstudentsjudgingtheirowninterviewingskills.Afterseeingvideotapesofothermedicalstudentsconductinginterviews,topperformersraisetheirself-evaluationstobettermatchwhattheirsupervisorsaresayingaboutthem.Bottomperformersadjustnotawhit(Hodgesetal.,20015.3.TheparadoxofgainingexpertiseOnefinalpredictionfollowsfromouranalysisoftheDunning–Krugereffect.Thereisanavenuebywhichbottomperformerscanbeguidedtowardmoreaccurateself-judgments.Iftheymisjudgethemselvesbecausetheydonothavetheintellectualresourcestojudgesuperiorversusinferiorperfor-mance,onehasmerelytoprovidethemwiththoseresources.Ofcourse,thisprocedureleadstoaparadox,inthatitrendersbottomperformersnolongerignorantorincompetent.Thatis,onewaytotrainincompetentpeopletorecognizetheirincompetenceistoridthemofthatincompetence.Wehaveshownthatoncepoorperformersa

28 reeducatedoutoftheirincompetence,theysho
reeducatedoutoftheirincompetence,theyshowampleabilityandwillingnesstorecognizetheerrorsoftheirpastways.Inonesuchstudy,weaskedparticipantstocompleteanumberofWasonselectiontasks—alogicalreasoningtaskfamiliartostudentsofpsychology(Wason,1966).Notsurprisingly, Table5.2ImpactofsocialcomparisoninformationonperceivedpercentileperformanceoftopandbottomquartileperformersQuartile/measureBeforeAfterChangeTopquartileGrammarabilityTestperformanceBottomquartileGrammarabilityTestperformanceKrugerandDunning(1999),bytheAmericanPsychologicalAssociation.Adaptedwith DavidDunning wefoundthatbottomperformersgrosslyoverestimatedtheirperformance,thinkingthattheirscoreonthetasklayinthe55thpercentilewhenit,infact,layinthe12th.However,nextwetookroughlyhalfofourparticipantsandgavethema20-mintrainingsessiononhowtosolveWasontasks—andthenaskedthemtore-ratehowwelltheyhaddoneontheoriginaltest.AsseeninFig.5.6participantsatthebottomdramaticallyrevisedtheirself-judgments.Theyratedtheirtestperformance19percentilepointsmoreharshlyandtheiroverallskillatlogicalreasoning10pointsmorenegatively—anirony,inthat,ifanything,the20-minlessonweprovideparticipantshadledthemtobemore,notless,skilledinlogicalreasoning.Butwithadequateintellectualresourcesinplace,participantsprovedquitewillingtoratethemselvesnegativelywhenfacedwithadeficientperformance(Kruger&Dunning,,Study4). 6.SourcesofSelf-evaluationSofar,inmakingthecasethatpeopledonotnecessarilyknowthescopeoftheirignorance,Ihavebeenmakinga“negative”account,showingwhypeoplecannotbeexpectedtoknowwhentheirresponses 01020304060708090100BottomPercentileObjective performance quartile Actual Before SecondThird Figure5.6Self-perceivedlogicalreasoningability(percentilerating)asafunctionofobjectiveperformancebeforeandafterbeinggivenatrainingsessiononhowtoaddressWasonselectiontasks.FromKrugerandDunning(1999),bytheAmericanPsychologicalAssociation.Adaptedwithpermission. TheDunning-KrugerEffect aremisguidedormisinformed.Butthisnegativeaccountleavesopenanimportantquestion.Peopleoftenthinktheirresponsesarereasonable;theyoftenhavesomelevelofconfidenceintheanswerstheyprovide.Ifpeoplecannotrecognizewhentheirresponsesaremistaken(thenegativeaccount),whatisthe“positivep

29 rocess”thatleadspeopletothinkgenera
rocess”thatleadspeopletothinkgenerallythattheirresponsesarecorrect(andinafewcasesthattheirresponsesaresuspect)?Tobeginthepositiveaccountofhowpeoplereachtheirself-evaluativejudgments,Imustfirstmakeclearwhatpeoplefailtohaveattheirdisposalwhenjudgingthewisdomoftheirjudgmentsandchoices.Whatpeopledonothaveisadirect-accesscuethattellsthemwhentheyarerightorwrongintheirconclusions.Theypossessnograndanswersheetthatinformsthemoftheaccuracyoftheirjudgments.ThereexistsnoPinocchio’snosetoindicateunequivocallywhenananswerisatruthoralie;whenitcomestogaugingtheaccuracyofmanyoflife’sdecisions,thereisnoiPhoneappforthat.6.1.TheissueofindirectindicatorsInstead,whatpeoplehaveareindirectcuesthatarecorrelatedwithaccu-racy,albeitinonlyanimperfectway(Koriat,2008a).Acrossmanydomains,forexample,peoplearemoreconfidentwhentheyreachanswersquicklyratherthanmoreslowly(Dunning&Stern,1994;Kelley&Lindsay,1993;Schwarz,2004),andjudgmentspeedappearstobeavalidindicatorofaccuracyunderusualcircumstances(Dunning&Perretta,2002;Koriat,2008a;Koriat,Ma’ayan,&Nussinson,2006).Thesamecanbesaidforfamiliaritywiththetaskathand—eitherthegeneraltopicorthespecificelementsincludedinthetask.Peoplewhoconsidertheoveralldomainortaskelementstobefamiliaralsoaremoreconfident,andaccurate,intheirresponses(Griffin,Jee,&Wiley,2009;Koriat,2008bHowever,underothercircumstances,theseusuallyvaluabletealeafindicatorsofaccuracycanmislead.Decisionspeed,forexample,canbeincreasedbyexposingpeopletoanswers—bothcorrectandincorrect—makingthemmoreconfidentinwhateverconclusiontheyreachwithoutanyconcomitantincreaseinaccuracy(Kelley&Lindsay,1993).Describingataskinatinyandunfamiliarfontmakespeoplelessconfidentthattheycansuccessfullycompleteatask(Song&Schwarz,2008),irrespectiveofactualability.Inasimilarvein,makingatopicoritselementsmorefamiliarbyexposingparticipantstothemalsoleadspeopletobemoreconfidentthattheycanprovidecorrectanswers,irrespectiveofactualaccuracy.Forexample,exposingparticipantstotheequation4556makespeoplemoreconfidenttheycancalculatetheequation4556(Schwartz&Metcalfe,1992).AskingpeoplequestionsaboutChinamakesthetopicmorefamiliartopeople,andtheybecomemoreconfidentthattheycananswerotherquestionsaboutthatcountry(Ar

30 kes,Boehm,&Xu,1991 DavidDunning 6.2.Thep
kes,Boehm,&Xu,1991 DavidDunning 6.2.Theproblemof“rationalerrors”Inworkinourlaboratory,wehavefoundthatanothertealeafindicatorthatpeoplerelyonishowrationaltheirdecisionsare.By“rational,”Imeanthatpeoplefollowsomeoverallruleoralgorithmtocomputetheirresponseacrosssimilarproblems.Themoretheysystematicallyapplythatoverallrule,theyendupmoreconfidentinthequalityofthoseresponses.Totheextentthattheyapproacheachproblemwithadifferentruleorstrategy,theyarelessconfident.Formanytasks,thismakessense.Mathematics,forexample,isaskillthatexactlyaskspeopletoapplysystematicoperationstonumbersacrosssimilarproblemstoachievesomesortofcalculativeresult.Andifoneisapplyingthesameoverallruletosolvesimilarsortsofmathproblems,thenonedoeshaveevidencethatoneissolvingthosemathproblemscorrectly.Thereis,however,aproblem.Peoplemaybeapplyingtherightalgorithmorruletosolveamathorlogicproblem,oranysortofpuzzle,buthowtheybesuretheyhaveappliedtherightalgorithmoronefraughtwitherror?Anobservationineducationalpsychologyisthatschoolroomerrorsofchildrenareoftennothaphazard,butarefrequentlyrationalinnature.Studentsareconscientiouslyfollowingsystematicrules,justthewrongones.Forexample,ifaskedtosolvetheequation3317,manystudentsstatetheansweris24.Theyarewrongnotbecausetheyaresloppy,butbecausetheyhaveanalgorithmintheirheadaboutwhatsubtractionis;itisjustamistakenalgorithm.Theyassumeyoutakethesmallernumberineachcolumnandsubtractitfromthelargerone,andsothe1iscorrectlysubtractedfromthefirst3,butthesecond3issubtractedfromthelarger7.Inshort,theirmistakesarerationalinthattheyfollowaruleoralgorithmthatcontainssomemisunderstandingorglitchthatissystematicallyappliedBen-Zeev,1995,1998Otherworkhasconnectedrationalitywithpositiveevaluationsofper-formance,evenifthosefavorableevaluationsareunwarranted.ThisnotionofrationalityisreminiscentofthedistinctionmadebyTetlock(2005)inhisstudyofexpertdecision-making.Foxesareflexibleandnuancedintheirthinkingwhentheystrivetopredictfutureevents.Hedgehogsapproachallpredictionswithagrand(i.e.,rational)theorythattheyareunwillingtodeviatefrom.Tetlockfoundthatfoxestendedtobemoreaccurateintheirpredictionsoffutureworldevents—andexpressedlessexuberantbutmoreappropriatelevelsofconfidence—

31 thandidhedgehogs.Inourinvestigations,weh
thandidhedgehogs.Inourinvestigations,wehavelookedtoseewhetherfollowingarigidalgorithmleadstomorefavorableperceptionsofperformance,irrespectiveofwhetherthatalgorithmwasrightorwrong.Inonesuchinvestigation,wereanalyzedthedatafromKrugerandDunning(1999,Study4,140),inwhichparticipantsstruggledwithWasonselectiontasks.Anexampleitemis TheDunning-KrugerEffect presentedinFig.5.7.Onthislogicalreasoningtask,weassessedhowconsistentlyparticipantsapproachedthetask,andobservedtwointerestingpatterns(Williams&Dunning,2010).First,participantswhogotnearlyeveryitemrightapproachedeachiteminasystematic,rule-basedway.Thisstandstoreason:GiventhattheWasontaskisalogicalreasoningtask,eachindividualinstanceofitbeapproachedinthesameway.However,wealsofoundthatparticipantswhogotnearlyeveryitemwrongapproachedtheWasontaskinanexactingalgorithmicway.Theyhadjustappliedthewrongalgorithm(see,e.g.,Fig.5.7),leadingthemtobemistakenineverysingleanswertheygave.Figure5.8illustratesoneconsequenceofthesetwopatterns,thatconsis-tencyinapproachingtheWasonselectiontaskwasassociatedwithextremeperformance—bothgoodandbad.Asseeninthecurvilineartrendlinefromapolynomialregressionanalysis,participantswhoweremostlyrightmostlywrongtendedtobethemostsystematicintheirapproachtothetask. AB47 Figure5.7AnexampleitemusedintheWasonselectiontask.:Thecorrectansweristurningoverthe“A”and“7”cards.Whenparticipantsmakeconsistent(i.e.,rational)errors,theytypicallyturnoveronlythe“A”card,orthe“A”and“4”card. Response consistency0102030405060708090100 Figure5.8Relationshipbetweenactualperformance(measuredinpercentiles)andconsistencyinresponding(Williams&Dunning,2010 DavidDunning ThosepostinglessextremeperformancestendedtobemorehaphazardintheirapproachtheWasontask.Asexpected,weobservedisthatconfidenceinone’sperformancemorecloselyfollowedhowsystematicparticipantswereintheirresponsesthanhowaccuratetheywere.ThisledtoanironythatisdisplayedinTable5.3inwhichwelookedatallparticipantswhowerecompletelyconsistentintheiranswers,thatis,theyrespondedtoallitemsinexactlythesameway.Oftheseparticipants,28solvedallitemscorrectly;8allitemsincorrectly.Asseeninthetable,bothgroupsofparticipantswereindistinguishableinho

32 wfavorablytheyviewedtheirperformance.Bot
wfavorablytheyviewedtheirperformance.Both,forexample,thoughttheyhadsolved8–9items(outof10)correctlywheninfactonegrouphadaperfectscoreandtheotheraperfectlyoppositeresult(Williams&Dunning,2010Furtherevidenceimplicatesconsistencywithfavorableviewsofperfor-mancewithnocommensurateriseinaccuracy.Inonestudy,wepresentedparticipantswithaseriesofcaricaturesdrawnbyHirschfeld,whowasfamousforembeddingthenameofhisdaughter,Nina,intohisdrawings.ParticipantswereaskedtofindalltheNina’stheycouldineachdrawing.Onegroupwasforcedtoapproachthetaskconsistently.Thecomputercoveredeachdrawingwithagridof20squares,andthenexposedeachsquareofthegridinthesameregularsequence.Theothergroupwasimpelledtoapproachthetaskmorehaphazardly.Foreachdrawing,thesequenceinwhichthesquareswereexposedwasdifferent.Later,partici-pantsinthefirstgroupwereratedtheirperformancemorepositivelythanthesecondgroupdid,eventhougheachperformedequivalently—thinking, Table5.3Perceivedandactualperformanceofcompletelyconsistentparticipantscomparingthoseansweringallitemsrightversusallitemswrong(adaptedfromWilliams&Dunning,2010AllwrongAllrightAbilitypercentile1.03Testscorepercentile1.73Rawscore1.01Aggregate(standardized)0.81.50PercentileRawscore TheDunning-KrugerEffect forexample,thattheyweremorelikelytohavefoundalltheNinasimbeddedinthecaricatures(Williams&Dunning,20106.3.TheimpactofpreconceivednotionsofskillButanothercuepeoplerelyonsuggeststhatagoodchunkofeveryone’sperformanceevaluationonanyspecifictaskisformedwellbeforetheyeverhearaboutthattask.Peoplecarrywiththempreconceivednotionsaboutwhethertheyaregoodorbadatmath,logic,counselingothers,publicspeaking—thelistisendless.Andthosepreconceivednotionscolorpeople’sevaluationsoftheirperformances—eventheirguessesabouthowwelltheyhaveobjectivelydone.Forexample,wegaveparticipantsalogicalreasoningtestand,attheend,askedparticipantstoestimatetheirrawscoreonthetestaswellastheirpercentilerankingamongtheirpeers.Theirestimatesoftheirrawscorewerejustasstronglycorrelatedwiththeirpreexistingnotionsoftheirlogicalreasoningskillaswiththeiractualrawscore.Theirpercentileself-estimateswereevenmorecloselyassociatedwiththeirpreconceivednotionsofabilitythanwiththeiractualperformance(Ehrli

33 nger&Dunning,,Study1).Thestrategyofconsu
nger&Dunning,,Study1).Thestrategyofconsulting“top-down”self-viewsofcompetencewouldseem—atfirst—tobearationalandappropriatestrategytouse.Anditwouldbe,ifpeople’spreconceivednotionswerestronglycorrelatedwithactualperformance.Asmentionedabove,however,decadesofresearchsuggestthattheimpressionspeoplehaveoftheirskillareonlyweaklytomodestlycorrelatedwithobjectiveperformance(forreviews,seeetal.,2006;Dunning,2005;Dunningetal.,2004;Mabe&West,1982makingthestrategyasuspectoneonwhichtorelyheavily.6.3.1.Theimpactofalteringpreexistingself-viewsOurworkhasshownotherwaysinwhichrelyingontop-downself-viewsmayinfluenceperformanceevaluationsthathavenothingtodowithobjectiveperformance.Inonestudy,wetookareasoningtask,basedonGREanalyticalitems,andfoundthatswitchingwhichself-viewwasrele-vanttothetestsignificantlyalteredhowwellparticipantsthoughttheyperformedonit.AsseeninFig.5.9,whenthe10-itemtestwasdescribedasfocusingon“abstractreasoning,”atraitourparticipantsstatedtheyhadinabundance,participantsestimatedthattheyanswered10%moreitemscorrectlyandrankedthemselves12percentilepointsmorefavorablythanwhenthetestwasdescribedasanexaminationof“computerprogrammingskills,”atraitourparticipantsdeniedhavingtoanypositivedegree.Thesedifferencesarosedespitethefactthatthetestwasidenticalregardlessofitslabel,anddespitethefactthatparticipantsachievedthesamescoresregardlessoflabel. DavidDunning Inanotherstudy,causingparticipantstoquestiontheirknowledgeofNorthAmericangeographybyaskingthem,forexample,whethertheyhadevervisitedWyomingorNebraska,madethemthinktheydidworseonasubsequentgeographyquizcomparedtoagroupaskedmorebenignques-tions,suchaswhethertheyhadevervisitedNewYorkCityorCalifornia.Suchdifferencesinperformanceestimatesaroseirrespectiveofactualper-formanceonthetest(Ehrlinger&Dunning,20036.3.2.ImplicationsforgenderandscienceThisrelianceonpreconceivedself-notionsmaypreventpeoplefromrealiz-ingcompetenciesthattheyhave—oratleastinhibitthemfromrecognizingthattheyaredoingjustaswellastheirpeers.Considerthefact,oftremen-douscurrentinterestandimportance,thatmenandwomenenterandsticktocareersincomputerscience,chemicalengineering,andearthsciencesatstunninglydifferentrates,withmenoverr

34 epresentedrelativetowomenNationalScience
epresentedrelativetowomenNationalScienceFoundation,2000).Womencompriseonly22%ofthelaborforceinscienceandengineering,despitebeing56%ofthelaborforceoverall(NationalScienceFoundation,2000),anddespitenoapparentdifferencesinabilitytohandlesuchcareers(Seymour,1992Couldmenandwomendivergeintheirenthusiasmforsciencebecausetheyholddifferentpreconceivednotionsoftheirscientifictalentthatbearnorelationtothetruth?Thereisevidencethatwomentendtothinklessoftheirscientificaptitudethanmenthinkoftheirs(Eccles,1987)—afindingwereplicatedwithinasamplefromourownuniversity.Couldthatdifferentself-impressionleadtoacascadeofpsychologicaleventsthatcausemenandwomentodivergeondifferentcareerpaths?Totestthisidea,webrought Percent/percentilePerceived percentilePerceived raw scoreActual raw scoreAbstract reasoningComputer programming Figure5.9Perceivedandactualperformanceasafunctionoflabelgiventotest(abstractreasoningvs.computerprogramming)(Ehrlinger&Dunning,2003 TheDunning-KrugerEffect menandwomencollegestudentsintothelaboratory,gavethematestonscientificconcepts,afterwardaskingthemtojudgehowwelltheyhadobjectivelydone.Menandwomendivergedintheirassessments,withwomenthinkingtheyanswered13%fewerquestionsrightthanthementhought,andalsobelievingthattheirperformancelay17percentilepointslowerthanwhatthementhoughtoftheirs(seeFig.5.10).Boththesedifferencesweretraceablebacktodifferencesinpreconceivedbeliefsaboutscientifictalent,andarosedespitethefactthatmaleandfemaleparticipantsperformedequallywellonthetest(Ehrlinger&Dunning,,Study4).Andthesedifferencesinperceptionmattered.Attheendofthesession,allparticipantswereaskediftheywantedtotakepartina“sciencejeopardy”gameshowcompetitionbeingheldlaterinthesessionbythechemistryandpsychologydepartments.Afull70%ofmaleparticipantsexpressedsomeinterest;only49%offemaleparticipantsdidlikewise.Thisdifferencewastraceablebacktotheperceptionbutnottherealityofhowwellparticipantsthoughttheyhaddoneonthetestjustcompleted(Ehrlinger&Dunning,,Study4).Onecanspeculateabouthowmanylifeandcareerdecisionsareguidedbyasimilarpsychologicalprocessthatbearsnorelationtoactualabilityorachievement.6.4.Preconceivednotions“versus”bottom-upexperienceInasense,theimpactofpreconceivedself-noti

35 onspresentstwomysteries.Thefirstistheexa
onspresentstwomysteries.Thefirstistheexactpsychologicalmechanismthatallowssuchviewstoinfluenceimpressionsofobjectiveperformance.Thesecondiswhythe Percent/percentilePerceived percentilePerceived raw scoreActual raw scoreWomen Figure5.10Perceivedandactualperformanceofmaleandfemalestudentsonasciencequiz(Ehrlinger&Dunning,2003 DavidDunning impactofsuchself-viewsisnotswampedbyconcrete“bottom-up”experi-encespeoplehaveastheycompleteatask—suchaswhethertheconceptsinthetestseemfamiliar,whetherpeopleanswerquestionsquicklyoronlyafterconsiderableeffort,orwhethertheystrugglebetweendifferentresponseoptionstheyareconsidering.Asnotedabove,theseconcreteexperiencesallinfluencepeople’sconfidenceintheirperformances,sowhydothesesignalsnot“crowdout”theimpactofmoreabstractsignalscomingfromtop-downnotionsofself?Wehavediscoveredthatthesetwoquestionscanbeaddressedbythesameanswer.Theimpactoftop-downviewsishardlydrownedoutbybottom-upexperiences.Instead,top-downviewssetupexpectationsthatactuallypeople’sbottom-upexperienceswithatask.Peoplewhothinktheyareskilledatatask,forexample,thinktheycometoanswersmorequicklyandwithlessstrugglethanpeoplewhobelievetheyarelessskilled.Peoplewhothinktheyareskilledfeeltheconceptsandquestionstheyconfrontaremorefamiliarthandothosewhoarelessconfidentintheirability.Inthisway,top-downviewsofcompetenceactmuchlikeotherabstractlabelsthataltertheconcretephenomenologicalexperiencespeoplehaveastheycompleteatask.Yogurtlabeledas“fullfat”ratherthan“lowfat”isratedastastier(Wardle&Solomons,1994).Abottleofwineisratedasmorepleasant,andactivatesmoreoftheorbitofrontalcortex,whenitspriceisdescribedas$90ratherand$10(Plassman,O’Doherty,Shiv,&Rangel,).PeopleliterallyseetheskincolorinafaceasdarkerwhenitislabeledasanAfrican-AmericanfaceratherthanaEuropeanAmericanone(Levin&Banaji,2006Acrossseveralstudies,wehaveshownthatpeople’stop-downself-viewsinfluencetheirexperienceswithatask,whichinturninfluencetheirimpressionsofobjectiveperformance.Inonesuchstudy,studentscom-pletedaninterpersonalperceptiontaskafterratingtheir“socialperceptionability.”Foreachitemonthetest,theyalsodescribedtheirexperienceincomingtoananswer—suchaswhethert

36 heyknewtheanswerimmediatelyorhadtogoback
heyknewtheanswerimmediatelyorhadtogobackandforthbetweenpossibleanswers.Attheendofthetest,theyalsoindicatedhowmanyitemstheythoughttheygotright.Statisticalanalysissubsequentlyrevealedthatparticipants’confidenceintheirsocialperceptionabilitysignificantlypredictedhowtheyratedtheirbottom-upexperiencewiththetask,whichinturnpredictedhowwelltheythoughttheyhadobjectivelyperformed(Critcher&Dunning,2009,Study2).Otherdataconfirmthattop-downself-viewscolorbottom-upexperi-ence,andthusimpressionsofobjectiveperformance.Inonestudy,partici-pantswereaskedtotaketwodifferenthistorytests—onedesignedforthehighschoollevelandoneforthegraduateschoollevel.Infact,thetwotestswereequivalentandparticipantsdidnotdifferintheirperformancebetweenthetwotests.(Indeed,wecounterbalancedacrossparticipantswhichexacttestwasgivenwhichlabel.)However,participantshelda TheDunning-KrugerEffect top-downexpectationthattheycouldbetterhandlethehighschooltest,anddescribedtheexperienceoftakingthehighschooltestasmorebenignandfamiliar(e.g.,“ThisquestiondealswithmaterialI’velearnedbefore”)thantheydidthegraduateschooltest.Asaconsequenceofthesedifferent“experiences,”participantsestimatedthattheyperformedsignificantlybet-teronthehighschooltestthantheydidthegraduateschoolversionCritcher&Dunning,2009,Study4).Onefinalstudyfirmlyestablishedthatthecapacityoftop-downviewstoinfluencebottom-upexperienceswasessentialtoultimatelyshapeperfor-manceestimates.Wereplicatedthestudyinwhichparticipantscompletedatestwedescribedasfocusedeitheronabstractreasoningoroncomputerprogrammingskills.However,wevariedthetimingofthislabel.Roughlyhalfofparticipantsweregiventhelabelbeforetheystartedthetest.Theremainderweregiventhelabelonlyaftertheyhadcompletedthetestbutbeforetheyjudgedtheirperformance.Iftop-downviewsinfluenceperfor-manceestimatesonlybecausetheyfirstmoldbottom-upexperiencewiththetask,theimpactofthelabelshouldariseonlyifparticipantswereinformedofthatlabeltheystartedthetest.Onlythendidthelabelhavethecapacitytoinfluencetheirconcreteexperienceswiththetest.Andinthisreplication,asevidencedinFig.5.11,thiswasexactlywhatwefound. Performance estimate (standardized)BeforeAbstract reasoningComputer programming Figure5.11Perceived

37 performance(standardizedcompositemeasure
performance(standardizedcompositemeasure)asafunctionoftestlabel(abstractreasoningversuscomputerprogramming)andtimingthatthelabelisapplied(beforeorafterthetest)(Critcher&Dunning,2009 DavidDunning Informingparticipantsofthelabelafterthetesthadnoimpactontheirperformanceestimates(Critcher&Dunning,2009,Study1). 7.OutstandingIssuesTherearenumerousoutstandingissuesthatdeservefutureresearchattentionregardingpeople’sinabilitytospottheirignoranceingeneralandtheDunning–Krugereffectmorespecifically.7.1.Individualdifferencesinmeta-ignoranceFirst,aretheregeneralindividualdifferencesinmeta-ignorance,orismeta-ignoranceaphenomenonthatarisesinamoredomain-specificway?InouroriginaltreatmentoftheDunning–Krugereffect,weproposedthatthephenomenonwasbestunderstoodasdomain-specific.Eachindividualhashisorherownpersonalpocketsofignoranceofwhichheorshewillbeunaware(Kruger&Dunning,1999).Istillfeelthisisausefulwaytothinkofthephenomenon.Tobesure,thereissuchathingas,thatis,generalintelligence,butmuchresearchshowsthatpeople’sperformanceonintel-lectualtaskscanverygreatlyfromsettingtosetting(e.g.,Ceci&Liker,).Thus,itislikelythatpocketsofincompetencearisequiteindepen-dentlyfromgeneralintellectualskill,andpeopleshouldbeprepared7.1.1.IntellectualcharacteristicsThatsaid,itmightbeusefultopursueworkexploringwhetherthereareanygeneralcharacteristicsthattendtoprovideordeprivepeopleofinsightintotheirshortcomings.Someofthesecharacteristicsmaybeintellectualinnature,andmayinvolvepracticalcompetenciesnecessarytomakeitinthecontemporaryworld.Forexample,literacyhasbeenshowntoinfluencehowpeopleperforminawidevarietyofsettings,fromhealthbehaviortojobsettingstofinancialdecision-making(UNESCO,2002).Itsclosecousin,numeracy,ortheabilitytoreasonwithnumbersandmathematicalconcepts,hasbeensimilarlylinkedtohealthandeconomicoutcomes(e.g.,Reyna,Nelson,Han,&Dieckmann,2009).Itmightbethecasethatthosewhoarelessliterateornumeratemaysuffernotonlyfromlackofskillbutalsofromnotknowingthatthereisinformationtheyneedtoseekout.Someempiricalevidencealreadysuggeststhatpeoplewhoaremoreeducated(whichwecantakeasaproxyforliteracy)arebetterabletoseparatewhattheyknowfromwhattheydonot.Inresearchonnonatti-tudes,highlyeducatedpeoplearemorelikelyt

38 oofferopinionsonrealtopicsbuttoclaimigno
oofferopinionsonrealtopicsbuttoclaimignoranceonnonexistentones,relativetotheirless TheDunning-KrugerEffect educatedpeers.Lesseducatedpeers,paradoxically,tendtoclaimgreaterignoranceonrealtopicsbuttooffermoreopinionsonnonexistentones,suggestingtheyhaveamoredifficulttimeseparatingknowledgefromignorance(Schuman&Presser,1980;Bishopetal.,1980,19867.1.2.MotivationalcharacteristicsOtherpotentialcharacteristicspreventingpeoplefromrecognizingtheirincompetencemaybemoremotivationalinnature,centeringonpeople’stendencytodefendtheirsenseofself-worth(seeKunda,1990;Mele,1997Todate,therehavebeensomeexplorationsofindividualdifferencesasso-ciatedwithself-esteemdefense—andtheseexplorationsshowthatpeoplepronetodefensivenessdobolsterthemselvesmorewhengivenachance.Narcissismandself-deceptiveenhancementpredictover-claimingofknowledgeaboutnonexistentconcepts(Paulhus,Harms,Bruce,&Lysy,).Narcissism,aswell,predictshowwellpeoplethinktheycanmind-readtheintentionsandemotionsofothers,irrespectiveofactualperfor-mance(Ames&Kammrath,2004However,narcissismappearstohaveanimpactthatisindependentofcompetence.Highnarcissistsratethemselvesmorepositively,buttheirjudgmentsarenotlesssensitivetoactuallevelofperformance.Thatis,itisnotuniquelythehighnarcissistswhomisshowpoorlytheyaredoingwhentheydobadly(Ames&Kammrath,2004).TheyarenottheonesresponsiblefortheDunning–Krugereffect;allpoorperformersare.7.2.PerseveranceinignoranceButtheremaybeawayinwhichmotivationalorself-defensivecharacter-isticsmatter.WhentalkingabouttheDunning–Krugereffectwithlaypeo-ple,itoftenbecomesapparentthatwhenpeopleexpressfrustrationabouttheeffect,itisnotsomuchtheincompetencethatbothersthemasitistheblowbacktheyreceivewhentheytrytointervene.Manypoorperformerspushback.Theyrebelagainsttheadvice;theyarguepointsofviewthatcontradicttheirown.Wehavefoundthatpointingoutpeople’sdeficitsdoesnecessarilyinducethemtostrivetoovercomethoselimitations.Inarecentstudyonemotionalintelligence,werevealedtobusinessschoolstudentstheirscorerelativetonationalnormsandaskediftheywantedabookonthe“emo-tionallyintelligentmanager”thatwecouldsellthemata50%discount.Ofthosescoringinthetopquartile,64%wantedthebook.Ofthoseinthebottomquartile,only19%did(Sh

39 eldonetal.,2010).Inasimilarvein,Prasadet
eldonetal.,2010).Inasimilarvein,Prasadetal.(2009)foundthatconfrontingpeoplewithevidencedidnotnecessarilyleadthemtoreconsidertheirmisbeliefsaboutSaddamHussein’sinvolvementinthe9/11tragedy.Amongthe49respondentsconfronted,only1changedhismind,and7deniedtheyhadeverclaimedthelinkinthe DavidDunning firstplace.Theremaining41allrefusedtochangetheirmind,insteadeithercounterarguingtheconfronter’sevidence,refusingtobelieveintheevi-dence’svalidity,bolsteringtheattitudestheyalreadyhad,orsimplyrefusingtoengageinanydiscussiononthematter.Otherworkhasshownthatpeopledonotnecessarilylearntoanticipatetheirincompetenceevenafterrepeatedfeedback.Althoughhighperformingstudentsinapsychologycoursebecamemoreaccurateinpredictingtheirtestperformanceinaclassfromtesttotest,lowperformingstudentsdidnot—remainingstubbornlyoptimisticabouthowwelltheywoulddoonthenexttest(Hacker,Bol,Horgan,&Rakow,2000),aresultreplicatedamongstudentsattendingundergraduateeconomicscoursesFerraro,2010Whatprocessesmightbethesourcesofpeople’sresistancetorecogniz-ingtheirownignoranceeveninthefaceofdirectfeedback?Motivationaldefensesaimedatkeepingself-esteemhighmayverywellbebehindahighlevelofpushback.Anothersourcemaybepeople’scentralworldviews.NyhanandReifler(2010)presentedvoterswithnewspaperarticlesthatcontainedfalseclaims,suchasthattheBushadministrationhadfoundweaponsofmassdestructioninIraqduringtheIraqwar.Introducingacorrectionofthatfalsefactintoanewspaperarticlealteredthebeliefsofliberalvotersbutnotconservativeones,whowereknowntosupportthewarmore.Similarly,whenreadingaboutafalseclaimthattheBushadministrationhadbannedstemcellresearch,theintroductionofacorrec-tionchangedthebeliefofconservativereadersbutnotofliberalones,whomaintainedtheirbeliefintheexistenceofthisnonexistentban.Tyingthisresistancemoredirectlytoself-esteemconcerns,NyhanandReifler(2009)foundthatconservativesweremorelikelytoacceptfactsandargumentsaboutwithdrawingthemilitaryfromIraqaftercompletingaself-affirmationexercisedesignedtoquellself-esteemconcerns.Butsourcesofresistanceneednotallbemotivationalinnature.Preex-istingknowledgeitselfmightbeasourceofpeople’spushback.People,forexample,counterarguepoliticalstancesthatopposetheirownmoretothe

40 extentthattheyarepoliticallysophisticate
extentthattheyarepoliticallysophisticatedandhavemorepoliticalknowl-edge(Tabor,Cann,&Kucsova,2009).Knowledgemaymakeitmoredifficultforpeopletoassimilatenewargumentsandtasks.Inarecentstudy,LondoncabdriverswereaskedtolearnaboutahypotheticalnewareathatexistedinthemiddleofLondon.TheirpriorknowledgeofLondongreatlyinterferedwiththeirabilitytolearnroutesthroughthisnewdistrict,andtheyunderperformedmatchedcontrols(Woollett&Maguire,2010Todate,whenresearchershavelookedathowpreexistingknowledgemightleadtoresistanceinlearning,theyhavelookedataccurateknowl-edge.Onemightpresumethatresistancemaybepromotedby“knowl-edge”thatisinaccurateinnatureaswell.Thatis,ifapersonhasamistaken TheDunning-KrugerEffect ideaofhowstreetsarelaidoutinLondon,wouldthatpreventhimorherfromlearningthecorrectlayout?Todate,noworkhasbeencompletedonthisissue,butonecanpredictthatanysortofknowledge—accurateorerroneous—mayinterferewithpeople’sabilitytoupdatethatknowledge.Oncepeoplebelievesomething,forbetterorworse,itmaybemoredifficulttoalterthatbeliefthanifthepersonknewnothingatall.7.3.BoundaryconditionstotheinvisibilityofincompetenceKeytotheanalysisguidingthischapteristhatoftentheexpertiseneededtoevaluateknowledgeisexactlythesameexpertiseneededtoactexpertly.Butsometimes,onedoesnothavetorelyonthesameexpertisetojudgeperformanceasonedoestoattainit.Couldthoseinstancesbeexceptionstotherule,whenpeoplebecomequitecompetentatspottingtheirincom-petence?Forexample,theskillsneededtoevaluateone’sfreethrowshootingabilityinbasketball(e.g.,anadequatepairofeyes)arequitedistinctfromthoseneededtoproducegoodfreethrows(e.g.,goodhand-eyecoordinationandpropertechnique).Pastworktendstoshowthatevaluationsofperformancecorrelatemorehighlywithrealityinthoseareasinwhichtheskillsneededtoevaluateperformanceareclearlydifferentfromthoseneededtoproduceperfor-mance.Whenitcomestoathletictasks,forexample,thecorrelationbetweenperceptionandrealityofperformancetendstohoveraround0.47.However,asonemovestodomainsthataremoreknowledge-based,thecorrelationtendstodissolve—to0.33forskilledtechnicalknowl-edge,0.17formedicalrelatedtasks,0.28forjobinterviewskills,0.20forgeneralmechanicalexpertise,0.17forinterpersonalability,and0.04formanageria

41 lskills(Mabe&West,1982).Inoneillustrativ
lskills(Mabe&West,1982).Inoneillustrativestudy,varsitycollegefootballplayersdidnotdifferfromtheircoachesinhowtheyevaluatedtheirstrength,speed,andsize—arguablybecausethemannerinwhichplayersevaluatedthosequalitiesdifferedfromthewaythatstrength,speed,andsizewereproduced.However,whenitcametotraitsinwhichonecouldarguethatthesameskillswereneededtoproduceandevaluateperformance—suchasmentaltoughness,coordination,and“footballsense”—varsityplayerstendedtoratethemselvesmorefavorablythandidtheircoaches(Felson,19817.4.Canignorancebebliss?Wecanalsorelyonthediscussionabovetoaddressoneenduringandunsettleddispute—whethertheoptimismandoverconfidencethatpeoplesooftenexhibitisbeneficialorcostlytothem(Colvin&Block,1994;Dunning,2005;Kurt&Paulhus,2008;Taylor&Brown,1988).Often,overconfidenceistakentobeanenergizerthatspurspeopleontotheir DavidDunning goals,helpingthemtoachieveevenunrealisticones(e.g.,Taylor&Brown,).Perhapsmeta-ignorance—notknowingalltheobstaclesandcom-plicationsalongthepathtoone’sgoal—isanadvantage.Thatis,atthemomentwhenpeopleneedtomotivatethemselvestoaction,itmayjustbefollytobewise.Perhaps,itmaybeimportanttomakeadistinctionaboutwhenigno-rance—andtheoverconfidenceitengenders—maybebeneficialorcostly.Theroadtoagoaloftencontainstwophases.Thefirstisaplanningandpreparationphase,inwhichpeoplemustmapouthowtheycanreachtheirgoal.Thesecondistheactualexecutionofaplan.Overconfidencemaybebeneficialinthesecondphase,whenpeoplepotentiallymustenergizeandperseveretopressontotheirgoals,butitmaybedeadlyinthefirstphase.Forexample,itmaybeappropriateforageneraltoincitehisorhertroopstosupremeconfidencewhenthedayofbattlearrives.However,onewouldnotwantthatgeneraltobeincompetentoroverconfidentintheweeksofplanningleadinguptothatbattle.Onewouldnotwantthatgeneraltodenythatmorereservetroopsareneeded,orthatprotectivegearisnotnecessary,orthatthetroopshaveenoughordnance.Onewouldwanttomakesurethatthegeneralhasthoughtoutallthecontingenciesofbattle,sothatheorshecanchangeplansifthecircumstancesofengagementchange.Thus,itispossibleforignoranceandoverconfidencetobebothanadvantageandadisadvantage,dependingonwhetheroneistalkingaboutplanningandpreparationversusexecution. 8.

42 ConcludingRemarksPlato,inhisenduringclas
ConcludingRemarksPlato,inhisenduringclassic,,describesapuzzlethathismentorSocratesoncehadtosolve.Socrates’sfriend,Chaerephon,hadgonetotheOracleofDelphiandaskedwhethertherewasanyonewiserthanSocrates,towhichtheOraclehadrepliedthattherewasnooneelse.ThisbothsurprisedandvexedSocrates,whofeltthatthereweremanymorecitizenswhoknewmorethanhedid,andsohewentoutonasearchtofindtheonewiserpersonthathecouldbringtotheOracleasacounterexample.Heinterviewedthepoliticians,poets,andartisansofAthens,andalthoughhefoundthemallknowledgeableandquiteskilled,healsofoundthemtobesupremelyconfidentintheirexpertiseandunwillingtoacknowledgewhentheirintelligencewaseitherfaultyorvalueless.Inthis,SocratesdiscoveredwhattheOraclehadbeentalkingabout.He,aloneamongallothercitizens,recognizedthathisknowledgeandwisdomwastrivialnexttothatofthegods.Heknewofhislimits,andthisinsightgainedhimtheslightestofadvantagesinwisdomoverothers. TheDunning-KrugerEffect Inthischapter,Ihaveassertedtheinevitabilityofeachindividual’signorance—andhavearguedthatwhenthisignorancevisitspeople’sdeci-sionsandactions,theyarelikelynottoknowit.Nowhereisthisblindnessmoreperceptiblethanintheimpressionsthatincompetentperformershaveoftheirownintellectualandsocialachievements,anditisacautionarytalefortherestofus,because,attimes,wearetheoneswhoexchangeroleswiththem.Ignorancemakesahabitofslyandartfulinvisibility.But,perhaps,onceweknowofthetrick,webecomealittlebitwiserinhowtolookoutforanddealwiththismischievous,significant,andhopefullynot-too-frequentcompanion.ACKNOWLEDGMENTSIacknowledgethemanyandessentialcontributionsofcoauthorsonpreviouschapters,citedthroughoutthischapter,whohelpedbreathelifeintomuchoftheresearchdescribedherein.SpecialthanksgotoJustinKruger,whocollaboratedontheinitialseriesofstudiesuponwhichthisreviewisbased.ThewritingofthischapterwassupportedbyNationalScienceFoundationGrant0745806.TheviewsexpressedinthischapterdonotnecessarilyreflectthoseoftheFoundation.Alicke,M.D.,&Govorun,O.(2005).Thebetter-than-averageeffect.InM.D.Alicke,D.A.Dunning,&J.I.Krueger(Eds.),Theselfinsocialjudgment(pp.85–106).NewYork:PsychologyPress.Alter,A.L.,Oppenheimer,D.M.,&Zemla,J.C.(2010).Missingthetreesfortheforest:Aconstruallevelaccountofthe

43 illusionofexplanatorydepth.JournalofPers
illusionofexplanatorydepth.JournalofPersonalityandSocial,436–451.Ames,D.R.,&Kammrath,L.K.(2004).Mind-readingandmetacognition:Narcissism,notactualcompetence,predictsself-estimatedability.JournalofNonverbalBehaviorAriely,D.,&Norton,M.I.(2007).Psychologyandexperimentaleconomics:AgapinCurrentDirectionsinPsychologicalScience,336–339.Arkes,H.R.,Boehm,L.E.,&Xu,G.(1991).Determinantsofjudgedvalidity.JournalofExperimentalSocialPsychology,576–605.Austin,Z.,Gregory,P.A.M.,&Galli,M.(2008).“Ijustdon’tknowwhatI’msupposedtoknow”:Evaluatingself-assessmentskillsofinternationalpharmacygraduatesinCanada.AdministrativePharmacy,115–124.Barnsley,L.,Lyon,P.,Ralson,S.,Hibbert,E.,Cunningham,I.,Gordon,F.,etal.(2004).Clinicalskillsinjuniormedicalofficers:Acomparisonofself-reportedconfidenceandobservedcompetence.MedicalEducation,358–367.Ben-Zeev,T.(1995).Thenatureandoriginofrationalerrorsinarithmeticthinking:Inductionfromexamplesandpriorknowledge.CognitiveScience,341–376.Ben-Zeev,T.(1998).Rationalerrorsandthemathematicalmind.ReviewofGeneralPsychol-,366–383.Bishop,G.F.,Oldendick,R.W.,Tuchbarber,A.J.,&Bennett,S.E.(1980).Pseudo-opinionsonpublicaffairs.PublicOpinionQuarterly,198–209. DavidDunning Bishop,G.F.,Tuchfarber,A.J.,&Oldendick,R.W.(1986).Opinionsonfictitiousissues:Thepressuretoanswersurveyquestions.PublicOpinionQuarterly,240–250.Burson,K.A.,Larrick,R.P.,&Klayman,J.(2006).Skilledorunskilled,butstillunawareofit:Howperceptionsofdifficultydrivemiscalibrationinrelativecomparisons.JournalofPersonalityandSocialPsychology,60–77.Camerer,C.,&Hogarth,R.(1999).Theeffectsoffinancialincentivesinexperiments:Areviewandcapital-laborproductionframework.JournalofRiskandUncertainty,7–42.Caputo,D.D.,&Dunning,D.(2005).Whatyoudon’tknow:Theroleplayedbyerrorsofomissioninimperfectself-assessments.JournalofExperimentalSocialPsychologyCarter,T.J.,&Dunning,D.(2008).Faultyself-assessment:Whyevaluatingone’sowncompetenceisanintrinsicallydifficulttask.PersonalityandSocialPsychologyCompassCeci,S.J.,&Liker,J.(1986).Adayattheraces:IQ,expertise,andcognitivecomplexity.JournalofExperimentalPsychology:General,255–266.Chi,M.T.H.,Glaser,R.,&Rees,E.(1982)

44 .Expertiseinproblemsolving.InR.Sternberg
.Expertiseinproblemsolving.InR.SternbergAdvancesinthepsychologicalofhumanintelligence(Vol.1,pp.17–76).Hillsdale,NJ:Chiu,M.M.,&Klassen,R.M.(2010).Relationsofmathematicsself-conceptanditscalibrationwithmathematicsachievement:Culturaldifferencesamongfifteen-year-oldsin34countries.LearningandInstruction,2–17.Colvin,C.R.,&Block,J.(1994).Dopositiveillusionsfostermentalhealth?AnexaminationoftheTaylorandBrownformulation.PsychologicalBulletin,3–20.Critcher,C.R.,&Dunning,D.(2009).Howchronicself-viewsinfluence(andmislead)self-evaluationsofperformance:Self-viewsshapebottom-upexperienceswiththetask.JournalofPersonalityandSocialPsychology,931–945.Davis,D.A.,Mazmanian,P.E.,Fordis,M.,Harrison,R.V.,Thorpe,K.E.,&Perrier,L.(2006).Accuracyofphysicianself-assessmentcomparedwithobservedmeasuresof,1094–1102.Downs,A.(1957).Aneconomictheoryofdemocracy.NewYork:Harper&Brothers.Dunning,D.(2005).Self-insight:Roadblocksanddetoursonthepathtoknowingthyself.NewYork:PsychologyPress.Dunning,D.,Heath,C.,&Suls,J.(2004).Flawedself-assessment:Implicationsforhealth,education,andtheworkplace.PsychologicalScienceinthePublicInterest,69–106.Dunning,D.,Johnson,K.,Ehrlinger,J.,&Kruger,J.(2003).Whypeoplefailtorecognizetheirowncompetence.CurrentDirectionsinPsychologicalScience,83–87.Dunning,D.,Meyerowitz,J.A.,&Holzberg,A.D.(1989).Ambiguityandself-evaluation:Theroleofidiosyncratictraitdefinitionsinself-servingassessmentsofability.JournalofPersonalityandSocialPsychology,1082–1090.Dunning,D.,&Perretta,S.F.(2002).Automaticityandeyewitnessaccuracy:A10-to12-secondrulefordistinguishingaccuratefromerroneouspositiveidentifications.JournalofAppliedPsychology,951–962.Dunning,D.,&Stern,L.B.(1994).Distinguishingaccuratefrominaccurateeyewitnessidentificationsviainquiriesaboutdecisionprocesses.JournalofPersonalityandSocialEccles,J.S.(1987).Genderrolesandwomen’sachievement-relateddecisions.PsychologyofWomenQuarterly,585–609.Edwards,R.K.,Kellner,K.R.,Sistrom,C.L.,&Magyari,E.J.(2003).Medicalstudentself-assessmentofperformanceonanobstetricsandgynecologyclerkship.JournalofObstetricsandGynecology,1078–1082.Ehrlinger,J.,&Dunning,D.(2003).Howchronicself-viewsinfluence(andpotentiallymisle

45 ad)assessmentsofperformance.JournalofPer
ad)assessmentsofperformance.JournalofPersonalityandSocialPsychology,5–17. TheDunning-KrugerEffect Ehrlinger,J.,Johnson,K.,Banner,M.,Dunning,D.,&Kruger,J.(2008).Whytheunskilledareunaware?Furtherexplorationsof(lackof)self-insightamongtheincompetent.OrganizationalBehaviorandHumanDecisionProcesses,98–121.Epstein,W.,Glenberg,A.M.,&Bradley,M.(1984).Coactivationandcomprehension:Contributionoftextvariablestotheillusionofknowing.Memory&Cognition,355–360.Everson,H.T.,&Tobias,S.(1998).Theabilitytoestimateknowledgeandperformanceincollege:Ametacognitiveanalysis.InstructionalScience,65–79.Felson,R.B.(1981).Ambiguityandbiasintheself-concept.SocialPsychologyQuarterlyFerraro,P.J.(2010).Knowthyself:Competenceandself-awareness.AtlanticEconomic,183–196.Fischhoff,B.(1975).Hindsightforesight:Theeffectofoutcomeknowledgeonjudgmentunderuncertainty.JournalofExperimentalPsychology.HumanPerceptionandPerformanceFischhoff,B.(1977).Perceivedinformativenessoffacts.JournalofExperimentalPsychology.HumanPerceptionandPerformance,349–358.Fussell,S.R.,&Krauss,R.M.(1991).Accuracyandbiasinestimatesofothers’knowledge.EuropeanJournalofSocialPsychology,445–454.Fussell,S.R.,&Krauss,R.M.(1992).Coordinationofknowledgeincommunication:Effectsofspeakers’assumptionsaboutwhatothersknow.JournalofPersonalityandSocial,378–391.Garb,H.N.(1989).Clinicaljudgment,clinicaltraining,andprofessionalexperience.PsychologicalBulletin,387–396.Glenberg,A.M.,&Epstein,W.(1987).Inexpertcalibrationofcomprehension.Memory&,84–93.Glenberg,A.M.,Wilkinson,A.,&Epstein,W.(1982).Theillusionofknowing:Failureintheself-assessmentofcomprehension.Memory&Cognition,597–602.Graeff,T.R.(2003).Exploringconsumers’answerstosurveyquestions:Areuninformedresponsestrulyuninformed?Psychology&Marketing,643–667.Griffin,T.D.,Jee,B.D.,&Wiley,J.(2009).Theeffectsofdomainknowledgeonmetacomprehensionaccuracy.Memory&Cognition,1001–1013.Hacker,D.J.,Bol,L.,Horgan,D.D.,&Rakow,E.A.(2000).Testpredictionandperformanceinaclassroomcontext.JournalofEducationalPsychology,160–170.Halpern,J.Y.,&Rego,L.C.(2006).Extensivegameswithpossiblyunawareplayers.Proceedingsofthefifthinternationaljointconferenceonautonomo

46 usagentsandmultiagentsystems(pp.744–
usagentsandmultiagentsystems(pp.744–751).NewYork:AssociationforComputingMachinery.Hansford,B.C.,&Hattie,J.A.(1982).Therelationshipbetweenselfandachievement/performancemeasures.ReviewofEducationalResearch,123–142.Harris,M.M.,&Schaubroeck,J.(1988).Ameta-analysisofself-supervisor,self-peer,andpeer-supervisorratings.,43–62.Haun,D.E.,Zeringue,A.,Leach,A.,&Foley,A.(2000).Assessingthecompetenceofspecimen-processingpersonnel.LaboratoryMedicine,633–637.Heath,C.,Larrick,R.P.,&Klayman,J.(1998).Cognitiverepairs:Howorganizationalpracticescancompensateforindividualshortcomings.InB.M.Staw&L.L.CummingsResearchinOrganizationalBehavior(Vol.20,pp.1–37).Hecht,H.,Bertamini,M.,&Gamer,M.(2005)Naiveoptics:Actinguponmirrorreflec-JournalofExperimentalPsychology:HumanPerceptionandPerformanceHertwig,R.,&Ortmann,A.(2001).Experimentalpracticesineconomics:Amethodologi-calchallengeforpsychologists?TheBehavioralandBrainSciences,383–403.Hodges,B.,Regehr,G.,&Martin,D.(2001).Difficultiesinrecognizingone’sownincompetence:Novicephysicianswhoareunskilledandunawareofit.AcademicMedicine,S87–S89. DavidDunning IntercollegiateStudiesInstitute(2008).Thediverginginfluencesofthecollegedegree&civiclearningonAmericanbeliefs.Wilmington,DE:IntercollegiateStudiesInstitute.Kaiser,M.K.,Jonides,J.,&Alexander,J.(1986).Intuitivereasoningaboutabstractandfamiliarphysicsproblems.MemoryandCognition,308–312.Kamen,A.(2002).Tentrent:$10aday.WashingtonPost,A25.Kelley,C.M.,&Lindsay,D.S.(1993).Rememberingmistakenforknowing—Easeofretrievalasabasisforconfidenceinanswerstogeneralknowledgequestions.JournalofMemoryandLanguage,1–24.Keren,G.B.(1987).Facinguncertaintyinthegameofbridge:Acalibrationstudy.OrganizationalBehaviorandHumanDecisionProcessesKoriat,A.(2008a).Subjectiveconfidenceinone’sanswers:Theconsensualityprinciple.JournalofExperimentalPsychology.Learning,Memory,andCognition,945–959.Koriat,A.(2008b).Whenconfidenceinachoiceisindependentofwhichchoiceismade.PsychonomicBulletin&Review,997–1001.Koriat,A.,Ma’ayan,H.,&Nussinson,R.(2006).Theintricaterelationshipsbetweenmonitoringandcontrolinmetacognition:Lessonsforthecause-and-effectrelationbetweensubjectiveexperienceandbeha

47 vior.JournalofExperimentalPsychology:Gen
vior.JournalofExperimentalPsychology:General,36–69.Kossiakoff,A.,&Sweet,W.N.(2003).Systemsengineering:Principlesandpractice.NewYork:Krueger,J.,&Mueller,R.A.(2002).Unskilled,unaware,orboth?Thecontributionofsocial-perceptualskillsandstatisticalregressiontoself-enhancementbiases.JournalofPersonalityandSocialPsychology,180–188.Kruger,J.M.,&Dunning,D.(1999).Unskilledandunawareofit:Howdifficultiesinrecognizingone’sownincompetenceleadtoinflatedself-assessments.JournalofPersonal-ityandSocialPsychology,1121–1134.Kruger,J.,&Dunning,D.(2002).Unskilledandunaware—Butwhy?AreplytoKruegerandMueller.JournalofPersonalityandSocialPsychology,189–192.Kuklinski,J.H.,Quirk,P.J.,Jerit,J.,Schwieder,D.,&Rich,R.F.(2000).Misinformationandthecurrencyofdemocraticcitizenship.JournalofPolitics,79–816.Kunda,Z.(1990).Thecaseformotivatedreasoning.PsychologicalBulletin,480–498.Kurt,A.,&Paulhus,D.L.(2008).Moderatorsoftheadaptivenessofself-enhancement:Operationalization,motivationaldomain,adjustmentfacet,andevaluator.JournalofResearchinPersonality,839–853.Lawson,R.(2006).Thescienceofcycology:FailurestounderstandhoweverydayobjectsMemoryandCognitionLeamy,E.,&Weber,V.(2009).HomeownersbewareofDIYrepairdangers:Do-it-yourselfhomeprojectsoftenleadtoinjuriesandincreasedcosts.http://abcnews.go.com/GMA/story?(accessedFebruary2,2011).Lerner,J.S.,&Tetlock,P.E.(1999).Accountingfortheeffectsofaccountability.logicalBulletin255–275.Levenberg,S.B.(1975).Professionaltraining,psychodiagnosticskill,andkineticfamilyJournalofPersonalityAssessment,389–393.Levin,D.T.,&Banaji,M.R.(2006).Distortionsintheperceivedlightnessoffaces:Theroleofracecategories.JournalofExperimentalPsychology:General,501–512.Lusardi,A.,&Mitchell,O.S.(2009).Howordinaryconsumersmakecomplexeconomicdecisions:Financialliteracyandretirementreactions.DartmouthCollege,Unpublishedmanuscript.Mabe,P.A.III,&West,S.G.(1982).Validityofself-evaluationofability:AreviewandTheJournalofAppliedPsychology,280–286.Maki,R.H.,&Berry,S.L.(1984).Metacomprehensionoftextmaterial.JournalofExperimentalPsychology.Learning,Memory,andCognition,663–679. TheDunning-KrugerEffect Maki,R.H.,Jonas,D.,&Kallod,M.(1994).Therelationshipbe

48 tweencomprehensionandmetacomprehensionab
tweencomprehensionandmetacomprehensionability.PsychonomicBulletin&Review,126–129.McPherson,S.L.,&Thomas,J.R.(1989).Relationofknowledgeandperformanceinboys’tennis:Ageandexpertise.JournalofExperimentalChildPsychology,190–211.Mele,A.R.(1997).Realself-deception.TheBehavioralandBrainSciences,91–136.Miller,J.D.(1998).Themeasurementofcivicscientificliteracy.PublicUnderstandingof,203–223.Mynttinsen,S.,Sundstrom,A.,Vissers,J.,Koivukoski,M.,Hakuli,K.,&Keskinen,E.(2009).Self-assesseddrivercompetenceamongnovicedrivers:AcomparisonofdrivingtestcandidateassessmentsandexaminerassessmentsinaDutchandFinnishsample.JournalofSafetyResearch,301–309.NationalScienceBoard(2010).Scienceandengineeringindicators2010.Washington,DC:NationalScienceFoundation.NationalScienceFoundation(2000).Women,minorities,andpersonswithdisabilitiesinscienceandengineering.Arlington,VA:NationalScienceFoundation.Nickerson,R.S.,Baddeley,A.,&Freeman,B.(1987).Arepeople’sestimatesofwhatotherpeopleknowinfluencedbywhattheythemselvesknow?ActaPsychologicaNielsen-Bohlman,L.,Panzer,A.M.,&Kindig,D.A.(2004).Healthliteracy:Aprescriptiontoendconfusion.Washington,DC:InstituteofMedicine.Nyhan,B.,&Reifler,J.(2009).Openingthepartisanmind:Self-affirmationandinformationprocessingaboutthesurgeinIraq.Unpublishedmanuscript,UniversityofMichigan.Nyhan,B.,&Reifler,J.(2010).Whencorrectionsfail:Thepersistenceofpoliticalmis-PoliticalBehavior,303–330.Park,Y.-J.,&Santos-Pinto,L.(2010).Overconfidenceintournaments:EvidencefromtheTheoryandDecision,143–166.Paulhus,D.L.,Harms,P.D.,Bruce,M.N.,&Lysy,D.C.(2003).Theover-claimingtechnique:Measuringself-enhancementindependentofability.JournalofPersonalityandSocialPsychology,890–904.Plassman,H.,O’Doherty,J.,Shiv,B.,&Rangel,A.(2008).Marketingactionscanmodulateneuralrepresentationsofexperiencedpleasantness.ProceedingsoftheNationalAcademyofScienceUnitedStatesofAmerica,1050–1054.Prasad,M.,Perrin,A.,Bezila,K.,Hoffman,S.,Kindleberger,K.,Manturuk,K.,etal.(2009).Theremustbeareason:Osama,Saddam,andinferredjustification.,142–162.Proffitt,D.R.(1999).Naivephysics.InR.Wilson&F.Keil(Eds.),TheMITencyclopediaofthecognitivesciences(pp.577–579).Cambridge,MA:MITPress.

49 Reyna,V.F.,Nelson,W.L.,Han,P.K.,&Dieckma
Reyna,V.F.,Nelson,W.L.,Han,P.K.,&Dieckmann,N.F.(2009).Hownumeracyinfluencesriskcomprehensionandmedicaldecisionmaking.PsychologicalBulletinRoss,L.D.,Greene,D.,&House,P.(1977).The“falseconsensuseffect”:Anegocentricbiasinsocialperceptionandattributionprocesses.JournalofExperimentalSocialPsychology,279–301.L.,&Keil,F.C.(2002).Themisunderstoodlimitsoffolkscience:Anillusionofexplanatorydepth.CognitiveScience,521–562.Schipper,B.C.(2010).Theunawarenessbibliographyhttp://www.econ.ucdavis.edu/faculty/(accessedNovember30,2010).Schuman,H.,&Presser,S.(1980).Publicopinionandpublicignorance:Thefinelinebetweenattitudesandnonattitudes.TheAmericanJournalofSociology,1214–1225.Schwartz,B.L.,&Metcalfe,J.(1992).Cuefamiliaritybutnottargetretrievabilityenhancesfeeling-of-knowingjudgments.JournalofExperimentalPsychology.Learning,Memory,and,1074–1083. DavidDunning Schwarz,N.(2004).MetacognitiveexperiencesinconsumerjudgmentanddecisionJournalofConsumerPsychology,332–348.Seymour,E.(1992).UndergraduateproblemswithteachingandadvisinginSMEmajors:Explaininggenderdifferencesinattributionrates.JournalofCollegeScienceTeachingShaughnessy,J.J.(1979).Confidencejudgmentaccuracyasapredictoroftestperformance.JournalofResearchinPersonality,505–514.Sheldon,O.,Ames,D.,&Dunning,D.(2010).Self-assessmentsofemotionalintelligence.Unpublisheddata,RutgersUniversity.Sinkavich,F.J.(1995).Performanceandmetamemory:Dostudentsknowwhattheydon’tInstructionalPsychology,77–87.Song,H.,&Schwarz,N.(2008).Ifit’shardtoread,it’shardtodo:Processingfluencyaffectseffortpredictionandmotivation.PsychologicalScience,986–988.Stajkovic,A.D.,&Luchins,F.(1998).Self-efficacyandwork-relatedperformance:Ameta-PsychologicalBulletin,240–261.Tabor,C.S.,Cann,D.,&Kucsova,S.(2009).ThemotivatedprocessingofpoliticalPoliticalBehavior,137–155.Taylor,S.E.,&Brown,J.D.(1988).Illusionandwell-being:Asocialpsychologicalperspectiveonmentalhealth.PsychologicalBulletin,193–210.Tetlock,P.E.(2005).Expertpoliticaljudgment.Princeton,NJ:PrincetonUniversityPress.Tetlock,P.E.,&Kim,J.I.(1987).Accountabilityandjudgmentprocessesinapersonalitypredictiontask.JournalofPersonalityandSocialPsychology,700–70

50 9.UNESCO,J.I.(2002).Educationforall:Isth
9.UNESCO,J.I.(2002).Educationforall:Istheworldontrack?EFAglobalmonitoringreport.Vnuk,A.,Owen,H.,&Plummer,J.(2006).Assessingproficiencyinadultbasiclifesupport:Studentandexpertassessmentandtheimpactofvideorecording.MedicalTeacherWagenaar,W.A.,&Keren,G.B.(1985).Calibrationofprobabilityassessmentsbyprofes-sionalblackjackdealers,statisticalexperts,andlaypeople.OrganizationalBehaviorandHumanDecisionProcesses,406–416.Wardle,J.,&Solomons,W.(1994).Naughtybutnice:Alaboratorystudyofhealthinformationandfoodpreferencesinacommunitysample.HealthPsychologyWason,P.C.(1966).Reasoning.InB.M.Foss(Ed.),Newhorizonsinpsychology(pp.135–151).Baltimore:PenguinPress.Watts,W.E.,Rush,K.,&Wright,M.(2009).Evaluatingfirst-yearnursingstudents’abilitytoself-assesspsychomotorskillsusingvideo.NursingEducationPerspectives,214–219.Weinstein,N.D.(1980).Unrealisticoptimismaboutfuturelifeevents.JournalofPersonalityandSocialPsychology,806–820.Williams,N.V.,Baker,D.W.,Honig,E.G.,Lee,T.M.,&Nowlan,A.(1998).Inadequateliteracyisabarriertoasthmaknowledgeandself-care.Williams,E.,&Dunning,D.(2010).Fromformulaetofaith:Aconsistencyheuristicunderliesmistakenconfidenceinself-performance.UniversityofFloridaunderreview.Wilson,T.D.,&Gilbert,D.T.(2003).Affectiveforecasting.InM.P.Zanna(Ed.),inexperimentalsocialpsychology(Vol.35,pp.345–411).SanDiego:AcademicPress.Winship,C.,&Korenman,S.(1997).Doesstayinginschoolmakeyousmarter?TheeffectofeducationonIQ.InB.Devlin,S.E.Fienberg,D.Resnick,&K.Roeder(Eds.),Intelligenceandsuccess:Isitallinthegenes?Scientistsrespondtothebellcurve(pp.215–234).NewYork:Springer-Verlag.Winship,C.,&Morgan,S.L.(2000).TheestimationofcausaleffectsfromobservationalAnnualReviewofSociology,659–706. TheDunning-KrugerEffect Woodzicka,J.A.,&LaFrance,M.(2001).Realversusimaginedgenderharassment.ofSocialIssues,15–30.Woollett,K.,&Maguire,E.(2010).Theeffectofnavigationalexpertiseonwayfindinginnewenvironments.JournalofEnvironmentalPsychology,565–573.Zenger,T.R.(1992).Whydoemployersonlyrewardextremeperformance?Examiningtherelationshipsamongperformance,pay,andturnover.AdministrativeScienceQuarterlyZizek,S.(2004).Iraq’sfalsepromise.ForeignAffairs,42–49,January/February. Davi