Journa o Experimenta Psychology Huma Perceptio an Performanc  Vol  No   Knowin wit Certainty Th Appropriatenes o Extrem Confidenc Baruc Fischhoff Pau Slovic an Sara Lichtenstei Decisio Research A Bra
155K - views

Journa o Experimenta Psychology Huma Perceptio an Performanc Vol No Knowin wit Certainty Th Appropriatenes o Extrem Confidenc Baruc Fischhoff Pau Slovic an Sara Lichtenstei Decisio Research A Bra

Fo a variet o generalknowledg question eg absinth i a a liqueu o b a preciou stone subject firs chos th most likel answe an the indicate thei degre o certaint tha th answe the ha selecte was i fact correct Acros severa differen questio an respons fo

Tags : variet
Download Pdf

Journa o Experimenta Psychology Huma Perceptio an Performanc Vol No Knowin wit Certainty Th Appropriatenes o Extrem Confidenc Baruc Fischhoff Pau Slovic an Sara Lichtenstei Decisio Research A Bra




Download Pdf - The PPT/PDF document "Journa o Experimenta Psychology Huma Per..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.



Presentation on theme: "Journa o Experimenta Psychology Huma Perceptio an Performanc Vol No Knowin wit Certainty Th Appropriatenes o Extrem Confidenc Baruc Fischhoff Pau Slovic an Sara Lichtenstei Decisio Research A Bra"— Presentation transcript:


Page 1
Journa o Experimenta Psychology Huma Perceptio an Performanc 1977 Vol 3 No 4 552-56 Knowin wit Certainty Th Appropriatenes o Extrem Confidenc Baruc Fischhoff Pau Slovic an Sara Lichtenstei Decisio Research A Branc o Perceptronic Eugene Orego Ho ofte ar peopl wron whe the ar certai tha the kno th answe t a questio ? Th studie reporte her sugges tha th answe i "to often. Fo a variet o general-knowledg question (e.g. absinth i [a a liqueu o [b a preciou stone) subject firs chos th most likel answe an the indicate thei degre o certaint tha th answe the ha selecte was i fact

correct Acros severa differen questio an respons formats subject wer consistentl overconfident The ha sufficien fait thei confidenc judgment t b willin t stak mone o thei validity Th psychologica base fo unwarrante certaint ar discusse i term o th inferentia processe whereb knowledge i constructe fro perception an memories Tw aspect o knowledg ar wha on believe t b tru an ho confiden on i tha belief Bot ar represente i a statemen like " a 70 certai tha Quit i th capita o Equador. Whil i ofte no difficul t asses th veridicalit a belie (e.g. b lookin i a definitiv atlas) evaluatin th validit o a

degre confidenc i mor difficult Fo example th 70 certaint i th abov statemen woul see mor appropriat i Quit i th capita tha i Quit isn' th capital bu tha i a rathe crud assessment I a sense onl statement o certaint (0 o 100% ca b evaluate individually accordin t whethe th belief t whic the ar at tache ar tru o false Thi researc wa supporte b th Advance Researc Project Agenc (ARPA o th De partmen o Defens an wa monitore b th Offic o Nava Researc unde Contrac N00014 76-007 (ARP Orde No 30S2 unde a sub contrac fro Decision an Designs Inc t Ore ga Researc Institute woul lik t than Bernar Corrigan

Roby Dawes War Edwards an Amo Tversk for thei contribution to thi project Request fo reprint shoul b sen t Baruc Fischhoff Decisio Research A Branc o Per ceptronics 120 Oa Street Eugene Orego 97401 On wa t validat degree o confidenc i loo a th calibratio o a se o suc con fidenc statements A individua i wel cali brate if ove th lon run fo al proposi tion assigne a give probability th pro portio tha i tru i equa t th probabilit assigned Fo example hal o thos state ment assigne a probabilit o .5 o bein tru shoul b true a shoul 60 o thos assigne .60 an al o thos abou whic th individua i 100

certain A burgeonin literatur o calibratio ha bee surveye Lichtenstein Fischhoff an Phillip (i press) The primar conclusio of thi re- vie i tha peopl ten t b overconfident tha is the exaggerat th exten t whic wha the kno i correct A fairl typica se o calibratio curves draw fro severa studies appear i Figur 1 W se tha whe peopl shoul b righ 70 o th time thei "hi rate i onl 60% whe the ar 90 certain the ar onl 75 right an s on People' poo calibratio ma be i part jus a questio o scaling Probabilitie (o odds ar a se o number tha peopl us wit som interna consistenc (e.g. th curve i Figur 1 ar mor o

les mono tonicall increasing bu no i accordanc 55
Page 2
KNOWIN WIT CERTAINT 55 1. 7 . Subjects Respons 1. Figure 1 Som representativ calibratio curves (Take fro Lichtenstein Fischhoff & Phillips press Copyrigh 197 b D Reide Publishin Co Reprinte b permission. wit th absolut critero o calibration Miscalibratio ca hav serious conse quence (se Lichtenstei e al. i press) ye people' inabilt t asses appropriatel probabilit o .8 ma b n mor surpris in tha th difficult the migh hav i estimatin brightnes i candle o tempera tur i degree Fahrenheit Degree o cer taint ar ofte use i everyda speec

(a are reference to temperature) but the are seldo expresse numericall no i th op portunit t validat the ofte availabl (Tversk & Kahneman 1974) Th extreme o th probabilit scal are however no suc foreig concepts Bein 100 certai tha a statemen i tru i readil understoo b mos people an it appropriatenes i readil evaluated Th followin studie examin th calibratio o people' expression o extrem certainty Th studie ask Ho ofte ar peopl wron whe the ar certai tha the ar right I Experimen 1 th answe i sough i probabilit judgment elicite b question pose i fou differen ways Experimen 1 Method Stimuli. Th

question covere a wid variet topics includin history music geography na ture, an literature The fou format use wer th following Open-ende format Subject wer presente wit a questio stem whic the wer aske t complete fo example "Absinth i a . Afte writin dow a answer the estimate th probabilit tha thei answe wa correct usin a numbe fro .0 t 1.00 2. One-alternativ format Subject wer aske asses th probabilit (fro .0 t 1.00 tha simpl statement wer correct fo example "Wha i th probabilit tha absinth i a preciou stone? Th statemen o fac bein judge wa sometime tru an sometime false Two-alternativ forma

(hal rang o re sponses) Fo eac question subject wer aske
Page 3
55 FISCHHOFF P SLOVIC AN S LIECHTENSTEI choos th correc answe fro tw tha wer offered Afte makin eac choice the judge th probabilit tha th choic wa correct fo ex ample "Absinth i (a a preciou ston o (b liqueur. Sinc the chos th mor likel an swer thei probabilitie wer limite t th rang fro .5 t 1.00 Two-alternativ forma (ful rang o re sponses) Instea o havin subject pic th an swe mos likel t b correc a i Forma 3 th experimenter randoml selecte on o th tw alternative (e.g. [b a liqueur an ha sub ject judg th probabilit tha

th selecte al ternativ wa correct Her th ful rang [.00 1.00 was used As in Forma 3, one answe was correct Subjects an procedure. Th subject wer 36 pai volunteer wh responde t a a i th Universit o Orego studen newspaper The wer assigne t th fou group accordin t preferenc fo experimen tim an date Eac grou receive th question i onl on o th fou formats Beside th difference i questio format th specifi question use differe some wha fro grou t group Instruction wer brie an straightforward askin subject t choos o produc a answe an assig a proba bilit o bein correc i accordanc wit th for ma used

Results Lichtenstei an Fischhof (i press an Fischhof an Lichtenstei (Not 1 hav re porte o th calibratio o th entir rang probabilit response o subject i Ex perimen 1 Her w examin onl thei ex trem responses Tabl 1 show (a th frequenc wit whic subject indicate 1.0 .0 a th probabilit a alternativ wa correc an (b th percentag o answer associate wit thes extrem probabilitie tha were i fact correct Answer assigne probabilit o 1.0 o bein correc wer righ betwee 20 an 30 o th time Answer assigne a probabilit o .0 wer righ betwee 20 an 30 o th time Format 2 an 4 wher response o 1.0 an .0 wer possible bot

response oc curre wit abou equa frequency Further more alternative judge certai t b cor rec wer wron abou a ofte a alterna tive judge certai t b wron wer cor rect Th percentag o fals certaintie range fro abou 17 (Forma 1 t abou 30 (Forma 2) bu comparison acros format shoul b mad wit cautio becaus th item differed Clearly ou sub ject wer wron al to ofte whe the wer certai o th correctnes o thei choic o answer Experimen 2 Experimen 1 migh b faulte becaus o th insensitivit o th respons mode Wit probabilities subject usin th stereotypi response o .50 .55 .60 an s on hav fe possibl response fo

indicatin differ en degree o hig certainty A th ex treme mos subject restricte themselve th response .90 .95 an 1.00 corre spondin t odd o 9:1 19:1 an o :1 Perhap wit a mor graduate respons mode subject woul be bette abl to ex- pres differen level o certainty I Ex perimen 2 subject wer presente wit general-knowledg question concerne wit singl topic—th incidenc o differen cause o deat i th Unite States—an Tabl 1 Analysis of Certainty Responses in Experiment 1 Questio forma Ope ende On alternativ Tw alternativ (hal range Tw alternativ (ful range No item No subject 12 13 Tota no response 1,29

6,45 9,00 6,50 Certaint response (P) 1.0 1.0 .0 1.0 1.0 .0 certaint response 19. 14. 13. 21. 17, 19. correc certaint response 83. 71. 29. 81. 80. 20.
Page 4
KNOWIN WIT CERTAINT 55 aske t expres thei confidenc i thei answer i odds Th odd scal i ope ende a th extremes easil allowin th expressio o man differen level o grea certaint (e.g. 20:1 50:1 100:1 500:1 etc.) Method Stimuli. Al item involve th relativ fre quencie o th 4 letha event show i Tabl 2 The wer chose becaus the wer easil under stoo an ha fairl stabl deat rate ove th las S year fo whic statistic wer available Th even

frequencie appearin i Tabl 2 wer estimate fro vita statistic report prepare b th Nationa Cente fo Healt Statistic an th "Statistica Bulletin o th Metropolita Lif In suranc Company Thes frequencie provide th correc answer fo th question pose t ou subjects Fro amon thes 4 cause o death 10 pair wer constructe accordin t th followin cri teria (a Eac caus appeare i approximatel si pair an (b th ratio o th statistica rate th more-frequen even t th less-frequen even varie systematicall fro 1.25: (e.g. ac cidenta fall vs emphysema t abou 100,000: (e.g. strok vs botulism) Procedure. Subjects

instruction rea a fol lows Eac ite consist o tw possibl cause o death Th questio yo ar t answe is Whic caus o deat i mor frequent i general i th Unite States Fo eac pai o possibl cause o death (a an (b) w wan yo t mar o you answe shee whic caus yo thin i mor frequent Next w wan t decid ha confiden yo ar tha yo have i fact chose th mor frequen caus o death Indicat you confidenc b th odd tha you answe i correct Odd o 2: mea tha yo ar twic a likel t b righ wrong Odd o 1,000: mea tha yo ar thousan time mor likel t b righ tha wrong Odd o 1: mea tha yo ar equall likel t b righ o wrong Tha is you an

swe i completel a guess th to o th answe shee w hav draw scal tha look lik this ! I 1: 10: 100: 1,000: I 10,000: 100,000: 1,000,000: etc coul writ 75:1 i yo thin tha i i 7 time mor likel tha yo ar righ tha yo ar wrong o 1.2: i yo thin tha i i onl 20 mor likel tha yo ar righ tha wrong no us odd les tha 1:1 Tha woul mea tha i i les likel tha yo ar righ tha tha yo ar wrong i whic cas yo shoul indi cat th othe caus o deat a mor frequent Tabl 2 Lethal Events Whose Relative Frequencies Were Judged by Subjects in Experiments 2 and 3 Actua death Letha even pe 10 million Thi scal i use t giv yo a ide o

th kind o number yo migh wan t use Yo don' hav t us exactl thes numbers Yo Smallpo 0 Poisonin b vitamin 0. Botulis 1 Measle 2. Firework 3 Smallpo vaccinatio 4 Whoopin coug 7. Poli 8. Venomou bit o stin 23. Tornad 4 Lightnin 5 Nonvenomou anima 6 Floo 10 Exces col 16 Syphili 20 Pregnancy childbirth an abortio 22 Infectiou hepatiti 33 Appendiciti 44 Electrocutio 50 Moto vehicl - trai collisio 74 Asthm 92 Firear acciden 1,10 Poisonin by soli or liqui 1,25 Tuberculosi 1,80 Fir an flame 3,60 Drownin 3,60 Leukemi 7,10 Accidenta fall 8,50 Homicid 9,20 Emphysem 10,60 Suicid 12,00 Breas cance 15,20

Diabete 19,00 Moto vehicl (car truck o bus acciden 27,00 Lun cance 37,00 Cance o th digestiv syste 46,40 Al accident 55,00 Strok 102,00 Al cancer 160,00 Hear diseas 360,00 Al disease 849,00 Per-yea deat rate ar base o 10 millio Unite State residents
Page 5
55 FISCHHOFF P SLOVIC AN S LICHTENSTEI Tabl 3 Percentage of Correct Answers for Major Odds Categories Letha event Experimen 2 Experimen 3 General-knowledg question Experimen 4 Odd 1: 1.5: 2: 3: 5: 10: 20: 50: 100: 1,000: 10,000: 100,000: 1,000,000: Tota Overal % -Tippiopriau correct 99. 10 10 10 correc 64 57 18 25 1,16 12 25 1,18

86 45 16 15 6,09 %N cor rec 71. 33 10 43 25 32 39 16 22 31 21 13 2,98 %N 2. cor rec 72. 86 21 45 15 19 37 37 33 26 13 36 3,85 %N 3. 1. 1. cor rec 73. Note. % T refer t th percentag o odd judgment tha fel i eac o th majo categories Ther wer 6 subject i Experimen 2 4 i Experimen 3 an 4 i Experimen 4 Fo well-calibrate subjects cas som o th cause o deat ar ambigu ou o no wel define b th brie phras tha describe them w hav include a glossar fo severa o thes items Rea thi glossar befor starting Subjects. Th subject wer 6 pai volunteer wh answere a a i th Universit o Orego studen newspaper Results

Tabl 3 show th percentage o correc answers groupe acros subjects fo eac th mos frequentl use (major odd categories A odd o 1:1 1.5:1 2:1 an 3:1 subject wer reasonabl wel cali brated However a odd increase fro 3: t 100:1 ther wa littl o n increas accuracy Onl 73% o th answer as signe odd o 100: wer correct Accurac jumpe t 81% a 1,000: an t 87 a 10,000:1 Fo th answer assigne odd o 1,000,000: o greater accurac wa 90%. Fo th latte responses th appropriat de gre o confidenc woul hav bee odd o 9:1 Th 12% o response tha ar no liste i Tabl 3 becaus the fel betwee th majo odd categorie showe simila

calibration i Experimen 1 subject i Experi men 2 exhibite grea overconfidence The wer frequentl wron a eve th highes odd levels Moreover the gav man ex trem odd responses O 6,99 odd judg ments 3,56 (51% wer greate tha 50:1 Almos on fourt o th response wer greate tha 1,000:1 Experimen 3 Althoug th task an instruction fo Experiment 1 an 2 seeme reasonabl straightforward w wer concerne tha subjects extrem overconfidenc migh b mor detaile descriptio o subjects per formance o thi tas an severa relate one ca foun i Lichtenstein Slovic Fischhoff Combs an Layma (Not 2)
Page 6
KNOWIN WIT

CERTAINT 55 du t lac o motivatio o misunderstand in o th respons scale Experimen 3 replicate Experimen 2 givin mor car an attentio t instructin an motivatin th subjects Method Experimen 3 use the 106 causes-of-deat ques tion an odd respons forma o Experimen 2 Th experimente starte th sessio wit a 20 minut lectur t th subjects I thi lecture th concept o probabilit an odd wer carefull explained Th subtletie o expressin one' feel ing o uncertaint a numerica odd judgment wer discussed wit specia emphasi o ho t us smal odd (betwee 1: an 2:1 whe on quit uncertai abou th correc answer A char wa

provide showin th relationshi be twee variou odd estimate an th correspond in probabilities Finally subject wer taugh th concep o calibratio an wer urge t mak odd judgment i a wa tha woul lea the t wel calibrated (Th complet tex o th in struction i availabl fro th authors. Th subject fo Experimen 3 wer 4 per son wh responde t a a i th Universit o Orego studen newspaper A i previou experi ments the wer pai fo participating Grou siz wa hel t abou 2 t increas th likelihoo tha subject woul as question abou an face th tas tha wa unclear Results Th proportio o correc answer fo eac o th mos frequen

odd categorie i show i th cente portio o Tabl 3 Th detaile instruction ha severa effects First subject wer muc mor pron t us atypica odd suc a 1.4:1 2.5:1 an on Onl 70 o thei judgment fel withi th majo odd categorie o Tabl 3 compare t 88 fo Experimen 2 Second thei odd estimate tende t b smaller Abou 43 o thei estimate wer 5: o less compare t 27 fo thi cate gor i Experimen 1 Third subject i thi experimen wer mor ofte correc a odd abov 10: an thu wer bette cali brated Nevertheless subject agai exhibite un warrante certainty The assigne odd greate tha o equa t 50: t approxi matel on thir o th

items Onl 83 o th answer associate wit thes odd wer correct Whe subject estimate odd o 50:1 the wer correc 74 o th tim an thu shoul hav bee givin odd o abou 3:1 A 1,000:1 the shoul hav bee sayin abou 5:1 Althoug onl 70 o th response fel th majo odd categorie o Tabl 3 inclusio o th remainin 32 woul no hav change th picture Odd estimate fallin betwee majo categorie wer cali brate similarl t estimate withi thos categories Elaborat instructio tempere subjects extrem overconfidence bu onl a limite extent Experimen 4 ther somethin peculia t th causes of-deat item tha induce suc overconfi dence

Experimen 4 replicate Experimen usin general-knowledg question (o th typ use i Experimen 1 matche i difficult wit th 10 causes-of-deat items addition subjects fait i thei odd judgment wa teste b thei willingnes participat i a gamblin gam base o thos judgments Method Th questionnair consiste o 10 two-alterna tiv item coverin a wid variet o topics fo example "Whic magazin ha th larges circu latio i 1970 (a Playboy o (b Time"; "Ade wa occupie i 183 b th (a Britis o (b French" "Bil pigment accumulat a a resul o conditio know a (a gangren o (b jaun dice. Thes item wer take fro a larg ite poo wit

know characteristics Availabilit o thi poo allowe u t selec item matche i difficulty questio b question wit th 10 item abou letha event studie i Experiment 2 an 3 Th subject wer 4 pai volunteers recruite a a i th Universit o Orego studen newspaper Th instruction parallele thos o Experimen 3 Subject firs receive th detaile lectur describin th concept o probability odds an calibration The the responde t th 10 general-knowledg items markin th answe the though t b correc an expressin thei cer taint abou tha answe wit a odd judgment Afte respondin t th 10 items the wer aske whethe the woul b willin

t accep gamble contingen o th correctnes o thei an swer an th appropriatenes o thei odd esti mates I subject reall believ i thei extrem (extremel overconfident odd responses i
Page 7
55 FISCHHOFF P SLOVIC AN S LICHTENSTEI shoul b possibl t construc gamble tha the ar eage t accep bu which i fact ar quit disadvantageou t them Th gam wa describe th followin instructions Th experimen i over Yo hav jus earne $2.50 whic yo wil b abl t collec soon Bu befor yo tak th mone an leave I' lik yo t conside whethe yo woul b will in t pla a certai gam i orde t possibl increas you earnings Th rule o

th gam ar a follows Loo a you answe sheet Fin th question wher yo estimate th odd o you bein cor rec a 50: o greate tha 50:1 Ho man suc question wer there (writ num ber I'l giv yo th correc answer t thes "50: o greater questions We'l coun ho man time you answer t thes question wer wrong Sinc a wron answe i th fac suc hig certaint woul b surprising we'l cal thes wron answer "you surprises. I hav a ba o poke chip i fron o me Ther ar 10 whit chip an 2 re chip i th bag I I reac i an randoml selec a chip th odd tha I wil selec a whit chi ar 100: o 50:1 jus lik th odd tha you "50:1 answer ar correct

Fo ever "50: o greater answe yo gave I'l dra a chi ou o th bag (I yo wish yo ca dra th chip fo me. I'l pu th chi bac i th ba befor I dra again s th odd won' change Th probabilit o m draw in a re chi i 1/51 Sinc drawin a re chi unlikely ever re chi I dra ca b con sidere "m surprise. Ever tim yo ar surprise b a wron an swe t a "50: o greater question yo pa $1 Ever tim I a surprise b drawin a re chip I'l pa yo $1 I yo ar wel calibrated thi gam i ad vantageou t you Thi i becaus I expec t los $ abou onc ou o ever 5 time I dra chip o th average Bu sinc you odd ar sometime highe tha 50:1 yo expec t

los les ofte tha that Ye ica uiic man uiai Woul yo pla thi game Circl one Subject wh decline wer the aske i the woul pla i th experimente raise th amoun woul pa the t $1.5 wheneve h dre a re chip whil the stil ha t pa onl $ i th even o a wron answer Thos wh stil re fuse wer offere $ an the a fina offe o $2.5 fo ever re chip Sinc th experimenter expecte th gam t b unfai t subject (b capitalizin o a "known judgmenta bias) i wa no actuall playe fo money Results Th proportio o correc answer asso ciate wit eac o th mos commo odd response i show i th right-han colum Tabl 3 Compare wit th previou

studies subject i Experimen 4 gav a highe proportio o 1: odd (19% o th tota responses) A fe difficul item le almos al o th subject t giv answer clos t 1:1 indicatin tha the wer tryin us smal odd whe the fel i wa ap propriat t d so However thi bi o re strain wa couple wit a hig a per centag o larg odd estimate a wa give th untutore subject i Experimen 2 Abou on quarte o al answer wer as signe odd equa t o greate tha 1,000:1 Onc again answer t whic extremel hig odd ha bee assigne wer frequentl wrong A odd o 10:1 subject wer cor rec o abou thre ou o ever fou ques tions appropriat t odd o 3:1 A

100:1 the shoul hav bee sayin 4:1 A 1,000 an a 100,000:1 estimate o abou 7: an 9: woul hav bee mor i keepin wit subjects actua abilities Ove th larg numbe o question fo whic peopl gav odd o 1,000,000: o higher the wer wron a averag o abou 1 tim ou ever 16 Th gambling game. O th 4 subjects agree t pla th gamblin gam de scribe abov fo $1 Si mor agree whe th stake wer raise t $1.5 ever tim th experimente dre a re chip O th holdouts 3 subject agree t pla a $ fo ever re chi an 2 mor agree whe th fina offe o $2.5 wa made Onl 3 sub ject refuse t participat a an leve o paymen pe re chip Afte subject

ha mad thei decision abou playin th game the wer aske whethe the woul chang thei mind i th gam wer t b played o th spot fo rea money N subjec indicate a desir t chang hi o he decision Tw subject approache th experimente afte th ex perimen requestin tha the b give a chanc t pla th gam fo cash Thei re ques wa refused
Page 8
KNOWIN WIT CERTAINT 55 course thi gam i strongl biase favo o th experimenter Sinc subject wer wron abou onc fo ever eigh an swer assigne odd o 50: o greater th gam woul hav bee approximatel fai ha th experimente remove 8 o th whit chip fro th bag leavin it con tent

a 1 whit an 2 re chips Th expecte outcom o playin th gam wit eac subjec wa simulated. Ever wron answe o a "50: o greater questio wa assume t cos th subjec $1 Th experimente wa assume t hav draw 1/5 o a re chi fo ever answe give a odd greate tha o equa t 50:1 hi expecte los wa the calculate i ac cordanc wit th be th subjec ha ac cepted Fo example i a subjec accepte th experimenter' firs offe ($ pe re chip an gav 1 "50: o greater an swers th experimenter' simulate los wa 17/5 dollar (33^5) The subject who agree to pla average 38. question wit odd greate tha o equa t 50:1 Thirty-si person ha ex

pecte monetar losses an thre ha ex pecte wins Individua expecte outcome range betwee a los o $25.6 an a gai $1.84 Th mea expecte outcom wa los o $3.6 pe perso an th media outcom wa a los o $2.35 Te person woul hav los mor tha $5 Th 3 sub ject woul hav los a tota o $142.1 acros 1,49 answer a odd greate tha o equa t 50:1 a averag los o 9.5$ fo ever suc answer Th tw person wh earnestl requeste specia permissio t pla th gam ha expecte losse totalin $33.3 betwee them Experimen 5 Playin fo Keep Subject i Experimen 4 viewe thei overconfiden odd judgment a faithfu enoug reflection o thei stat o knowl

edg tha the wer willin t accep hypo thetica bet mor disadvantageou tha man tha ca b foun i a La Vega casino Befor concludin tha ther i mone t b mad i "trivi hustling, w decide t replicat Experimen 4 wit rea gamblin a th end Method Ninetee subject participate i Experimen 5 differe fro Experimen 4 onl i tha th gamblin gam wa presente a a rea game Afte respondin t th 10 items subject hear th gamblin gam instruction an decide whethe o no the woul play The wer tol tha the coul los al th mone the ha earne th experimen an possibl eve mor tha that Afte the mad thei decision abou playin th game subject

wer tol tha an earning fro th gam woul b adde t thei pa fo th experiment bu tha i the los money non th mone initiall promise the fo partici patin woul b confiscated Th gam wa the playe o thos terms Si o th 1 subject agree t pla th gam firs specifie (wit a $ paymen fo eac "ex perimenter' surprise") Thre mor agree t pla whe th experimente offere t increas th paymen t $1.5 pe re chip Increasin th pay men t $ brough i on additiona player an thre mor agree t pla a $2.50 Si subject consistentl refuse t participate som becaus the fel the wer no wel calibrated other be caus the di no lik t gamble

Results Whe th gam wa actuall played th participatin subject misse 4 o th 38 answer (11.9% t whic the ha as signe odd greate tha o equa t 50:1 Al subject woul hav los money rangin fro $ t $1 (i par because b chance re chip wer drawn) Whe th ex perimenter' par o th gam wa simulate i Experimen 4 fou subject woul hav los mor tha $6 an th averag participatin subjec woul hav los $2.64 Thus th hypothetica natur o th gambl Experimen 4 apparentl ha minima influenc o subjects willingnes t bet Genera Subjec an Ite Analyse undu confidenc foun onl i a fe subject o onl fo a fe specia items I case o extrem

overconfidenc ar concen trate i onl a fe subjects then th gen eralit o ou conclusion woul b limited Pathologica overconfidenc o th par o smal secto o th publi woul b wort
Page 9
56 FISCHHOFF P SLOVIC AN S LICHTENSTEI Tabl 4 Frequency of Extreme Overconfidence (Odds Greater Than or Equal to 50:1 That Were Assigned to Wrong Answers) No case extrem overconfidenc Mor tha 17 Numbe Experimen 3 (32 subject Experimen 4 (27 No extremel overconfiden subject Numbe Experimen 3 33 item Experimen 4 Ther wer fiv subject i Experimen 3 wh neve showe extrem overconfidence Ther wer 3 item i Experimen 3

fo whic n subjec showe extrem overconfidence Actua numbe o case i i parentheses explorin furthe bu woul no tel u muc abou cognitiv functionin i general Th result o th gamblin game reporte abov sho tha thi wa no th case Mos subject wer willin t pla an mos woul hav los mone becaus the wer to ofte wron whe usin extrem odds Th lef column o Tabl 4 sho th dis tributio o case o extrem overconfidenc (define a givin odd o 50: o greate an bein wrong ove subject fo Experi ment 3 an 4 Th grea majorit o sub ject ha on o mor case o extrem overconfidence The media numbe was 4 Experimen 4 an betwee 3 an 4 i

Experimen 3 wel ove wha woul b ex pecte wit well-calibrate subjects I eac experiment on subjec appeare t b a outlie (thos subject havin 3 an 2 cases) Reanalyzin th dat afte removin thos tw subject ha n effec o ou con clusions Th righ column o Tabl 4 sho th distributio o case o extrem overconfi denc ove items I mos case wer concen trate i onl a fe items th situatio woul b rathe differen tha i a broa sectio o item foole som o th peopl som o th time I woul no necessaril les interesting fo i woul remai t explaine wh peopl wen astra o thos fe items A th result i Tabl 4 Tabl 5 Percentage Wrong with

Deceptive and Nondeceptive Items Percentag wron associate wit odd o Experimen an ite >50: >1000: Experimen 3 Al item (106 16. 14. 12. Deceptiv item (18 73. 75.5 72. Nondeceptiv item (88 8. 6. 6. Experimen 4 Al item (106 13. 13. 10. Deceptiv item (17 73. 76. 70. Nondeceptiv item (89 7. 6. 5. Expecte wit perfec calibratio <1.9 <.9 <.1
Page 10
KNOWIN WIT CERTAINT 56 indicate bot situation see to hav bee true Ther ar som item o whic man peopl gav hig odd t th wron answer bu mos item di sho a fe suc cases Th item o whic si o mor subject showe extrem overconfidenc wer al item tha migh b

describe a "decep tive, one whic les tha 50 o th sub ect answere correctly Som correlatio betwee deceptivenes an extrem overcon fidenc i inevitable man subject mus ge answe wron befor man ca ge i wron an b certai tha the ar right Ther wer 1 item i Experimen 3 an item i Experimen 4 answere cor rectl b les tha 50 o ou subjects Tabl 5 show th incidenc o case o extrem overconfidenc wit deceptiv an nondeceptiv items Althoug extrem over confidenc i disproportionatel prevalen wit th deceptiv items i i stil abundan wit th nondeceptiv ones I th deceptiv item ar remove fro th sample the th remainin

distributio o case o ex trem overconfidenc ove item closel re semble a Poisso distribution whic i wha on woul expec i suc case wer distribute a rando ove items On thir th easies items thos answere cor rectl b 90 o mor o ou subjects ha leas on cas o a subjec answerin wrongl an givin odd o bein correc o 1,000: o greater Deletin th on ex trem subjec fro eac o Experiment 3 an 4 ha littl effec o thi result Clearly fe subject o item ar no responsibl fo th extrem overconfidenc effect Genera Discussio Thes fiv experiment hav show peo pl t b wron to ofte whe the ar certai tha the ar right Thi resul wa

obtaine wit bot probabilit an odd re sponses wit minima an extensiv in struction an wit tw rathe differen type o questions Subject wer sufficientl comfortabl wit thei expression o cer taint tha the wer willin t ris mone the i bot hypothetica an rea gam bles Finally case o extrem overconfi denc wer widel distribute ove subject an items Althoug thes studie hav show th effec t b a robus one the hav cer tainl no close th topic Furthe researc wit differen subjects differen items an differen instruction woul b mos useful Som moderatel informe guesse a th result o suc additiona studie ar pos sible

Lichtenstei an Fischhof (i press hav foun tha th calibratio o probabilit response associate wit general-knowledg question i relativel invarian wit regar severa factor no considere here in cludin subjects intelligence subjects ex pertis i th subject-matte are o th question an subjects relianc o th ste reotypi response o .5 an 1.00 The did however fin tha calibratio varie wit ite difficulty crucia questio fo generalit i ho wel th leve o ite difficult foun i thes experiment represent th leve foun i th world Althoug n simpl answe t thi questio i possible i i wort notin tha th item i Experiment 2

an 3 wer no constructe wit th intentio o elicit in extrem overconfidence Rather the wer constructe t var i difficult fro ver har t ver easy a define b th rati o th statistica frequencie o deat fro eac o th tw causes Item i Ex periment 4 an 5 wer matche t thes item i difficulty explai thes results w mus under stan bot ho peopl answe question an ho the asses th validit o thei answer in process Collin (Collins Warnock Aiello & Miller 1975 Collins Not 3 ha show tha peopl us man differen strate gie i answerin questions W suspect therefore tha extrem overconfidenc ca com fro a variet o sources Ever

an swerin procedur ma hav it ow way leadin peopl astra an it ow way hidin tha misguidanc whe peopl tr asses answe validity Som possibl pathway t overconfidenc ar describe below Man o th item w presente t ou subject ar o topic fo whic the d no
Page 11
56 FISCHHOFF P SLOVIC AN S LICHTENSTEI Tabl 6 Deceptive Items in Experiment 3 Cause o deat compared Pregnancy abortion childbirt versu appendiciti Al accident versu strok Homicid versu suicid Measle versu firework Suicid versu diabete Breas cance versu diabete Percen correc 17. 27. No case o extrem overconfidence Subject judge th firs

caus o deat listed t b les fre quen tha th second Dat ar th numbe o subject (ou o 40 wh gav odd greate tha o equa t 50: t th wron alternative hav a read answe store i memory The mus infe th answe fro othe in formatio know t them Bu peopl ma insufficientl critica o thei inferenc processes The ma fai t as "Wha wer assumption i derivin tha infer ence? o "Ho goo a I a makin suc inferences? Fo example whe peopl dra a fe instance o a categor fro memor t ge a ide o th propertie o th category the ma no realiz tha readil availabl example nee no b rep resentativ o th categor (Tversk & Kahneman 1973)

Waso an Johnson Lair (1972 hav show tha peopl hav Tabl 7 Deceptive Items in Experiment 4 considerabl confidenc i thei ow errone ou syllogisti reasoning Collin e al (1975 hav describe a variet o inferen tia strategie tha peopl us i producin answer withou realizin thei limitations Summarizin he studie o th inferenc proces i perception Johnson-Abercrombi (1960 concluded "Thefs erroneous in ference wer no arrive a a a serie o logica step bu swiftl an almos uncon sciously Th validit o th inference wa usuall no inquire into indeed th pro ces wa usuall accompanie b a feelin o certaint o bein right (p

89) Pit (1974) wh als observe overconfidenc probabilit estimates elaborate a simila hypothesis H propose tha peopl ten t trea th result o inferentia processe a thoug ther wa n uncertaint associate wit th earl stage o th inference Suc strateg i simila t th "best-guess heuristi tha ha bee foun t describ th behavio o subject i cascade inferenc task (e.g. Gettys Kelly & Peterson 1973) Fo othe questions peopl believ tha the ar answerin directl fro memor withou makin an inferences Peopl com monl vie thei memorie a exac (al General-knowledg question Answers' No case Percen o extrem correc

overconfidence Thre fourth o th world' caca come fro Whic cause mor death i th U.S. Africa o Sout Americ Appendicitis o preg 4. 19. Whe wa th firs ai raid Adoni wa th go o Kahli Gibra wa mos inspire b whic religion Dido an Aeneas i a oper writte b Potatoe ar nativ t nancy abortion an childbirt 1849 o 193 26. Lov o vegetation 31. Buddhis o Christian 33. Berlio o Purcell 33. Irelan o Peru 35. Som question hav bee abbreviate slightly Correc answe carrie a asterisk Dat ar th numbe o subject (ou o 42 wh gav odd greate tha o equa t 50: t th wron alternative
Page 12
KNOWIN WIT CERTAINT 56

thoug perhap faded copie o thei origi na experiences However considerabl evi denc ha demonstrate tha memor i mor tha jus a copyin proces (e.g. Neisser 1967) Accordin t thi view peo pl reac conclusion abou wha the hav see o wha the remembe b reconstruct in thei knowledg fro fragment o in formation muc a a paleontologis infer th appearanc o a dinosau fro fragment bone Durin reconstruction a variet o cognitive social an motivationa factor ca introduc erro an distortio int th outpu o th process Example o thi ar th foible o eyewitnes testimon docu mente b Buckhou (1974) Loftu (1974) Miinsterber

(1908) an others peopl ar unawar o th reconstruc tiv natur o memor an perceptio an canno distinguis betwee assertion an inference (Harri & Monaco i press) the wil no criticall evaluat thei in ferre knowledge I general an proces tha change th content o memor un beknowns t peopl wil kee the fro askin relevan validit question an ma lea t overconfidence I hi classi studie reconstructiv processe i memory Bartlet (1932 foun tha subject no onl create ne materia bu wer ofte highl certai abou tha whic the ha invented. presen thes idea mor a a frame wor fo futur researc an conceptuali zatio tha a a

explanatio fo ou re sults Nonetheless i thes speculation hav som validity i shoul b possibl t fin apparen example i ou data Table 6 an presen th mos deceptiv item fro Ex periment 3 an 4 respectively Althoug case o extrem overconfidenc wer dis tribute ove mos i'tems thes "deceptive item produce a disproportionat share I th absenc o detaile protocol fro sub jects thes case wher man peopl wen astra ma provid bette clue t ou in tuition tha situation wher jus on o tw subject ha troubl wit a item Lookin a th deceptiv item i Experi men 3 (se Tabl 6) w fin tha i man case th caus o deat incorrectl

judge b mor frequen (th firs on liste i eac pair i a dramatic well-publicize event wherea th underestimate caus i mor "quiet killer Considerin th firs thre examples (a pregnancy abortion an childbirth (b accidents an (c ho micid see disproportionatel mor news worth an bette reporte tha thei com pariso caus o death. I thes cases peo pl ma b relyin o th greate avail abilit i memor o example o th "flash ier cause o deat withou realizin tha availabilit i a imperfec inferentia rul (Tversk & Kahneman 1973). Othe item sugges othe answerin processes Subjects confident—bu erroneous—belief (no show i

Tabl 6 tha ther wer fewe death fro smallpo vaccin tha fro th diseas itsel ma hav bee base th generall vali assumptio tha vac cine ar safe tha th disease the ar mean t prevent Wit smallpox however th vaccin ha bee s successfu tha n on ha die o th diseas i th U.S sinc 1949 whil fro 6 t 1 peopl hav die A exampl o th subtl rol o assumption th reconstructio o knowledg come fro th experienc o on o th author wh becam em broile i a friendl debat wit a colleagu abou th date o a forthcomin conference Bot par tie agree tha th conferenc wa t las abou t S days Bu th disput centere abou whethe thes date wer

Marc 3 t Apri 3 o Apri 3 Ma 3 Th autho wa certai o th forme date becaus h specificall recalle th dat Marc 3 th organizer' letter Hi colleagu wa certai th latte perio becaus h specificall recalle th dat Ma 3 i th letter Bet wer placed an th lette wa consulte t resolv th dispute T th surpris o bot parties th lette state th date a Marc 3 t Ma 3 a obviou mistake Thus bot partie wer correc regardin th frag men o informatio the recalled bu on frag men le t th wron conclusion Thi speculatio ha bee empiricall affirme Lichtenstein Slovic Fischhoff Combs an Lay ma (Not 2) Subject i Experimen 3 wer aske

t selec on answe abou whic the wer certai an t writ a shor statemen indicatin wh the wer confident On subjec explaine odd o 2,000: tha deat fro pregnanc wa mor frequen tha death b appendiciti b writin "I'v neve hear a perso dyin o appendicitis bu I hav man time hear o person dyin durin childbirt an abortion.
Page 13
56 FISCHHOFF P SLOVIC AN S LICHTENSTEI annuall fro complication arisin fro vaccination Fo th general-knowledg question o Experimen 4 i Tabl 7 w wil giv a fe interpretation o th varie way tha un recognize o inadequatel questione as sumption ca obscur th tenuousnes o

erroneou beliefs th reade ca surel pro vid others Regardin Ite 1 caca i na tiv t Sout America Subject wh kne thi fac (o guesse i fro th Spanish soundin name ma hav bee misle b assumin tha th continen o origi i als th continen o greates production Simila reasonin ma hav bee involve wit Ite 7 Th potato' prominenc i Iris histor doe no mea tha i originate there Regardin Ite 3 i ma no hav occurre t subject tha a ai rai coul conducte b balloons whic wer use Austri t bom Venic i 1849 Th fac tha Adoni wa a handsom yout wh ha a affai wit Venus th Goddes o Love ma hav suggeste tha he too wa diet o lov

(Ite 4) An s on Finally le u ad a warnin tha ex trem overconfidenc cut bot ways Ou source fo th answer t general-knowl edg question wer a variet o encyclo pedia an dictionaries W viewe th an swer the provide wit grea confidence Muc t ou chagrin w discovere o sev era occasion tha thes authorativ source disagreed a possibilit w ha neve con sidered Fortunately ou ow overconfi denc wa discovere befor conducting thes experiments th offendin item wer delete an th remainin one double an triple-checke unti w wer certain o thei accuracy Referenc Note Fischhoff B. & Lichtenstein S Th effect o response

mode and question format on calibra- tion (Rep No 77-1) Eugene Oregon Deci sio Research 1977 Lichtenstein S. Slovic P. Fischhoff B. Combs B. & Layman M Perceived frequency lo probability, lethal events (Rep No 76 2) Eugene Oregon Decisio Research 1976 Collins A Processes i acquiring knowledge (Rep No 3231) Cambridge Mass. Bol Bera ne an Newman Inc. 1976 Reference Bartlett F C Remembering. Cambridge Eng land Cambridg Universit Press 1932 Buckhout R Eyewitnes testimony Scientific American, 1974 231, 23-31 Collins A. Warnock E H. Aiello N. & Miller Reasonin fro incomplet knowledge I Bobro & A

Collin (Eds.) Representation an understanding. Ne York Academi Press 1975 Gettys C F. Kelly C. Ill & Peterson C R Th bes gues hypothesi i multistag infer ence Organizational Behavior an Human Per- formance, 1973 10 364-373 Harris R ]., & Monaco G E Psycholog o pragmati implication Informatio processin betwee th lines Journal o Experimental Psy- chology: General, i press Johnson-Abercrombie M L Th anatomy o judgment. Ne York Basi Books 1960 Lichtenstein S. & Fischhoff B D thos wh kno mor als kno mor abou ho muc the know Th calibratio o probabilit judgments Organizational Behavior and Human

Perform- ance, i press Lichtenstein S. Fischhoff B. & Phillips L D Calibratio o probabilities Th stat o th art H Jungerma & G d Zeeu (Eds.) Deci- sion making an change i human affairs. Am sterdam Th Netherlands Reidel i press Loftus E Th incredibl eyewitness Psychology Today, Decembe 1974 pp 116-119 Miinsterberg H O th witness stand. Ne York Doubleday Page 1908 Neisser U Cognitive psychology. Ne York Appleton-Century-Crofts 1967 Pitz G Subjectiv probabilit distribution fo imperfectl know quantities I G W Greg (Ed.) Knowledge an cognition. Ne York Wiley 1974 Tversky A. & Kahneman D Availability

A heuristi fo judgin frequenc an probability Cognitive Psychology, 1973 5 207-232 Tversky A. & Kahneman D Judgmen unde uncertainty Heuristic an biases Science, 1974 185, 1124-1131 Wason P C. & Johnson-Laird P N Psychology reasoning: Structure an content. Cambridge Mass. Harvar Universit Press 1972 Receive Decembe 14 197