Measuring Belief Bias with Ternary Response Sets - PowerPoint Presentation

Slide1

Measuring Belief Bias with Ternary Response Sets

Samuel Winiger, Henrik Singmann, David Kellen

Syllogism

Logical arguments:Premise:Premise:Putative Conclusion

No oaks are

jubs

.

Some trees are jubs.Therefore, some trees are not oaks.

Valid

Inv

alid

Belief Bias in Syllogistic Reasoning

Believability

Validity

Believable

Unbelievable

Valid

No oaks are jubs.No trees are jubs.

Some trees are

jubs

.Some oaks are jubs.Therefore, some trees are not oaks.Therefore, some oaks are not trees.InvalidNo trees are jubs.No oaks are jubs.Some oaks are jubs.Some trees are jubs.Therefore, some trees are not oaks.Therefore, some oaks are not trees.

Belief Bias in Syllogistic Reasoning

Believability

Validity

Believable

Unbelievable

Valid

No oaks are jubs.No trees are jubs.

Some trees are

jubs

.Some oaks are jubs.Therefore, some trees are not oaks.Therefore, some oaks are not trees.InvalidNo trees are jubs.No oaks are jubs.Some oaks are jubs.Some trees are jubs.Therefore, some trees are not oaks.Therefore, some oaks are not trees.

89

%

71%

56%

10%

Data: Evans et al. (1983)

Belief Bias in Syllogistic Reasoning

Believability

Validity

Believable

Unbelievable

Valid

No oaks are jubs.No trees are jubs.

Some trees are

jubs

.Some oaks are jubs.Therefore, some trees are not oaks.Therefore, some oaks are not trees.InvalidNo trees are jubs.No oaks are jubs.Some oaks are jubs.Some trees are jubs.Therefore, some trees are not oaks.Therefore, some oaks are not trees.

89

%

71%

56%

10%

Data: Evans et al. (1983)

Two possible explanations:

Believability affects reasoning processes

(e.g.,

more effort for unbelievable syllogisms)

Believability

affects response bias

(e.g., higher propensity for accepting believable

conclusions)

Threshold-Model for Belief Bias

Klauer,

Musch

&

Naumer

, (2000)

Threshold-Model for Belief Bias

Klauer,

Musch

&

Naumer

, (2000)

Problem:

Data provides 4 independent data points

Model has 6 free parameters

(4 reasoning and 2 guessing parameters)Model parameters not uniquely identifiedKlauer et al.'s solution: response bias manipulationParticipants in one of three bias condition:17% versus 50% versus 83% validFix reasoning parameters across conditions, but allow for different response biasBelief bias solely affected reasoning processes.

Signal Detection Model of Belief Bias

Dube,

Rotello

, &

Heit

(2010)

Signal Detection Model of Belief Bias

Dube,

Rotello

, &

Heit

(2010)

Signal detection based analysis with confidence-rating:Belief bias mainly a response bias effect!(also Trippas, Kellen, Singmann, et al. in press, PB&R)

Experiment

Syllogism evaluation task with 3 response options:"valid""I don't know""invalid"

Belief Bias: Driven by reasoning processes or response processes?

Logical validity (valid vs. invalid)

Conclusion believability (believable vs. unbelievable)

354 Participants (online study) 8 syllogisms per participant7/26/2018Title of the presentation, AuthorPage 10

Results: Response Frequencies

7/26/2018Title of the presentation, Author

Page

11

r

vu

1-

r

vu

n

u

1-

n

ugu1-guI don’t knowvalidvalidinvalid

valid and unbelievable

r

vb

1-

r

vb

n

b

1-

n

b

g

b

1-

g

b

I don’t know

valid

valid

invalid

valid and believable

r

ib

1-

r

ib

n

b

1-

n

b

g

b

1-

g

b

I don’t know

invalid

valid

invalid

invalid and believable

r

iu

1-

r

iu

n

u

1-

n

u

g

u

1-

g

u

I don’t know

invalid

valid

invalid

invalid and unbelievable

Extended Threshold Model

Modeling Results: Extended belief bias MPT

7/26/2018Title of the presentation, Author

Page

13

Modeling Results: Extended belief bias MPT

7/26/2018Title of the presentation, Author

Page

14

BF

0: 1.4 – 2.0

BF0: 0.8 – 1.6

BF

0

: 2.1 – 4.5 BFAlt: 16 – 230

Summary

Binary "Valid"/"Invalid" response format does not provide independent data points for comprehensive measurement model of belief bias

Klauer et al. (2000): threshold model

Response bias manipulation

Believability affects reasoning processes

Dube et al. (2010); Trippas et al. (in press); Stephens, Dunn, & Hayes (2017): signal-detection modelConfidence-ratingsBelievability affects response processesOur results: extended threshold modelTernary response setsBelievability affects response processes (i.e., larger propensity for responding "valid" for believable syllogisms)Differences in conclusions not dependent on model, but manipulation for achieving parameter identifiability.Substantive theories of belief bias must include response bias!7/26/2018Title of the presentation, Author

Page 15

7/26/2018

Title of the presentation, AuthorPage

16

Thank you for your attention.

Questions?

Syllogisms

Structures

Complex

structures (indeterminately invalid)

-

Dube et al. (2010) experiments 1-3- Klauer et al. (2000) experiments 3, 4, and 7.This set includes Trippas et al. (2013) and Stephens et al. (2017)

ContentsRated contents from all around the literatureKlauer et al., 2000

Dube

et al., 2010

Ball, Phillips, Wade, & Quayle, 2006Oakhill & Johnson-Laird, 1985Quayle & Ball, 2000; Evans et al., 19837/26/2018Title of the presentation, AuthorPage 17

Results: Bayes Factors

Bayes factors derived from the difference parameters between conditions* Bayes factors in favor of the null Hypothesis

7/26/2018

Title of the presentation, Author

Page

18Parameterrvrin

gnarrow prior1.4*1.22.1*232.0medium prior1.5*1.3*3.1*

70.3

wide

prior2.0*1.6*4.5*16.1