/
Martin J. Pickering, Simon Martin J. Pickering, Simon

Martin J. Pickering, Simon - PowerPoint Presentation

olivia-moreira
olivia-moreira . @olivia-moreira
Follow
371 views
Uploaded On 2016-03-06

Martin J. Pickering, Simon - PPT Presentation

Garrod Behavioral and Brain Sciences 2012 An Integrated Theory of Language Production and Comprehension Computer Science amp Engineering 201220835 SangWoo Lee Background Aphasia Expressive aphasia ID: 244327

production action language comprehension action production comprehension language perception predict model implementer amp utterance prediction interactive joint process movements

Share:

Link:

Embed:

Download Presentation from below link

Download Presentation The PPT/PDF document "Martin J. Pickering, Simon" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Slide1

Martin J. Pickering, Simon Garrod, Behavioral and Brain Sciences, 2012.

An Integrated Theory of Language Production and Comprehension

Computer Science & Engineering

2012-20835

Sang-Woo LeeSlide2

Background - AphasiaSlide3

Expressive aphasiaAgrammatic aphasiaUnderstand what other people say, but cannot speak the sentence well.

Caused by damage to, or developmental issues in the anterior regions of the brainIncluding (but not limited to) the Broca’s area

3

Broca’s

AphasiaSlide4

Also known as Receptive AphasiaFluent apahsia

, or sensory aphasiaSpeak the sentence fluently, but not well-organized sense in their speechTraditionally associated with neurological damage to Wernicke’s area in the brain(Actually it is not just simply associated to

Wernicke’s

area in current experimental result, but anyway…)

4

Wernicke’s

AphasiaSlide5

5

There are modules which specify some function

The “classical

Lichtheim-Broca-Wernicke

” Model

perception

actionSlide6

Background - AphasiaTraditional independence of production and comprehensionInterweaving in action and action perception

Perception process in actionPredict next action of otherJoint ActionInterweaving in Production and ComprehensionComprehension process in ProductionPredict next speech of other

Interactive Language

Professor’s Question

Contents

6Slide7

Traditional independence of

production and comprehensionSlide8

8

Traditional model of communication

Discrete stages

A produces, B comprehends

B

produces, A comprehendsSlide9

Assumes “

horizontal split” between production and comprehensionArrows-within-arrows indicate feedback (in interactive accounts)

But this feedback is internal to production or comprehension

It may involve

general knowledge

But production does not involve comprehension processes

And comprehension does not involve production processes

9

Horizontal SplitSlide10

Interlocutors are not static, as the traditional model assumes, but are “moving targets” performing a joint activitiy (

Garrod & Pickering, 2009)10

Example of predict

in Behavioral InstanceSlide11

11

Example of predict in Neuroscience

Big(neuter)

Painting (neuter)

Big (common)

Bookcase (common)

(Pickering &

Garrod

, 2007)

große

Gemälde

großen

BücherschrankSlide12

Also, many experiments demonstrate effects of one on the otherPicture-word interference (Schriefers

et al., 1990)Word identification affected by externally controlled cheek movement (Ito et al., 2009)And strongly overlapping neural circuits for production and comprehension (e.g., Pulvermüller

&

Fadiga

, 2010;

Scott et al., 2009)

12

Other CounterexampleSlide13

Prediction process could be naturally understood with production moduleWhen they comprehend the utterance, they also use production model internally.

13

Result

-

There is forward model

To p

redict

perception

caused by their own

utterance

- Fast alert when you say something wrongSlide14

Interweaving in action and action perception

Perception process in action

Predict next action of other

Joint ActionSlide15

Interweaving in action and action perception

Close links between action and action perception, e.g.participants

arm movements affected by observing another person

s arm movements (

Kilner

et al., 2003)And making hand movements can facilitate concurrent visual discrimination of deviant hand postures (

Miall

et al., 2006)

Such links could have various purposes

Supporting overt imitation

facilitating memory or understanding (

postdictively

)

But

authors

propose that they aid

prediction

of own and others

actions, by use of a

forward model

Based on computational neuroscience (

Wolpert

, 1997; see

Grush

, 2004)Slide16

Forward modelling in actionIn our terms, the action command

causes the action implementer to move the hand and the perceptual implementer to construct the perceptAnd the efference copy causes the forward action model

to generate the predicted hand movement and the

forward perceptual model

to construct the predicted percept Slide17

Forward Modeling in Action

Just act

Feel involved percepts of your own act

e.g. Own coordination info

Feeling of wind blowing to your arm

Gravity info … Slide18

18

Efference CopySlide19

Forward Modeling in Action

- Predict perception caused by their own action

e.g. Own coordination info

Feeling of wind blowing to your arm

Gravity info … Slide20

Prediction-by-Simulation

- Predict perception of other’s next actionby seeing other’s current actionSlide21

Joint Action

People are highly adept at joint activities (Sebanz et al., 2006).

ballroom dancing, playing a duet, carrying a large object together

Precise timing is crucial

To succeed,

A

predicts

B’

s action and

B

predicts

B

’s actionSlide22

22

Joint ActionSlide23

Interweaving in Production and Comprehension

Comprehension process in Production

Predict next speech of other

Interactive LanguageSlide24

Forward modeling in language production

Action implementer 

production implementer

Perceptual implementer

comprehension implementer

Action command

production command

Drives the production implementer

Efference

copy drives the forward models

Comparator

monitor

compares the utterance percept and the predicted utterance perceptSlide25

Unifying production and comprehension

Production and comprehension are interwovenTight coupling in dialogue (Clark, 1996; Pickering & Garrod

, 2004)

Behavioural experiments show effects of comprehension processes on production and vice versa (e.g.,

Schriefers

et al., 1990)

Overlap of brain circuits for production and comprehension (e.g.,

Pulvermuller & Fadiga, 2010)

Such interweaving facilitates prediction of self and other

s utterancesSlide26

26

Classical modeling in language production

Just say utterance,

Listen what you say.Slide27

27

Forward modeling in language production

- Predict perception

caused by their own

utterance

- Fast alert when you say something wrongSlide28

Self-monitoringSpeaker wishes to say

kiteIn the past, she has always constructed the kite-concept and then uttered /k/

She therefore constructs forward model p^[phon](t) =

/k/

If she then incorrectly constructs p[phon] =

/g/

, the monitor notices the mismatch

If she believes the forward model, she will detect an error (and perhaps reformulate)

Otherwise, she will alter her forward modelSlide29

29

Prediction-by-simulation

- Predict perception of other’s next

utterance

by

listening

other’s current

utterance

Big(neuter)

Painting (neuter)Slide30

30

Interactive Language

Joint action involves combining accounts of action and action perception

Similarly, interactive language involves combining accounts of production and comprehension

Facilitates coordination (e.g., short intervals between speakers; Wilson & Wilson, 2005)

Facilitates alignment (developing same representations; Pickering &

Garrod

, 2004)Alignment in turn facilitates comprehension (better prediction of others)Slide31

31

Interactive LanguageSlide32

ConclusionWe propose that language production and comprehension are interwoven

It assumes a central role to prediction in production, comprehension, and dialogueSpeakers construct forward models to predict aspects of their upcoming utterancesListeners covertly imitate speakers and use forward models to predict the speakersOur account helps explain the efficiency of production and comprehension and the remarkable fluidity of dialogueSlide33

Thank youSlide34

Professor’s

Question

Comprehension process in Production

Predict next speech of other

Interactive LanguageSlide35

Q1: Give the evidence for how language production and comprehension are tightly interwoven. How does this relate to the perception-action cycle theory of cognitive systems?

35Question 1Slide36

Q2: Explain and give the evidence for how action, action perception, and joint action are interwoven. Explain how the authors use this to develop accounts of production, comprehension, and interactive language.

36Question 2

Action implementer

production implementer

Perceptual implementer

comprehension implementer

Action command

production command

Comparator

monitor

participants

arm movements affected by observing another person

s arm movements (

Kilner

et al., 2003)

And making hand movements can facilitate concurrent visual discrimination of deviant hand postures (

Miall

et al., 2006)Slide37

Q3: Give examples of what behavioral and neuroscientific data on language processing can be explained by the integrated theory of language production and comprehension explains, while modular theory does not.

37

Question 3

- Behavioral data

-

Neuroscientific

data (Pickering &

Garrod

, 2007)