Garrod Behavioral and Brain Sciences 2012 An Integrated Theory of Language Production and Comprehension Computer Science amp Engineering 201220835 SangWoo Lee Background Aphasia Expressive aphasia ID: 244327
Download Presentation The PPT/PDF document "Martin J. Pickering, Simon" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Martin J. Pickering, Simon Garrod, Behavioral and Brain Sciences, 2012.
An Integrated Theory of Language Production and Comprehension
Computer Science & Engineering
2012-20835
Sang-Woo LeeSlide2
Background - AphasiaSlide3
Expressive aphasiaAgrammatic aphasiaUnderstand what other people say, but cannot speak the sentence well.
Caused by damage to, or developmental issues in the anterior regions of the brainIncluding (but not limited to) the Broca’s area
3
Broca’s
AphasiaSlide4
Also known as Receptive AphasiaFluent apahsia
, or sensory aphasiaSpeak the sentence fluently, but not well-organized sense in their speechTraditionally associated with neurological damage to Wernicke’s area in the brain(Actually it is not just simply associated to
Wernicke’s
area in current experimental result, but anyway…)
4
Wernicke’s
AphasiaSlide5
5
There are modules which specify some function
The “classical
Lichtheim-Broca-Wernicke
” Model
perception
actionSlide6
Background - AphasiaTraditional independence of production and comprehensionInterweaving in action and action perception
Perception process in actionPredict next action of otherJoint ActionInterweaving in Production and ComprehensionComprehension process in ProductionPredict next speech of other
Interactive Language
Professor’s Question
Contents
6Slide7
Traditional independence of
production and comprehensionSlide8
8
Traditional model of communication
Discrete stages
A produces, B comprehends
B
produces, A comprehendsSlide9
Assumes “
horizontal split” between production and comprehensionArrows-within-arrows indicate feedback (in interactive accounts)
But this feedback is internal to production or comprehension
It may involve
“
general knowledge
”
But production does not involve comprehension processes
And comprehension does not involve production processes
9
Horizontal SplitSlide10
Interlocutors are not static, as the traditional model assumes, but are “moving targets” performing a joint activitiy (
Garrod & Pickering, 2009)10
Example of predict
in Behavioral InstanceSlide11
11
Example of predict in Neuroscience
Big(neuter)
Painting (neuter)
Big (common)
Bookcase (common)
(Pickering &
Garrod
, 2007)
große
Gemälde
großen
BücherschrankSlide12
Also, many experiments demonstrate effects of one on the otherPicture-word interference (Schriefers
et al., 1990)Word identification affected by externally controlled cheek movement (Ito et al., 2009)And strongly overlapping neural circuits for production and comprehension (e.g., Pulvermüller
&
Fadiga
, 2010;
Scott et al., 2009)
12
Other CounterexampleSlide13
Prediction process could be naturally understood with production moduleWhen they comprehend the utterance, they also use production model internally.
13
Result
-
There is forward model
To p
redict
perception
caused by their own
utterance
- Fast alert when you say something wrongSlide14
Interweaving in action and action perception
Perception process in action
Predict next action of other
Joint ActionSlide15
Interweaving in action and action perception
Close links between action and action perception, e.g.participants
’
arm movements affected by observing another person
’
s arm movements (
Kilner
et al., 2003)And making hand movements can facilitate concurrent visual discrimination of deviant hand postures (
Miall
et al., 2006)
Such links could have various purposes
Supporting overt imitation
facilitating memory or understanding (
“
postdictively
”
)
But
authors
propose that they aid
prediction
of own and others
’
actions, by use of a
forward model
Based on computational neuroscience (
Wolpert
, 1997; see
Grush
, 2004)Slide16
Forward modelling in actionIn our terms, the action command
causes the action implementer to move the hand and the perceptual implementer to construct the perceptAnd the efference copy causes the forward action model
to generate the predicted hand movement and the
forward perceptual model
to construct the predicted percept Slide17
Forward Modeling in Action
Just act
Feel involved percepts of your own act
e.g. Own coordination info
Feeling of wind blowing to your arm
Gravity info … Slide18
18
Efference CopySlide19
Forward Modeling in Action
- Predict perception caused by their own action
e.g. Own coordination info
Feeling of wind blowing to your arm
Gravity info … Slide20
Prediction-by-Simulation
- Predict perception of other’s next actionby seeing other’s current actionSlide21
Joint Action
People are highly adept at joint activities (Sebanz et al., 2006).
ballroom dancing, playing a duet, carrying a large object together
Precise timing is crucial
To succeed,
A
predicts
B’
s action and
B
predicts
B
’s actionSlide22
22
Joint ActionSlide23
Interweaving in Production and Comprehension
Comprehension process in Production
Predict next speech of other
Interactive LanguageSlide24
Forward modeling in language production
Action implementer
production implementer
Perceptual implementer
comprehension implementer
Action command
production command
Drives the production implementer
Efference
copy drives the forward models
Comparator
monitor
compares the utterance percept and the predicted utterance perceptSlide25
Unifying production and comprehension
Production and comprehension are interwovenTight coupling in dialogue (Clark, 1996; Pickering & Garrod
, 2004)
Behavioural experiments show effects of comprehension processes on production and vice versa (e.g.,
Schriefers
et al., 1990)
Overlap of brain circuits for production and comprehension (e.g.,
Pulvermuller & Fadiga, 2010)
Such interweaving facilitates prediction of self and other
’
s utterancesSlide26
26
Classical modeling in language production
Just say utterance,
Listen what you say.Slide27
27
Forward modeling in language production
- Predict perception
caused by their own
utterance
- Fast alert when you say something wrongSlide28
Self-monitoringSpeaker wishes to say
kiteIn the past, she has always constructed the kite-concept and then uttered /k/
She therefore constructs forward model p^[phon](t) =
/k/
If she then incorrectly constructs p[phon] =
/g/
, the monitor notices the mismatch
If she believes the forward model, she will detect an error (and perhaps reformulate)
Otherwise, she will alter her forward modelSlide29
29
Prediction-by-simulation
- Predict perception of other’s next
utterance
by
listening
other’s current
utterance
Big(neuter)
Painting (neuter)Slide30
30
Interactive Language
Joint action involves combining accounts of action and action perception
Similarly, interactive language involves combining accounts of production and comprehension
Facilitates coordination (e.g., short intervals between speakers; Wilson & Wilson, 2005)
Facilitates alignment (developing same representations; Pickering &
Garrod
, 2004)Alignment in turn facilitates comprehension (better prediction of others)Slide31
31
Interactive LanguageSlide32
ConclusionWe propose that language production and comprehension are interwoven
It assumes a central role to prediction in production, comprehension, and dialogueSpeakers construct forward models to predict aspects of their upcoming utterancesListeners covertly imitate speakers and use forward models to predict the speakersOur account helps explain the efficiency of production and comprehension and the remarkable fluidity of dialogueSlide33
Thank youSlide34
Professor’s
Question
Comprehension process in Production
Predict next speech of other
Interactive LanguageSlide35
Q1: Give the evidence for how language production and comprehension are tightly interwoven. How does this relate to the perception-action cycle theory of cognitive systems?
35Question 1Slide36
Q2: Explain and give the evidence for how action, action perception, and joint action are interwoven. Explain how the authors use this to develop accounts of production, comprehension, and interactive language.
36Question 2
Action implementer
production implementer
Perceptual implementer
comprehension implementer
Action command
production command
Comparator
monitor
participants
’
arm movements affected by observing another person
’
s arm movements (
Kilner
et al., 2003)
And making hand movements can facilitate concurrent visual discrimination of deviant hand postures (
Miall
et al., 2006)Slide37
Q3: Give examples of what behavioral and neuroscientific data on language processing can be explained by the integrated theory of language production and comprehension explains, while modular theory does not.
37
Question 3
- Behavioral data
-
Neuroscientific
data (Pickering &
Garrod
, 2007)