CSCTR Session 8 Dana Retov á group at UC Berkeley amp Uni of Hawaii Nancy Chang Benjamin Bergen Jerome Feldman General assumption Semantic relations could be extracted from language input ID: 413632
Download Presentation The PPT/PDF document "Embodied construction grammar" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Embodied construction grammar
CSCTR Session 8
Dana
Retov
áSlide2
group at UC Berkeley
&
Uni of HawaiiNancy ChangBenjamin BergenJerome Feldman, …General assumptionSemantic relations could be extracted from language input“In its communicative function, language is a set of tools with which we attempt to guide another mind to create within itself a mental representation that approximates one we have.” (Delancey 1997)
NTLSlide3
Listener and speaker have to share enough experienceLanguage can be expressed by a discrete set of parameters and by semantic relations among entities and actions.
How these relations are encoded in the sequences of letters and sounds?
LanguageSlide4
A word that conveys some meaning
“in, on, through”
Word order“red fire engine” vs. “fire engine red”Some change in a base word -”ed” ending for the past tenseSystematic change in spelling (“car”-> “cars”)
Converting a verb to a noun (“evoke”->”evocation”)
3 mechanisms for conveying a semantic relationSlide5
Context Free GrammarSlide6
Analysis of simple sentence by CFGSlide7
S -> VP NPVP.person
<->
NP.personVP.gender <-> NP.genderVP.number <-> NP.numberSolutionSlide8
Context
The meaning of
indexicals“here”, “now”Referents of expressions“they”, “that question”Ambiguous sentences“Harry waked into the café with the singer”MetaphorsIntonation (e.g. stress, irony,…)
“HARRY walked into the café.”“Harry WALKED into the café.”Gestures
What CFG cannot cover?Slide9
Language understandingSlide10
Meanings reside in words
Each word has multiple fixed meanings –
word sensesRules of grammar are devoid of meaning and only specify which combinations of words are allowedMeaning of any combination of words can be determined by first detecting which sense of each word is involved and then using the appropriate rule for each word sense.“stone lion”Should each animal name like “lion” have another word sense covering animal-shaped objectsTraditional theorySlide11
Each word activates alternative meaning
subnetwork
These subnetworks themselves are linked to other circuits representing the semantics of words and frames that are active in the current context.The meaning of a word in context is captured by the joint activity of all of the relevant circuitryNTL – alternative theorySlide12
To write down rules of grammar that are understandable by people and computer programs and that also characterize the way our brains actually process language
The job of grammar is to specify which semantic schemas are being evoked, how they are parameterized and how they are liked together in the semantic specification.
To formalize cognitive linguisticsGoal of NTL’s embodied grammarSlide13
Construction = pairing of linguistic form and meaning
All levels of linguistic form (prefixes, words, phrases, sentences, stories, etc.) can be represented as mapping from some regularities of form to some semantic relations in the semantic specification
“embodied”Semantic part of a construction is composed of various kinds of embodied schemasImageForce dynamicactionEmbodied construction grammarSlide14
Simulation-based language understanding
Analysis Process
Semantic
Specification
“Harry walked into the cafe.”
Utterance
CAFE
Simulation
Belief State
General Knowledge
Constructions
construction
W
ALKED
form
self
f
.phon
[wakt]
meaning :
Walk-Action
constraints
self
m
.time
before
Context.speech-time
self
m
..aspect
encapsulatedSlide15
“Harry strolled to Berkeley”Individual word
simplest construction (lexical)
Lexical construction To |From subcase of Spatial Preposition evokes SPG as s form “to” |“from”
meaning Trajector-Landmark
lm <-> s.goal |lm <-> s.source
traj <-> s.traj
ExampleSlide16
Construction
Spatial PP
subcase of Destination constituents r: Spatial Preposition base: NP form r < base
meaning
r.lm <-> baseIn CFG: Spatial PP -> Spatial Preposition NP
Spatial Prepositional PhrasesSlide17
SemSpec
for “Harry strolled into Berkley”Slide18
Lexical construction Harry
subcase of NP form “Harry” meaning Referent Schema type <-> person gender <-> male
count <-> one specificity <-> known
resolved <-> harry2“Harry”Slide19
SemSpec
for “Harry strolled into Berkeley”Slide20
Lexical construction Strolled
subcase of Motion Verb, Regular Past form “stroll+ed” meaning WalkX speed <-> slow
tense <-> past aspect <-> completed
“Strolled”Slide21
Only single parameter controls the rate of moving one leg after the other
Leg moves only after the other is stable
As opposed to runningWalkX schemaSlide22
SemSpec
for “Harry strolled into Berkeley”Slide23
Lexical construction Strolled
subcase of Motion Verb, Regular Past form “stroll+ed” meaning WalkX speed <-> slow
tense <-> past aspect <-> completed
“Strolled”Slide24
SemSpec
for “Harry strolled into Berkley”Slide25
Construction
Self-Directed Motion
subcase of Motion Clause constituents movA: NP actV: Motion Verb locPP
: Spatial PP form mover < action < direction
meaning Self-Motion Schema mover <-> movA
action <-> actV direction <->
locPPSelf-directed motionSlide26
SemSpec
for “Harry strolled into Berkley”Slide27
ECG’s formalized schemas are just a way of writing down hypothesized neural connections and bindings.
These schemas are connected to semantic specification (
SemSpec).The simulation semantics process uses SemSpec and other activated knowledge to achieve conceptual integration and the resulting inferencesWhat is the difference between
ECG and other formal notations of gramar rules?Slide28
Normally “sneeze” is intransitive
Traditional grammar would suggest separate word sense for sneeze as a transitive verb
ECG would need caused motion constructionConstruction Caused Motion subcase of Motion Clause constituents
causer: Agent action: Motion
trajector: Movable object direction:
SpatialSpec form causer < action < trajector < direction
meaning Caused Motion Schema causer <-> action.actor
direction <-> action.location
“She sneezed the tissue off the table”Slide29
In traditional view “opened” refers to one sense of beer while “drank” to another
“Beer” sometimes stands for a “container of beer”
In ECG we use measure phrase constructionConstruction Measure NP subcase of NP constituents
m: Measure NP “of”
s: Substance NP form m < “of” < s meaning
Containment Schema vessel <-> m contents <-> s
“She opened and drank an expensive large beer”Slide30
Schema
Construction
MapmetaphorsMental spaceCan formalize “Josh said that Harry strolled to Berkeley”Talking about other times, places, other people’s thoughts, etc.
4 basic formal structures to formalize cognitive linguisticsSlide31
Computer understanding systemsNarayanan (1997)
Analysis of metaphors in news articles
Used pre-processed semanticsBryant (2004)Program that derives semantic relations that underlie English sentencesLater Bryant, Narayanan and Sinha combined the two modelsUse of ECGSlide32
Human processing:
What can ECG tell us about natural intelligence?
Garden-path sentences“The horse raced past the barn fell”Narayanan et al. 1988 – computer model that gives detailed predictions of how various factors (frequency of individual words, likelihood that they appear in certain constructions, etc.) interact in determining the difficulty of a garden-path situation.“The witness examined by the lawyer turned out to be unreliable”“The evidence examined by the lawyer turned out to be unreliable”
Chang (2006)Model how children learn grammar
Use of ECGSlide33
Learning constructionsSlide34
First wordsSlide35
Image schemasTopological
E.g. a container
OrientationalE.g. “in front of”Force-dynamicE.g. “against”Reference object and smaller objectLandmark and trajectorUnderstanding prepositionsSlide36
English
ON
AROUND
OVER
IN
Bowerman & PedersonSlide37
Dutch
Bowerman & Pederson
ANN
OM
BOVEN
IN
OPSlide38
Chinese
Bowerman & Pederson
SHANG
ZHOU
LISlide39
“Into” binds
inside
to a goalSlide40
Language and thought
“El jam
ón prueba salado“ Computational modelsConnectionist networksNeural systemsLevels of descriptionSlide41
Emulates a child viewing a simple geometric scene and being told a word that describes something about that scene
Has universal structure – visual system
2 classes of visual featuresQuantitative geometric features (e.g. angles)Qualitative topological features (e.g. contact)ComponentsCenter-surround cells, edge-sensitive cells, etc.Trained with a series of word-image pairsStandard back-propagation learningLater extended with motion prepositions (into, through, around)
Reiger (1996)Slide42
ModelSlide43
Children perform and plan actions long before they learn to describe them
Idea of characterizing actions by
parametersMotor control has its hierarchyLower levelCoordination, inhibitionHigher levelDesired speedWe can create abstract neural models of motor control systemsexecuting schemas
Action wordsSlide44
“Push” and “walk” schemasSlide45
Child learning of action wordsPerforming an action and hearing her parent’s label
Restricted to actions that can be carried out by one hand on a table
Bailey (1997)Slide46
Intermediate set of feature structures
Parameterization of action
Chosen to fit the basic X-schemasBi-directional arrowsLabeling pathwayCommand pathwayModelSlide47
4 steps in learning “push”Slide48
Model how children learn their first rules of grammar and generalize them in more adult-like rules
Chang (2006)Slide49
Suppose the child knows lexical construction for words “throw” and “ball”
But does not know construction for the phrase “throw ball”
“You throw the ball”Slide50Slide51
She learned that the second word determines which object fills the thrown role of a throw action
Only later learns generalization of this construction that works for any transitive verb
New grammar ruleSlide52
Key to understanding grammar acquisition is not the famous poverty of stimulus
but rather the
richness of the substrateChild already has rich base of conceptual and embodied experienceThe reason why understanding is ahead of productionChild can understand complex sentences by matching constructions to only parts of the utteranceConstructions are the same in bothGrammar learningSlide53
Decay of unused knowledgePeople always choose the set of constructions that best fits an input
If you keep track of best matches and
Increase the potential value of successful constructionsDecrease probability of trying not-useful constructionsThere would always be a better choiceBest-matchGiven a sentence S and a grammar G, the best analysis is the one that maximizes the probability of sentence S being generated by grammar GNo need for negative evidenceSlide54
Lifting (learning
superordinate
categories)Taking a collection of relations of similar form and replacing the common element with a parameterAfter learning that cows, dogs, horses and pigs all move and eat and make noises, a good learning system will postulate a category (animal) and just remember what goes in the category and what relations to apply to membersOccurs also in grammar learningVery early child generalizes e.g. throw-ball to other small objectsGeneralisation