Focused Entailment Graphs for Open IE Propositions
GB
Published · 52 slides · 0 views
1 / 1
Description
Focused Entailment Graphs for Open IE Propositions Omer Levy Ido Dagan Jacob Goldberger Bar-Ilan University, Israel Open IE Whats missing in Open IE? Adding Structure to Open IE Which structure? Build a graph of Open IE propositions and
Related Topics
Share
Embed code
Download this presentation From Below
"Focused Entailment Graphs for Open IE Propositions" is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Presentation Transcript
01
Focused Entailment Graphs for Open IE Propositions Omer Levy Ido Dagan Jacob Goldberger
Bar-Ilan University,
Israel<br>
Bar-Ilan University,
Israel<br>
02
Open IE<br>
03
What’s missing in Open IE?<br>
04
Adding Structure to Open IE Which structure?
Build a graph of Open IE propositions and their semantic relations<br>
Build a graph of Open IE propositions and their semantic relations<br>
05
Adding Structure to Open IE<br>
06
aspirin, eliminate, headache aspirin, cure, headache headache, control with, aspirin drug, relieve, headache drug, treat, headache analgesic, banish, headache headache, respond to, painkiller headache, treat with, caffeine coffee, help, headache tea, soothe, headache Original Open IE Output<br>
07
aspirin, eliminate, headache aspirin, cure, headache headache, control with, aspirin drug, relieve, headache drug, treat, headache analgesic, banish, headache headache, respond to, painkiller headache, treat with, caffeine coffee, help, headache tea, soothe, headache Consolidated Open IE Output<br>
08
Semantic Applications Example: Structured Queries
“What relieves headaches?”<br>
“What relieves headaches?”<br>
09
Semantic Applications<br>
10
aspirin, eliminate, headache aspirin, cure, headache headache, control with, aspirin drug, relieve, headache drug, treat, headache analgesic, banish, headache headache, respond to, painkiller headache, treat with, caffeine coffee, help, headache tea, soothe, headache<br>
11
aspirin, eliminate, headache aspirin, cure, headache headache, control with, aspirin drug, relieve, headache drug, treat, headache analgesic, banish, headache headache, respond to, painkiller headache, treat with, caffeine coffee, help, headache tea, soothe, headache<br>
12
aspirin drug analgesic painkiller caffeine coffee tea<br>
13
Our Contributions Structuring Open IE with Proposition Entailment Graphs
Dataset: 30 gold-standard graphs, 1.5 million entailment annotations
Algorithm for constructing Focused Proposition Entailment Graphs
Analysis: Predicate entailment is not quite what we thought<br>
Dataset: 30 gold-standard graphs, 1.5 million entailment annotations
Algorithm for constructing Focused Proposition Entailment Graphs
Analysis: Predicate entailment is not quite what we thought<br>
14
Proposition Entailment Graphs<br>
15
Related Work: Predicate Entailment Graphs Berant et al. (2010,2011,2012)
We extend Berant et al.’s work from predicates to propositions<br>
We extend Berant et al.’s work from predicates to propositions<br>
16
Focused Proposition Entailment Graphs Nodes: Open IE propositions
Edges: Textual Entailment<br>
Edges: Textual Entailment<br>
17
Focused Proposition Entailment Graphs<br>
18
Focused Proposition Entailment Graphs<br>
19
aspirin, eliminate, headache aspirin, cure, headache headache, control with, aspirin drug, relieve, headache drug, treat, headache analgesic, banish, headache headache, respond to, painkiller headache, treat with, caffeine coffee, help, headache tea, soothe, headache<br>
20
aspirin, eliminate, headache aspirin, cure, headache headache, control with, aspirin drug, relieve, headache drug, treat, headache analgesic, banish, headache headache, respond to, painkiller headache, treat with, caffeine coffee, help, headache tea, soothe, headache<br>
21
Focused Proposition Entailment Graphs<br>
22
Constructing Proposition Entailment Graphs<br>
23
Dataset<br>
24
Dataset: High-Quality Open IE Propositions Google’s Syntactic N-grams
Based on millions of books
Filter for subject-verb-object
Including prepositional objects and passive
Result: 68 million high-quality propositions<br>
Based on millions of books
Filter for subject-verb-object
Including prepositional objects and passive
Result: 68 million high-quality propositions<br>
25
Dataset: Annotating Entailment Graphs<br>
26
Algorithm<br>
27
How do we recognize proposition entailment?<br>
28
How do we recognize proposition entailment?<br>
29
How do we recognize proposition entailment?<br>
30
How do we recognize proposition entailment?<br>
31
How do we recognize proposition entailment?<br>
32
Lexical Entailment (Logistic) Lexical Entailment Lexical Entailment Features<br>
33
Lexical Entailment (Logistic) Lexical Entailment Features
WordNet Relations
UMLS
Distributional Similarity
String Edit Distance Lexical Entailment Features Supervision<br>
WordNet Relations
UMLS
Distributional Similarity
String Edit Distance Lexical Entailment Features Supervision<br>
34
From Lexical to Proposition Entailment Lexical Entailment (Logistic) Lexical Entailment Features Supervision<br>
35
Argument Entailment (Logistic) Predicate Entailment (Logistic) From Lexical to Proposition Entailment Argument Entailment Features Predicate Entailment Features Supervision Supervision<br>
36
Argument Entailment (Logistic) Predicate Entailment (Logistic) From Lexical to Proposition Entailment Argument Entailment Features Predicate Entailment Features Supervision Supervision Proposition Entailment (Conjunction)<br>
37
Following Snow (2005), Berant (2012) Argument Entailment (Logistic) Predicate Entailment (Logistic) Distant Supervision (WordNet)? Argument Entailment Features Predicate Entailment Features WordNet WordNet Proposition Entailment (Conjunction)<br>
38
Argument Entailment (Logistic) Proposition Entailment (Conjunction) Predicate Entailment (Logistic) Direct Supervision (30 Annotated Graphs) Argument Entailment Features Predicate Entailment Features Annotated Graphs<br>
39
Proposition Entailment (Conjunction) Direct Supervision (30 Annotated Graphs) Argument Entailment Features Predicate Entailment Features Hidden Layer Annotated Graphs<br>
40
Flat Model Argument Entailment Features Proposition Entailment (Logistic) Predicate Entailment Features Annotated Graphs<br>
41
Compared Methods Component-Level Distant Supervision (WordNet)
Predicates & Arguments
Predicates Only
Arguments Only
Proposition-Level Direct Supervision (30 Annotated Graphs)
Hierarchical (our method)
Flat
All methods used Berant et al.’s Global Optimization method<br>
Predicates & Arguments
Predicates Only
Arguments Only
Proposition-Level Direct Supervision (30 Annotated Graphs)
Hierarchical (our method)
Flat
All methods used Berant et al.’s Global Optimization method<br>
42
Results<br>
43
Direct Supervision: Flat vs Hierarchical Hierarchal model performs better than flat model
Better to model predicate and argument entailment separately<br>
Better to model predicate and argument entailment separately<br>
44
Distant vs Direct Supervision Direct supervision is better
Although WordNet provides more training examples<br>
Although WordNet provides more training examples<br>
45
Predicate Entailment with Distant Supervision Ignoring predicates improves distant supervision baselines<br>
46
Are WordNet relations capturing real-world predicate entailments?<br>
47
Predicate Entailment vs WordNet Relations Over a predicate inference subset, how many predicate entailments are covered by WordNet?
Positive indicators
synonyms, hypernyms, entailment<br>
Positive indicators
synonyms, hypernyms, entailment<br>
48
Why isn’t WordNet capturing predicate entailment? Predicate Entailment vs WordNet Relations Over a predicate inference subset, how many predicate entailments are covered by WordNet?
Positive indicators
synonyms, hypernyms, entailment
Negative Indicators
antonyms, hyponyms, cohyponyms<br>
Positive indicators
synonyms, hypernyms, entailment
Negative Indicators
antonyms, hyponyms, cohyponyms<br>
49
Predicate Entailment is Context-Sensitive<br>
50
Predicate Entailment is Context-Sensitive<br>
51
Investigating Context-Sensitive Entailment Recent work on context-sensitive lexical inference
e.g. (Melamud et al., 2013)
Previous datasets
Lexical substitution (McCarthy and Navigli, 2007)
Predicate inference (Zeichner et al., 2012)
We offer a new dataset of real-world lexical entailments in context!
Sample: synthetic vs naturally occurring
Size: several thousands vs 1.5 million<br>
e.g. (Melamud et al., 2013)
Previous datasets
Lexical substitution (McCarthy and Navigli, 2007)
Predicate inference (Zeichner et al., 2012)
We offer a new dataset of real-world lexical entailments in context!
Sample: synthetic vs naturally occurring
Size: several thousands vs 1.5 million<br>
52
Conclusion<br>
53
Conclusion Structuring Open IE with Proposition Entailment Graphs
Algorithm for constructing Focused Proposition Entailment Graphs
Analysis: Predicate entailment is extremely context-sensitive
Dataset: 1.5 million proposition entailment decisions
Thank you for listening!<br>
Algorithm for constructing Focused Proposition Entailment Graphs
Analysis: Predicate entailment is extremely context-sensitive
Dataset: 1.5 million proposition entailment decisions
Thank you for listening!<br>
54
Next Steps Predicate entailment in context is an open problem
Improve coverage of argument entailment
Investigate more complex proposition and graph structures
Thank you for listening!<br>
Improve coverage of argument entailment
Investigate more complex proposition and graph structures
Thank you for listening!<br>
55
Berant et al.’s Method<br>
56
Berant et al.’s Method affect treat cure trigger A set of predicates<br>
57
Local estimation of entailment probabilities Berant et al.’s Method affect treat cure trigger<br>
58
Global optimization of entailment edges Berant et al.’s Method affect treat cure trigger<br>
59
From Predicates to Propositions<br>
60
From Predicates to Propositions<br>
61
From Predicates to Propositions<br>
62
From Predicates to Propositions Infer proposition entailment from lexical features?<br>
63
Component Entailment Conjunction (CEC)<br>
64
Component Entailment Conjunction (CEC)<br>
65
Component Entailment Conjunction (CEC) Learn component-level classifiers from proposition-level supervision
Expectation Maximization (EM)
E-Step: Estimate component-level labels from proposition-level label
M-Step: Use estimates as “soft” labels to train component weights<br>
Expectation Maximization (EM)
E-Step: Estimate component-level labels from proposition-level label
M-Step: Use estimates as “soft” labels to train component weights<br>
66
Component Entailment Conjunction (CEC) Argument Entailment Features Argument Entailment (Logistic) Proposition Entailment (Conjunction) Predicate Entailment Features Predicate Entailment (Logistic)<br>
67
Component Entailment Conjunction (CEC) Argument Entailment Features Argument Entailment (Logistic) Proposition Entailment (Conjunction) Predicate Entailment Features Predicate Entailment (Logistic)<br>
68
How do we learn the weights? Learn lexical classifiers with distant supervision (WordNet)
Berant et al.
Snow et al.
Doesn’t work well in practice!
Learn lexical classifiers with direct supervision (30 annotated graphs)
Propagate proposition-level supervision to lexical features with EM<br>
Berant et al.
Snow et al.
Doesn’t work well in practice!
Learn lexical classifiers with direct supervision (30 annotated graphs)
Propagate proposition-level supervision to lexical features with EM<br>
69
Creating a Predicate Entailment Dataset<br>
70
Creating a Predicate Entailment Dataset<br>
71
Predicate Entailment: Syntactic Glue?<br>
72
Argument Entailment<br>
73
Open IE does not consolidate information<br>