CSC 594 Topics in AI –

CSC 594 Topics in AI – CSC 594 Topics in AI – - Start

Added : 2018-01-17 Views :39K

Download Presentation

CSC 594 Topics in AI –




Download Presentation - The PPT/PDF document "CSC 594 Topics in AI –" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.



Presentations text content in CSC 594 Topics in AI –

Slide1

CSC 594 Topics in AI –Natural Language Processing

Spring 2016/1711. Context-Free Parsing(Some slides adapted from Jurafsky & Martin)

Slide2

Parsing

Parsing with CFGs refers to the task of assigning proper trees to input stringsProper here means a tree that covers all and only the elements of the input and has an S at the topIt doesn’t mean that the system can select the correct tree from among all the possible trees

Speech and Language Processing - Jurafsky and Martin

2

Slide3

For Now

Let’s assume…You have all the words for a sentence already in some bufferThe input is not POS tagged prior to parsingWe won’t worry about morphological analysisAll the words are knownThese are all problematic in various ways, and would have to be addressed in real applications.

Speech and Language Processing - Jurafsky and Martin

3

Slide4

Search Framework

It’s productive to think about parsing as a form of search…A search through the space of possible trees given an input sentence and grammarThis framework suggests that heuristic search methods and/or dynamic programming methods might be applicableIt also suggests that notions such as the direction of the search might be useful

Speech and Language Processing - Jurafsky and Martin

4

Slide5

Top-Down Search

Since we’re trying to find trees rooted with an S (Sentences), why not start with the rules that give us an S.Then we can work our way down from there to the words.

Speech and Language Processing - Jurafsky and Martin

5

Slide6

Top Down Space

Speech and Language Processing -

Jurafsky

and Martin

6

Slide7

Bottom-Up Parsing

Of course, we also want trees that cover the input words. So we might also start with trees that link up with the words in the right way.Then work your way up from there to larger and larger trees.

Speech and Language Processing - Jurafsky and Martin

7

Slide8

Bottom-Up Search

Speech and Language Processing -

Jurafsky and Martin

8

Slide9

Bottom-Up Search

Speech and Language Processing -

Jurafsky and Martin

9

Slide10

Bottom-Up Search

Speech and Language Processing -

Jurafsky and Martin

10

Slide11

Bottom-Up Search

Speech and Language Processing -

Jurafsky and Martin

11

Slide12

Bottom-Up Search

Speech and Language Processing -

Jurafsky and Martin

12

Slide13

Top-Down and Bottom-Up

Top-downOnly searches for trees that can be answers (i.e. S’s)But also suggests trees that are not consistent with any of the wordsBottom-upOnly forms trees consistent with the wordsBut suggests trees that make no sense globally

Speech and Language Processing - Jurafsky and Martin

13

Slide14

Control

Of course, in both cases we left out how to keep track of the search space and how to make choicesWhich node to try to expand nextWhich grammar rule to use to expand a nodeOne approach is called backtracking.Make a choice, if it works out then fineIf not then back up and make a different choiceSame as with ND-Recognize

Speech and Language Processing - Jurafsky and Martin

14

Slide15

Problems

Even with the best filtering, backtracking methods are doomed because of two inter-related problemsAmbiguity and search control (choice)Shared sub-problems

Speech and Language Processing - Jurafsky and Martin

15

Slide16

Ambiguity

Speech and Language Processing -

Jurafsky and Martin

16

Slide17

Shared Sub-Problems

No matter what kind of search (top-down or bottom-up or mixed) that we choose...We can’t afford to redo work we’ve already done.Without some help naïve backtracking will lead to such duplicated work.

Speech and Language Processing - Jurafsky and Martin

17

Slide18

Shared Sub-Problems

Consider

“A flight from Indianapolis to Houston on TWA”

Speech and Language Processing - Jurafsky and Martin

18

Slide19

Sample L1 Grammar

Speech and Language Processing -

Jurafsky and Martin

19

Slide20

Shared Sub-Problems

Assume a top-down parse that has already expanded the NP rule (dealing with the Det) Now its making choices among the various Nominal rulesIn particular, between these twoNominal -> NounNominal -> Nominal PPStatically choosing the rules in this order leads to the following bad behavior...

Speech and Language Processing - Jurafsky and Martin

20

Slide21

Shared Sub-Problems

Speech and Language Processing -

Jurafsky

and Martin

21

Slide22

Shared Sub-Problems

Speech and Language Processing -

Jurafsky and Martin

22

Slide23

Shared Sub-Problems

Speech and Language Processing -

Jurafsky

and Martin

23

Slide24

Shared Sub-Problems

Speech and Language Processing -

Jurafsky and Martin

24

Slide25

Dynamic Programming

DP search methods fill tables with partial results and therebyAvoid doing avoidable repeated workSolve exponential problems in polynomial time (ok, not really)Efficiently store ambiguous structures with shared sub-parts.We’ll cover one approach that corresponds to a bottom-up strategyCKY

Speech and Language Processing - Jurafsky and Martin

25

Slide26

CKY Parsing

First we’ll limit our grammar to epsilon-free, binary rules (more on this later)Consider the rule A  BCIf there is an A somewhere in the input generated by this rule then there must be a B followed by a C in the input.If the A spans from i to j in the input then there must be some k st. i<k<jIn other words, the B splits from the C someplace after the i and before the j.

Speech and Language Processing - Jurafsky and Martin

26

Slide27

Problem

What if your grammar isn’t binary?As in the case of the TreeBank grammar?Convert it to binary… any arbitrary CFG can be rewritten into Chomsky-Normal Form automatically.What does this mean?The resulting grammar accepts (and rejects) the same set of strings as the original grammar.But the resulting derivations (trees) are different.

Speech and Language Processing - Jurafsky and Martin

27

Slide28

Problem

More specifically, we want our rules to be of the formA  B COrA  wThat is, rules can expand to either 2 non-terminals or to a single terminal.

Speech and Language Processing - Jurafsky and Martin

28

Slide29

Binarization Intuition

Eliminate chains of unit productions.Introduce new intermediate non-terminals into the grammar that distribute rules with length > 2 over several rules. So… S  A B C turns into S  X C andX  A BWhere X is a symbol that doesn’t occur anywhere else in the grammar.

Speech and Language Processing - Jurafsky and Martin

29

Slide30

Sample L1 Grammar

Speech and Language Processing -

Jurafsky and Martin

30

Slide31

CNF Conversion

Speech and Language Processing -

Jurafsky and Martin

31

Slide32

CKY

Since each LHS non-terminal consists of exactly two RHS non-terminals, a 2-dimensional table can be used to record constituents found.Each cell [i,j] contains the set of (LHS) non-terminals that span i through j in the sentence. 0 Book 1 that 2 flight 3So a non-terminal spanning an entire sentence will sit in cell [0, n].Then for a sentence of length n, the algorithm uses only the upper-triangular portion of an (n+1) x (n+1) table.

Speech and Language Processing - Jurafsky and Martin

32

Slide33

CKY

For a rule like A  B C we should look for a B in [i,k] and a C in [k,j].In other words, if we think there might be an A spanning i,j in the input… AND A  B C is a rule in the grammar THENThere must be a B in [i,k] and a C in [k,j] for some k such that i<k<j

Speech and Language Processing - Jurafsky and Martin

33

Slide34

CKY

So to fill the table loop over the cells [i,j] values in some systematic wayThen for each cell, loop over the appropriate k values to search for things to add.Add all the derivations that are possible for each [i,j] for each k

Speech and Language Processing - Jurafsky and Martin

34

Slide35

CKY Table

Speech and Language Processing -

Jurafsky and Martin

35

Slide36

Example

Speech and Language Processing -

Jurafsky and Martin

36

Slide37

CKY Algorithm

Looping over

columns

Filling bottom cell

Filling row i in column j

Looping over the possible split locations between i and j.

Check the grammar for rules that link the constituents in [i,k] with those in [k,j]. For each rule found store the LHS of the rule in cell [i,j].

Speech and Language Processing - Jurafsky and Martin

37

Slide38

Example

Filling column 5

Speech and Language Processing -

Jurafsky and Martin

38

Slide39

Example

Speech and Language Processing -

Jurafsky and Martin

39

Slide40

Example

Speech and Language Processing -

Jurafsky and Martin

40

Slide41

Example

Speech and Language Processing -

Jurafsky

and Martin

41

Slide42

Example

Speech and Language Processing -

Jurafsky and Martin

42

Slide43

Example

Since there’s an S in [0,5] we have a valid parse.Are we done? We sort of left something out of the algorithm.

Speech and Language Processing -

Jurafsky and Martin

43

Slide44

CKY Notes

Since it’s bottom up, CKY hallucinates a lot of silly constituents.Segments that by themselves are constituents but cannot really occur in the context in which they are being suggested.To avoid this we can switch to a top-down control strategyOr we can add some kind of filtering that blocks constituents where they can not happen in a final analysis.

Speech and Language Processing - Jurafsky and Martin

44

Slide45

CKY Notes

We arranged the loops to fill the table a column at a time, from left to right, bottom to top. This assures us that whenever we’re filling a cell, the parts needed to fill it are already in the table (to the left and below)It’s somewhat natural in that it processes the input a left to right a word at a timeKnown as onlineCan you think of an alternative strategy?

Speech and Language Processing - Jurafsky and Martin

45

Slide46

Note

An alternative is to fill a diagonal at a time.That still satisfies our requirement that the component parts of each constituent/cell will already be available when it is filled in.Filling a diagonal at a time corresponds naturally to what task?

Speech and Language Processing -

Jurafsky and Martin

46


About DocSlides
DocSlides allows users to easily upload and share presentations, PDF documents, and images.Share your documents with the world , watch,share and upload any time you want. How can you benefit from using DocSlides? DocSlides consists documents from individuals and organizations on topics ranging from technology and business to travel, health, and education. Find and search for what interests you, and learn from people and more. You can also download DocSlides to read or reference later.
Youtube