Distributional Semantic Models in Lexical
Description: Distributional Semantic Models in Lexical Typology: Constructing a typological questionnaire Daria Ryzhova School of Linguistics NRU HSE Outline Lexical Typology: Frame-based Approach Ideology Typological questionnaire Constructing a
Related Topics
Download Presentation
"Distributional Semantic Models in Lexical" is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Presentation Transcript
slide1. Distributional Semantic Models in Lexical Typology: Constructing a typological questionnaire Daria Ryzhova
School of Linguistics
NRU HSE<br>
slide2. Outline Lexical Typology: Frame-based Approach
Ideology
Typological questionnaire
Constructing a questionnaire with Distributional Semantic Models
tasks
DSModels
Clustering
Evaluation
Conclusions<br>
slide3. Lexical Typology: Frame-based Approach<br>
slide4. MLexT E. Rakhilina
T. Reznikova
B. Orekhov
A. Vyrenkova
D. Ryzhova
M. Kyuseva
E. Kashkin
L. Kholkina
L. Khokhlova
A. Panina E. Rudnickaja
E. Parina
V. Krugljakova
E. Kozlova
M. Tagabileva
M. Shapiro
I. Stenin
А. Ladygina
E. Luchina A. Kozlov
M. Privizentseva
E. Kuzmenko
E. Mustakimova
Maksimova
E. Baskakova
E. Pavlova
…<br>
slide5. MLexT: it works! Majsak,
Rakhilina (eds.)
2007 Britsyn, Rakhilina, Reznikova, Yavorska (eds.) 2010 Work in progress:
Verbs of rotation
Verbs of oscillation
Quality concepts (mainly physical qualities:
SOFT / HARD, HEAVY / LIGHT, FULL / EMPTY, SHARP / BLUNT, ROUGH, DIRECT, …<br>
slide6. Ideology Distributional hypothesis:
Meaning via the prism of co-occurrence
Moscow Semantic School tradition (Apresjan 1974)<br>
slide7. Meaning = co-occurrence: deep well
deep river
deep plate
deep + ‘container’ => size deep sympathy
deep impression
deep grief
deep + ‘emotion’ => intensifier deep blue
deep red
deep + ‘colour’ => ‘dark’<br>
slide8. Typological extension: Russian glubokij ‘deep’ deep well
deep river
deep plate
deep + ‘container’ => size deep sympathy
deep impression
deep grief
deep + ‘emotion’ => intensifier deep blue
deep red
deep + ‘color’ => saturation<br>
slide9. Typological questionnaire<br>
slide10. Typological questionnaire: “shuttle” method Russian ostryj
Serbian oštar:
‘sharp knife’, ‘sharp spear’, ‘sharp contrast’, ‘sharp picture’, ‘prickly blanket’
Russian ostryj, rezkij, kolyuchij<br>
slide11. Typological questionnaire Very labor-consuming!<br>
slide12. Typological questionnaire Very labor-consuming!
Is it possible to construct a questionnaire automatically?<br>
slide13. Constructing a typological questionnaire with Distributional Semantic Models<br>
slide14. Tasks to collect a set of collocations (= examples)
to divide it into frames<br>
slide15. Step 1: List of collocations The main subcorpus of Russian National Corpus (lemmatized)
List of nouns co-occurring with the adjective in question in the corpus [window = +1]
Collocations occurring more than 10 times only<br>
slide16. Step 2: Dividing into frames Vector representation for every collocation (Distributional Semantic Models)
Clustering of the vector space<br>
slide17. DSModels: parameters Vectors of co-occurrences
Dimensions: 10 000 most frequent content words
Dimension values: number of co-occurrences
Window: ±5 content words<br>
slide18. ±5 window Вместо этого он [раскрыл свой кожаный чемоданчик, достал из него несколько острых ножей и соединил их один с другим, так что получилась длинная] сабля.
Instead, he [opened his leathern case, took from it several sharp knives, and joined them one after another, until they made a long] sword.<br>
slide19. Distributional Semantic Models Vectors of co-occurrences:<br>
slide20. Clustering: parameters Predetermined number of clusters:
(N1 + N2 + … + Nn) * 2, where N1, N2, …, Nn – numbers of meanings of every adjective of the field
5 clustering algorithms with different modifications (CLUTO clustering toolkit)<br>
slide21. Clustering algorithms Automatic determination of the number of clusters:
Affinity propagation
DBScan
Predetermined number of clusters [CLUTO]:
rb (repeated bisections)
rbr
graph
direct
agglo
bagglo<br>
slide22. Resulting clusters Example 1
koljuchij_kustarnik ‘thorny bushes’
koljuchij_kust ‘thorny bush’
koljuchij_trava ‘thorny grass’
…<br>
slide23. Resulting clusters Example 2:
prjamoj_linija ‘straight line’
prjamoj_efir ‘live broadcast’
prjamoj_navodka ‘direct laying’
prjamoj_popadanije ‘direct hit’
prjamoj_kishka ‘rectum (lit.: straight intestine)’
prjamoj_potomok ‘direct descendant’
…<br>
slide24. Clustering: post-processing Extracting of 3 examples (centroids) from every big cluster
Eliminating of small clusters (less than 3 members)<br>
slide25. Resulting clusters: centroids Example 1:
prjamoj_stolb ‘straight pole’
prjamoj_dorozhka ‘straight path’
prjamoj_alleja ‘straight avenue’
Example 2:
prjamoj_potomok ‘direct descendant’
prjamoj_predshestvennik ‘direct predecessor’
prjamoj_nasledije ‘direct heritage’<br>
slide26. Clustering: centroids ‘sharp’: fragment of the semantic map Piercing instruments
‘sharp spear’, ‘sharp arrow’ Sharp form
‘sharp nose’ ‘sharp elbow’<br>
slide27. Clustering: centroids ‘straight’: fragment of an experts mark up
prjamoj rjad ‘straight row’ 1
prjamaja linija ‘straight line’ 1
prjamoj udar ‘straight/direct attack’ 1|4
prjamoj dostup ‘direct access’ 6
prjamoj razgovor ‘direct/frank conversation’ 6|7
prjamaja ugroza ‘direct threat’ 7<br>
slide28. Evaluation Recall, R: how many frames are presented
Precision, P: whether every cluster is homogenous or not
F-measure:
1.09*PR / (0.09*P+R)
(precision is more important than recall) 1 (max)
0.9 (max)
0.89<br>
slide29. Conclusions Automatic construction of a questionnaire is possible
It accelerates significantly a typological research
Resulting clusters are very similar to typological frames<br>
slide30. Thank you for your attention!<br>
School of Linguistics
NRU HSE<br>
slide2. Outline Lexical Typology: Frame-based Approach
Ideology
Typological questionnaire
Constructing a questionnaire with Distributional Semantic Models
tasks
DSModels
Clustering
Evaluation
Conclusions<br>
slide3. Lexical Typology: Frame-based Approach<br>
slide4. MLexT E. Rakhilina
T. Reznikova
B. Orekhov
A. Vyrenkova
D. Ryzhova
M. Kyuseva
E. Kashkin
L. Kholkina
L. Khokhlova
A. Panina E. Rudnickaja
E. Parina
V. Krugljakova
E. Kozlova
M. Tagabileva
M. Shapiro
I. Stenin
А. Ladygina
E. Luchina A. Kozlov
M. Privizentseva
E. Kuzmenko
E. Mustakimova
Maksimova
E. Baskakova
E. Pavlova
…<br>
slide5. MLexT: it works! Majsak,
Rakhilina (eds.)
2007 Britsyn, Rakhilina, Reznikova, Yavorska (eds.) 2010 Work in progress:
Verbs of rotation
Verbs of oscillation
Quality concepts (mainly physical qualities:
SOFT / HARD, HEAVY / LIGHT, FULL / EMPTY, SHARP / BLUNT, ROUGH, DIRECT, …<br>
slide6. Ideology Distributional hypothesis:
Meaning via the prism of co-occurrence
Moscow Semantic School tradition (Apresjan 1974)<br>
slide7. Meaning = co-occurrence: deep well
deep river
deep plate
deep + ‘container’ => size deep sympathy
deep impression
deep grief
deep + ‘emotion’ => intensifier deep blue
deep red
deep + ‘colour’ => ‘dark’<br>
slide8. Typological extension: Russian glubokij ‘deep’ deep well
deep river
deep plate
deep + ‘container’ => size deep sympathy
deep impression
deep grief
deep + ‘emotion’ => intensifier deep blue
deep red
deep + ‘color’ => saturation<br>
slide9. Typological questionnaire<br>
slide10. Typological questionnaire: “shuttle” method Russian ostryj
Serbian oštar:
‘sharp knife’, ‘sharp spear’, ‘sharp contrast’, ‘sharp picture’, ‘prickly blanket’
Russian ostryj, rezkij, kolyuchij<br>
slide11. Typological questionnaire Very labor-consuming!<br>
slide12. Typological questionnaire Very labor-consuming!
Is it possible to construct a questionnaire automatically?<br>
slide13. Constructing a typological questionnaire with Distributional Semantic Models<br>
slide14. Tasks to collect a set of collocations (= examples)
to divide it into frames<br>
slide15. Step 1: List of collocations The main subcorpus of Russian National Corpus (lemmatized)
List of nouns co-occurring with the adjective in question in the corpus [window = +1]
Collocations occurring more than 10 times only<br>
slide16. Step 2: Dividing into frames Vector representation for every collocation (Distributional Semantic Models)
Clustering of the vector space<br>
slide17. DSModels: parameters Vectors of co-occurrences
Dimensions: 10 000 most frequent content words
Dimension values: number of co-occurrences
Window: ±5 content words<br>
slide18. ±5 window Вместо этого он [раскрыл свой кожаный чемоданчик, достал из него несколько острых ножей и соединил их один с другим, так что получилась длинная] сабля.
Instead, he [opened his leathern case, took from it several sharp knives, and joined them one after another, until they made a long] sword.<br>
slide19. Distributional Semantic Models Vectors of co-occurrences:<br>
slide20. Clustering: parameters Predetermined number of clusters:
(N1 + N2 + … + Nn) * 2, where N1, N2, …, Nn – numbers of meanings of every adjective of the field
5 clustering algorithms with different modifications (CLUTO clustering toolkit)<br>
slide21. Clustering algorithms Automatic determination of the number of clusters:
Affinity propagation
DBScan
Predetermined number of clusters [CLUTO]:
rb (repeated bisections)
rbr
graph
direct
agglo
bagglo<br>
slide22. Resulting clusters Example 1
koljuchij_kustarnik ‘thorny bushes’
koljuchij_kust ‘thorny bush’
koljuchij_trava ‘thorny grass’
…<br>
slide23. Resulting clusters Example 2:
prjamoj_linija ‘straight line’
prjamoj_efir ‘live broadcast’
prjamoj_navodka ‘direct laying’
prjamoj_popadanije ‘direct hit’
prjamoj_kishka ‘rectum (lit.: straight intestine)’
prjamoj_potomok ‘direct descendant’
…<br>
slide24. Clustering: post-processing Extracting of 3 examples (centroids) from every big cluster
Eliminating of small clusters (less than 3 members)<br>
slide25. Resulting clusters: centroids Example 1:
prjamoj_stolb ‘straight pole’
prjamoj_dorozhka ‘straight path’
prjamoj_alleja ‘straight avenue’
Example 2:
prjamoj_potomok ‘direct descendant’
prjamoj_predshestvennik ‘direct predecessor’
prjamoj_nasledije ‘direct heritage’<br>
slide26. Clustering: centroids ‘sharp’: fragment of the semantic map Piercing instruments
‘sharp spear’, ‘sharp arrow’ Sharp form
‘sharp nose’ ‘sharp elbow’<br>
slide27. Clustering: centroids ‘straight’: fragment of an experts mark up
prjamoj rjad ‘straight row’ 1
prjamaja linija ‘straight line’ 1
prjamoj udar ‘straight/direct attack’ 1|4
prjamoj dostup ‘direct access’ 6
prjamoj razgovor ‘direct/frank conversation’ 6|7
prjamaja ugroza ‘direct threat’ 7<br>
slide28. Evaluation Recall, R: how many frames are presented
Precision, P: whether every cluster is homogenous or not
F-measure:
1.09*PR / (0.09*P+R)
(precision is more important than recall) 1 (max)
0.9 (max)
0.89<br>
slide29. Conclusions Automatic construction of a questionnaire is possible
It accelerates significantly a typological research
Resulting clusters are very similar to typological frames<br>
slide30. Thank you for your attention!<br>