Part of the book series:Lecture Notes in Computer Science ((LNCS,volume 14486))
Included in the following conference series:
319Accesses
Abstract
The lack of annotated data is one of the challenging issues in an ultra-fine entity typing, which is the task to assign semantic types for a given entity mention. Hence, automatic type generation is receiving increased interest, typically to be used as distant supervision data. In this study, we investigate an unsupervised way based on distributionally induced word senses. The types or labels are obtained by selecting the appropriate sense cluster for a mention. Experimental results on an ultra-fine entity typing task demonstrate that combining our predictions with the predictions of an existing neural model leads to a slight improvement over the ultra-fine types for mentions that are not pronouns.
This is a preview of subscription content,log in via an institution to check access.
Access this chapter
Subscribe and save
- Get 10 units per month
- Download Article/Chapter or eBook
- 1 Unit = 1 Article or 1 Chapter
- Cancel anytime
Buy Now
- Chapter
- JPY 3498
- Price includes VAT (Japan)
- eBook
- JPY 8579
- Price includes VAT (Japan)
- Softcover Book
- JPY 10724
- Price includes VAT (Japan)
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
Our code can be found at:https://github.com/uhh-lt/unsupervised-ultra-fine-entity-typing.
References
Anwar, S., Shelmanov, A., Panchenko, A., Biemann, C.: Generating lexical representations of frames using lexical substitution. In: Proceedings of the Probability and Meaning Conference (PaM 2020), Gothenburg, pp. 95–103 (2020).https://aclanthology.org/2020.pam-1.13
Biemann, C.: Chinese whispers - an efficient graph clustering algorithm and its application to natural language processing problems. In: Proceedings of TextGraphs: the First Workshop on Graph Based Methods for Natural Language Processing, New York City, pp. 73–80 (2006).https://aclanthology.org/W06-3812
Biemann, C., Coppola, B., Glass, M.R., Gliozzo, A., Hatem, M., Riedl, M.: JoBimText visualizer: a graph-based approach to contextualizing distributional similarity. In: Proceedings of TextGraphs-8 Graph-based Methods for Natural Language Processing, Seattle, WA, USA, pp. 6–10 (2013).https://aclanthology.org/W13-5002
Biemann, C., Riedl, M.: Text: now in 2D! A framework for lexical expansion with contextual similarity. J. Lang. Modell.1(1), 55–95 (2013).https://doi.org/10.15398/jlm.v1i1.60
Bird, S., Klein, E., Loper, E.: Natural Language Processing with Python. O’Reilly Media Inc. (2009).https://www.nltk.org/book
Choi, E., Levy, O., Choi, Y., Zettlemoyer, L.: Ultra-fine entity typing. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Melbourne, Australia, pp. 87–96 (2018).https://www.aclweb.org/anthology/P18-1009
Dai, H., Song, Y., Wang, H.: Ultra-fine entity typing with weak supervision from a masked language model. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 1790–1799 (2021).https://doi.org/10.18653/v1/2021.acl-long.141
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA, pp. 4171–4186 (2019).https://doi.org/10.18653/v1/N19-1423
Ding, N., et al.: Prompt-learning for fine-grained entity typing. In: Findings of the Association for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates, pp. 6888–6901 (2022).https://aclanthology.org/2022.findings-emnlp.512
Gillick, D., Lazic, N., Ganchev, K., Kirchner, J., Huynh, D.: Context-dependent fine-grained entity type tagging (2016)
Hearst, M.A.: Automatic acquisition of hyponyms from large text corpora. In: COLING 1992 Volume 2: The 14th International Conference on Computational Linguistics, pp. 539–545 (1992).https://aclanthology.org/C92-2082
Huang, L., May, J., Pan, X., Ji, H.: Building a fine-grained entity typing system overnight for a new x (x = language, domain, genre) (2016)
Huang, L., et al.: Liberal entity extraction: rapid construction of fine-grained entity typing systems. Big Data5(1), 19–31 (2017).https://doi.org/10.1089/big.2017.0012
Jana, A., Goyal, P.: Can network embedding of distributional thesaurus be combined with word vectors for better representation? In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), New Orleans, LO, USA, pp. 463–473 (2018).https://doi.org/10.18653/v1/N18-1043
Jiang, C., Jiang, Y., Wu, W., Xie, P., Tu, K.: Modeling label correlations for ultra-fine entity typing with neural pairwise conditional random field. In: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Abu Dhabi, United Arab Emirates, pp. 6836–6847 (2022).https://aclanthology.org/2022.emnlp-main.459
Kilgarriff, A., Rychly, P., Smrz, P., Tugwell, D.: ITRI-04-08 the sketch engine. Inf. Technol.105(116), 105–116 (2004)
Lee, C., Dai, H., Song, Y., Li, X.: A Chinese corpus for fine-grained entity typing. In: Proceedings of the Twelfth Language Resources and Evaluation Conference, Marseille, France, pp. 4451–4457 (2020).https://aclanthology.org/2020.lrec-1.548
Li, B., Yin, W., Chen, M.: Ultra-fine entity typing with indirect supervision from natural language inference. Trans. Associat. Computat. Linguist.10, 607–622 (2022).https://doi.org/10.1162/tacl_a_00479
Ling, X., Weld, D.: Fine-grained entity recognition. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 26, no. 1, pp. 94–100 (2021).https://doi.org/10.1609/aaai.v26i1.8122
Liu, L., et al.: TexSmart: a system for enhanced natural language understanding. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations, pp. 1–10 (2021).https://doi.org/10.18653/v1/2021.acl-demo.1
Liu, Q., Lin, H., Xiao, X., Han, X., Sun, L., Wu, H.: Fine-grained entity typing via label reasoning. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online and Punta Cana, Dominican Republic, pp. 4611–4622 (2021).https://doi.org/10.18653/v1/2021.emnlp-main.378
López, F., Heinzerling, B., Strube, M.: Fine-grained entity typing in hyperbolic space. In: Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019), Florence, Italy, pp. 169–180 (2019).https://doi.org/10.18653/v1/W19-4319
Obeidat, R., Fern, X., Shahbazi, H., Tadepalli, P.: Description-based zero-shot fine-grained entity typing. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA, pp. 807–814 (2019).https://doi.org/10.18653/v1/N19-1087
Onoe, Y., Boratko, M., McCallum, A., Durrett, G.: Modeling fine-grained entity types with box embeddings. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 2051–2064 (2021).https://doi.org/10.18653/v1/2021.acl-long.160
Onoe, Y., Durrett, G.: Learning to denoise distantly-labeled data for entity typing. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA, pp. 2407–2417 (2019).https://doi.org/10.18653/v1/N19-1250
Pan, W., Wei, W., Zhu, F.: Automatic noisy label correction for fine-grained entity typing. In: Raedt, L.D. (ed.) Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, pp. 4317–4323 (2022).https://doi.org/10.24963/ijcai.2022/599
Panchenko, A., et al.: Unsupervised, knowledge-free, and interpretable word sense disambiguation. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Copenhagen, Denmark, pp. 91–96 (2017).https://doi.org/10.18653/v1/D17-2016
Panchenko, A., Ruppert, E., Faralli, S., Ponzetto, S.P., Biemann, C.: Building a web-scale dependency-parsed corpus from Common Crawl. In: Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan, pp. 1816–1823 (2018).https://aclanthology.org/L18-1286
Panchenko, A., Ruppert, E., Faralli, S., Ponzetto, S.P., Biemann, C.: Unsupervised does not mean uninterpretable: the case for word sense induction and disambiguation. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, Valencia, Spain, pp. 86–98 (2017).https://aclanthology.org/E17-1009
Pelevina, M., Arefiev, N., Biemann, C., Panchenko, A.: Making sense of word embeddings. In: Proceedings of the 1st Workshop on Representation Learning for NLP, Berlin, Germany, pp. 174–183 (2016).https://doi.org/10.18653/v1/W16-1620
Qi, P., Zhang, Y., Zhang, Y., Bolton, J., Manning, C.D.: Stanza: a python natural language processing toolkit for many human languages. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 101–108 (2020).https://doi.org/10.18653/v1/2020.acl-demos.14
Qian, J., et al.: Fine-grained entity typing without knowledge base. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online and Punta Cana, Dominican Republic, pp. 5309–5319 (2021).https://doi.org/10.18653/v1/2021.emnlp-main.431
Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using Siamese BERT-networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, Hong Kong, pp. 3982–3992 (2019).https://doi.org/10.18653/v1/D19-1410
Riedl, M., Biemann, C.: Scaling to large\(^3\) data: an efficient and effective method to compute distributional thesauri. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, WA, USA, pp. 884–890 (2013).https://aclanthology.org/D13-1089
Ruppert, E., Kaufmann, M., Riedl, M., Biemann, C.: JoBimViz: a web-based visualization for graph-based distributional semantic models. In: Proceedings of ACL-IJCNLP 2015 System Demonstrations, Beijing, China, pp. 103–108 (2015).https://doi.org/10.3115/v1/P15-4018
Sui, X., et al.: Improving zero-shot entity linking candidate generation with ultra-fine entity type information. In: Proceedings of the 29th International Conference on Computational Linguistics, pp. 2429–2437. Gyeongju, Republic of Korea (2022).https://aclanthology.org/2022.coling-1.214
Xiong, W., et al.: Imposing label-relational inductive bias for extremely fine-grained entity typing. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA, pp. 773–784 (2019).https://doi.org/10.18653/v1/N19-1084
Zhang, T., Xia, C., Lu, C.T., Yu, P.: MZET: memory augmented zero-shot fine-grained named entity typing. In: Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, Spain, pp. 77–87 (2020).https://doi.org/10.18653/v1/2020.coling-main.7
Zhou, B., Khashabi, D., Tsai, C.T., Roth, D.: Zero-shot open entity typing as type-compatible grounding. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, pp. 2065–2076 (2018).https://doi.org/10.18653/v1/D18-1231
Zuo, X., Liang, H., Jing, N., Zeng, S., Fang, Z., Luo, Y.: Type-enriched hierarchical contrastive strategy for fine-grained entity typing. In: Proceedings of the 29th International Conference on Computational Linguistics, Gyeongju, Republic of Korea, pp. 2405–2417 (2022).https://aclanthology.org/2022.coling-1.212
Acknowledgements
The work was partially supported by a Deutscher Akademischer Austauschdienst (DAAD) doctoral stipend and the DFG funded JOIN-T project BI 1544/4.
Author information
Authors and Affiliations
Language Technology Group, Universität Hamburg, Hamburg, Germany
Özge Sevgili, Steffen Remus & Chris Biemann
School of Electrical Sciences, Indian Institute of Technology Bhubaneswar, Bhubaneswar, India
Abhik Jana
Skolkovo Institute of Science and Technology, Moscow, Russia
Alexander Panchenko
Artificial Intelligence Research Institute, Moscow, Russia
Alexander Panchenko
- Özge Sevgili
You can also search for this author inPubMed Google Scholar
- Steffen Remus
You can also search for this author inPubMed Google Scholar
- Abhik Jana
You can also search for this author inPubMed Google Scholar
- Alexander Panchenko
You can also search for this author inPubMed Google Scholar
- Chris Biemann
You can also search for this author inPubMed Google Scholar
Corresponding author
Correspondence toÖzge Sevgili.
Editor information
Editors and Affiliations
National Research University Higher School of Economics, Moscow, Russia
Dmitry I. Ignatov
Krasovskii Institute of Mathematics and Mechanics of Russian Academy of Sciences, Yekaterinburg, Russia
Michael Khachay
University of Oslo, Oslo, Norway
Andrey Kutuzov
American University of Armenia, Yerevan, Armenia
Habet Madoyan
Artificial Intelligence Research Institute, Moscow, Russia
Ilya Makarov
University of Hamburg, Hamburg, Germany
Irina Nikishina
Skolkovo Institute of Science and Technology, Moscow, Russia
Alexander Panchenko
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates
Maxim Panov
University of Florida, Gainesville, FL, USA
Panos M. Pardalos
National Research University Higher School of Economics, Nizhny Novgorod, Russia
Andrey V. Savchenko
Apptek, Aachen, Germany
Evgenii Tsymbalov
Kazan Federal University, Kazan, Russia
Elena Tutubalina
MTS AI, Moscow, Russia
Sergey Zagoruyko
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Sevgili, Ö., Remus, S., Jana, A., Panchenko, A., Biemann, C. (2024). Unsupervised Ultra-Fine Entity Typing with Distributionally Induced Word Senses. In: Ignatov, D.I.,et al. Analysis of Images, Social Networks and Texts. AIST 2023. Lecture Notes in Computer Science, vol 14486. Springer, Cham. https://doi.org/10.1007/978-3-031-54534-4_9
Download citation
Published:
Publisher Name:Springer, Cham
Print ISBN:978-3-031-54533-7
Online ISBN:978-3-031-54534-4
eBook Packages:Computer ScienceComputer Science (R0)
Share this paper
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative