- Jinglei Zhang13,14,
- Guochang Wen13,14,
- NingLin Liao16,
- DongDong Du15,
- Qing Gao13,
- Minghui Zhang17 &
- …
- XiXin Cao14
Part of the book series:Lecture Notes in Computer Science ((LNAI,volume 14886))
Included in the following conference series:
500Accesses
Abstract
We study the problem of few-shot learning in Name Entity Recognition(FS-NER). Specifically, unlike other sequence labeling-based models, that mainly focus on better representations, we leverage logit adjustment technology to alleviate the problem that the different distribution between training and test dataset. Furthermore, we propose a simple but effective method, called Logit Adjustment with Normalization and Augmentation (LANA), for FS-NER. In detail, LANA first combines moving average and logit adjustment to retain the information of pre-training to overcome the representation drop problem in FS-NER. We also involve logit normalization to deal with the overfitting problem in FS-NER, and further improve the generalization ability of LANA. Our method achieves competitive performance on seven widely used FS-NER datasets and significantly reduces the influence of overfitting and representation drop.
This is a preview of subscription content,log in via an institution to check access.
Access this chapter
Subscribe and save
- Get 10 units per month
- Download Article/Chapter or eBook
- 1 Unit = 1 Article or 1 Chapter
- Cancel anytime
Buy Now
- Chapter
- JPY 3498
- Price includes VAT (Japan)
- eBook
- JPY 26311
- Price includes VAT (Japan)
- Softcover Book
- JPY 10581
- Price includes VAT (Japan)
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Atefeh, F., Khreich, W.: A survey of techniques for event detection in twitter. Comput. Intell.31(1), 132–164 (2015)
Bach, N., Badaskar, S.: A review of relation extraction. Lit. Rev. Lang. Stat.II(2), 1–15 (2007)
Brown, T., et al.: Language models are few-shot learners. Adv. Neural. Inf. Process. Syst.33, 1877–1901 (2020)
Budzianowski, P., et al.: Multiwoz–a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling (2018). arXiv preprintarXiv:1810.00278
Chen, C., Zhan, Y., Yu, B., Liu, L., Luo, Y., Du, B.: Resistance training using prior bias: toward unbiased scene graph generation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 212–220 (2022)
Chen, J., Zhang, R., Mao, Y., Xu, J.: Contrastnet: a contrastive learning framework for few-shot text classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 10492–10500 (2022)
Coucke, A., et al.: Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces (2018). arXiv preprintarXiv:1805.10190
Das, S.S.S., Katiyar, A., Passonneau, R.J., Zhang, R.: Container: few-shot named entity recognition via contrastive learning (2021). arXiv preprintarXiv:2109.07589
Derczynski, L., Nichols, E., Van Erp, M., Limsopatham, N.: Results of the wnut2017 shared task on novel and emerging entity recognition. In: Proceedings of the 3rd Workshop on Noisy User-generated Text, pp. 140–147 (2017)
Ding, N., et al.: Few-nerd: a few-shot named entity recognition dataset (2021). arXiv preprintarXiv:2105.07464
Fritzler, A., Logacheva, V., Kretov, M.: Few-shot classification in named entity recognition task. In: Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing, pp. 993–1000 (2019)
Geng, R., Li, B., Li, Y., Zhu, X., Jian, P., Sun, J.: Induction networks for few-shot text classification (2019). arXiv preprintarXiv:1902.10482
Hakkani-Tür, D., et al.: Multi-domain joint semantic frame parsing using bi-directional rnn-lstm. In: Interspeech, pp. 715–719 (2016)
Hofer, M., Kormilitzin, A., Goldberg, P., Nevado-Holgado, A.: Few-shot learning for named entity recognition in medical text (2018). arXiv preprintarXiv:1811.05468
Huang, J., et al.: Few-shot named entity recognition: a comprehensive study (2020). arXiv preprintarXiv:2012.14978
Kowsari, K., Jafari Meimandi, K., Heidarysafa, M., Mendu, S., Barnes, L., Brown, D.: Text classification algorithms: a survey. Information10(4), 150 (2019)
Li, X., Feng, J., Meng, Y., Han, Q., Wu, F., Li, J.: A unified MRC framework for named entity recognition (2019). arXiv preprintarXiv:1910.11476
Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of the AAAI conference on artificial intelligence, vol. 29 (2015)
Liu, J., Pasupat, P., Cyphers, S., Glass, J.: Asgard: a portable architecture for multilingual dialogue systems. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 8386–8390. IEEE (2013)
Liu, J., Pasupat, P., Wang, Y., Cyphers, S., Glass, J.: Query understanding enhanced by hierarchical parsing structures. In: 2013 IEEE Workshop on Automatic Speech Recognition and Understanding, pp. 72–77. IEEE (2013)
Liu, Y., et al.: Roberta: a robustly optimized bert pretraining approach (2019). arXiv preprintarXiv:1907.11692
Lu, D., Weng, Q.: A survey of image classification methods and techniques for improving classification performance. Int. J. Remote Sens.28(5), 823–870 (2007)
Menon, A.K., Jayasumana, S., Rawat, A.S., Jain, H., Veit, A., Kumar, S.: Long-tail learning via logit adjustment (2020). arXiv preprintarXiv:2007.07314
Ren, M., et al.: Meta-learning for semi-supervised few-shot classification (2018). arXiv preprintarXiv:1803.00676
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature323(6088), 533–536 (1986)
Sang, E.F., De Meulder, F.: Introduction to the CONLL-2003 shared task: language-independent named entity recognition (2003). arXiv preprint cs/0306050
Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P.H., Hospedales, T.M.: Learning to compare: relation network for few-shot learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1199–1208 (2018)
Wang, Y., Chu, H., Zhang, C., Gao, J.: Learning from language description: low-shot named entity recognition via decomposed framework (2021). arXiv preprintarXiv:2109.05357
Wang, Y., Zhang, B., Hou, W., Wu, Z., Wang, J., Shinozaki, T.: Margin calibration for long-tailed visual recognition (2021). arXiv preprintarXiv:2112.07225
Wei, H., Xie, R., Cheng, H., Feng, L., An, B., Li, Y.: Mitigating neural network overconfidence with logit normalization. In: International Conference on Machine Learning, pp. 23631–23644. PMLR (2022)
Yang, Y., Katiyar, A.: Simple and effective few-shot named entity recognition with structured nearest neighbor learning (2020). arXiv preprintarXiv:2010.02405
You, Y., Chen, T., Sui, Y., Chen, T., Wang, Z., Shen, Y.: Graph contrastive learning with augmentations. Adv. Neural. Inf. Process. Syst.33, 5812–5823 (2020)
Yu, M., et al.: Diverse few-shot text classification with multiple metrics (2018). arXiv preprintarXiv:1805.07513
Acknowledgment
This work is supported by the HeiBei Province Major Science and Technology Project(No. 23260101Z) and the Research and Application of Intelligent Regional Industrial Brain Platform.
Author information
Authors and Affiliations
National Engineering Research Center for Software Engineering, Peking University, Beijing, China
Jinglei Zhang, Guochang Wen & Qing Gao
School of Software and Microelectronics, Peking University, Beijng, China
Jinglei Zhang, Guochang Wen & XiXin Cao
China Academy of Industrial Internet, Beijing, China
DongDong Du
Beijing Institute of Control and Electronic Technology, Beijing, China
NingLin Liao
Handan Institute of Innovation, Peking University, Hebei, Handan, China
Minghui Zhang
- Jinglei Zhang
You can also search for this author inPubMed Google Scholar
- Guochang Wen
You can also search for this author inPubMed Google Scholar
- NingLin Liao
You can also search for this author inPubMed Google Scholar
- DongDong Du
You can also search for this author inPubMed Google Scholar
- Qing Gao
You can also search for this author inPubMed Google Scholar
- Minghui Zhang
You can also search for this author inPubMed Google Scholar
- XiXin Cao
You can also search for this author inPubMed Google Scholar
Corresponding authors
Correspondence toDongDong Du orQing Gao.
Editor information
Editors and Affiliations
Chinese Academy of Sciences, Beijing, China
Cungeng Cao
Zhejiang University, Zhejiang, China
Huajun Chen
Emory University, Atlanta, GA, USA
Liang Zhao
Birmingham City University, Birmingham, UK
Junaid Arshad
Monash University, Banten, Indonesia
Taufiq Asyhari
Birmingham City University, Birmingham, UK
Yonghao Wang
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zhang, J.et al. (2024). Logit Adjustment with Normalization and Augmentation in Few-Shot Named Entity Recognition. In: Cao, C., Chen, H., Zhao, L., Arshad, J., Asyhari, T., Wang, Y. (eds) Knowledge Science, Engineering and Management. KSEM 2024. Lecture Notes in Computer Science(), vol 14886. Springer, Singapore. https://doi.org/10.1007/978-981-97-5498-4_31
Download citation
Published:
Publisher Name:Springer, Singapore
Print ISBN:978-981-97-5497-7
Online ISBN:978-981-97-5498-4
eBook Packages:Computer ScienceComputer Science (R0)
Share this paper
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative