Movatterモバイル変換


[0]ホーム

URL:


Skip to main content

Advertisement

Springer Nature Link
Log in

Logit Adjustment with Normalization and Augmentation in Few-Shot Named Entity Recognition

  • Conference paper
  • First Online:

Abstract

We study the problem of few-shot learning in Name Entity Recognition(FS-NER). Specifically, unlike other sequence labeling-based models, that mainly focus on better representations, we leverage logit adjustment technology to alleviate the problem that the different distribution between training and test dataset. Furthermore, we propose a simple but effective method, called Logit Adjustment with Normalization and Augmentation (LANA), for FS-NER. In detail, LANA first combines moving average and logit adjustment to retain the information of pre-training to overcome the representation drop problem in FS-NER. We also involve logit normalization to deal with the overfitting problem in FS-NER, and further improve the generalization ability of LANA. Our method achieves competitive performance on seven widely used FS-NER datasets and significantly reduces the influence of overfitting and representation drop.

This is a preview of subscription content,log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 26311
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 10581
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide -see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Similar content being viewed by others

References

  1. Atefeh, F., Khreich, W.: A survey of techniques for event detection in twitter. Comput. Intell.31(1), 132–164 (2015)

    Article MathSciNet  Google Scholar 

  2. Bach, N., Badaskar, S.: A review of relation extraction. Lit. Rev. Lang. Stat.II(2), 1–15 (2007)

    Google Scholar 

  3. Brown, T., et al.: Language models are few-shot learners. Adv. Neural. Inf. Process. Syst.33, 1877–1901 (2020)

    Google Scholar 

  4. Budzianowski, P., et al.: Multiwoz–a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling (2018). arXiv preprintarXiv:1810.00278

  5. Chen, C., Zhan, Y., Yu, B., Liu, L., Luo, Y., Du, B.: Resistance training using prior bias: toward unbiased scene graph generation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 212–220 (2022)

    Google Scholar 

  6. Chen, J., Zhang, R., Mao, Y., Xu, J.: Contrastnet: a contrastive learning framework for few-shot text classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 10492–10500 (2022)

    Google Scholar 

  7. Coucke, A., et al.: Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces (2018). arXiv preprintarXiv:1805.10190

  8. Das, S.S.S., Katiyar, A., Passonneau, R.J., Zhang, R.: Container: few-shot named entity recognition via contrastive learning (2021). arXiv preprintarXiv:2109.07589

  9. Derczynski, L., Nichols, E., Van Erp, M., Limsopatham, N.: Results of the wnut2017 shared task on novel and emerging entity recognition. In: Proceedings of the 3rd Workshop on Noisy User-generated Text, pp. 140–147 (2017)

    Google Scholar 

  10. Ding, N., et al.: Few-nerd: a few-shot named entity recognition dataset (2021). arXiv preprintarXiv:2105.07464

  11. Fritzler, A., Logacheva, V., Kretov, M.: Few-shot classification in named entity recognition task. In: Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing, pp. 993–1000 (2019)

    Google Scholar 

  12. Geng, R., Li, B., Li, Y., Zhu, X., Jian, P., Sun, J.: Induction networks for few-shot text classification (2019). arXiv preprintarXiv:1902.10482

  13. Hakkani-Tür, D., et al.: Multi-domain joint semantic frame parsing using bi-directional rnn-lstm. In: Interspeech, pp. 715–719 (2016)

    Google Scholar 

  14. Hofer, M., Kormilitzin, A., Goldberg, P., Nevado-Holgado, A.: Few-shot learning for named entity recognition in medical text (2018). arXiv preprintarXiv:1811.05468

  15. Huang, J., et al.: Few-shot named entity recognition: a comprehensive study (2020). arXiv preprintarXiv:2012.14978

  16. Kowsari, K., Jafari Meimandi, K., Heidarysafa, M., Mendu, S., Barnes, L., Brown, D.: Text classification algorithms: a survey. Information10(4), 150 (2019)

    Article  Google Scholar 

  17. Li, X., Feng, J., Meng, Y., Han, Q., Wu, F., Li, J.: A unified MRC framework for named entity recognition (2019). arXiv preprintarXiv:1910.11476

  18. Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of the AAAI conference on artificial intelligence, vol. 29 (2015)

    Google Scholar 

  19. Liu, J., Pasupat, P., Cyphers, S., Glass, J.: Asgard: a portable architecture for multilingual dialogue systems. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 8386–8390. IEEE (2013)

    Google Scholar 

  20. Liu, J., Pasupat, P., Wang, Y., Cyphers, S., Glass, J.: Query understanding enhanced by hierarchical parsing structures. In: 2013 IEEE Workshop on Automatic Speech Recognition and Understanding, pp. 72–77. IEEE (2013)

    Google Scholar 

  21. Liu, Y., et al.: Roberta: a robustly optimized bert pretraining approach (2019). arXiv preprintarXiv:1907.11692

  22. Lu, D., Weng, Q.: A survey of image classification methods and techniques for improving classification performance. Int. J. Remote Sens.28(5), 823–870 (2007)

    Article  Google Scholar 

  23. Menon, A.K., Jayasumana, S., Rawat, A.S., Jain, H., Veit, A., Kumar, S.: Long-tail learning via logit adjustment (2020). arXiv preprintarXiv:2007.07314

  24. Ren, M., et al.: Meta-learning for semi-supervised few-shot classification (2018). arXiv preprintarXiv:1803.00676

  25. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature323(6088), 533–536 (1986)

    Google Scholar 

  26. Sang, E.F., De Meulder, F.: Introduction to the CONLL-2003 shared task: language-independent named entity recognition (2003). arXiv preprint cs/0306050

    Google Scholar 

  27. Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P.H., Hospedales, T.M.: Learning to compare: relation network for few-shot learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1199–1208 (2018)

    Google Scholar 

  28. Wang, Y., Chu, H., Zhang, C., Gao, J.: Learning from language description: low-shot named entity recognition via decomposed framework (2021). arXiv preprintarXiv:2109.05357

  29. Wang, Y., Zhang, B., Hou, W., Wu, Z., Wang, J., Shinozaki, T.: Margin calibration for long-tailed visual recognition (2021). arXiv preprintarXiv:2112.07225

  30. Wei, H., Xie, R., Cheng, H., Feng, L., An, B., Li, Y.: Mitigating neural network overconfidence with logit normalization. In: International Conference on Machine Learning, pp. 23631–23644. PMLR (2022)

    Google Scholar 

  31. Yang, Y., Katiyar, A.: Simple and effective few-shot named entity recognition with structured nearest neighbor learning (2020). arXiv preprintarXiv:2010.02405

  32. You, Y., Chen, T., Sui, Y., Chen, T., Wang, Z., Shen, Y.: Graph contrastive learning with augmentations. Adv. Neural. Inf. Process. Syst.33, 5812–5823 (2020)

    Google Scholar 

  33. Yu, M., et al.: Diverse few-shot text classification with multiple metrics (2018). arXiv preprintarXiv:1805.07513

Download references

Acknowledgment

This work is supported by the HeiBei Province Major Science and Technology Project(No. 23260101Z) and the Research and Application of Intelligent Regional Industrial Brain Platform.

Author information

Authors and Affiliations

  1. National Engineering Research Center for Software Engineering, Peking University, Beijing, China

    Jinglei Zhang, Guochang Wen & Qing Gao

  2. School of Software and Microelectronics, Peking University, Beijng, China

    Jinglei Zhang, Guochang Wen & XiXin Cao

  3. China Academy of Industrial Internet, Beijing, China

    DongDong Du

  4. Beijing Institute of Control and Electronic Technology, Beijing, China

    NingLin Liao

  5. Handan Institute of Innovation, Peking University, Hebei, Handan, China

    Minghui Zhang

Authors
  1. Jinglei Zhang

    You can also search for this author inPubMed Google Scholar

  2. Guochang Wen

    You can also search for this author inPubMed Google Scholar

  3. NingLin Liao

    You can also search for this author inPubMed Google Scholar

  4. DongDong Du

    You can also search for this author inPubMed Google Scholar

  5. Qing Gao

    You can also search for this author inPubMed Google Scholar

  6. Minghui Zhang

    You can also search for this author inPubMed Google Scholar

  7. XiXin Cao

    You can also search for this author inPubMed Google Scholar

Corresponding authors

Correspondence toDongDong Du orQing Gao.

Editor information

Editors and Affiliations

  1. Chinese Academy of Sciences, Beijing, China

    Cungeng Cao

  2. Zhejiang University, Zhejiang, China

    Huajun Chen

  3. Emory University, Atlanta, GA, USA

    Liang Zhao

  4. Birmingham City University, Birmingham, UK

    Junaid Arshad

  5. Monash University, Banten, Indonesia

    Taufiq Asyhari

  6. Birmingham City University, Birmingham, UK

    Yonghao Wang

Rights and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, J.et al. (2024). Logit Adjustment with Normalization and Augmentation in Few-Shot Named Entity Recognition. In: Cao, C., Chen, H., Zhao, L., Arshad, J., Asyhari, T., Wang, Y. (eds) Knowledge Science, Engineering and Management. KSEM 2024. Lecture Notes in Computer Science(), vol 14886. Springer, Singapore. https://doi.org/10.1007/978-981-97-5498-4_31

Download citation

Publish with us

Access this chapter

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 26311
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 10581
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide -see info

Tax calculation will be finalised at checkout

Purchases are for personal use only


[8]ページ先頭

©2009-2025 Movatter.jp