Movatterモバイル変換


[0]ホーム

URL:


Skip to main content

Advertisement

Springer Nature Link
Log in

Pattern Filtering Attention for Distant Supervised Relation Extraction via Online Clustering

  • Conference paper
  • First Online:

Part of the book series:Lecture Notes in Computer Science ((LNISA,volume 11881))

Included in the following conference series:

  • 2566Accesses

Abstract

Distant supervised relation extraction has been widely used to extract relational facts in large-scale corpus but inevitably suffers from the wrong label problem. Many methods use attention mechanisms to address this issue. However, the attention weights in these models are not discriminative and precise enough to fully filter out noise. In this paper, we propose a novel Pattern Filtering Attention (PFA), which can filter noise effectively. Firstly, we adopt an online clustering algorithm on the instances labeled with the same relation to extract potential semantic centers (positive patterns) of each relation, and these patterns have less noise statistically. Then, we build a sentence-level attention based on the similarities of instances and positive patterns. Due to the large differences between these similarities, our model can assign more discriminative weights to instances to reduce the influence of noisy data. Experimental results on the New York Times (NYT) dataset show that our model can effectively improve the performance of relation extraction compared with state-of-the-art methods.

This material is supported partially by National Key R&D Program of China under Grant No. 2018YFC1604000 and No. 2018YFC1604003, partially by National Science Foundation of China (NSFC) under Grant No. 61872272 and No. 61772382.

This is a preview of subscription content,log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 11439
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 14299
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide -see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Similar content being viewed by others

References

  1. Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pp. 1247–1250. AcM (2008)

    Google Scholar 

  2. Du, J., Han, J., Way, A., Wan, D.: Multi-level structured self-attentions for distantly supervised relation extraction. In: EMNLP (2018)

    Google Scholar 

  3. Duda, R.: Sequential k-means clustering.http://www.cs.princeton.edu/courses/archive/fall08/cos436/Duda/C/sk_means.htm

  4. Feng, J., Huang, M., Zhao, L., Yang, Y., Zhu, X.: Reinforcement learning for relation classification from noisy data. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)

    Google Scholar 

  5. GuoDong, Z., Jian, S., Jie, Z., Min, Z.: Exploring various knowledge in relation extraction. In: Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, pp. 427–434. Association for Computational Linguistics (2005)

    Google Scholar 

  6. Han, X., Yu, P., Liu, Z., Sun, M., Li, P.: Hierarchical relation extraction with coarse-to-fine grained attention. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2236–2245 (2018)

    Google Scholar 

  7. Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.R.: Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprintarXiv:1207.0580 (2012)

  8. Hoffmann, R., Zhang, C., Ling, X., Zettlemoyer, L., Weld, D.S.: Knowledge-based weak supervision for information extraction of overlapping relations. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, vol. 1, pp. 541–550. Association for Computational Linguistics (2011)

    Google Scholar 

  9. Ji, G., Liu, K., He, S., Zhao, J., et al.: Distant supervision for relation extraction with sentence-level attention and entity descriptions. In: AAAI, pp. 3060–3066 (2017)

    Google Scholar 

  10. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprintarXiv:1412.6980 (2014)

  11. Lin, Y., Shen, S., Liu, Z., Luan, H., Sun, M.: Neural relation extraction with selective attention over instances. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2124–2133 (2016)

    Google Scholar 

  12. Liu, T., Wang, K., Chang, B., Sui, Z.: A soft-label method for noise-tolerant distantly supervised relation extraction. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 1790–1795 (2017)

    Google Scholar 

  13. Luo, B., et al.: Learning with noise: enhance distantly supervised relation extraction with dynamic transition matrix. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 430–439 (2017)

    Google Scholar 

  14. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprintarXiv:1301.3781 (2013)

  15. Mintz, M., Bills, S., Snow, R., Jurafsky, D.: Distant supervision for relation extraction without labeled data. In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, vol. 2, pp. 1003–1011. Association for Computational Linguistics (2009)

    Google Scholar 

  16. Mooney, R.J., Bunescu, R.C.: Subsequence kernels for relation extraction. In: Advances in Neural Information Processing Systems, pp. 171–178 (2006)

    Google Scholar 

  17. Peng, M., et al.: Improving distant supervision of relation extraction with unsupervised methods. In: Cellary, W., Mokbel, M.F., Wang, J., Wang, H., Zhou, R., Zhang, Y. (eds.) WISE 2016. LNCS, vol. 10041, pp. 561–568. Springer, Cham (2016).https://doi.org/10.1007/978-3-319-48740-3_42

    Chapter  Google Scholar 

  18. Qin, P., Xu, W., Wang, W.Y.: DSGAN: generative adversarial training for distant supervision relation extraction. arXiv preprintarXiv:1805.09929 (2018)

  19. Riedel, S., Yao, L., McCallum, A.: Modeling relations and their mentions without labeled text. In: Balcázar, J.L., Bonchi, F., Gionis, A., Sebag, M. (eds.) ECML PKDD 2010. LNCS (LNAI), vol. 6323, pp. 148–163. Springer, Heidelberg (2010).https://doi.org/10.1007/978-3-642-15939-8_10

    Chapter  Google Scholar 

  20. Surdeanu, M., Tibshirani, J., Nallapati, R., Manning, C.D.: Multi-instance multi-label learning for relation extraction. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 455–465. Association for Computational Linguistics (2012)

    Google Scholar 

  21. Takamatsu, S., Sato, I., Nakagawa, H.: Reducing wrong labels in distant supervision for relation extraction. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1, pp. 721–729. Association for Computational Linguistics (2012)

    Google Scholar 

  22. Zeng, D., Liu, K., Chen, Y., Zhao, J.: Distant supervision for relation extraction via piecewise convolutional neural networks. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1753–1762 (2015)

    Google Scholar 

  23. Zeng, D., Liu, K., Lai, S., Zhou, G., Zhao, J.: Relation classification via convolutional deep neural network. In: Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pp. 2335–2344 (2014)

    Google Scholar 

  24. Zhang, N., Deng, S., Sun, Z., Chen, X., Zhang, W., Chen, H.: Attention-based capsule networks with dynamic routing for relation extraction. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 986–992 (2018)

    Google Scholar 

  25. Zhang, Y., Qi, P., Manning, C.D.: Graph convolution over pruned dependency trees improves relation extraction. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2205–2215 (2018)

    Google Scholar 

  26. Zhou, P., et al.: Attention-based bidirectional long short-term memory networks for relation classification. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 207–212 (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science, Wuhan University, Wuhan, China

    Min Peng, Qingwen Liao, Weilong Hu & Gang Tian

  2. Centre for Applied Informatics, Victoria University, Melbourne, Australia

    Hua Wang & YanChun Zhang

Authors
  1. Min Peng

    You can also search for this author inPubMed Google Scholar

  2. Qingwen Liao

    You can also search for this author inPubMed Google Scholar

  3. Weilong Hu

    You can also search for this author inPubMed Google Scholar

  4. Gang Tian

    You can also search for this author inPubMed Google Scholar

  5. Hua Wang

    You can also search for this author inPubMed Google Scholar

  6. YanChun Zhang

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence toMin Peng.

Editor information

Editors and Affiliations

  1. University of Hong Kong, Hong Kong SAR, China

    Reynold Cheng

  2. University of Ioannina, Ioannina, Greece

    Nikos Mamoulis

  3. University of California, Los Angeles, CA, USA

    Yizhou Sun

  4. Hong Kong Baptist University, Kowloon Tong, Hong Kong SAR, China

    Xin Huang

Rights and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Peng, M., Liao, Q., Hu, W., Tian, G., Wang, H., Zhang, Y. (2019). Pattern Filtering Attention for Distant Supervised Relation Extraction via Online Clustering. In: Cheng, R., Mamoulis, N., Sun, Y., Huang, X. (eds) Web Information Systems Engineering – WISE 2019. WISE 2020. Lecture Notes in Computer Science(), vol 11881. Springer, Cham. https://doi.org/10.1007/978-3-030-34223-4_20

Download citation

Publish with us

Access this chapter

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 11439
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 14299
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide -see info

Tax calculation will be finalised at checkout

Purchases are for personal use only


[8]ページ先頭

©2009-2025 Movatter.jp