Movatterモバイル変換


[0]ホーム

URL:


Skip to main content

Advertisement

Springer Nature Link
Log in

Neural-Symbolic Reasoning with External Knowledge for Machine Reading Comprehension

  • Conference paper
  • First Online:

Abstract

Machine reading comprehension is a fundamental in natural language understanding. Existing large-scale pre-trained language models and graph neural network-based models have achieved good gains on logical reasoning of text. However, neither of them can give a complete reasoning chain, while symbolic logic-based reasoning is explicit and explainable. Therefore, we propose a framework LoGEK that integrates symbolicLogic andGraph neural networks for reasoning, while leveragingExternalKnowledge to augment the logical graph. The LoGEK model consists of three parts: logic extraction and extension, logical graph reasoning and answer prediction. Specifically, LoGEK extracts and extends logic set from the unstructured text. Then the logical graph reasoning module uses external knowledge to extend the original logical graph. After that, the model uses a path-based relational graph neural network to model the extended logical graph. Finally, the prediction module performs answer prediction based on graph embeddings and text embeddings. We conduct experiments on benchmark datasets for logical reasoning to evaluate the performance of LoGEK. The experimental results show that the accuracy of the method in this paper is better than the baseline models, which verifies the effectiveness of the method.

This work was supported by the Open Research Project of the Hubei Key Laboratory of Intelligent Geo-Information Processing (No. KLIGIP-2022-B11), and in part by the National Natural Science Foundation of China (NSFC) (No. 61972365, 42071382).

This is a preview of subscription content,log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 11439
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 14299
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide -see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Similar content being viewed by others

References

  1. Asai, A., Hashimoto, K., Hajishirzi, H., Socher, R., Xiong, C.: Learning to retrieve reasoning paths over wikipedia graph for question answering. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, 26–30 April 2020. OpenReview.net (2020)

    Google Scholar 

  2. Bang, Y., et al.: A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. CoRR abs/arXiv: 2302.04023 (2023)

  3. Bubeck, S., et al.: Sparks of artificial general intelligence: Early experiments with GPT-4. CoRR abs/arXiv: 2303.12712 (2023)

  4. Chen, J., Zhang, Z., Zhao, H.: Modeling hierarchical reasoning chains by linking discourse units and key phrases for reading comprehension. In: Proceedings of the 29th International Conference on Computational Linguistics, COLING 2022, Gyeongju, Republic of Korea, 12–17 October 2022, pp. 1467–1479. International Committee on Computational Linguistics (2022)

    Google Scholar 

  5. Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL-HLT, pp. 4171–4186. Association for Computational Linguistics (2019)

    Google Scholar 

  6. Feng, Y., Chen, X., Lin, B.Y., Wang, P., Yan, J., Ren, X.: Scalable multi-hop relational reasoning for knowledge-aware question answering. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, 16–20 November 2020, pp. 1295–1309. Association for Computational Linguistics (2020)

    Google Scholar 

  7. Hamilton, W.L., Bajaj, P., Zitnik, M., Jurafsky, D., Leskovec, J.: Embedding logical queries on knowledge graphs. In: Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, 3–8 December 2018, Montréal, Canada, pp. 2030–2041 (2018)

    Google Scholar 

  8. Jiao, F., Guo, Y., Song, X., Nie, L.: Merit: meta-path guided contrastive learning for logical reasoning. In: Findings of the Association for Computational Linguistics: ACL 2022, Dublin, Ireland, 22–27 May 2022, pp. 3496–3509. Association for Computational Linguistics (2022)

    Google Scholar 

  9. Joshi, V., Peters, M.E., Hopkins, M.: Extending a parser to distant domains using a few dozen partially annotated examples. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1190–1199 (2018)

    Google Scholar 

  10. Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., Soricut, R.: ALBERT: a lite BERT for self-supervised learning of language representations. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, 26–30 April 2020. OpenReview.net (2020)

    Google Scholar 

  11. Li, X., Cheng, G., Chen, Z., Sun, Y., Qu, Y.: Adalogn: adaptive logic graph network for reasoning-based machine reading comprehension. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 7147–7161 (2022)

    Google Scholar 

  12. Lin, B.Y., Chen, X., Chen, J., Ren, X.: Kagnet: knowledge-aware graph networks for commonsense reasoning. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 2829–2839 (2019)

    Google Scholar 

  13. Liu, H., Singh, P.: Conceptnet-a practical commonsense reasoning tool-kit. BT Technol. J.22(4), 211–226 (2004)

    Article  Google Scholar 

  14. Liu, J., Cui, L., Liu, H., Huang, D., Wang, Y., Zhang, Y.: LogiQA: a challenge dataset for machine reading comprehension with logical reasoning. In: Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020, pp. 3622–3628. ijcai.org (2020)

    Google Scholar 

  15. Liu, L., et al.: On the variance of the adaptive learning rate and beyond. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, 26–30 April 2020. OpenReview.net (2020)

    Google Scholar 

  16. Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., Neubig, G.: Pre-train, prompt, and predict: a systematic survey of prompting methods in natural language processing. ACM Comput. Surv.55(9), 195:1–195:35 (2023)

    Google Scholar 

  17. Radford, A., Narasimhan, K., Salimans, T., Sutskever, I., et al.: Improving language understanding by generative pre-training (2018)

    Google Scholar 

  18. Radford, A., et al.: Language models are unsupervised multitask learners. OpenAI blog1(8), 9 (2019)

    Google Scholar 

  19. Ren, H., Hu, W., Leskovec, J.: Query2box: reasoning over knowledge graphs in vector space using box embeddings. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, 26–30 April 2020. OpenReview.net (2020)

    Google Scholar 

  20. Ren, H., Leskovec, J.: Beta embeddings for multi-hop logical reasoning in knowledge graphs. In: Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, 6–12 December 2020, virtual (2020)

    Google Scholar 

  21. Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach, 4th edn. Pearson (2020)

    Google Scholar 

  22. Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., Welling, M.: Modeling relational data with graph convolutional networks. In: Gangemi, A., et al. (eds.) ESWC 2018. LNCS, vol. 10843, pp. 593–607. Springer, Cham (2018).https://doi.org/10.1007/978-3-319-93417-4_38

    Chapter  Google Scholar 

  23. Wang, S., et al.: Logic-driven context extension and data augmentation for logical reasoning of text. arXiv preprintarXiv:2105.03659 (2021)

  24. Wang, X., Wei, J., Schuurmans, D., Le, Q.V., Chi, E.H., Zhou, D.: Self-consistency improves chain of thought reasoning in language models. CoRR abs/arXiv: 2203.11171 (2022)

  25. Yang, Y., Song, L.: Learn to explain efficiently via neural logic inductive learning. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, 26–30 April 2020. OpenReview.net (2020)

    Google Scholar 

  26. Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R.R., Le, Q.V.: Xlnet: generalized autoregressive pretraining for language understanding. In: Advances in Neural Information Processing Systems 32 (2019)

    Google Scholar 

  27. Yu, W., Jiang, Z., Dong, Y., Feng, J.: Reclor: a reading comprehension dataset requiring logical reasoning. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, 26–30 April 2020. OpenReview.net (2020)

    Google Scholar 

  28. Zhang, Y., et al.: Efficient probabilistic logic reasoning with graph neural networks. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, 26–30 April 2020. OpenReview.net (2020)

    Google Scholar 

  29. Zhao, J., Rudnick, E.M., Patel, J.H.: Static logic implication with application to redundancy identification. In: 15th IEEE VLSI Test Symposium (VTS’97), Monterey, California, USA, 27 April -1 May 1997, pp. 288–295. IEEE Computer Society (1997)

    Google Scholar 

  30. Zhou, D., et al.: Least-to-most prompting enables complex reasoning in large language models. CoRR abs/arXiv: 2205.10625 (2022)

Download references

Author information

Authors and Affiliations

  1. School of Computer Science, China University of Geosciences, Wuhan, 430074, China

    Yilin Duan, Sijia Zhou, Xiaoyue Peng, Xiaojun Kang & Hong Yao

  2. Hubei Key Laboratory of Intelligent Geo-Information Processing, China University of Geosciences, Wuhan, 430074, China

    Xiaojun Kang & Hong Yao

Authors
  1. Yilin Duan

    You can also search for this author inPubMed Google Scholar

  2. Sijia Zhou

    You can also search for this author inPubMed Google Scholar

  3. Xiaoyue Peng

    You can also search for this author inPubMed Google Scholar

  4. Xiaojun Kang

    You can also search for this author inPubMed Google Scholar

  5. Hong Yao

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence toXiaojun Kang.

Editor information

Editors and Affiliations

  1. School of Automation, Central South University, Changsha, China

    Biao Luo

  2. Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Long Cheng

  3. Institute of Cyber-Systems and Control, Zhejiang University, Hangzhou, China

    Zheng-Guang Wu

  4. School of Automation, Guangdong University of Technology, Guangzhou, China

    Hongyi Li

  5. School of Electrical Engineering and Telecommunications, UNSW Sydney, Sydney, NSW, Australia

    Chaojie Li

Rights and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Duan, Y., Zhou, S., Peng, X., Kang, X., Yao, H. (2024). Neural-Symbolic Reasoning with External Knowledge for Machine Reading Comprehension. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Communications in Computer and Information Science, vol 1965. Springer, Singapore. https://doi.org/10.1007/978-981-99-8145-8_34

Download citation

Publish with us

Access this chapter

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 11439
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 14299
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide -see info

Tax calculation will be finalised at checkout

Purchases are for personal use only


[8]ページ先頭

©2009-2025 Movatter.jp