- Yilin Duan ORCID:orcid.org/0009-0001-1940-958110,
- Sijia Zhou ORCID:orcid.org/0009-0002-4348-657610,
- Xiaoyue Peng ORCID:orcid.org/0009-0002-2574-085X10,
- Xiaojun Kang ORCID:orcid.org/0000-0002-5628-054510,11 &
- …
- Hong Yao ORCID:orcid.org/0000-0002-0367-952810,11
Part of the book series:Communications in Computer and Information Science ((CCIS,volume 1965))
Included in the following conference series:
856Accesses
Abstract
Machine reading comprehension is a fundamental in natural language understanding. Existing large-scale pre-trained language models and graph neural network-based models have achieved good gains on logical reasoning of text. However, neither of them can give a complete reasoning chain, while symbolic logic-based reasoning is explicit and explainable. Therefore, we propose a framework LoGEK that integrates symbolicLogic andGraph neural networks for reasoning, while leveragingExternalKnowledge to augment the logical graph. The LoGEK model consists of three parts: logic extraction and extension, logical graph reasoning and answer prediction. Specifically, LoGEK extracts and extends logic set from the unstructured text. Then the logical graph reasoning module uses external knowledge to extend the original logical graph. After that, the model uses a path-based relational graph neural network to model the extended logical graph. Finally, the prediction module performs answer prediction based on graph embeddings and text embeddings. We conduct experiments on benchmark datasets for logical reasoning to evaluate the performance of LoGEK. The experimental results show that the accuracy of the method in this paper is better than the baseline models, which verifies the effectiveness of the method.
This work was supported by the Open Research Project of the Hubei Key Laboratory of Intelligent Geo-Information Processing (No. KLIGIP-2022-B11), and in part by the National Natural Science Foundation of China (NSFC) (No. 61972365, 42071382).
This is a preview of subscription content,log in via an institution to check access.
Access this chapter
Subscribe and save
- Get 10 units per month
- Download Article/Chapter or eBook
- 1 Unit = 1 Article or 1 Chapter
- Cancel anytime
Buy Now
- Chapter
- JPY 3498
- Price includes VAT (Japan)
- eBook
- JPY 11439
- Price includes VAT (Japan)
- Softcover Book
- JPY 14299
- Price includes VAT (Japan)
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Asai, A., Hashimoto, K., Hajishirzi, H., Socher, R., Xiong, C.: Learning to retrieve reasoning paths over wikipedia graph for question answering. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, 26–30 April 2020. OpenReview.net (2020)
Bang, Y., et al.: A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. CoRR abs/arXiv: 2302.04023 (2023)
Bubeck, S., et al.: Sparks of artificial general intelligence: Early experiments with GPT-4. CoRR abs/arXiv: 2303.12712 (2023)
Chen, J., Zhang, Z., Zhao, H.: Modeling hierarchical reasoning chains by linking discourse units and key phrases for reading comprehension. In: Proceedings of the 29th International Conference on Computational Linguistics, COLING 2022, Gyeongju, Republic of Korea, 12–17 October 2022, pp. 1467–1479. International Committee on Computational Linguistics (2022)
Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL-HLT, pp. 4171–4186. Association for Computational Linguistics (2019)
Feng, Y., Chen, X., Lin, B.Y., Wang, P., Yan, J., Ren, X.: Scalable multi-hop relational reasoning for knowledge-aware question answering. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, 16–20 November 2020, pp. 1295–1309. Association for Computational Linguistics (2020)
Hamilton, W.L., Bajaj, P., Zitnik, M., Jurafsky, D., Leskovec, J.: Embedding logical queries on knowledge graphs. In: Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, 3–8 December 2018, Montréal, Canada, pp. 2030–2041 (2018)
Jiao, F., Guo, Y., Song, X., Nie, L.: Merit: meta-path guided contrastive learning for logical reasoning. In: Findings of the Association for Computational Linguistics: ACL 2022, Dublin, Ireland, 22–27 May 2022, pp. 3496–3509. Association for Computational Linguistics (2022)
Joshi, V., Peters, M.E., Hopkins, M.: Extending a parser to distant domains using a few dozen partially annotated examples. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1190–1199 (2018)
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., Soricut, R.: ALBERT: a lite BERT for self-supervised learning of language representations. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, 26–30 April 2020. OpenReview.net (2020)
Li, X., Cheng, G., Chen, Z., Sun, Y., Qu, Y.: Adalogn: adaptive logic graph network for reasoning-based machine reading comprehension. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 7147–7161 (2022)
Lin, B.Y., Chen, X., Chen, J., Ren, X.: Kagnet: knowledge-aware graph networks for commonsense reasoning. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 2829–2839 (2019)
Liu, H., Singh, P.: Conceptnet-a practical commonsense reasoning tool-kit. BT Technol. J.22(4), 211–226 (2004)
Liu, J., Cui, L., Liu, H., Huang, D., Wang, Y., Zhang, Y.: LogiQA: a challenge dataset for machine reading comprehension with logical reasoning. In: Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020, pp. 3622–3628. ijcai.org (2020)
Liu, L., et al.: On the variance of the adaptive learning rate and beyond. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, 26–30 April 2020. OpenReview.net (2020)
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., Neubig, G.: Pre-train, prompt, and predict: a systematic survey of prompting methods in natural language processing. ACM Comput. Surv.55(9), 195:1–195:35 (2023)
Radford, A., Narasimhan, K., Salimans, T., Sutskever, I., et al.: Improving language understanding by generative pre-training (2018)
Radford, A., et al.: Language models are unsupervised multitask learners. OpenAI blog1(8), 9 (2019)
Ren, H., Hu, W., Leskovec, J.: Query2box: reasoning over knowledge graphs in vector space using box embeddings. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, 26–30 April 2020. OpenReview.net (2020)
Ren, H., Leskovec, J.: Beta embeddings for multi-hop logical reasoning in knowledge graphs. In: Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, 6–12 December 2020, virtual (2020)
Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach, 4th edn. Pearson (2020)
Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., Welling, M.: Modeling relational data with graph convolutional networks. In: Gangemi, A., et al. (eds.) ESWC 2018. LNCS, vol. 10843, pp. 593–607. Springer, Cham (2018).https://doi.org/10.1007/978-3-319-93417-4_38
Wang, S., et al.: Logic-driven context extension and data augmentation for logical reasoning of text. arXiv preprintarXiv:2105.03659 (2021)
Wang, X., Wei, J., Schuurmans, D., Le, Q.V., Chi, E.H., Zhou, D.: Self-consistency improves chain of thought reasoning in language models. CoRR abs/arXiv: 2203.11171 (2022)
Yang, Y., Song, L.: Learn to explain efficiently via neural logic inductive learning. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, 26–30 April 2020. OpenReview.net (2020)
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R.R., Le, Q.V.: Xlnet: generalized autoregressive pretraining for language understanding. In: Advances in Neural Information Processing Systems 32 (2019)
Yu, W., Jiang, Z., Dong, Y., Feng, J.: Reclor: a reading comprehension dataset requiring logical reasoning. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, 26–30 April 2020. OpenReview.net (2020)
Zhang, Y., et al.: Efficient probabilistic logic reasoning with graph neural networks. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, 26–30 April 2020. OpenReview.net (2020)
Zhao, J., Rudnick, E.M., Patel, J.H.: Static logic implication with application to redundancy identification. In: 15th IEEE VLSI Test Symposium (VTS’97), Monterey, California, USA, 27 April -1 May 1997, pp. 288–295. IEEE Computer Society (1997)
Zhou, D., et al.: Least-to-most prompting enables complex reasoning in large language models. CoRR abs/arXiv: 2205.10625 (2022)
Author information
Authors and Affiliations
School of Computer Science, China University of Geosciences, Wuhan, 430074, China
Yilin Duan, Sijia Zhou, Xiaoyue Peng, Xiaojun Kang & Hong Yao
Hubei Key Laboratory of Intelligent Geo-Information Processing, China University of Geosciences, Wuhan, 430074, China
Xiaojun Kang & Hong Yao
- Yilin Duan
You can also search for this author inPubMed Google Scholar
- Sijia Zhou
You can also search for this author inPubMed Google Scholar
- Xiaoyue Peng
You can also search for this author inPubMed Google Scholar
- Xiaojun Kang
You can also search for this author inPubMed Google Scholar
- Hong Yao
You can also search for this author inPubMed Google Scholar
Corresponding author
Correspondence toXiaojun Kang.
Editor information
Editors and Affiliations
School of Automation, Central South University, Changsha, China
Biao Luo
Institute of Automation, Chinese Academy of Sciences, Beijing, China
Long Cheng
Institute of Cyber-Systems and Control, Zhejiang University, Hangzhou, China
Zheng-Guang Wu
School of Automation, Guangdong University of Technology, Guangzhou, China
Hongyi Li
School of Electrical Engineering and Telecommunications, UNSW Sydney, Sydney, NSW, Australia
Chaojie Li
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Duan, Y., Zhou, S., Peng, X., Kang, X., Yao, H. (2024). Neural-Symbolic Reasoning with External Knowledge for Machine Reading Comprehension. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Communications in Computer and Information Science, vol 1965. Springer, Singapore. https://doi.org/10.1007/978-981-99-8145-8_34
Download citation
Published:
Publisher Name:Springer, Singapore
Print ISBN:978-981-99-8144-1
Online ISBN:978-981-99-8145-8
eBook Packages:Computer ScienceComputer Science (R0)
Share this paper
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative