This articlecontainsweasel words: vague phrasing that often accompaniesbiased orunverifiable information. Such statements should beclarified or removed.(May 2024) |
Neuro-symbolic AI is a type ofartificial intelligence that integratesneural andsymbolic AI architectures to address the weaknesses of each, providing a robust AI capable ofreasoning, learning, andcognitive modeling. As argued byLeslie Valiant[1] and others,[2][3] the effective construction of rich computationalcognitive models demands the combination ofsymbolic reasoning and efficientmachine learning.
Gary Marcus argued, "We cannot construct richcognitive models in an adequate, automated way without the triumvirate of hybrid architecture, rich prior knowledge, and sophisticated techniques for reasoning."[4] Further, "To build a robust, knowledge-driven approach to AI we must have the machinery of symbol manipulation in our toolkit. Too much useful knowledge is abstract to proceed without tools that represent and manipulate abstraction, and to date, the only known machinery that can manipulate such abstract knowledge reliably is the apparatus of symbol manipulation."[5]
Angelo Dalli,[6]Henry Kautz,[7]Francesca Rossi,[8] andBart Selman[9] also argued for such a synthesis. Their arguments attempt to address the two kinds of thinking, as discussed inDaniel Kahneman's bookThinking Fast and Slow. It describes cognition as encompassing two components: System 1 is fast, reflexive, intuitive, and unconscious. System 2 is slower, step-by-step, and explicit. System 1 is used forpattern recognition. System 2 handles planning, deduction, and deliberative thinking. In this view,deep learning best handles the first kind of cognition while symbolic reasoning best handles the second kind. Both are needed for a robust, reliable AI that can learn, reason, and interact with humans to accept advice and answer questions. Such dual-process models with explicit references to the two contrasting systems have been worked on since the 1990s, both in AI and in Cognitive Science, by multiple researchers.[10]
Approaches
editApproaches for integration are diverse.[11]Henry Kautz's taxonomy of neuro-symbolic architectures[12] follows, along with some examples:
- Symbolic Neural symbolic is the current approach of many neural models innatural language processing, where words or subword tokens are the ultimate input and output oflarge language models. Examples includeBERT, RoBERTa, andGPT-3.
- Symbolic[Neural] is exemplified byAlphaGo, where symbolic techniques are used to invoke neural techniques. In this case, the symbolic approach isMonte Carlo tree search and the neural techniques learn how to evaluate game positions.
- Neural | Symbolic uses a neural architecture to interpret perceptual data as symbols and relationships that are reasoned about symbolically. Neural-Concept Learner[13] is an example.
- Neural: Symbolic → Neural relies on symbolic reasoning to generate or labeltraining data that is subsequently learned by a deep learning model, e.g., to train a neural model for symbolic computation by using aMacsyma-likesymbolic mathematics system to create or label examples.
- NeuralSymbolic uses aneural net that is generated from symbolic rules. An example is the Neural Theorem Prover,[14] which constructs a neural network from anAND-OR proof tree generated from knowledge base rules and terms. Logic Tensor Networks[15] also fall into this category.
- Neural[Symbolic] allows a neural model to directly call a symbolic reasoning engine, e.g., to perform an action or evaluate a state. An example would beChatGPT using aplugin to queryWolfram Alpha.
These categories are not exhaustive, as they do not consider multi-agent systems. In 2005, Bader andHitzler presented a more fine-grained categorization that considered, e.g., whether the use of symbols included logic and if it did, whether the logic waspropositional or first-order logic.[16] The 2005 categorization and Kautz's taxonomy above are compared and contrasted in a 2021 article.[12] Recently,Sepp Hochreiter argued thatGraph Neural Networks "...are the predominant models of neural-symbolic computing"[17] since "[t]hey describe the properties of molecules, simulate social networks, or predict future states in physical and engineering applications with particle-particle interactions."[18]
Artificial general intelligence
editGary Marcus argues that "...hybrid architectures that combine learning and symbol manipulation are necessary for robust intelligence, but not sufficient",[19] and that there are
...four cognitive prerequisites for building robust artificial intelligence:
- hybrid architectures that combine large-scale learning with the representational and computational powers of symbol manipulation,
- large-scale knowledge bases—likely leveraging innate frameworks—that incorporate symbolic knowledge along with other forms of knowledge,
- reasoning mechanisms capable of leveraging those knowledge bases in tractable ways, and
- richcognitive models that work together with those mechanisms andknowledge bases.[20]
This echoes earlier calls for hybrid models as early as the 1990s.[21][22]
History
editGarcez and Lamb described research in this area as ongoing at least since the 1990s.[23][24] At that time, the termssymbolic and sub-symbolic AI were popular.
A series of workshops on neuro-symbolic AI has been held annually since 2005 Neuro-Symbolic Artificial Intelligence.[25] In the early 1990s, an initial set of workshops on this topic were organized.[21]
Research
editKey research questions remain,[26] such as:
- What is the best way to integrate neural and symbolic architectures?
- How should symbolic structures be represented within neural networks and extracted from them?
- How should common-sense knowledge be learned and reasoned about?
- How can abstract knowledge that is hard to encode logically be handled?
Implementations
editImplementations of neuro-symbolic approaches include:
- AllegroGraph: an integrated Knowledge Graph based platform for neuro-symbolic application development.[27][28][29]
- Scallop: a language based onDatalog that supports differentiable logical and relational reasoning. Scallop can be integrated inPython and with aPyTorch learning module.[30]
- Logic Tensor Networks: encode logical formulas as neural networks and simultaneously learn term encodings, term weights, and formula weights.
- DeepProbLog: combines neural networks with the probabilistic reasoning ofProbLog.
- SymbolicAI: a compositional differentiable programming library.
- Explainable Neural Networks (XNNs): combine neural networks with symbolichypergraphs and trained using a mixture of backpropagation and symbolic learning called induction.[31]
See also
editCitations
edit- ^Valiant 2008.
- ^Garcez et al. 2015.
- ^D'Avila Garcez, Artur S.; Lamb, Luis C.; Gabbay, Dov M. (2009).Neural-symbolic cognitive reasoning. Cognitive technologies. Springer.ISBN 978-3-540-73245-7.
- ^Marcus 2020, p. 44.
- ^Marcus & Davis 2019, p. 17.
- ^Dalli 2025.
- ^Kautz 2020.
- ^Rossi 2022.
- ^Selman 2022.
- ^Sun 1995.
- ^"Disentangling visual attributes with neuro-vector-symbolic architectures, in-memory computing, and device noise".IBM Research. 2021-02-09. Retrieved2024-10-20.
- ^abSarker, Md Kamruzzaman; Zhou, Lu; Eberhart, Aaron;Hitzler, Pascal (2021)."Neuro-symbolic artificial intelligence: Current trends".AI Communications.34 (3):197–209.doi:10.3233/AIC-210084.S2CID 239199144.
- ^Mao et al. 2019.
- ^Rocktäschel, Tim; Riedel, Sebastian (2016)."Learning Knowledge Base Inference with Neural Theorem Provers".Proceedings of the 5th Workshop on Automated Knowledge Base Construction. San Diego, CA: Association for Computational Linguistics. pp. 45–50.doi:10.18653/v1/W16-1309. Retrieved2022-08-06.
- ^Serafini, Luciano; Garcez, Artur d'Avila (2016). "Logic Tensor Networks: Deep Learning and Logical Reasoning from Data and Knowledge".arXiv:1606.04422 [cs.AI].
- ^Bader & Hitzler 2005.
- ^L.C. Lamb, A.S. d'Avila Garcez, M.Gori, M.O.R. Prates, P.H.C. Avelar,M.Y. Vardi (2020). "Graph Neural Networks Meet Neural-Symbolic Computing: A Survey and Perspective." CoRR abs/2003.00330 (2020)
- ^Hochreiter, Sepp (April 2022)."Toward a broad AI".Communications of the ACM.65 (4):56–57.doi:10.1145/3512715.ISSN 0001-0782.
- ^Marcus 2020, p. 50.
- ^Marcus 2020, p. 48.
- ^abSun & Bookman 1994.
- ^Honavar 1995.
- ^Garcez & Lamb 2020, p. 2.
- ^Garcez et al. 2002.
- ^"Neuro-Symbolic Artificial Intelligence".people.cs.ksu.edu. Retrieved2023-09-11.
- ^Sun 2001.
- ^Harper, Jelani (2023-12-29)."AllegroGraph 8.0 Incorporates Neuro-Symbolic AI, a Pathway to AGI".The New Stack. Retrieved2024-06-13.
- ^"Neuro-Symbolic AI and Large Language Models Introduction | AllegroGraph 8.1.1".franz.com. Retrieved2024-06-13.
- ^"Franz Inc. Introduces AllegroGraph Cloud: A Managed Service for Neuro-Symbolic AI Knowledge Graphs".Datanami. Retrieved2024-06-13.
- ^Li, Ziyang; Huang, Jiani; Naik, Mayur (2023). "Scallop: A Language for Neurosymbolic Programming".arXiv:2304.04812 [cs.PL].
- ^"Model Induction Method for Explainable AI". USPTO. 2021-05-06.
References
edit- Bader, Sebastian;Hitzler, Pascal (2005-11-10). "Dimensions of Neural-symbolic Integration – A Structured Survey".arXiv:cs/0511042.
- Garcez, Artur S. d'Avila; Broda, Krysia; Gabbay, Dov M.; Gabbay (2002).Neural-Symbolic Learning Systems: Foundations and Applications. Springer Science & Business Media.ISBN 978-1-85233-512-0.
- Garcez, Artur; Besold, Tarek; De Raedt, Luc; Földiák, Peter;Hitzler, Pascal; Icard, Thomas; Kühnberger, Kai-Uwe; Lamb, Luís; Miikkulainen, Risto; Silver, Daniel (2015).Neural-Symbolic Learning and Reasoning: Contributions and Challenges. AAAI Spring Symposium - Knowledge Representation and Reasoning: Integrating Symbolic and Neural Approaches. Stanford, CA.doi:10.13140/2.1.1779.4243.
- Garcez, Artur d'Avila; Gori, Marco; Lamb, Luis C.; Serafini, Luciano; Spranger, Michael; Tran, Son N. (2019). "Neural-Symbolic Computing: An Effective Methodology for Principled Integration of Machine Learning and Reasoning".arXiv:1905.06088 [cs.AI].
- Garcez, Artur d'Avila; Lamb, Luis C. (2020). "Neurosymbolic AI: The 3rd Wave".arXiv:2012.05876 [cs.AI].
- Hitzler, Pascal; Sarker, Md Kamruzzaman (2022).Neuro-Symbolic Artificial Intelligence: The State of the Art. IOS Press.ISBN 978-1-64368-244-0.
- Hitzler, Pascal; Sarker, Md Kamruzzaman; Eberhart, Aaron (2023).Compendium of Neurosymbolic Artificial Intelligence. IOS Press.ISBN 978-1-64368-406-2.
- Hochreiter, Sepp. "Toward a Broad AI." Commun. ACM 65(4): 56–57 (2022).Toward a broad AI
- Honavar, Vasant (1995).Symbolic Artificial Intelligence and Numeric Artificial Neural Networks: Towards a Resolution of the Dichotomy. The Springer International Series In Engineering and Computer Science. Springer US. pp. 351–388.doi:10.1007/978-0-585-29599-2_11.
- Kautz, Henry (2020-02-11).The Third AI Summer, Henry Kautz, AAAI 2020 Robert S. Engelmore Memorial Award Lecture. Retrieved2022-07-06.
- Kautz, Henry (2022)."The Third AI Summer: AAAI Robert S. Engelmore Memorial Lecture".AI Magazine.43 (1):93–104.doi:10.1609/aimag.v43i1.19122.ISSN 2371-9621.S2CID 248213051. Retrieved2022-07-12.
- Mao, Jiayuan; Gan, Chuang; Kohli, Pushmeet; Tenenbaum, Joshua B.; Wu, Jiajun (2019). "The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision".arXiv:1904.12584 [cs.CV].
- Marcus, Gary; Davis, Ernest (2019).Rebooting AI: Building Artificial Intelligence We Can Trust. Vintage.
- Marcus, Gary (2020). "The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence".arXiv:2002.06177 [cs.AI].
- Dalli, Angelo (2025-02-13)."WAICF2025: Why neurosymbolic AI is the future of trustworthy AI (WAICF 2025 Keynote)". Retrieved2025-03-06.
- Rossi, Francesca (2022-07-06)."AAAI2022: Thinking Fast and Slow in AI (AAAI 2022 Invited Talk)". Retrieved2022-07-06.
- Selman, Bart (2022-07-06)."AAAI2022: Presidential Address: The State of AI". Retrieved2022-07-06.
- Serafini, Luciano; Garcez, Artur d'Avila (2016-07-07). "Logic Tensor Networks: Deep Learning and Logical Reasoning from Data and Knowledge".arXiv:1606.04422 [cs.AI].
- Sun, Ron (1995). "Robust reasoning: Integrating rule-based and similarity-based reasoning".Artificial Intelligence.75 (2):241–296.doi:10.1016/0004-3702(94)00028-Y.
- Sun, Ron; Bookman, Lawrence (1994).Computational Architectures Integrating Neural and Symbolic Processes. Kluwer.
- Sun, Ron; Alexandre, Frederic (1997).Connectionist Symbolic Integration. Lawrence Erlbaum Associates.
- Sun, R (2001). "Hybrid systems and connectionist implementationalism".Encyclopedia of Cognitive Science (MacMillan Publishing Company, 2001).
- Valiant, Leslie G (2008)."Knowledge Infusion: In Pursuit of Robustness in Artificial Intelligence".IARCS Annual Conference on Foundations of Software Technology and Theoretical Computer Science.doi:10.4230/LIPIcs.FSTTCS.2008.1770.