Repository files navigation

Rational OpenCog Controlled Agent, or ROCCA, is a project aiming atcreating an opencog agent that acts rationally in OpenAI Gymenvironments (including Minecraft via MineRL and Malmo).

At its core it relies on PLN (Probabilistic Logic Networks) for bothlearning and planning. In practice most of the learning is howeverhandled by the pattern miner, which can be seen as a specialized formof PLN reasoning. Planning, the discovery of cognitive schematics, ishandled by PLN and its temporal reasoning rule base. Decision iscurrently a hardwired module, heavily inspired by OpenPsi with a morerational sampling procedure (Thompson Sampling for better exploitationvs exploration tradeoff).

For now learning is able to

1. Discover temporal patterns based on directly observable events viathe pattern miner.
2. Turn these temporal patterns into plans (cognitive schematics).
3. Combine these plans to form new plans, possibly composed of newaction sequences, via temporal deduction.

The next steps are

1. Add more sophisticated temporal (including dealing with longs lagsbetween cause and effect) and then spatial inference rules.
2. Integrate ECAN, for Attention Allocation, to dynamically restrictthe atomspace to subsets of items to process/pay-attention-to.
3. Record attention spreading to learn/improve Hebbian links.
4. Carry concept creation and schematization (crystallized attentionallocation).
5. Record internal processes, not just attention spreading, aspercepta to enable deeper forms of instrospective reasoning.
6. Plan internal actions, not just external, to enable self-growth.

OpenCog tools

• cogutil (tested with revision 555a003)
• atomspace (tested with revision 396e1e7)
• unify (tested with revision 1e93141)
• ure (tested with revision 4e01b02)
• spacetime (tested with revision 962862c)
• pln (tested with revision 08c100f)
• miner (tested with revision 15befc4)

Third party tools

• Python 3.10 (or Python 3.8 see below)
• jupyter notebook
• python-orderedmultidicthttps://pypi.org/project/orderedmultidict/
• fastcorehttps://fastcore.fast.ai
• OpenAI Gymhttps://gym.openai.com/
• MineRLhttps://minerl.io
• nbdevhttps://nbdev.fast.ai
• blackhttps://pypi.org/project/black/

Python 3.10 offers a better out-of-the-box type annotation system thanPython 3.8 and is thus the default required version. However you maystill use Python 3.8 by checking out thepython-3.8-compatiblebranch. Beware that such Python 3.8 branch may not be as wellmaintained as the master.

In the root folder enter the following command (you might need to beroot depending on your system):

pip install -e.

For the tools used for development:

pip install -r requirements-dev.txt

An OpencogAgent defined under therocca/agents folder is providedthat can used to implement agents for given environments. See theexamples under theexamples folder.

There are Jupyter notebooks provided for experimentation as well. Torun them call jupyter notebook on a ipynb file, such as

jupyter notebook 01_cartpole.ipynb

Some experiments, notably the notebooks, use TensorBoard via thetensorboardX library to store event files that show certain metricsover time for training / testing (for now it's just rewards).

By default, event files will be created under theruns/<datetime><comment> directory. You can invoketensorboard --logdir runs from the project root to start an instance that willsee all the files under that directory. Open your browser tohttp://localhost:6006 to see its interface.

If you write code in notebooks that is exported (has the#exportcomment on top of the cell), remember to invokenbdev_build_lib toupdate the library. Remember to useblack for formatting, you caninvokeblack . from the project root to format everything.

You can also use the Makefile for your convenience, invokingmake rocca will do both of the above in sequence.

The.devcontainer folder has configuration forVS Codedevcontainerfunctionality. You can use it to setup a development environment veryquickly and regardless of the OS you use.

• The container has a JupyterLab instance running on the port 8888.
• The container has a VNC server running on the port 5901.
• The password for the VNC server started in the container isvncpassword. You can use any VNC client to see the results ofrendering Gym environments this way.

Usingtype annotationsis highly encouraged. One can type check the entire Python ROCCA codeby calling

tests/mypy.sh

from the root folder.

To only type check some subfolder, you may callmypy.sh from thatsubfolder. For instance to type check theexamples subfolder

cd examples../tests/mypy.sh

Or directly runmypy on files or directories, such as

mypy rocca/rocca/agents/core.py

tests/mypy.sh merely calls mypy on all python files in the directoryfrom which it is called while filtering out some error messages.

Simply runpytest in the root folder.

There is no ROCCA paper per se yet. In the meantime here is a list of related references

• An Inferential Approach to Mining Surprising Patterns in Hypergraphs, Nil Geisweiller et Ben Goertzel
• Partial Operator Induction with Beta Distributions, Nil Geisweiller
• Thompson Sampling is Asymptotically Optimal in General Environments, Jan Leike et al
• Draft about temporal reasoning in PLN, Nil Geisweiller
• Presentation and Demo of Temporal and Procedural Reasoning with OpenCog, AGI-21, Nil Geisweiller et Hedra Yusuf
• References about OpenCog including PLN