Repository files navigation

BrainChulo is a powerful Chat application with an emphasis on its memory management system. Our work is inspired by the camel family's resilience and adaptability. The repository contains a custom LTM (Long-Term Memory) approach for Llama-based LLMs (Large Language Models), and is designed to help users enhance their experience when using Open-Source LLMs.

• Custom LTM implementation based on Open Source Large Language Models
• Memory retrieval algorithms for quick recall
• Easy-to-use interface for storing and managing memories
• Modern User Interface
• One-Shot, Few-Shot, and Tool-Capable Agents with Full Vector-Based Memory Integration
• Ability to handle multiple conversations
• Ability to upvote or downvote AI answers for fine-tuning

Update 06.06.2023 As of today, we're dropping support toOobabooga Text Generation WebUI.The reason being is it does not offer enough support for theguidance library features.

Update 08.06.2023 We've raised a PR to add an extension that would enable us to again use oobabooga. If that's important for you,leave a thumbs up so it gets merged.

Note that currently, we only support models on GPU (GPTQ or Hugging Face), because the guidance does not (yet) fully support llama cpp bindings, or any GGML model.This would be the case untilthis PR is merged on guidance library.

We plan to reintroduce CPU support through GGML / llamacpp-bindings - at least for the LLaMA models - when this PR is fully merged into guidance.

To use BrainChulo, there are four required steps.

1. Docker Engine
2. Docker Compose v2
3. nvidia-docker

If you want to use docker-compose v1, you might run into an issue with the port binding - though there's an easy workaround here:#39

Choose a model that either implements the Hugging Face API, if you're downloading them from TheBloke, typically they have the-HF suffix, or a GPTQ one (-GPTQ)

There a couple of scripts to help you with this:

• bootstrap_models.py
• download-model.py (highly-inspired script from Oobabooga's repo)

If you simply executepython bootstrap_models.py it will create the required directory and download a pair of recommended models to get started. You can override the default model by passing additional arguments:

For instance, downloading WizardLM-30B-GPTQ

python bootstrap_models.py TheBloke/WizardLM-30B-GPTQ

Under the hood, this is roughly equivalent to this:

mkdir modelscd modelsgit clone https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2git clone https://huggingface.co/TheBloke/wizardLM-7B-HFcd ..

Copy the.env.example file. In Linux, that is done:

cp .env.example .env

You should then override the following variables to match your downloaded model:

MODEL_PATH=/models/vicuna-AlekseyKorshuk-7B-GPTQ-4bit-128g# General SettingsGENERAL_LOADING_METHOD=GPTQGENERAL_BOOL_CPU_OFFLOADING=true# HuggingFaceHF_BOOL_USE_QUANT=trueHF_BOOL_USE_4_BIT=true# GPTQGPTQ_INT_WBITS=4GPTQ_INT_GROUP_SIZE=128GPTQ_INT_PRE_LOADED_LAYERS=20GPTQ_DEVICE="cuda"

Note that you only need to set the variables according to your desired model loading method.GENERAL_LOADING_METHOD expects eitherGPTQ orHUGGING_FACE. Most of these parameters are self-explanatory,exceptGPTQ_INT_PRE_LOADED_LAYERS which only changes how many layers are preloaded for theCPU OFFLOADING when loading a GPTQ model.Setting a number too high for the model e.g., 50 for a 7B model triggers an error.

docker-compose up --build

The easiest way to start BrainChulo is using Docker withdocker-compose:

# from the root directory of the project, start with:docker-compose up --build# To shut it downdocker compose down# After pulling the latest code, make sure to run the database migrationsdocker-compose exec backend alembic upgrade head

We have a experimental feature that enables what we call flow agents. To activate it, set the feature flag to true in the .env file:

USE_FLOW_AGENTS=true

1. Create a new directory for your plugin, e.g./app/plugins/my_plugin

2. Inside the new directory, create a new Python module, e.g.myplugin_main.py

3. Inmyplugin_main.py, define your plugin routes using FastAPI. See the/app/plugins/sample_plugin directory for an example.

4. Create adatabase.py file to define your SQLModel models. Data for hese models will be persisted in the main database.

5. Once you have defined your models, run the following commandwith your own message to generate a new Alembic migration file. Migration files allow the application to add your models to the database schema:

alembic revision --autogenerate -m "Add Sample Plugin Demo Model"

6. Runalembic upgrade head to run your migration. This will update your database.

Note: If you wish to run these commands using the stood-up containers, prefix them as such:

# To create a migrationdocker-compose exec backend alembic revision --autogenerate -m "Add Sample Plugin Demo Model"# To run a migrationdocker-compose exec backend alembic upgrade head

Developers: While you may use the container-based approach since it is a development container with hot reloading you may also wish to start BrainChulo's services manually. To do so, run themain.py script from within theapp directory, then start the frontend:

cd ./apppython main.py# if within the ./app directorycd ../frontendnpm run dev

This will eventually launch the BrainChulo application. Point your web browser to the following default URL:

http://localhost:5173/

This interface allows you to chat or load text documents which will be used as context in the BrainChulo application.

As we develop BrainChulo the application is gaining abilities which allow it to access and change files on the filesystem. This is a major security risk. We strongly recommend that you run BrainChulo in a Docker container, using the providedDockerfile.

To run BrainChulo in a Docker container, you just need to run:

docker-compose up

To shut down the container:

docker-compose down

Here's a detailed roadmap for the BrainChulo project:

1. Create an End-to-End capable of creating, persisting, and using an index which can be loaded as context to a conversation usinglangchain memory module orllama-index package. This will allow the agent to maintain context and continuity between conversations and ensure that information is not lost over time.

2. Implement a Long-Term Memory (LTM) mechanism using a Vector Database layer with fine-tuning/training capabilities for LTM Management. This will allow the agent to remember past conversations and information for longer periods of time, and provide a more personalized experience to the user.

3. Implement Tools for the agent (ability to browse the web, send a tweet, read a file, use the REPL). These tools will enable the agent to perform various tasks and make it more versatile.

4. Create an Integration point for external systems. This will allow other systems to interact with the BrainChulo agent and expand its capabilities.

5. Integrate into existing Text Generation Systems such as oobabooga and Kobold. This will enable the BrainChulo agent to work with other text generation systems and provide a wider range of outputs to the user.

Please note that this roadmap is subject to change based on community feedback and contributions. We welcome your input and ideas as we work together to add long-term memory to custom LLMs!

We welcome contributions to BrainChulo from the open source community! If you would like to contribute to the project, please fork the repository and submit a pull request. For more information on how to contribute, please see theCONTRIBUTING.md file.

BrainChulo is licensed under the MIT license. SeeLICENSE.md for more information.

More commands are available on Docker. They are not necessary but can help developersBuild the Docker image:

docker build -t brainchulo .

Run the Docker container:

docker run -p 7865:7865 --name brainchulo brainchulo

If you wish to mount the BrainChulo local repository to the running container, run the following command:

docker run -p 7865:7865 -v /path/to/local/code:/app brainchulo

If you want hot reloading when coding, start the app with the following command:

gradio main.py

https://medium.com/@paolorechia/creating-my-first-ai-agent-with-vicuna-and-langchain-376ed77160e3

https://medium.com/@paolorechia/building-a-question-answer-bot-with-langchain-vicuna-and-sentence-transformers-b7f80428eadc

https://medium.com/@paolorechia/fine-tuning-my-first-wizardlm-lora-ca75aa35363d

https://betterprogramming.pub/a-deep-dive-into-guidances-source-code-16681a76fb20

https://betterprogramming.pub/prototyping-a-small-guidance-clone-with-exllama-logits-bias-de26f85623ab