Repository files navigation

Hal9 is a deployment platform purpose-built for generative AI, enabling you to create and deploy generative (LLMs anddiffusers) applications (chatbots, agents, APIs, and apps). Key features:

• Flexible: Use any library (LangChain,DSPy), and any model (OpenAI,Llama,Groq,MidJourney).
• Intuitive: No need to learn app frameworks (Flask), simply useinput() andprint(), or write file to disk.
• Scalable: Designed to integrate your app with scalable technologies (Docker,Kubernetes, etc).
• Powerful: Support for long-running agents, multiple programming languages, complex system dependencies, and running arbitrary code in a secure Kubernetes pods.
• Open: The code behindhal9, is also open source and open for contributions under atapps/hal9.

Focus on AI (RAG, fine-tuning, alignment, training) and skip engineering tasks (frontend development, backend integration, deployment, operations).

Create and share a chatbot in seconds as follows:

pip install hal9hal9 create chatbothal9 deploy chatbot

Notice thatdeploy needs aHAL9_TOKEN environment variable with an API token you can get fromhal9.com/devs. You can use this token to deploy from your local computer, a notebook or automate from GitHub.

HAL9_TOKEN=H9YOURTOKEN hal9 deploy chatbot --name my_first_chatbot

As easy as that you have created your first chatbot!

The code inside/chatbot/app.py contains a "Hello World" chatbot that reads the user prompt and echos the result back:

prompt=input()print(f"Echo:{prompt}")

We designed this package with simplicity in mind, the job of the code is to read input and write output, that's about it. That said, you can create chatbots that use LLMs, generate images, or even use tools that connect to databases, or even build websites and games!

By defaulthal9 create defaults to the--template echo template, but you can choose different ones as follows:

hal9 create chatbot-openai --template openaihal9 create chatbot-groq --template groq

A template provides ready to use code with specific technologies and use cases. Is very popular to use OpenAI's ChatGPT-like template with--template openai, the code generated will look as follows:

importhal9ash9fromopenaiimportOpenAImessages=h9.load("messages", [])prompt=h9.input(messages=messages)completions=OpenAI().chat.completions.create(model="gpt-4",messages=messages,stream=True)h9.complete(completions,messages=messages)h9.save("messages",messages,hidden=True)

Thelearn code section explains in detail how this code works, but will provide a quick overview. Thehal9 package contains a helper functions to simplify your generative AI code. You can choose to not usehal9 at all and useinput() andprint() statements yourself, or even sue tools likelangchain. Theh9.load() andh9.save() functions load and save data across chat sessions, our platform is stateless by default. Theh9.input() function is a slim wrapper overinput() that also stores the user input in themessages. Thenh9.complete() is a helper function to help parse the completion results and save the result inmessages. That's about it!

To make changes to your project, openchatbot/ in your IDE and modifychatbot/app.py.

You can then run your project as follows:

hal9 run chatbot

If you customized your template with--template make sure to set the correct key, for example, if you are using the OpenAI template use for Linux or macOS:

export OPENAI_KEY=YOUR_OPENAI_KEY.

For Windows use:

set OPENAI_KEY=YOUR_OPENAI_KEY.

For more information on obtaining and using your OpenAI API key, please refer to theOpenAI API Key documentation.

You can then run your application locally with:

hal9 run chatbot

This command is just a convenience wrapper to running the code yourself with something likepython app.py.

The deploy command will prepare for deployment your generative app.

For example, you can prepare deployment as a generative app (Hal9). We have plans to also provide deployment to Docker and the open source community can expand this even further.

hal9 deploy chatbot --target hal9

Each command is tasked with preparing the deployment of your project folder. For example,--target docker should create aDockerfile file that gets this project ready to run in cloud containers.

For personal use,--target hal9 supports a free tier athal9.com; enterprise support is also available to deploy with--target hal9 --url hal9.yourcompany.com

Apart from deploying your apps directly onhal9.com, you can collaborate with our community by contributing new ones to the/apps directory in this repository. Additionally, you can improve Hal9’s core capabilities by refining the code in theapps/hal9 folder.

Thehal9 Python package is located in the/python directory, while the documentation website resides under/website. We encourage contributors to focus on enhancing apps first before proposing more complex changes.

Keep in mind that the philosophy of thehal9 package is to remain a lightweight wrapper aroundinput() andprint(). The Python community already offers many excellent frameworks, and we aim to encourage their use rather than creating another one.