ChatOctoAI
OctoAI offers easy access to efficient compute and enables users to integrate their choice of AI models into applications. TheOctoAI
compute service helps you run, tune, and scale AI applications easily.
This notebook demonstrates the use oflangchain.chat_models.ChatOctoAI
forOctoAI endpoints.
Setup
To run our example app, there are two simple steps to take:
Get an API Token fromyour OctoAI account page.
Paste your API token in the code cell below or use the
octoai_api_token
keyword argument.
Note: If you want to use a different model than theavailable models, you can containerize the model and make a custom OctoAI endpoint yourself, by followingBuild a Container from Python andCreate a Custom Endpoint from a Container and then updating yourOCTOAI_API_BASE
environment variable.
import os
os.environ["OCTOAI_API_TOKEN"]="OCTOAI_API_TOKEN"
from langchain_community.chat_modelsimport ChatOctoAI
from langchain_core.messagesimport HumanMessage, SystemMessage
Example
chat= ChatOctoAI(max_tokens=300, model_name="mixtral-8x7b-instruct")
messages=[
SystemMessage(content="You are a helpful assistant."),
HumanMessage(content="Tell me about Leonardo da Vinci briefly."),
]
print(chat(messages).content)
Leonardo da Vinci (1452-1519) was an Italian polymath who is often considered one of the greatest painters in history. However, his genius extended far beyond art. He was also a scientist, inventor, mathematician, engineer, anatomist, geologist, and cartographer.
Da Vinci is best known for his paintings such as the Mona Lisa, The Last Supper, and The Virgin of the Rocks. His scientific studies were ahead of his time, and his notebooks contain detailed drawings and descriptions of various machines, human anatomy, and natural phenomena.
Despite never receiving a formal education, da Vinci's insatiable curiosity and observational skills made him a pioneer in many fields. His work continues to inspire and influence artists, scientists, and thinkers today.
Related
- Chat modelconceptual guide
- Chat modelhow-to guides