Javelin AI Gateway
The Javelin AI Gateway service is a high-performance, enterprise grade API Gateway for AI applications.
It is designed to streamline the usage and access of various large language model (LLM) providers,such as OpenAI, Cohere, Anthropic and custom large language models within an organization by incorporatingrobust access security for all interactions with LLMs.
Javelin offers a high-level interface that simplifies the interaction with LLMs by providing a unified endpointto handle specific LLM related requests.
See the Javelin AI Gatewaydocumentation for more details.
Javelin Python SDK is an easy to use client library meant to be embedded into AI Applications
Installation and Setup
Installjavelin_sdk to interact with Javelin AI Gateway:
pip install 'javelin_sdk'
Set the Javelin's API key as an environment variable:
export JAVELIN_API_KEY=...
Completions Example
from langchain.chainsimport LLMChain
from langchain_community.llmsimport JavelinAIGateway
from langchain_core.promptsimport PromptTemplate
route_completions="eng_dept03"
gateway= JavelinAIGateway(
gateway_uri="http://localhost:8000",
route=route_completions,
model_name="text-davinci-003",
)
llmchain= LLMChain(llm=gateway, prompt=prompt)
result= llmchain.run("podcast player")
print(result)
Embeddings Example
from langchain_community.embeddingsimport JavelinAIGatewayEmbeddings
from langchain_openaiimport OpenAIEmbeddings
embeddings= JavelinAIGatewayEmbeddings(
gateway_uri="http://localhost:8000",
route="embeddings",
)
print(embeddings.embed_query("hello"))
print(embeddings.embed_documents(["hello"]))
Chat Example
from langchain_community.chat_modelsimport ChatJavelinAIGateway
from langchain_core.messagesimport HumanMessage, SystemMessage
messages=[
SystemMessage(
content="You are a helpful assistant that translates English to French."
),
HumanMessage(
content="Artificial Intelligence has the power to transform humanity and make the world a better place"
),
]
chat= ChatJavelinAIGateway(
gateway_uri="http://localhost:8000",
route="mychatbot_route",
model_name="gpt-3.5-turbo"
params={
"temperature":0.1
}
)
print(chat(messages))