AzureChatOpenAI
This guide will help you get started with AzureOpenAIchat models. For detailed documentation of all AzureChatOpenAI features and configurations head to theAPI reference.
Azure OpenAI has several chat models. You can find information about their latest models and their costs, context windows, and supported input types in theAzure docs.
Azure OpenAI refers to OpenAI models hosted on theMicrosoft Azure platform. OpenAI also provides its own model APIs. To access OpenAI services directly, use theChatOpenAI integration.
Overview
Integration details
| Class | Package | Local | Serializable | JS support | Package downloads | Package latest |
|---|---|---|---|---|---|---|
| AzureChatOpenAI | langchain-openai | ❌ | beta | ✅ |
Model features
| Tool calling | Structured output | JSON mode | Image input | Audio input | Video input | Token-level streaming | Native async | Token usage | Logprobs |
|---|---|---|---|---|---|---|---|---|---|
| ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ |
Setup
To access AzureOpenAI models you'll need to create an Azure account, create a deployment of an Azure OpenAI model, get the name and endpoint for your deployment, get an Azure OpenAI API key, and install thelangchain-openai integration package.
Credentials
Head to theAzure docs to create your deployment and generate an API key. Once you've done this set the AZURE_OPENAI_API_KEY and AZURE_OPENAI_ENDPOINT environment variables:
import getpass
import os
if"AZURE_OPENAI_API_KEY"notin os.environ:
os.environ["AZURE_OPENAI_API_KEY"]= getpass.getpass(
"Enter your AzureOpenAI API key: "
)
os.environ["AZURE_OPENAI_ENDPOINT"]="https://YOUR-ENDPOINT.openai.azure.com/"
To enable automated tracing of your model calls, set yourLangSmith API key:
# os.environ["LANGSMITH_API_KEY"] = getpass.getpass("Enter your LangSmith API key: ")
# os.environ["LANGSMITH_TRACING"] = "true"
Installation
The LangChain AzureOpenAI integration lives in thelangchain-openai package:
%pip install-qU langchain-openai
Instantiation
Now we can instantiate our model object and generate chat completions.
- Replace
azure_deploymentwith the name of your deployment, - You can find the latest supported
api_versionhere:https://learn.microsoft.com/en-us/azure/ai-services/openai/reference.
from langchain_openaiimport AzureChatOpenAI
llm= AzureChatOpenAI(
azure_deployment="gpt-35-turbo",# or your deployment
api_version="2023-06-01-preview",# or your api version
temperature=0,
max_tokens=None,
timeout=None,
max_retries=2,
# other params...
)
Invocation
messages=[
(
"system",
"You are a helpful assistant that translates English to French. Translate the user sentence.",
),
("human","I love programming."),
]
ai_msg= llm.invoke(messages)
ai_msg
AIMessage(content="J'adore la programmation.", response_metadata={'token_usage': {'completion_tokens': 8, 'prompt_tokens': 31, 'total_tokens': 39}, 'model_name': 'gpt-35-turbo', 'system_fingerprint': None, 'prompt_filter_results': [{'prompt_index': 0, 'content_filter_results': {'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}}], 'finish_reason': 'stop', 'logprobs': None, 'content_filter_results': {'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}}, id='run-bea4b46c-e3e1-4495-9d3a-698370ad963d-0', usage_metadata={'input_tokens': 31, 'output_tokens': 8, 'total_tokens': 39})
print(ai_msg.content)
J'adore la programmation.
Chaining
We canchain our model with a prompt template like so:
from langchain_core.promptsimport ChatPromptTemplate
prompt= ChatPromptTemplate.from_messages(
[
(
"system",
"You are a helpful assistant that translates {input_language} to {output_language}.",
),
("human","{input}"),
]
)
chain= prompt| llm
chain.invoke(
{
"input_language":"English",
"output_language":"German",
"input":"I love programming.",
}
)
AIMessage(content='Ich liebe das Programmieren.', response_metadata={'token_usage': {'completion_tokens': 6, 'prompt_tokens': 26, 'total_tokens': 32}, 'model_name': 'gpt-35-turbo', 'system_fingerprint': None, 'prompt_filter_results': [{'prompt_index': 0, 'content_filter_results': {'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}}], 'finish_reason': 'stop', 'logprobs': None, 'content_filter_results': {'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}}, id='run-cbc44038-09d3-40d4-9da2-c5910ee636ca-0', usage_metadata={'input_tokens': 26, 'output_tokens': 6, 'total_tokens': 32})
Specifying model version
Azure OpenAI responses containmodel_name response metadata property, which is name of the model used to generate the response. However unlike native OpenAI responses, it does not contain the specific version of the model, which is set on the deployment in Azure. E.g. it does not distinguish betweengpt-35-turbo-0125 andgpt-35-turbo-0301. This makes it tricky to know which version of the model was used to generate the response, which as result can lead to e.g. wrong total cost calculation withOpenAICallbackHandler.
To solve this problem, you can passmodel_version parameter toAzureChatOpenAI class, which will be added to the model name in the llm output. This way you can easily distinguish between different versions of the model.
%pip install-qU langchain-community
from langchain_community.callbacksimport get_openai_callback
with get_openai_callback()as cb:
llm.invoke(messages)
print(
f"Total Cost (USD): ${format(cb.total_cost,'.6f')}"
)# without specifying the model version, flat-rate 0.002 USD per 1k input and output tokens is used
Total Cost (USD): $0.000063
llm_0301= AzureChatOpenAI(
azure_deployment="gpt-35-turbo",# or your deployment
api_version="2023-06-01-preview",# or your api version
model_version="0301",
)
with get_openai_callback()as cb:
llm_0301.invoke(messages)
print(f"Total Cost (USD): ${format(cb.total_cost,'.6f')}")
Total Cost (USD): $0.000074
API reference
For detailed documentation of all AzureChatOpenAI features and configurations head to the API reference:https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.azure.AzureChatOpenAI.html
Related
- Chat modelconceptual guide
- Chat modelhow-to guides