AzureAIChatCompletionsModel
This will help you get started with AzureAIChatCompletionsModelchat models. For detailed documentation of all AzureAIChatCompletionsModel features and configurations, head to theAPI reference
The AzureAIChatCompletionsModel class uses the Azure AI Foundry SDK. AI Foundry has several chat models, including AzureOpenAI, Cohere, Llama, Phi-3/4, and DeepSeek-R1, among others. You can find information about their latest models and their costs, context windows, and supported input types in theAzure docs.
Overview
Integration details
| Class | Package | Local | Serializable | JS support | Package downloads | Package latest |
|---|---|---|---|---|---|---|
| AzureAIChatCompletionsModel | langchain-azure-ai | ❌ | ✅ | ✅ |
Model features
| Tool calling | Structured output | JSON mode | Image input | Audio input | Video input | Token-level streaming | Native async | Token usage | Logprobs |
|---|---|---|---|---|---|---|---|---|---|
| ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ |
Setup
To access AzureAIChatCompletionsModel models, you'll need to create anAzure account, get an API key, and install thelangchain-azure-ai integration package.
Credentials
Head to theAzure docs to see how to create your deployment and generate an API key. Once your model is deployed, you click the 'get endpoint' button in AI Foundry. This will show you your endpoint and api key. Once you've done this, set the AZURE_INFERENCE_CREDENTIAL and AZURE_INFERENCE_ENDPOINT environment variables:
import getpass
import os
ifnot os.getenv("AZURE_INFERENCE_CREDENTIAL"):
os.environ["AZURE_INFERENCE_CREDENTIAL"]= getpass.getpass(
"Enter your AzureAIChatCompletionsModel API key: "
)
ifnot os.getenv("AZURE_INFERENCE_ENDPOINT"):
os.environ["AZURE_INFERENCE_ENDPOINT"]= getpass.getpass(
"Enter your model endpoint: "
)
If you want to get automated tracing of your model calls, you can also set yourLangSmith API key by uncommenting below:
# os.environ["LANGSMITH_TRACING"] = "true"
# os.environ["LANGSMITH_API_KEY"] = getpass.getpass("Enter your LangSmith API key: ")
Installation
The LangChain AzureAIChatCompletionsModel integration lives in thelangchain-azure-ai package:
%pip install-qU langchain-azure-ai
Instantiation
Now we can instantiate our model object and generate chat completions:
from langchain_azure_ai.chat_modelsimport AzureAIChatCompletionsModel
llm= AzureAIChatCompletionsModel(
model_name="gpt-4",
temperature=0,
max_tokens=None,
timeout=None,
max_retries=2,
)
Invocation
messages=[
(
"system",
"You are a helpful assistant that translates English to French. Translate the user sentence.",
),
("human","I love programming."),
]
ai_msg= llm.invoke(messages)
ai_msg
AIMessage(content="J'adore programmer.", additional_kwargs={}, response_metadata={'model': 'gpt-4o-2024-05-13', 'token_usage': {'input_tokens': 31, 'output_tokens': 4, 'total_tokens': 35}, 'finish_reason': 'stop'}, id='run-c082dffd-b1de-4b3f-943f-863836663ddb-0', usage_metadata={'input_tokens': 31, 'output_tokens': 4, 'total_tokens': 35})
print(ai_msg.content)
J'adore programmer.
Chaining
We canchain our model with a prompt template like so:
from langchain_core.promptsimport ChatPromptTemplate
prompt= ChatPromptTemplate(
[
(
"system",
"You are a helpful assistant that translates {input_language} to {output_language}.",
),
("human","{input}"),
]
)
chain= prompt| llm
chain.invoke(
{
"input_language":"English",
"output_language":"German",
"input":"I love programming.",
}
)
AIMessage(content='Ich liebe Programmieren.', additional_kwargs={}, response_metadata={'model': 'gpt-4o-2024-05-13', 'token_usage': {'input_tokens': 26, 'output_tokens': 5, 'total_tokens': 31}, 'finish_reason': 'stop'}, id='run-01ba6587-6ff4-4554-8039-13204a7d95db-0', usage_metadata={'input_tokens': 26, 'output_tokens': 5, 'total_tokens': 31})
API reference
For detailed documentation of all AzureAIChatCompletionsModel features and configurations, head to the API reference:https://python.langchain.com/api_reference/azure_ai/chat_models/langchain_azure_ai.chat_models.AzureAIChatCompletionsModel.html
Related
- Chat modelconceptual guide
- Chat modelhow-to guides