title:Open AI recipes for Azure API Management Service

title:AzureOpen AI recipes for Azure API Management Service

date:2025-01-01 10:00

tags:[Azure, networking, API manager, OpenAI ]

excerpt:"xxx"

excerpt:"A collection of recipes for API Management for those who need to expose one or more instances of Azure OpenAI"

header:

overlay_image:https://live.staticflickr.com/65535/52755090506_6cf0808a3c_h.jpg

caption:"Photo credit: [**nicola since 1972**](https://www.flickr.com/photos/15216811@N06/52755090506)"

overlay_image:https://live.staticflickr.com/65535/54308248466_dc89a5f3d7_h.jpg

caption:"Photo credit: [**nicola since 1972**](https://www.flickr.com/photos/15216811@N06/54308248466/)"

Azure OpenAI Service provides REST API access to OpenAI's powerful language models including o1, o1-mini, GPT-4o, GPT-4o mini, GPT-4 Turbo with Vision, GPT-4, GPT-3.5-Turbo, and Embeddings model series. These models can be easily adapted to your specific task including but not limited to content generation, summarization, image understanding, semantic search, and natural language to code translation.

Combining Azure Open AI with Azure API Manager enables enterprises to efficiently distribute and manage their AI capabilities while maintaining high standards of security and performance.

obiettivo di questa serie di post é quello di mostrate alcuni dei pattern da usare per esporre Azure Open AI attraversoAPIM.

In this context, the following architecturea and implementation in Azure Architecture Center are**developed** and**maintained** by Azure Pattern and Practices team. I suggest yout to bookmark them for further information.

The lab used for these walk-throughs consists of the following resources:

in questi post ci focalizzeremo sulla configurazione delle policy e di implementazione di alcuni pattern di utilizzo con esse. La parte di configurazione della rete e l'integrazione con una ESLZ é out of scope. Per un walktrough che guidi l'integrazione di APIM ed AOAI in un contesto di hub & spoke, è possibile fare riferimento anche[al mio articolo](https://github.com/nicolgit/hub-and-spoke-playground/blob/main/scenarios/aoai.md) disponibile nell'ambito del progetto[the hub-and-spoke playground](https://github.com/nicolgit/hub-and-spoke-playground)

Il laboratorio su cui lavoró prevede le seguenti risorse:

* Azure API Management service (developer SKU) enterpriseapim

* Azure API Management service (developer SKU)`nicold-apim`

* 2 x Azure OpenAI service (S0 SKU)`apimaoai01` and`apimaoai02`

come mostrato nello schema seguente:


Here are the key recipes we'll cover in this article:

-[Add Azure Open AI as backend resource](#add-azure-open-ai-as-backend-resource)

-[Show apim-001 endpoint as root API](#show-apim-001-endpoint-as-root-api)

-[Implement throttling](#implement-throttling)

-[Show in a Response header (_aoai-origin_) the host of the OpenAI API Called](#show-in-a-response-header-aoai-origin-the-host-of-the-openai-api-called)

-[Round robin calls between 2 instances of Open AI](#round-robin-calls-between-2-instances-of-open-ai)

-[Fallback on a second openAI instance for 30 secs. If the first send a 4xx error](#fallback-on-a-second-openai-instance-for-30-secs-if-the-first-send-a-4xx-error)

Go to API Management services >`nicold-apim` > Backends > Add

here also a powershell script to test this configuration:

The following policy limits the access**at 10 requests per minute**. Paste the xml in: API Management Service >

`nicold-apim` > APIs > All APIs >`/` > all operations > inbound processing > policies (code editor)

<policies>

<inbound>

<base />

To limit at 2 calls**per IP** in 60 seconds, use the following rate-limit xml:

<rate-limit-by-keycalls="2"renewal-period="60"counter-key="@(context.Request.IpAddress)" />

To limit at 2 calls per API KEY in 60 seconds, use the following rate-limit xml:

<rate-limit-by-keycalls="2"renewal-period="60"counter-key="@(context.Request.Headers.GetValueOrDefault("api-key"))" />

Add the following XML in the**outbound** policy:

<outbound>

<base />

<set-headername="aoai-origin"exists-action="override">

Use the following**inbound** policy:

<inbound>

<base />

<cache-lookup-valuekey="backend-rr"variable-name="backend-rr" />

* Azure APIM to scale AOAI -https://github.com/Azure/aoai-apim

* Manage Azure OpenAI using APIM -https://github.com/microsoft/AzureOpenAI-with-APIM

[def]:#recipes-list

title:Azure OpenAI powershell chat-completition call sample

date:2025-02-24 10:00

tags:[Azure, powershell, OpenAI ]

excerpt:"sampletocopy and paste"

excerpt:"a powershell script that shows howtocall an Azure OpenAI chat completition endpoint API"

header:

overlay_image:https://live.staticflickr.com/65535/52755090506_6cf0808a3c_h.jpg

caption:"Photo credit: [**nicola since 1972**](https://www.flickr.com/photos/15216811@N06/52755090506)"

The followingpowershellsamples shows how to call an Azure OpenAI chat completition endpoint API

Apowershellscript that shows how to call an Azure OpenAI chat completition endpoint API