• Reduced Latency: Serve responses faster to your users by avoiding a round trip to the origin AI provider for repeated requests.
• Cost Savings: Minimize the number of paid requests made to your AI provider, especially for frequently accessed or non-dynamic content.
• Increased Throughput: Offload repetitive requests from your AI provider, allowing it to handle unique requests more efficiently.

This caching behavior will be uniformly applied to all requests that support caching. If you need to modify the cache settings for specific requests, you have the flexibility to override this setting on a per-request basis.

To check whether a response comes from cache or not,cf-aig-cache-status will be designated asHIT orMISS.

While your gateway's default cache settings provide a good baseline, you might need more granular control. These situations could be data freshness, content with varying lifespans, or dynamic or personalized responses.

To address these needs, AI Gateway allows you to override default cache behaviors on a per-request basis using specific HTTP headers. This gives you the precision to optimize caching for individual API calls.

The following headers allow you to define this per-request cache behavior:

Skip cache refers to bypassing the cache and fetching the request directly from the original provider, without utilizing any cached copy.

You can use the headercf-aig-skip-cache to bypass the cached version of the request.

As an example, when submitting a request to OpenAI, include the header in the following manner:

curlhttps://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai/chat/completions\
--header"Authorization: Bearer$TOKEN"\
--header'Content-Type: application/json'\
--header'cf-aig-skip-cache: true'\
--data' {
"model": "gpt-4o-mini",
"messages": [
{
"role": "user",
"content": "how to build a wooden spoon in 3 short steps? give as short as answer as possible"
}
]
}
'

Cache TTL, or Time To Live, is the duration a cached request remains valid before it expires and is refreshed from the original source. You can usecf-aig-cache-ttl to set the desired caching duration in seconds. The minimum TTL is 60 seconds and the maximum TTL is one month.

For example, if you set a TTL of one hour, it means that a request is kept in the cache for an hour. Within that hour, an identical request will be served from the cache instead of the original API. After an hour, the cache expires and the request will go to the original API for a fresh response, and that response will repopulate the cache for the next hour.

As an example, when submitting a request to OpenAI, include the header in the following manner:

curlhttps://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai/chat/completions\
--header"Authorization: Bearer$TOKEN"\
--header'Content-Type: application/json'\
--header'cf-aig-cache-ttl: 3600'\
--data' {
"model": "gpt-4o-mini",
"messages": [
{
"role": "user",
"content": "how to build a wooden spoon in 3 short steps? give as short as answer as possible"
}
]
}
'

Custom cache keys let you override the default cache key in order to precisely set the cacheability setting for any resource. To override the default cache key, you can use the headercf-aig-cache-key.

When you use thecf-aig-cache-key header for the first time, you will receive a response from the provider. Subsequent requests with the same header will return the cached response. If thecf-aig-cache-ttl header is used, responses will be cached according to the specified Cache Time To Live. Otherwise, responses will be cached according to the cache settings in the dashboard. If caching is not enabled for the gateway, responses will be cached for 5 minutes by default.

As an example, when submitting a request to OpenAI, include the header in the following manner:

curlhttps://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai/chat/completions\
--header'Authorization: Bearer {openai_token}'\
--header'Content-Type: application/json'\
--header'cf-aig-cache-key: responseA'\
--data' {
"model": "gpt-4o-mini",
"messages": [
{
"role": "user",
"content": "how to build a wooden spoon in 3 short steps? give as short as answer as possible"
}
]
}
'