Purging explicitly removes content from a cache, rather than allowing it to expire or to beevicted. Once content has been purged, subsequent lookups for that content will not be able to be satisfied from the cache and, in thereadthrough cache, will trigger a request to an origin server.
Purging is typically used in a number of scenarios:
Manually: As an occasional, manual exercise to remove errant content or data, or to clear old versions of data in the case of a manual deployment. Typically this is donevia the web interface orfastly purge CLI command.
On deploy: Incorporated into your application's deployment process such that, when you release a new version of your site, the edge cache is purged. Typically this is apurge all operation and makes use of thepurge API or thefastly purge CLI command.
On content update: As a way to purge content when it changes in your content management system. This approach is particularly useful when combined with setting a very long TTL in aCache-Control header so that traffic to origin is minimized, but end users are still assured of up to date content.
The purge-all operation invalidates all cached data for a Fastly service, regardless of which cache interface was used to put the data in cache. This purge is available via theweb interface, via theAPI, or thefastly purge command in the CLI.
WARNING: Purging a large amount of content from a high traffic service is likely to result in a rapid increase in traffic to origin.
Purge-all operations take up to 2 minutes to complete. Purge-all is not compatible withsoft purge orbulk purge.
URL purge
A single URL purge invalidates a single object (identified by URL) that was cached by thereadthrough cache. This can be donevia the web interface, by using theFastly API, or by sending aPURGE request to the URL to be purged. For example, usingcurl:
Single URL purges may also be executed using thefastly purge command in the fastly CLI (and internally this will send an HTTPPURGE request similar to the curl command above).
WARNING: URL purges are unauthenticated by default. To require a valid Fastly API token for URL purges, configure your service to add aFastly-Purge-Requires-Auth header to the client request:
Single URL purges are processed by your Fastly service like normal requests. The purge takes effect when your service forwards the request to the readthrough cache layer. In VCL services, this happens by default, but can be obstructed if your code prevents the request being forwarded. Take care to ensure that requests with aFASTLYPURGE method are not blocked.
In Compute services, HTTPPURGE requests must be forwarded explicitly to thereadthrough cache by calling the relevant fetch method:
Rust
JavaScript
Go
usefastly::{Error,Request,Response};
#[fastly::main]
fnmain(req:Request)->Result<Response,Error>{
// Forward purge requests immediately to a backend
if req.get_method()=="PURGE"{
returnOk(req.send("any_backend_name")?);
}
// ... process normal, non-purge requests here
Ok(Response::from_status(200))
}
Currently, manually forwardingPURGE requests so that they are visible to the cache is the only way to enable URL purging in Compute services. For such requests, any backend can be used, since the request will not be forwarded beyond the cache layer.
HINT: In Compute services, it's often better to usesurrogate keys to label cached objects.
IMPORTANT: The HTTP method ofPURGE requests is changed toFASTLYPURGE before the request is processed by VCL services. No such transformation happens in Compute services.
Surrogate key purge
A surrogate key purge invalidates all objects which share a specified token. One key can appear on multiple objects, and an object can have multiple keys, making this a very flexible way to label cached content for potential future purging.
Purging of items by surrogate key is available via theweb interface, via theAPI, thefastly purge command in the CLI, and also via methods in supported edge code SDKs (currentlyRust)
Surrogate key purges take around 150ms to complete, and supportsoft purging as an option. Surrogate key purges can also bedone in bulk.
Adding surrogate keys
Tagging content with surrogate keys is a great way to reduce the number of purge operations you need to trigger when content is updated. It is often useful to be able to perform a single purge operation for agroup of content that shares a common trait; for example, all pages that mention a specific product.
Surrogate keys can also be added explicitly in your edge code. In VCL services, surrogate keys can be added to objectsusing VCL code in thevcl_fetch subroutine, before they are written to cache:
Fastly automatically removes anySurrogate-Key headers present on a response before delivering it to the end user (unlessFastly-Debug is included in the request).
Surrogate keys are subject to size limitations. Individual tokens may not exceed 1KB in length andSurrogate-Key header values (comprising one or more space-separated tokens) may not exceed 16KB in length. If either of the key or key header value limits are reached while parsing aSurrogate-Key header, the token currently being parsed and all tokens following it within the same header will be ignored.
Purge by cache key
The simple cache interface provides an edge-accessible mechanism for purging an object by key, available in each languageSDK. For example,SimpleCache.purge in #"Copy to clipboard" aria-roledescription="status" data-is-floaty="true" data-is-vertically-centered="true">
SimpleCache.purge("my-key",{scope:"global"});
IMPORTANT: Unlike all other purges, executing a purge using simple cache, from edge code, will by default purgeonly the local POP. All SDKs offer an option to expand the purge globally, as shown in the example above.
Soft vs hard purging
Purge operations default tohard purging. These purge operations will cause content to be invalidated, making it immediately unusable for future requests. It may not be immediately removed from disk but will be permanently inaccessible and will in time be overwritten by new content.
In contrast, asoft purge marks the content asstale. This means that it will not automatically be available to serve in response to a request, but remains available to use in some circumstances. Learn more aboutstaleness and revalidation.
Soft purging is available for URL purges and surrogate key purges, but not for purge-all or cache key purges:
In the web interface, select the 'Soft purge' option.
Soft purging, especially when combined withstale-while-revalidate, is a great way to reduce origin traffic spikes and to provide more reliable performance for end users. For more information about the benefits of stale content, see the blog post"Prevent application and network instability by serving stale content".
Bulk purges
A bulk purge accepts more than one content identifier in a single request and purges them all at once. Bulk purge is available only for surrogate key purges, via theAPI and the--file flag on thefastly purge CLI command. Bulk purges can purge multiple surrogate keys, but always operate on a single service.
Invalidation, eviction, and purging
Content stored in Fastly caches is ephemeral by design; our systems manage the content that we store so that we maximize the proportion of requests that can be successfully served from the cache. When an object is in cache and available to be used, it is "fresh". Purging causes an object to be "invalidated", which means although it may still exist on disk, it cannot be used. "Eviction" describes the automatic invalidation of content from the cache by Fastly, as part of our cache management process.
The amount of time that objects remain fresh in the cache is affected by multiple things:
The size and popularity of the object. Larger objects, and those accessed less frequently, are more susceptible to being evicted from cache before the expiry of their cache lifetime.
Whether the cache server storing the object is the primary storage node for that object. Seeclustering.
WhetherPCI compliance mode is enabled. Enabling PCI mode prevents Fastly from storing content on disk. It can still be cached in memory but is likely to be evicted sooner.
Content that is no longer fresh, either because it has reached the end of its allotted lifetime (TTL) or because it has been invalidated by a purge, may becomestale.
Advanced and best practices
The following sections describe special considerations that may apply to your service depending on your use case.
Shielding
If your service usesshielding, then a request from an end user may traversetwo FastlyPOPs before being forwarded to your origin server. This can cause a number of issues that prevent purging from working as expected:
Race conditions
Since the order in which caches are purged is not deterministic, it is possible that a request might reach a purged edge POP first, and be forwarded to a shield POP that has not yet been purged. The shield POP will deliver the cached content to the edge POP, which will cache it, resulting in the pre-purge content remaining in Fastly cache.
One solution to this race condition problem is simply to purge twice. For purge-all operations, the two purges should be around 30 seconds apart and, for single object and surrogate key purges, around 2 seconds apart.
However, a more elegant solution is possible with the addition of some edge logic. A purge-all operation will cause the cachegeneration value to change. This can be read in VCL usingreq.vcl.generation and in Compute using theFASTLY_CACHE_GENERATION environment variable. This value can be added to a response at the shield POP and then compared with the local value at the edge POP. If they differ, the edge POP should not cache the response. The following example shows how this can be implemented in VCL:
There is no way to mitigate the race condition issue in the case of surrogate key or URL purges, but these take far less time to propagate and as a result encounter the problem much more rarely.
Stale content becoming fresh
If you usesoft purging, a request for a recently purged object will encounter stale content in the cache and, if that content supportsstale-while-revalidate, it will be served immediately to the client while Fastly sends a revalidation request upstream. With shielding enabled, the upstream server can be another Fastly cache, which also may have a stale copy of the object that also supportsstale-while-revalidate. In this scenario, the shield POP will serve the stale content to the edge POP. Forexpired content, this is not a problem, because the content'sAge header will show it to be already older than itsmax-age, and the edge POP will decline to cache it. Forsoft purged content, however, the edge POP may re-cache the object for the remainder of itsmax-age-defined TTL asfresh.
TheVary header is used on responses to indicate that the response can only be used for requests with a specific header value. Commonly this is used to differentiate compressed responses from uncompressed ones by varying onAccept-Encoding, but can also be used in a variety of other situations, including caching logged-in and logged-out variants of a page separately.
In the case ofURL purges, the purge targets objects by cache key, and will therefore invalidateall variants of the object.
Surrogate key purges target objects by the surrogate key, not the cache key, so will invalidate only variants that have the key. In general, distinct variants of the same cache object usually have the same surrogate keys, but this is not a requirement. If an object has multiple variants in cache and only some of them match the key you have requested to purge, only those variants will be invalidated.
Purge-all invalidates all content in the service, so implicitly all variants of all objects are purged.
Versioned URLs
As well as explicitly purging content from the Fastly cache, an effective means of delivering an updated version of content is to publish it at a new URL.
When an HTML page is updated, its URL typically does not change, to avoid breaking inbound links and optimize SEO. However, assets that are loaded by the page, such as videos, images, scripts and stylesheets, are typically only linked to from pages that you control and don't need to maintain a constant URL. Therefore, when a page is updated, it is often possible to change the URLs of all the assets loaded by the page. The page you are reading right now uses this technique.
This is an optimal caching strategy because Fastly will send requests related to the updated page directly to origin to get a new version of the assets, whereas requests from old versions of the page will continue to benefit from assets cached at the edge.
Versioned URLs can take many forms. Here are some examples:
/script.js?v=2021-01-01T00:00:00: Using thetime when the site was updated is a simple mechanism, but if you don't update the entire site in one atomic operation, you may create cache fragmentation.
/script.js?v=12: Using aversion number works well if your site has a single overall version number.
/script.195bb21f.js: Putting ahash of the content in the filename is an effective stateless solution and favored by many web frameworks.
All these URL paths would be considered separate objects by Fastly. Serve these kinds of assets from your origin server with a long cache TTL, and consider setting theimmutable cache directive:
Cache-Control:max-age=31556952, immutable
HINT: When using versioned URLs for assets, avoid performing purge-all operations, since this will also remove all versioned assets. Keeping older versions of versioned assets in the cache can improve user experience during deployments, especially if the older versions are not retained on your origin servers. Instead, consider using asurrogate key purge.
Surrogate key techniques
While we offer three types of purge, the surrogate key purge is the most powerful, flexible, and performant. It is, on average, 100x faster than a purge-all, supports soft purging, and unlike URL purge, offers a way to purge multiple keys in one operation.
With the addition of some edge code, it is possible to purge all objects (the equivalent of a purge-all) or purge a single URL (the equivalent of a URL purge) using surrogate key purge simply by ensuring that all objects share a single common key and that each has a key representing its URL or cache digest.
HINT: If using versioned URLs as described above, implementing a custom 'purge all' using surrogate keys offers the opportunity to not purge versioned assets - for example, by not adding the "all" surrogate key to responses that include animmutableCache-Control directive.
With this code in place you can replace a purge-all with a surrogate key purge for the key "all", and you can replace any individual URL purge with a surrogate key purge for a key matching the URL path of the object you want to purge (e.g., "/products/t-shirt").
Logging
Purge-all operations are logged automatically in theevent log. Surrogate key purges and URL purges are not recorded by default.
If your service has alogging endpoint connected, you can log URL purges to that endpoint by emitting log events in edge code. For example, in a VCL service, this code would log a query string-formatted log line with the service ID when a purge is received.
Surrogate key and URL purges are not counted as part of API rate limits but are separately limited to an average of 100,000 purges per hour, per customer. In the case of surrogate key purges, each key targeted counts as one purge regardless of how many objects are tagged with that key.
Purge mechanism
When you send a purge request to Fastly, it is received and initially processed by a cache server in aPOP close to your location.
URL purges and surrogate key purges are distributed from that machine out to the entire Fastly cache network usinga variant of a gossip protocol, and take around 150ms to reach every cache server in the network.
Purge-all requests are different. Every Fastly service includes a hash function that generates the cache key for a given request. In addition to request properties such as the URL, query string and request method, this function also includes a unique identifier for the service, and a number we call the 'cache generation'. In a purge-all operation, Fastly will recompile your service configuration, incrementing the cache generation, and redeploy it to the network. This means that two identical requests received before and after the purge has completed will result in different cache keys. By this mechanism the previous cache generation is placed 'out of reach' and the space it occupies will be reclaimed by the normal cache management process.
Because purge-all works by changing a property of the service, not by modifying cached content itself, it is possible to revert a purge-all, bydecrementing the cache generation. This operation can only be executed by a Fastly employee. If you purge-all by accident you can request a revert by reaching out toFastly support. Reverting a purge does not guarantee that all previously cached content will return. The sooner a purge is reverted, the more of the cached content will be restored.
It is not possible to revert a URL purge or a surrogate key purge.
Diagnosing purge failures
If a purge appears to have no effect, consider the following:
URL purges:
Hash algorithm: If you have changed your service's hashing algorithm (vcl_hash in VCL services), take care to ensure that your purge request is causing the same lookup in cache as the request that caused the content to be cached.
Rewriting URLs: If your service configuration rewrites URLs, or changes any other property of the request in a way that affects the cache key, you may not be targeting it correctly.
Shielding: If your service has shielding enabled, ensure that the request forwarded to the shield POP has the same request properties as the one originally received at the edge POP (i.e., a POP acting as an edge in a shielding configuration should avoid modifying the request)
Stale content: If your purge was done as asoft purge, you may continue to be served the pre-purge content for some time after the purge is completed. To understand why stale content is being served seeStaleness and revalidation.
Surrogate key purges:
Capitalization: Surrogate keys are case sensitive. Ensure the case of the key specified in your purge request matches that on the content.
Checking keys present: To determine which keys are present on a piece of cached content, request the content URL with an additionalFastly-Debug: 1 header. The response will include theSurrogate-Key header, helping you to understand which keys would cause the content to be purged.
Stale content: If your purge was done as asoft purge, you may continue to be served the pre-purge content for some time after the purge is completed. To understand why stale content is being served seeStaleness and revalidation.
