1. Home
2. Guides
3. Core concepts

When Fastly cannot satisfy end-user requests directly, we usually forward the request on to abackend. If you wish you can spread the traffic from Fastly across multiple backends, to balance the load, protect against backend failure and avoid overwhelming any one server.

In VCL services, support for load balancing is built-in viadirectors. In Compute services it can be performed explicitly by writing load balancing strategies in your own edge code.

Directors

Directors are entities in VCL services that group together a list ofbackends, or other directors, with a policy (also known as a 'strategy') for choosing one. These can be created automatically for you by usingautomatic load balancing, or defined explicitly in VCL code using thedirector declaration, for example:

director my_director random {
    { .backend= F_backend1; .weight=2; }
    { .backend= F_backend2; .weight=1; }
}

Ultimately all load balancing in VCL uses directors defined similarly to the one above, but there are several ways to create these configurations with different trade offs:

Regardless of the method you use, to provideredundancy and failover within any load balancing group, ensure that all backends that participate in load balancing have ahealth check configured. Directors will redistribute traffic to other backends in the group if one backend is unhealthy.Learn more about failover and redundancy.

Automatic load balancing

The simplest way to perform load balancing is to set theauto_loadbalance property of each backend totrue. This can be done via theAPI,CLI, orin the web interface.

Auto load balancing generates a director with therandom policy and equal weights for each backend, and groups backends based on groups of identicalconditions. The order of priority for selecting backends for a request is:

1. Evaluate conditions that are attached to backends, in order of increasing priority. If a condition is matched, do one of the following:
   • if the condition is associated with only one backend, select that backend
   • if the condition is associated with multiple backends and at least two of them haveauto_loadbalance enabled, choose one of the load balanced ones at random
   • (rarely) if the condition is associated with multiple backends and no more than one hasauto_loadbalance enabled, ignore the load balancing setting and choose the backend that was added to the service first.
2. Since no conditions match, determine if there are two or more backends with auto load balancing enabled and which do not have any conditions attached.
   • If so, choose one of those backends randomly
3. Otherwise, consider backends without attached conditions and with auto load balancing disabled, and select the one that was created earliest

Auto load-balancing is simple but has some limitations:

• If your service usesshielding, the backends participating in auto load balancing should all be configured with the sameshield POP. If they are not, the shield that is used will depend on the order in which the backends were added to your service.
• All backends will have equalweight, i.e. roughly equal amounts of traffic will go to each backend. To configure unequal shares, use acustom director.
• The director will have therandompolicy. For other policies, use acustom director.
• The director will have aquorum of 1: so the director itself will be healthy as long as there is at least one healthy member. To configure a higher quorum, use acustom director.

Custom directors

Explicitly defined directors allow customization of director policies, and complex load balancing logic. They may be defined via API or in custom VCL and require custom VCL to invoke. They are not by default compatible withshielding.

To define a director:

• use adirector declaration in an 'init'VCL snippet or in the init space of a custom VCL file (i.e. outside of any VCL subroutines); or
• use thedirectors API to have Fastly generate the VCL code for you.

To select a custom director, set thereq.backend variable in thevcl_recv subroutine of a custom VCL file:

setreq.backend= my_director;

Custom directors support configurablepolicies andquorum. Therandom,hash,client, orchash policies are good choices for load balancing and will also automatically adapt to the failure of individual backends as long as all backends in the director have ahealth check configured. Thefallback policy creates a director whose only purpose isfailover.

A director can be used in any context in which aBACKEND type is expected, including as a member of another director.

Combining with shielding

Directors don't work easily with shielding because they require you to setreq.backend explicitly in VCL, which will override Fastly's shielding logic and therefore never forward a request to a shield POP. You can work around this:

1. Create a placeholder backend just for shielding.
   • You can name it anything you like, e.g.shielding_iad.
   • Use a hostname that won't route outside of Fastly, e.g.127.0.0.1.
   • Configure it to shield at your desired POP, e.g. Ashburn, Virginia (IAD)
   • If you want some requests to shield in a different place, add more shielding placeholder backends, e.g.shielding_fra.
2. Create the 'real' backends you want to use in a director.
   • You can name them anything you like, e.g.prod,prod_backup,eu1,eu2.
   • Don't enable shielding or load balancing on these backends.
3. Add a condition to all backends except one.
   • On all 'real' backends, add a condition which is always false. The conditional expression can simply be the wordfalse.
   • If you have more than one shielding placeholder backend, add conditions to all but one to differentiate when they should be used, e.g.,client.geo.country_code == "US".
   • You should end up with one backend (a shielding placeholder) that is not subject to any conditions.
4. Write VCL at the top of your custom VCL file (or in an "init" VCL snippet), to define adirector that contains your real backends, for example:

   director origin_director fallback {
     { .backend= F_prod; }
     { .backend= F_prod_backup; }
   }

5. Write VCL invcl_miss andvcl_pass to choose the director if the current backend is the shielding placeholder:

   if (req.backend== F_shielding_placeholder) {
   setreq.backend= origin_director;
   }

The effect of this is that when a request is received, Fastly's automatic backend allocation logic will assign the shielding placeholder backend (because your real backends are disabled by thefalse condition), and if the POP currently processing the request is not the designated shield, the backend will be set to the shield POP instead. Invcl_miss, if the backend is set to the shielding placeholder, you know that you are on the shield POP and about to make a fetch to origin, so can swap out the placeholder backend for the director that you want to use.

Dynamic server pools

Creating or editing custom directors, whether created via the API or explicitly in VCL, requires you to create a new version of your service - even to add or remove backends in the director. If you prefer, Fastly can regenerate the list of backends within the director dynamically, without generating a new version of your service.

To use this capability, you must define the director and the constituent backends using thepools API. Also note that:

• To select the dynamic server pool director, use custom VCL in exactly the same way as for a custom director.
• Dynamic server pools, like custom directors, are not compatible with shielding.
• Backends in a dynamic server pool are not listed in the web interface and do not appear in the service's backends list via the backends API.
• Dynamic server pools are not listed in the web interface

Fastly Compute

The Compute platform does not support directors, but load balancing strategies can be written in edge code directly. Here are some examples for implementing load balancing logic in Compute programs.

A simple random load balancer could be written like so:

For more complex load balancing, you can make use of consistent hashing to map requests to backends based on certain request properties, such as the request's path or a user ID. The implementation for this approach is roughly the same regardless of which property of the request you use as the input to the hash function.

WARNING: Take care when designing load balancing logic in Compute programs that you are aware of how requests will be redistributed if you change the number of backends. Libraries for performingconsistent hashing exist in most programming languages, and can be used to implement more stable mappings between requests and backends.

The following JavaScript examples use theconsistent-hash library to implement hashing.

In this example, the requests are mapped to backends using the paths and queries of the requests:

Here, requests are mapped using a cookie containing a user ID:

Thehealth status of backends is exposed in some Compute SDKs and allows for the creation of customfailover logic in Compute programs.