Repository files navigation

fabio is a fast, modern, zero-conf load balancing HTTP(S) routerfor deploying microservices managed by consul.

Both hardware and software routers like Citrix Netscaler, F5 Big IP, haproxy,varnish, nginx or apache require some form of configuration - the routingtable - to route incoming traffic to services which can handle them. Therouting table has to be kept in sync with the actual deployed set of servicesand instances during each deployment and each outage on every environment.This makes the routing table a crucial part of the configuration without theapplication cannot function.

Managing the routing table can be automated via API calls or tools likeconsul-template but that alsorequires configuration and/or tools. In the case of consul-template the configfile template itself has to be kept in sync with the actual setup of theapplication. Finally, updating the routing table without loss of existingconnections can be [challenging](http://engineeringblog.yelp.com/2015/04/true-zero-downtime-haproxy-reloads.html).

fabio solves this problem by making the services themselves responsible forupdating the routing table. Services already know which routes they serve sincethey have handlers who can handle requests for them. Once services push theroutes they handle into the service registry (in this case consul) fabio canbuild the routing table and can re-configure itself on every changeautomatically without restart and without the loss of existing connections.

The motivation is also outlined in the presentation I've given at the dotGo EUpre-party in Paris on 9 Nov 2015. You can watch ithere.

fabio was developed at theeBay Classifieds Groupin Amsterdam and routes traffic formarktplaats.nl andkijiji.it. Marktplaats is running all of its trafficthrough fabio which is several thousand requests per second distributed overseveral fabio instances. We don't observe any measurable additional latency.

• Single binary in Go. No external dependencies.
• Zero-conf
• Hot-reloading of routing table through backend watchers
• Round robin and random distribution
• Traffic Shaping (send 5% of traffic to new instances)
• Graphite metrics
• Request tracing
• WebUI
• Fast
• v1.0.4: SSL client certificate authentication support (seeproxy.addr infabio.properties)
• v1.0.5:X-Forwarded-For andForwarded header support
• v1.0.5: Websocket support (experimental)
• v1.0.6: Raw websocket support as default
• v1.0.6: Experimental HTTP api
• v1.0.6: Improved UI
• v1.0.6: fabio registers itself in consul

• Installation
• Quickstart
• Deployment
• Configuration (documented fabio.properties file)
• Performance
• Service configuration
• Manual overrides
• Routing
• Traffic shaping
• Websockets
• Debugging
• Request tracing
• Web UI
• Changelog
• License

This is how you use fabio in your setup:

1. Register your service in consul
2. Register ahealth check in consul as describedhere.Make sure the health check ispassing since fabio will only watch serviceswhich have a passing health check.
3. Register oneurlprefix- tag perhost/path prefix it serves,e.g.urlprefix-/css,urlprefix-/static,urlprefix-mysite.com/
4. Start fabio without a config file (assuming a consul agent onlocalhost:8500)Watch the log output how fabio picks up the route to your service.Try starting/stopping your service to see how the routing table changes instantly.
5. Send all your HTTP traffic to fabio on port9999
6. Done

To start a sample server to test the routing run thedemo/server like this:

./server -addr 127.0.0.1:5000 -name svc-a -prefix /foo

and access the server direct and via fabio

curl 127.0.0.1:5000/foo   # directcurl 127.0.0.1:9999/foo   # via fabio

If you want fabio to handle SSL as well set theproxy.addr along with thepublic/private key files infabio.propertiesand runfabio -cfg fabio.properties. You might also want to set theproxy.header.clientip,proxy.header.tls andproxy.header.tls.valueoptions.

Check theDebugging section to see how to test fabio withcurl.

See fabio in action

[](https://www.youtube.com/watch?v=gvxxu0PLevs"fabio demo - Click to Watch!")

Thefabio-example project is now in thedemo/server directory.

To install fabio run (you need Go 1.4 or higher)

go get github.com/eBay/fabio

To start fabio run

./fabio

which will run it with the default configuration which is describedinfabio.properties. To run it with a config file run itwith

./fabio -cfg fabio.properties

or use the official Docker image and mount your own config file to/etc/fabio/fabio.properties

docker run -d -p 9999:9999 -p 9998:9998 -v $PWD/fabio/fabio.properties:/etc/fabio/fabio.properties magiconair/fabio

If you want to run the Docker image with one or more SSL certificates thenyou can store your configuration and certificates in/etc/fabio and mountthe entire directory, e.g.

$ cat ~/fabio/fabio.propertiesproxy.addr=:443;/etc/fabio/ssl/mycert.pem;/etc/fabio/ssl/mykey.pemdocker run -d -p 443:443 -p 9998:9998 -v $PWD/fabio:/etc/fabio magiconair/fabio

The official Docker image contains the root CA certificates from a recent and updatedUbuntu 12.04.5 LTS installation.

The main use-case for fabio is to distribute incoming HTTP(S) requestsfrom the internet to frontend (FE) services which can handle these requests.In this scenario the FE services then use the service discovery feature inconsul to find backend (BE) services they need in order to fulfil therequest.

That means that fabio is currently not used as an FE-BE or BE-BE router toroute traffic among the services themselves since the service discovery ofconsul already solves that problem. Having said that, there is nothing thatinherently prevents fabio from being used that way. It just means that weare not doing it.

In the following setup fabio is configured to listen on the public ip(s)where it can optionally terminate SSL traffic for one or more domains - one ip per domain.

                                           +--> service-a                                           |internet -- HTTP/HTTPS --> fabio -- HTTP --+--> service-b                                           |                                           +--> service-c

To scale fabio you can deploy it together with the frontend services which provideshigh-availability and distributes the network bandwidth.

           +- HTTP/HTTPS -> fabio -+- HTTP -> service-a (host-a)           |                       |internet --+- HTTP/HTTPS -> fabio -+- HTTP -> service-b (host-b)           |                       |           +- HTTP/HTTPS -> fabio -+- HTTP -> service-c (host-c)

In the following setup fabio is configured receive all incoming trafficfrom an existing gateway which also terminates SSL for one or more domains.fabio supports SSL Client Certificate Authentication to support theAmazon API Gateway

                                                          +--> service-a                                                          |internet -- HTTP/HTTPS --> LB -- HTTP --> fabio -- HTTP --+--> service-b                                                          |                                                          +--> service-c

Again, to scale fabio you can deploy it together with the frontend serviceswhich provides high-availability and distributes the network bandwidth.

                               +- HTTP -> fabio -+-> service-a (host-a)                               |                 |internet -- HTTP/HTTPS --> LB -+- HTTP -> fabio -+-> service-b (host-b)                               |                 |                               +- HTTP -> fabio -+-> service-c (host-c)

fabio has been tested to deliver up to 15.000 req/sec on a single 16core host with moderate memory requirements (~ 60 MB).

To achieve the performance fabio sets the following defaults whichcan be overwritten with the environment variables:

• GOMAXPROCS is set toruntime.NumCPU() since this is not thedefault for Go 1.4 and before
• GOGC=800 is set to reduce the pressure on the garbage collector

When fabio is compiled with Go 1.5 and run with default settings it can be upto 40% slower than the same version compiled with Go 1.4. TheGOGC=100default puts more pressure on the Go 1.5 GC which makes the fabio spend 10% ofthe time in the GC. WithGOGC=800 this drops back to 1-2%. Higher valuesdon't provide higher gains.

As usual, don't rely on these numbers and perform your own benchmarks. You cancheck the time fabio spends in the GC withGODEBUG=gctrace=1.

Each service can register one or more URL prefixes for which it servestraffic. A URL prefix is ahost/path combination without a scheme since SSLhas already been terminated and all traffic is expected to be HTTP. Toregister a URL prefix add a tagurlprefix-host/path to the servicedefinition.

By default, traffic is distributed evenly across all service instances whichregister a URL prefix but you can set the amount of traffic a set of instanceswill receive ("Canary testing"). See [Traffic Shaping](#Traffic Shaping)below.

A background process watches for service definition and health status changesin consul. When a change is detected a new routing table is constructed usingthe commands described in [Config Commands](#Config Commands).

Since an automatically generated routing table can only be changed with aservice deployment additional routing commands can be stored manually in theconsul KV store which get appended to the automatically generated routingtable. This allows fine-tuning and fixing of problems without a deployment.

The [Traffic Shaping](#Traffic Shaping) commands are also stored in the KVstore.

The routing table is configured with the following commands:

route add <svc> <src> <dst> weight <w> tags "<t1>,<t2>,..."  - Add route for service svc from src to dst and assign weight and tagsroute add <svc> <src> <dst> weight <w>  - Add route for service svc from src to dst and assign weightroute add <svc> <src> <dst> tags "<t1>,<t2>,..."  - Add route for service svc from src to dst and assign tagsroute add <svc> <src> <dst>  - Add route for service svc from src to dstroute del <svc> <src> <dst>  - Remove route matching svc, src and dstroute del <svc> <src>  - Remove all routes of services matching svc and srcroute del <svc>  - Remove all routes of service matching svcroute weight <svc> <src> weight <w> tags "<t1>,<t2>,..."  - Route w% of traffic to all services matching svc, src and tagsroute weight <src> weight <w> tags "<t1>,<t2>,..."  - Route w% of traffic to all services matching src and tagsroute weight <svc> <src> weight <w>  - Route w% of traffic to all services matching svc and srcroute weight service host/path weight w tags "tag1,tag2"  - Route w% of traffic to all services matching service, host/path and tags    w is a float > 0 describing a percentage, e.g. 0.5 == 50%    w <= 0: means no fixed weighting. Traffic is evenly distributed    w > 0: route will receive n% of traffic. If sum(w) > 1 then w is normalized.    sum(w) >= 1: only matching services will receive traffic    Note that the total sum of traffic sent to all matching routes is w%.

The order of commands matters but routes are always ordered from most to leastspecific by prefix length.

The routing table contains first all routes with a host sorted by prefixlength in descending order and then all routes without a host again sorted byprefix length in descending order.

For each incoming request the routing table is searched top to bottom for amatching route. A route matches if eitherhost/path or - if there was nomatch - just/path matches.

The matching route determines the target URL depending on the configuredstrategy.rnd andrr are available withrnd being the default.

The auto-generated routing table is

route add service-a www.mp.dev/accounts/ http://host-a:11050/ tags "a,b"route add service-a www.kjca.dev/accounts/ http://host-a:11050/ tags "a,b"route add service-a www.dba.dev/accounts/ http://host-a:11050/ tags "a,b"route add service-b www.mp.dev/auth/ http://host-b:11080/ tags "a,b"route add service-b www.kjca.dev/auth/ http://host-b:11080/ tags "a,b"route add service-b www.dba.dev/auth/ http://host-b:11080/ tags "a,b"

The manual configuration under/fabio/config is

route del service-b www.dba.dev/auth/route add service-c www.somedomain.com/ http://host-z:12345/

The complete routing table then is

route add service-a www.mp.dev/accounts/ http://host-a:11050/ tags "a,b"route add service-a www.kjca.dev/accounts/ http://host-a:11050/ tags "a,b"route add service-a www.dba.dev/accounts/ http://host-a:11050/ tags "a,b"route add service-b www.mp.dev/auth/ http://host-b:11080/ tags "a,b"route add service-b www.kjca.dev/auth/ http://host-b:11080/ tags "a,b"route add service-c www.somedomain.com/ http://host-z:12345/ tags "a,b"

fabio allows to control the amount of traffic a set of service instances willreceive. You can use this feature to direct a fixed percentage of traffic to anewer version of an existing service for testing ("Canary testing"). SeeManual Overrides for a complete description of theroute weight command.

The following command will allocate 5% of traffic towww.kjca.dev/auth/ toall instances ofservice-b which match tagsversion-15 anddc-fra. Thisis independent of the number of actual instances running. The remaining 95%of the traffic will be distributed evenly across the remaining instancespublishing the same prefix.

route weight service-b www.kjca.dev/auth/ weight 0.05 tags "version-15,dc-fra"

Websocket support works but is considered experimental since I don't have anin-house use case for it at the moment. I would like to hear from users whether itworks in their environments beyond my simple test case before I declare it stable.It has been implemented with the websocket library fromgolang.org/x/net/websocket

You can test the websocket support with thedemo/wsclient anddemo/server whichimplements a simple echo server.

./server -addr 127.0.0.1:5000 -name ws-a -prefix /echo -proto ws./wsclient -url ws://127.0.0.1:9999/echo

You can also run multiple web socket servers on different ports but the same endpoint.

fabio detects on whether to forward the request as HTTP or WS based on thevalue of theUpgrade header. If the value iswebsocket it will attempt awebsocket connection to the target. Otherwise, it will fall back to HTTP.

One limitation of the current implementation is that the accepted set ofprotocols has to be symmetric across all services handling it. Only thefollowing combinations will work reliably:

svc-a and svc-b register /foo and accept only HTTP traffic theresvc-a and svc-b register /foo and accept only WS traffic theresvc-a and svc-b register /foo and accept both HTTP and WS traffic there

The following setup (or variations thereof) will not work reliably:

svc-a registers /foo and accept only WS traffic theresvc-b registers /foo and accept only HTTP traffic there

This is not a limitation of the routing itself but because the currentconfiguration does not provide fabio with enough information to make therouting decision since the services do not advertise the protocols they handleon a given endpoint.

This does not look like a big restriction but is also not difficult to extendin a later version assuming there are use cases which require this behavior.For now the services have to be symmetric in the protocols they accept.

To send a request from the command line via the fabio usingcurlyou should send it as follows:

curl -v -H 'Host: foo.com' 'http://localhost:9999/path'

The-x or--proxy options will most likely not work as you expect as theysend the full URL instead of just the request URI which usually does not matchany route but the default one - if configured.

To trace how a request is routed you can add aTrace header with an non-empty value which is truncated at 16 characters to keep the log output short.

$ curl -v -H 'Trace: abc' -H 'Host: foo.com' 'http://localhost:9999/bar/baz'2015/09/28 21:56:26 [TRACE] abc Tracing foo.com/bar/baz2015/09/28 21:56:26 [TRACE] abc No match foo.com/bang2015/09/28 21:56:26 [TRACE] abc Match foo.com/2015/09/28 22:01:34 [TRACE] abc Routing to http://1.2.3.4:8080/

fabio contains a simple web ui to examine the routing table and manage themanual overrides. By default it is accessible onhttp://localhost:9998/

MIT licensed