Repository files navigation

dhcplb is Facebook's implementation of:

• a DHCP v4/v6 relayer with load balancing capabilities
• a DHCP v4/v6 server framework

Both modes currently only support handling messages sent by a relayer which isunicast traffic. It doesn't support broadcast (v4) and multicast (v6) requests.Facebook currently uses it in production, and it's deployed at global scaleacross all of our data centers.It is based on@insomniacslkdhcp library.

Facebook uses DHCP to provide network configuration to bare-metal machines atprovisioning phase and to assign IPs to out-of-band interfaces.

dhcplb was created because the previous infrastructure surrounding DHCP ledto very unbalanced load across the DHCP servers in a region when simply usingAnycast+ECMP alone (for example 1 server out of 10 would receive >65% ofrequests).

Facebook's DHCP infrastructure waspresented at SRECon15 Ireland.

Later, support for making it responsible for serving dhcp requests (server mode)was added. This was done because having a single threaded application (ISC KEA)queuing up packets while doing backend calls to another services wasn't scalingwell for us.

• All the relayer implementations available on the internet lack the loadbalancing functionality.
• Having control of the code gives you the ability to:
  • perform A/B testing on new builds of our DHCP server
  • implement override mechanism
  • implement anything additional you need

We needed a server implementation which allow us to have both:

• Multithreaded design, to avoid blocking requests when doing backend calls
• An interface to be able to call other services for getting the IP assignment,boot file url, etc.

This picture shows how we have deployeddhcplb in our productioninfrastructure:

TORs (Top of Rack switch) at Facebook run DHCP relayers, these relayers areresponsible for relaying broadcast DHCP traffic (DISCOVERY and SOLICITmessages) originating within their racks to anycast VIPs, one DHCPv4 and onefor DHCPv6.

In a Cisco switch the configuration would look like this:

ip helper-address 10.127.255.67ipv6 dhcp relay destination 2401:db00:eef0:a67::

We have a bunch ofdhcplbTupperware instances in every region listening onthose VIPs.They are responsible for received traffic relayed by TORs agents and loadbalancing them amongst the actualdhcplb servers distributed across clustersin that same region.

Having 2 layers allows us to A/B test changes of the server implementation.

The configuration fordhcplb consists of 3 files:

• json config file: contains the main configuration for the server as explained in theGetting Started section
• host lists file: contains a list of dhcp servers, one per line, those are the serversdhcplb will try to balance on
• overrides file: a file containing per mac overrides. See theGetting Started section.

dhcplb does not support relaying/responding broadcasted DHCPv4 DISCOVERYpackets or DHCPv6 SOLICIT packets sent toff02::1:2 multicast address. Wedon't need this in our production environment but adding that support should betrivial though.

TODOs and improvements are trackedhere

PRs are welcome!

When operating in v4dhcplb will relay relayed messages coming from otherrelayers (in our production network those are rack switches), the response fromthe server will be relayed back to the rack switches:

dhcp client <---> rsw relayer ---> dhcplb (relay) ---> dhcplb (server)                      ^                                      |                      |                                      |                      +--------------------------------------+

In DHCPv6 responses by the dhcp server will traverse the load balancer.

To installdhcplb into$GOPATH/bin/dhcplb, simply run:

$ go install github.com/facebookincubator/dhcplb@latest

If you wish to clone the repo you can do the following:

$ mkdir -p $GOPATH/src/github.com/facebookincubator$ cd $_$ git clone https://github.com/facebookincubator/dhcplb$ go install github.com/facebookincubator/dhcplb

You can run tests with:

$ cd $GOPATH/src/github.com/facebookincubator/dhcplb/lib$ go test

dhcplb can be run out of the box after compilation.

To start immediately, you can runsudo dhcplb -config config.json -version 6.That will start the relay in v6 mode using the default configuration.

Should you need to integratedhcplb with your infrastructure pleaseseeExtending DHCPLB.

You can bring up a virtual lab using vagrant. This will replicate our productionenvironment, you can spawn VMs containing various components like:

• N instances ofISC dhcpd
• An instance ofdhcplb
• An instance ofdhcrelay, simulating a top of rack switch.
• a VM where you can rundhclient orISC perfdhcp

All of that is managed byvagrant andchef-solo cookbooks.You can use this lab to test yourdhcplb changes.For more information have a look at thevagrant directory.

dhcplb started in April 2016 during a 3 days hackathon in the FacebookDublin office, the hackathon project proved the feasibility of the tool.In June we were joined by Vinnie Magro (@vmagro) for a 3 months internship inwhich he worked with two production engineers on turning the hack into aproduction ready system.

Original Hackathon project members:

• Angelo Failla (@pallotron), Production Engineer
• Roman Gushchin (@rgushchin), Production Engineer
• Mateusz Kaczanowski (@mkaczanowski), Production Engineer
• Jake Bunce, Network Engineer

Internship project members:

• Vinnie Magro (@vmagro), Production Engineer intern
• Angelo Failla (@pallotron), Intern mentor, Production Engineer
• Mateusz Kaczanowski (@mkaczanowski), Production Engineer

Other contributors:

• Emre Cantimur, Production Engineer, Facebook, Throttling support
• Andrea Barberio, Production Engineer, Facebook
• Pablo Mazzini, Production Engineer, Facebook

BSD License. See the LICENSE file.