- Notifications
You must be signed in to change notification settings - Fork0
simula/dynmhs
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
https://www.nntb.no/~dreibh/dynmhs/
Dynamic Multi-Homing Setup (DynMHS) dynamically sets up IP routing rules, to allow for using multiple network connections simultaneously. That is, for each relevant network interface, a separate routing table is created and maintained. For each source address of a network interface managed by DynMHS, routing rules are maintained to point to the corresponding routing table. Software binding to a specific interface address can then use a specific network. In addition, multi-homing-capable network protocols like the Multi-Path TCP (MPTCP) or the Stream Control Transmission Protocol (SCTP) can take advantage of multi-homing for redundancy and load balancing.
A Linux PC is connected to two NAT networks, configuration is dynamic via IPv4 DHCP and IPv6 auto-configuration:
- Network #1 on interface enp0s8: 172.30.255.4 / fdff:b44d:605c:0:a00:27ff:fedb:ad69
- Network #2 on interface enp0s9: 192.168.255.4 / fdc9:dc25:8e35:0:a00:27ff:feaa:bc91
Network settings forNetplan (in /etc/netplan/testpc.yaml):
network: version: 2 ethernets: enp0s8: accept-ra: true dhcp4: true dhcp4-overrides: route-metric: 200 enp0s9: accept-ra: true dhcp4: true dhcp4-overrides: route-metric: 300IPv4 routes:
user@testpc:~$ ip -4 route showdefault via 172.30.255.1 dev enp0s8 proto dhcp src 172.30.255.4 metric 200default via 192.168.255.1 dev enp0s9 proto dhcp src 192.168.255.4 metric 300172.30.255.0/24 dev enp0s8 proto kernel scope link src 172.30.255.4 metric 200172.30.255.1 dev enp0s8 proto dhcp scope link src 172.30.255.4 metric 200192.168.255.0/24 dev enp0s9 proto kernel scope link src 192.168.255.4 metric 300192.168.255.1 dev enp0s9 proto dhcp scope link src 192.168.255.4 metric 300IPv6 routes:
user@testpc:~$ ip -6 route showfc88:1::/64 dev hostonly101 proto kernel metric 256 linkdown pref mediumfdc9:dc25:8e35::/64 dev enp0s9 proto ra metric 300 pref mediumfdff:b44d:605c::/64 dev enp0s8 proto ra metric 200 pref mediumfe80::/64 dev enp0s8 proto kernel metric 256 pref mediumfe80::/64 dev enp0s9 proto kernel metric 256 pref mediumdefault via fe80::5054:ff:fe12:3500 dev enp0s8 proto ra metric 200 expires 788sec pref mediumdefault via fe80::5054:ff:fe12:3500 dev enp0s9 proto ra metric 300 expires 772sec pref mediumNote the two default routes with their different metrics (200, 300).
The expectation is that, according to the chosen source address, a packet is routed via the corresponding interface (enp0s8 or enp0s9). This can be tested by usingHiPerConTracer 2.0, running HiPerConTracer Ping to the Google DNS servers (8.8.8.8, 2001:4860:4860::8888) from all four source addresses:
user@testpc:~$ sudo hipercontracer -P \ -S 172.30.255.4 -S 192.168.255.4 \ -S fdff:b44d:605c:0:a00:27ff:fedb:ad69 -S fdc9:dc25:8e35:0:a00:27ff:feaa:bc91 \ -D 8.8.8.8 -D 2001:4860:4860::8888Connectivity is always over the primary interface, i.e. 172.30.255.4 and fdff:b44d:605c:0:a00:27ff:fedb:ad69. The reason is: This default route has the lowest metric! Also, simply using the same metric for both routes doesnot fix the issue. Then, just the first default route in the routing table would get used.
To get the setup working as expected, it is necessary to configure separate routing tables for each network, and routing rules to select a routing table according to thesource IP address. For example:
- Rule #2000: for packets from 172.30.255.4 use routing table #2000.
- Rule #2000: for packets from fdff:b44d:605c:0:a00:27ff:fedb:ad69 use routing table #2000.
- Rule #3000: for packets from 192.168.255.4 use routing table #3000.
- Rule #3000: for packets from fdc9:dc25:8e35:0:a00:27ff:feaa:bc91 use routing table #3000.
Rules:
user@testpc:~$ ip rule show0: from all lookup local2000: from 172.30.255.4 lookup 20003000: from 192.168.255.4 lookup 300032766: from all lookup main32767: from all lookup defaultuser@testpc:~$ ip -6 rule show0: from all lookup local2000: from fdff:b44d:605c:0:a00:27ff:fedb:ad69 lookup 20003000: from fdc9:dc25:8e35:0:a00:27ff:feaa:bc91 lookup 300032766: from all lookup mainTables:
user@testpc:~$ ip route show table 2000default via 172.30.255.1 dev enp0s8 proto dhcp src 172.30.255.4 metric 200172.30.255.0/24 dev enp0s8 proto kernel scope link src 172.30.255.4 metric 200172.30.255.1 dev enp0s8 proto dhcp scope link src 172.30.255.4 metric 200user@testpc:~$ ip -6 route show table 2000fdff:b44d:605c::/64 dev enp0s8 proto ra metric 200 pref mediumfe80::/64 dev enp0s8 proto kernel metric 256 pref mediumdefault via fe80::5054:ff:fe12:3500 dev enp0s8 proto ra metric 200 pref mediumuser@testpc:~$ ip route show table 3000default via 192.168.255.1 dev enp0s9 proto dhcp src 192.168.255.4 metric 300192.168.255.0/24 dev enp0s9 proto kernel scope link src 192.168.255.4 metric 300192.168.255.1 dev enp0s9 proto dhcp scope link src 192.168.255.4 metric 300user@testpc:~$ ip -6 route show table 3000fdc9:dc25:8e35::/64 dev enp0s9 proto ra metric 300 pref mediumfe80::/64 dev enp0s9 proto kernel metric 256 pref mediumdefault via fe80::5054:ff:fe12:3500 dev enp0s9 proto ra metric 300 pref mediumIt would be possible to configurestatic rules/tables in Netplan. But DHCP and IPv6 auto-configuration usedynamic addresses. So, they may change!
Dynamic Multi-Homing Setup (DynMHS) is the solution for dynamically creating, adapting, and destroying routing tables and rules. DynMHS monitors the system's network configuration for changes, and applies the necessary settings for additional routing tables and the corresponding routing rules. This works for IPv4 and IPv6, including multiple addresses as well as additional routes over the monitored interfaces.
sudo dynmhs --interface enp0s8:2000 --interface enp0s9:3000 --loglevel 2Configuration in /etc/dynmhs/dynmhs.conf:
# ====== Logging Verbosity ==================================================# 0=trace, 1=debug, 2=info, 3=warning, 4=error, 5=fatalLOGLEVEL=2# ====== Options ============================================================NETWORK1="enp0s8:2000"NETWORK2="enp0s9:3000"NETWORK3=""NETWORK4=""NETWORK5=""These settings map interface enp0s8 to routing table #2000, and interface enp0s9 to routing table #3000. DynMHS will maintain the tables, and the corresponding rules.
To enable and start the DynMHS service:
sudo systemctl daemon-reloadsudo systemctl enable dynmhssudo systemctl start dynmhsTo observe the logs of the DynMHS service:
sudo journalctl -f -u dynmhsAnother test withHiPerConTracer 2.0, running HiPerConTracer Ping to the Google DNS servers from all four source addresses:
user@testpc:~$ sudo hipercontracer -P \ -S 172.30.255.4 -S 192.168.255.4 \ -S fdff:b44d:605c:0:a00:27ff:fedb:ad69 -S fdc9:dc25:8e35:0:a00:27ff:feaa:bc91 \ -D 8.8.8.8 -D 2001:4860:4860::8888...2025-02-28 13:59:50.422: Ping ICMP 192.168.255.4 8.8.8.8 Success s: 27µs q: 4µs r: 54µs A:8.892ms S:8.808ms H:---2025-02-28 13:59:50.422: Ping ICMP fdff:b44d:605c:0:a00:27ff:fedb:ad69 2001:4860:4860::8888 Success s: 51µs q: 6µs r:246µs A:8.634ms S:8.331ms H:---2025-02-28 13:59:50.422: Ping ICMP 172.30.255.4 8.8.8.8 Success s: 38µs q: 4µs r:125µs A:9.272ms S:9.105ms H:---2025-02-28 13:59:50.422: Ping ICMP fdc9:dc25:8e35:0:a00:27ff:feaa:bc91 2001:4860:4860::8888 Success s: 29µs q: 2µs r:162µs A:8.651ms S:8.458ms H:---...Now, there is connectivity over both interfaces!
Please use the issue tracker athttps://github.com/dreibh/dynmhs/issues to report bugs and issues!
For ready-to-install Ubuntu Linux packages of DynMHS, seeLaunchpad PPA for Thomas Dreibholz!
sudo apt-add-repository -sy ppa:dreibh/ppasudo apt-get updatesudo apt-get install dynmhsFor ready-to-install Fedora Linux packages of DynMHS, seeCOPR PPA for Thomas Dreibholz!
sudo dnf copr enable -y dreibh/ppasudo dnf install dynmhsDynMHS is released under the GNU General Public Licence (GPL).
Please use the issue tracker athttps://github.com/dreibh/dynmhs/issues to report bugs and issues!
The Git repository of the DynMHS sources can be found athttps://github.com/dreibh/dynmhs:
git clone https://github.com/dreibh/dynmhscd dynmhscmake .makeContributions:
Issue tracker:https://github.com/dreibh/dynmhs/issues.Please submit bug reports, issues, questions, etc. in the issue tracker!
Pull Requests for DynMHS:https://github.com/dreibh/dynmhs/pulls.Your contributions to DynMHS are always welcome!
CI build tests of DynMHS:https://github.com/dreibh/dynmhs/actions.
Coverity Scan analysis of DynMHS:https://scan.coverity.com/projects/dreibh-dynmhs.