mode

The mode option allows you to select among the two operating modes, one of "balanced" (the default) or "failover". In the latter case, all ISP interfaces will be pinged periodically, but only the one that is up and running and which has the highest weight will be selected for network traffic. If the preferred ISP becomes inaccessible, then the ISP interface with the next highest weight will be used until such time as the preferred one becomes available again.

forwarding_group

The forwarding_group configuration option defines a set of services that the router is allowed to forward packets among. Provide a space-delimited set of service names or one or more of the abbreviations ":isp" and ":lan". ":isp" is an abbreviation for all ISP services, while ":lan" is an abbreviation for all LAN services. So for example, the two configuration lines below will allow forwarding of packets between LAN1, LAN2, LAN3 and both ISPs. LAN4 will be granted access to both ISPs but won't be able to exchange packets with LANs 1 through 3:

      forwarding_group=LAN1 LAN2 LAN3 :isp
      forwarding_group=LAN4 :isp

If no forwarding_group options are defined, then the router will forward packets among all LANs and ISP interfaces. It is equivalent to this:

      forwarding_group=:lan :isp

warn_email

Provides an email address to send notification messages to if the status of a link changes (goes down, or comes back up). You must have the "mail" program installed and configured for this to work.

interval_ms

Indicates how often to check the ping host for each ISP.

min_packet_loss

max_packet_loss

These define the minimum and maximum packet losses required to declare a link up or down.

min_successive_pkts_rcvd

max_successive_pkts_recvd

These define the minimum and maximum numbers of successively-transmitted pings that must be returned in order to declare a link up or down.

long_down_time

This is a value in seconds after a service that has gone down is considered to have been down for a long time. You may optionally run a series of shell scripts when this has occurred (see below).

4. Make edits to the firewall and route rules (optional)

Net-ISP-Balance allows you to add customized entries to the routing and firewall tables. See Further Configuration for more details.

5. Test load_balance.pl in debug mode (optional)

If you wish to check how the balancing script will configure your system when you execute it, then run (as a regular user) the following command:

 % sudo load_balance.pl  -d > commands.sh    # Ubuntu/Debian
 % su -c load_balance.pl -d > commands.sh    # RedHat/CentOS

The "-d" argument puts the script into debug mode. All commands that it would run on your behalf are placed into 'commands.sh' for your inspection. If you wish, you may pass these commands to the shell in order to preview how your system will perform under load balancing. An example of how to do this is shown below. Note that this doesn't start the link status monitoring daemon needed for automatic failover.

 % /bin/sh commands.sh

6. Start load_balance.pl

Become the superuser and run load_balance.pl

 sudo load_balance.pl    # Ubuntu/Debian
 su -c load_balance.pl   # RedHat/CentOS

This will configure the system for load balancing, installing a restrictive set of firewall rules, and launch the load status monitor (lsm) daemon to monitor each of the ISPs for activity.

7. Arrange for load_balance.pl to be run on system startup time.

You may do this by adding an entry in rc.local:

 if [ -x /etc/network/load_balance.pl ]; then
     /etc/network/load_balance.pl
 fi

Modify as needed for RedHat/CentOS.

Alternatively, my preference is to invoke the script when the LAN interface comes up. On Ubuntu/Debian systems, edit /etc/network/interfaces (Ubuntu/Debian), find the reference to the LAN interface, and edit it to add a "post-up" option as shown here:

 auto eth2
 iface eth2 inet static
    post-up /etc/network/load_balance.pl
    ...

On RedHat/CentOS systems, create an executable script named /sbin/ifup-local, and populate it with the following code:

#!/bin/sh

LANDEV=eth2;

if [ "$1" eq "$LANDEV" ] ; then
   /etc/sysconfig/network-scripts/load_balance.pl
fi

Be sure to change "eth2" to the correct device for the LAN interface.

Further Configuration

The default is to establish a reasonably restrictive firewall which allows incoming ssh services to the router from the Internet and rejects all other incoming services. You may modify this if you wish by adding additional firewall rules and routes.

The routes and rules are located in these subdirectories on Ubuntu/Debian systems:

 /etc/network/balance/firewall       # firewall rules
 /etc/network/balance/routes         # routes

and in these directories on RedHat/CentOS systems:

 /etc/sysconfig/network-scripts/balance/firewall       # firewall rules
 /etc/sysconfig/network-scripts/balance/routes         # routes

Any files you put into these directories will be read in alphabetic order and added to the routes and/or firewall rules emitted by the load balancing script.

A typical routing rules file will look like the example shown below.

 # filename: /net/network/balance/routes/01.local_routes.conf
 ip route add 192.168.100.1  dev eth0 src 198.162.1.14
 ip route add 192.168.1.0/24 dev eth2 src 10.0.0.4

Each line will be sent to the shell, and it is intended (but not required) that these be calls to the "ip" command. General shell scripting constructs are not allowed here.

A typical firewall rules file will look like the example shown here:

 # filename: /net/network/balance/firewall/02.accept.conf
 # accept incoming telnet connections to the router
 iptable -A INPUT -p tcp --syn --dport telnet -j ACCEPT

 # masquerade connections to the DSL modem's control interface
 iptables -t nat -A POSTROUTING -o eth2 -j MASQUERADE

You may also insert routing and firewall rules via fragments of Perl code, which is convenient because you can get the configured service and interface names from the configuration file and can make use of a variety of shortcuts. To do this, simply end the file's name with .pl and make it executable.

Here's an example of a file named balance/firewall/02.forwardings.pl that defines a series of port forwarding rules for incoming connections:

 $B->forward(80 => '192.168.10.35'); # forward port 80 to internal web server
 $B->forward(443=> '192.168.10.35'); # forward port 443 to 
 $B->forward(23 => '192.168.10.35:22'); # forward port 23 to ssh on  web sever

The main thing to know is that on entry to the script the global variable $B will contain an initialized instance of a Net::ISP::Balance object. You may then make method calls on this object to emit firewall and routing rules. Please read the manual page for Net::ISP::Balance for further information ("man Net::ISP::Balance" after the package is installed).

Calling the Script by Hand

You can invoke load_balance.pl from the command line to manually bring up and down ISP services. The format is simple:

/etc/network/load_balance.pl ISP1 ISP2 ISP3 ...                     # Ubuntu/Debian
/etc/sysconfig/network-scripts/load_balance.pl ISP1 ISP2 ISP3 ...   # RedHat/CentOS

ISP1, etc are service names defined in the configuration file. All ISPs indicated on the command line will be maked as "up", others will not be used for load balancing. If no services are indicated on the command line, then ALL the ISP services will be marked up initially and lsm will be launched to monitor their connectivity periodically.

Adding a -d option will print the routing and firewall commands to standard output for inspection.

How it Works

The script uses two load balancing techniques. The first is to set up a multipath default routing destination as described at http://lartc.org/howto/lartc.rpdb.multiple-links.html

 ip route add default \
    nexthop via 206.250.80.122  dev ppp0 weight 1 \
    nexthop via 198.5.13.201    dev eth0 weight 1

This balances network sessions originating from the router, but does usually not work for forwarded (NAT-ed) sessions from the LAN. To accomplish the latter, the script uses a combination of ip routing tables for outgoing connections, the firewall mark (fwmark) mechanism to select tables, and the iptables "mangle" chain to randomly select which ISP to use for outgoing connections:

 iptables -t mangle -A PREROUTING -i eth2 -m conntrack --ctstate NEW \
          -m statistic --mode random --probability 1 -j MARK-ISP1
 iptables -t mangle -A PREROUTING -i eth2 -m conntrack --ctstate NEW \
          -m statistic --mode random --probability 0.5 -j MARK-ISP2

This strategy is described at https://home.regit.org/netfilter-en/links-load-balancing/. The module always gives each ISP equal weight; a future version may support the ability to weight traffic towards one ISP or another.

How Do I...?

...Load balance when I only have a single ethernet port on my router?

For simplicity of configuration I recommend that you use one physical (real) network interface for each of the three components of the network (the two ISP modems and the LAN); USB-based ethernet interfaces are quite inexpensive these days. However, it is also possible to set up balancing in which all components of the network share the same physical wiring by using virtual network interfaces on the router (see this getting started guide). Create virtual network interfaces for each of ISP1, ISP2 and the LAN and give each one a distinct IP address and subnetwork.

Here's a simple example using /etc/network/interfaces:

  # the LAN
  auto eth0:0
  iface eth0:0 inet static
        address 192.168.0.1/24

  # ISP1
  auto eth0:1
  iface eth0:1 inet static
        address 192.168.1.1/24


  # ISP2
  auto eth0:2
  iface eth0:2 inet static
        address 192.168.2.1/24

You will of course also have to configure the modems and the machines on the LAN with the correct addresses and netmasks for the chosen subnets. Now use a balance.conf configuration similar to this one:

 ##service    device   role     ping-ip           weight
 LAN         eth0:0    lan                        
 CABLE       eth0:1    isp      173.194.43.95     1
 DSL         eth0:2    isp      173.194.43.95     1

Net-ISP-Balance version 1.14 is required for this tactic to work. Previous versions contained a bug that prevented virtual network interfaces from being recognized.

Net-ISP-Balance also works with VLANs. Simply use the appropriate VLAN device numbers (e.g. eth0.10)

...Allow an incoming connection to a web server running on my router/firewall machine?

Create a file named /etc/network/balance/firewall/04.webserver.conf. Set the contents to:

      iptables -A INPUT -p tcp --syn --dport 80  -j ACCEPT
      iptables -A INPUT -p tcp --syn --dport 443 -j ACCEPT
    

Connections will be accepted on any of public IP addresses that is assigned to your router machine by your ISPs. You will likely want to publish a domain name for this machine using a dynamic DNS hosting service that allows you to run a "round-robin" on the domain name. See Robin-robin DNS for information on the technique, and perform a Google search for "Round-robin dynamic DNS" for a list of paid DNS hosting services that implement this service.

You must run load_balance.pl once after making this edit in order to have the changes take effect.

Be aware that the location of the firewall rules directory different on RedHat and Debian-derived systems. See Further Configuration for the path on your system.

...Allow an incoming connection to a web server running on my router/firewall machine, but prevent flooding attacks by limiting incoming connections to 5 per second?

Change the contents of /etc/network/balance/firewall/04.webserver.conf to:

      iptables -A INPUT -p tcp --syn --dport 80  -m limit --limit 5/s --limit-burst 20 -j ACCEPT
      iptables -A INPUT -p tcp --syn --dport 80  -j DROPFLOOD
      iptables -A INPUT -p tcp --syn --dport 443 -m limit --limit 5/s --limit-burst 20 -j ACCEPT
      iptables -A INPUT -p tcp --syn --dport 443 -j DROPFLOOD
    

DROPFLOOD is a firewall rule defined by Net-ISP-Balance that drops the connection and logs it to the system log.

...Allow an incoming connection to a web server running on another machine in my LAN?

This is a firewall forwarding rule that has a Perl shortcut to help implement it. Assuming that the web server running on your LAN is at address 192.168.1.10, create the file /etc/balance/firewall/04.webserver.pl, make it executable, and write these contents:

      $B->forward(80  => '192.168.1.10');
      $B->forward(443 => '192.168.1.10');
    

The $B is a Perl variable that holds the Net::ISP::Balance code object. "forward" is the name of a subroutine call.

...Forward connections to port 80 on one of my router's public IP addresses to port 8080 on a web server machine running on another machine in my LAN?

This is similar to the previous recipe, except that you indicate the destination IP address on the internal machine:

      $B->forward(80  => '192.168.1.10:8080');
    

...Reconfigure load_balance for interface(s) whose IP addresses change periodically.

This can be an issue with network interfaces whose IP addresses are set by DHCP, as well as with PPP connections that reset periodically. When the IP address changes, load_balance will detect that the interface can no longer be used for outbound pings, but will not reset the firewall rules to accomodate the new IP address. This can be fixed with the addition of simple scripts that invoke load_balance.pl when the interface changes:

For DHCP interfaces, put the following script into /etc/dhcp/dhclient-exit-hooks.d:

    #!/bin/sh
    case $reason in
         BOUND|RENEW|REBIND|REBOOT|EXPIRE|FAIL|RELEASE|STOP)
             /etc/network/load_balance.pl
             ;;
    esac
  

For PPP and PPPOE interfaces, put the following executable script into /etc/ppp/ip-up.d:

    #!/bin/sh
    /etc/network/load_balance.pl
  

...Allow machines on the LAN to access the control interface of a cable and/or DSL modem attached to the router?

Cable and DSL modems usually provide a web-based control interface associated with the modem's IP address. Sometimes the routing rules that Net-ISP-Balance installs do not provide access to the modem, and attempting to connect to it will fail. There are two alternative techniques to correct this, depending on the type of modem you are using.

Modems in "bridge" mode:

This works when the modem's control interface is separate from its data interface. This is the case for a DSL modem that runs PPPOE in bridge mode. Its primary data interface is ppp0 and its control interface is an ethernet interface such as eth2. Create the file /etc/network/balance/routes/01.modem_route.pl containing the following:

	$B->add_route('192.168.1.1/32'=>'eth2',1);
      

Replace "192.168.1.1" with the modem's IP address and "eth2" with the network interface that the modem is attached to. The "1" argument causes connections to the modem to be masqueraded, which is handy in the case that the modem doesn't correctly route its responses back to your LAN. Make the file executable, and run load_balance.pl.

Modems in "router" mode:

In other cases, the modem performs routing on its own and shares the same interface as an ISP connection. In this case, you have to tell the router to make sure that connections intended for the modem always go out through this interface. Open /etc/network/balance/routes/01.modem_route.pl and enter the following:

	  $B->force_route('CABLE','-p tcp --syn -d 192.168.1.100/32');
	

Replace "192.168.1.100" with the modem's management IP address, and "CABLE" with the ISP service name that the modem is attached to.

...Force outgoing mail connections to go to a particular ISP?

You may wish for all your outgoing email to be routed through a mail transfer host provided by one of your ISPs. However, many ISPs only accept mail from IP addresses within the blocks that they assign. To force all outgoing mail to go to the "CABLE" ISP, create an executable file named 02.outgoing_mail.pl in $ETC_NETWORK/balance/firewall/ with contents similar to this one:

	  $B->force_route('CABLE','-p tcp --syn --dport 25');
    

...Run an OpenVPN server on my router?

VPNs and other applications that manipulate the routing table must be run after load_balance.pl does its thing. Put a command that restarts OpenVPN into $ETC_NETWORK/balance/post-run/99.openvpn:

      /etc/init.d/openvpn restart
    

Adjust as needed for your distribution. In addition you will need to define firewall rules that allow OpenVPN to route between the VPN tunneling interface (typically tun0) and your LAN. Create the file $ETC_NETWORK/balance/firewall/99.openvpn.pl, containing these lines of code:

      $B->iptables("-I INPUT -p $_ --dport 1194 -j ACCEPT") foreach ('udp','tcp');
      $B->iptables("-I INPUT  -i tun0 -j ACCEPT');
      $B->iptables("-I OUTPUT -o tun0 -j ACCEPT');
      foreach ($B->lan_services) {
         $B->iptables('-I FORWARD -i tun0 -o',$B->dev($_),'-j ACCEPT');
         $B->iptables('-I FORWARD -o tun0 -i',$B->dev($_),'-j ACCEPT');
      }
    

...Run custom commands when an ISP goes up or down?

You will find a series of directories in $ETC_NETWORK/balance/lsm named "up.d", "down.d" and "long_down.d". These correspond to ISP state events; any executable scripts found in these directories will be executed when the ISP's state changes. The contents of the "up.d" directory are executed when an ISP goes up, "down.d" will be executed when an ISP goes down, and "long_down.d" will be executed when an ISP remains down for a long time. The long time is determined by the contents of the "long_down_time" option in the balance.conf configuration file. The default is 120 seconds (2 minutes).

Each script will be called with a series of 16 arguments describing the event:

1. the current state
2. the service name
3. the ping IP
4. the network device
5. the email to be notified
6. number of replies to the last set of pings
7. number of waiting pings
8. number of timed out pings
9. number of late pings
10. consecutive packets which received replies
11. consecutive packets waiting for a reply
12. consecutive packets that timed out
13. average round-trip time
14. IP of the device
15. the previous state
16. a timestamp

...Load balance across two OpenVPN tunnels?

This recipe is designed for the case in which you have two or more interfaces on the machine that is serving as a router, you wish to run a OpenVPN tunnel across each of the interfaces, and you wish the router machine and all the hosts on the LAN to load balance across the two (or more) tunnels. The instructions and the example setup script are Ubuntu/Mint/Debian specific. Some work will be required for other distributions.

1. First, create a different OpenVPN client configuration file for each of the ISP interfaces. It is important to understand that each interface requires a distinct tunnel. One tunnel might work, but it will choose one of the ISP interfaces at random and will not load balance across them. You are free to use the same OpenVPN provider multiple times (if your provider allows it), or to use two or more different providers. Choose sensible names for the two OpenVPN configuration file. In this recipe, I am going to call them "vpn1" and "vpn2". Now, before going any further please confirm that with just one ISP interface up and running, you can start and stop the OpenVPN clients using the "service" commands service openvpn start vpn1 and service openvpn start vpn2. Make sure that the VPNs are not set to autostart at boot time.
2. Second, find the script named openvpn-prerun-script.pl that is located in the examples/openvpn-client-balancing directory of the Net-ISP-Balance distribution. Make a copy of it and open it up with a text editor. Toward the top, you will find a little embedded configuration table that looks like this:

   # service    physical_interface   tunnel_interface                                                                                                                                                    
   vpn1         eth0                 tun0
   vpn2         eth1                 tun1
   	 

   Without changing the line above or below, edit the columns to match how your system is configured. Change "vpn1" and "vpn2" to the names of the OpenVPN configuration files you created for the two services, and alter the physical interface names ("eth0", "eth1") to match the interfaces that your router uses to connect to your ISPs. The names of the tunnels ("tun0", "tun1") probably do not need to change. You may also add additional VPNs and interfaces by adding additional rows using the same format.
3. Third, copy the edited script into /etc/network/balance/pre-run/. You can change the script's name if you wish.
4. Last step! Edit /etc/network/balance.conf to tell the load balancer to balance across the VPN interfaces (tun0, tun1...) rather than across the physical interfaces. It should look something like this:

   #service    device   role     ping-ip           weight
   VPN1        tun0     isp      173.194.43.95     1
   VPN2        tun1     isp      173.194.43.95     1
   LAN         eth1     lan                        
   

   It doesn't actually matter what you name the VPN services, but it will be less confusing if you use the same names as used by OpenVPN configuration files.

You should now be able to run

load_balance.pl

as root. The pre-run script will do the last needed bit of adjustment of the OpenVPN configuration files (matching the local address of the interface that the client binds to the one indicated in the pre-run script), setting up the routing tables appropriately, and launching the VPNs. The load_balance script will take care of the rest.

Caveat emptor: The pre-run script is a slightly complicated piece of code that makes various assumptions about your system. It has not been tested exhaustively and may break. Please test extensively before entrusting this to a production router.

...Load balance across IPv6 interfaces>

Net-ISP-Balance does not currently support IPv6, but the underlying link status monitor (lsm) application does. As of March 2021, the requirements for adding IPv6 balancing are under investigation.