Network interface bonding in Linux
Bonding Ethernet interfaces in Linux is pretty straight forward. There is bunch of articles out there on it already, but since this is where I keep some of my notes, I decided to write a post on it. Plus I do not have to bother with Google and I can come straight here for instructions.
This was done on Poweredge running CentOS 5.2. Here are things that need to be done to make this happen:
- tell OS to load bonding.ko module on boot
- set up configuration files for members of the bonded interface and the bonded interface itself
- restart networking services or reboot
The following is /etc/modprobe.conf file. To get the OS to load bonding module on boot, you will need to add the alias bond0 bonding line. You can also pass some options to the bonding module. In this case I wanted the driver to check for link loss every 100ms. I also wanted the bond0 interface to perform adaptive load balancing, hence mode=6. Adaptive load balancing does not require any configuration on the switch side. If you choose a different mode, you might have to do additional configuration on the switch.
[root@bigfoot etc]# cat /etc/modprobe.conf
alias eth0 e1000
alias eth1 e1000
alias bond0 bonding
alias scsi_hostadapter qla1280
alias scsi_hostadapter1 megaraid_mbox
alias scsi_hostadapter2 ata_piix
options bond0 miimon=100 mode=6
Next, you need to set up configuration files for physical interfaces to be included in the bond0 interface. In my case bond0 consists of eth0 and eth1. Configuration files for both interfaces are identical except for DEVICE= lines.
[root@bigfoot etc]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
# Intel Corporation 82541GI Gigabit Ethernet Controller
DEVICE=eth0
BOOTPROTO=none
MASTER=bond0
SLAVE=yes
ONBOOT=yes
USERCTL=no
The last step is to configure bond0 interface itself:
[root@bigfoot etc]# cat /etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
BOOTPROTO=none
IPADDR=192.168.11.200
NETMASK=255.255.255.0
NETWORK=192.168.11.0
ONBOOT=yes
USERCTL=no
That is all. You can now do either /etc/init.d/networking restart or reboot the box.
This time I actually ran into a problem, where the physical interfaces were not being “enslaved” properly:
May 4 11:17:40 bigfoot kernel: ADDRCONF(NETDEV_UP): bond0: link is not ready
May 4 11:17:40 bigfoot kernel: bonding: bond0: Adding slave eth0.
May 4 11:17:40 bigfoot kernel: bonding: bond0: enslaving eth0 as an active interface with a down link.
May 4 11:17:40 bigfoot kernel: bonding: bond0: link status definitely up for interface eth0.
May 4 11:17:40 bigfoot kernel: ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready
May 4 11:17:40 bigfoot kernel: bonding: bond0: Adding slave eth1.
May 4 11:17:40 bigfoot kernel: bonding: bond0: enslaving eth1 as an active interface with a down link.
May 4 11:17:45 bigfoot kernel: bonding: bond0: Removing slave eth0
May 4 11:17:45 bigfoot kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:11:43:D8:AF:63 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts.
May 4 11:17:45 bigfoot kernel: bonding: bond0: releasing active interface eth0
May 4 11:17:47 bigfoot kernel: bonding: bond0: Adding slave eth0.
May 4 11:17:48 bigfoot kernel: bonding: bond0: Warning: failed to get speed and duplex from eth0, assumed to be 100Mb/sec and Full.
May 4 11:17:48 bigfoot kernel: bonding: bond0: enslaving eth0 as an active interface with an up link.
I have never had this problem before and quick googlage revealed that I am not alone. I came across this guy who had the same problem. He also links to the solution. Basically it seems Xen is causing the issue and to fix it you will need to edit /etc/xen/xend-config.sxp and force the network device to be used for network bridge in Xen:
(network-script 'network-bridge netdev=bond0')
Once I had that in place everything worked as advertised. Oh, and for thorough documentation check out Documentation included with kernel source. The file is called bonding.txt. Here is an online version of it. Continue Reading
