Solaris Link Aggregation
Link aggregation takes a bunch of network interfaces and creates a big pipe out of them.
Aggregation also provides redundancy. If all interfaces but one go down, the server will remain connected to the network.
Before starting make sure that:
- interfaces to be aggregated are of the following type: xge, e1000g, and bge
- interfaces to be aggregated are not plumbed
- they run in full duplex mode at the same speeds
- eeprom’s local-mac-address? variable is set to true
The following will create aggr1 interface with bge1 as one of its members:
bash-3.00# dladm create-aggr -d bge1 1
Next plumb the aggregate interface, configure an IP address on it and bring it up:
bash-3.00# ifconfig aggr1 plumb 192.168.1.5 netmask 255.255.255.0 up
At this point you can list aggregations:
bash-3.00# dladm show-aggr
key: 1 (0x0001) policy: L4 address: 0:3:ba:56:7f:ba (auto)
device address speed duplex link state
bge1 0:3:ba:56:7f:ba 0 Mbps unknown down standby
Now add bge0 as a second member of aggr1 aggregation interface and list aggregate interfaces:
bash-3.00# dladm add-aggr -d bge0 1
bash-3.00# dladm show-aggr
key: 1 (0x0001) policy: L4 address: 0:3:ba:56:7f:ba (auto)
device address speed duplex link state
bge1 0:3:ba:56:7f:ba 0 Mbps unknown down standby
bge0 0:3:ba:56:7f:b9 1000 Mbps full up attached
To keep the configuration persistent across reboots, create /etc/hostname.aggr1 with appropriate content and remove any hostname.* files pertaining to the interfaces that are now members of aggr1.
To have link aggregation working properly you need to have the switch to which server is connected to properly configured with LACP.
Another thing to consider is load balancing policy for outgoing traffic. You can load balance on layers 2,3 and 4. Load balancing policy can be changed using dladm command. Here is a quick example that will modify load balancing policy to combination of L3 and L4:
bash-3.00# dladm modify-aggr -P L3,L4 1
bash-3.00# dladm show-aggr -L
key: 1 (0x0001) policy: L3,L4 address: 0:3:ba:56:7f:ba (auto)
LACP mode: off LACP timer: short
device activity timeout aggregatable sync coll dist defaulted expired
bge1 passive short yes no no no no no
bge0 passive short yes no no no no no
And finally, command that will allow you to see utilisation of individual links within aggregation. Note the %ipkts column, I did not have LACP turned on on the switch at that time:
bash-3.00# dladm show-aggr -s
key: 1 ipackets rbytes opackets obytes %ipkts %opkts
Total 2723785 2287233197 1481682 710633551
bge1 618712 115674760 870443 636559150 22.7 58.7
bge0 2105073 2171558437 611239 74074401 77.3 41.3
More info on link aggregation is here. Continue Reading
