unixpowered.com » networking

Finding out length of a UTP cable using Cisco IOS

somedude — Tue, 05 Jan 2010 12:21:03 +0000

Well, this is just cool. Everyone has been there; sitting at a Cisco switch console wondering how long the unmarked UTP plugged into port 17 is… Thankfully Cisco IOS might be able to tell you:

core# test cable-diagnostics tdr interface gigabitethernet0/17

Unfortunately it’s not available on all switches. I dug this up on here Cisco site. And here is the command reference…

Network interface bonding in Linux

somedude — Mon, 04 May 2009 14:42:15 +0000

Bonding Ethernet interfaces in Linux is pretty straight forward. There is bunch of articles out there on it already, but since this is where I keep some of my notes, I decided to write a post on it. Plus I do not have to bother with Google and I can come straight here for instructions.
This was done on Poweredge running CentOS 5.2. Here are things that need to be done to make this happen:

tell OS to load bonding.ko module on boot
set up configuration files for members of the bonded interface and the bonded interface itself
restart networking services or reboot

The following is /etc/modprobe.conf file. To get the OS to load bonding module on boot, you will need to add the alias bond0 bonding line. You can also pass some options to the bonding module. In this case I wanted the driver to check for link loss every 100ms. I also wanted the bond0 interface to perform adaptive load balancing, hence mode=6. Adaptive load balancing does not require any configuration on the switch side. If you choose a different mode, you might have to do additional configuration on the switch.

[root@bigfoot etc]# cat /etc/modprobe.conf alias eth0 e1000 alias eth1 e1000 alias bond0 bonding alias scsi_hostadapter qla1280 alias scsi_hostadapter1 megaraid_mbox alias scsi_hostadapter2 ata_piix options bond0 miimon=100 mode=6

Next, you need to set up configuration files for physical interfaces to be included in the bond0 interface. In my case bond0 consists of eth0 and eth1. Configuration files for both interfaces are identical except for DEVICE= lines.

[root@bigfoot etc]# cat /etc/sysconfig/network-scripts/ifcfg-eth0 # Intel Corporation 82541GI Gigabit Ethernet Controller DEVICE=eth0 BOOTPROTO=none MASTER=bond0 SLAVE=yes ONBOOT=yes USERCTL=no

The last step is to configure bond0 interface itself:

[root@bigfoot etc]# cat /etc/sysconfig/network-scripts/ifcfg-bond0 DEVICE=bond0 BOOTPROTO=none IPADDR=192.168.11.200 NETMASK=255.255.255.0 NETWORK=192.168.11.0 ONBOOT=yes USERCTL=no

That is all. You can now do either /etc/init.d/networking restart or reboot the box.

This time I actually ran into a problem, where the physical interfaces were not being “enslaved” properly:

May 4 11:17:40 bigfoot kernel: ADDRCONF(NETDEV_UP): bond0: link is not ready May 4 11:17:40 bigfoot kernel: bonding: bond0: Adding slave eth0. May 4 11:17:40 bigfoot kernel: bonding: bond0: enslaving eth0 as an active interface with a down link. May 4 11:17:40 bigfoot kernel: bonding: bond0: link status definitely up for interface eth0. May 4 11:17:40 bigfoot kernel: ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready May 4 11:17:40 bigfoot kernel: bonding: bond0: Adding slave eth1. May 4 11:17:40 bigfoot kernel: bonding: bond0: enslaving eth1 as an active interface with a down link. May 4 11:17:45 bigfoot kernel: bonding: bond0: Removing slave eth0 May 4 11:17:45 bigfoot kernel: bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:11:43:D8:AF:63 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts. May 4 11:17:45 bigfoot kernel: bonding: bond0: releasing active interface eth0 May 4 11:17:47 bigfoot kernel: bonding: bond0: Adding slave eth0. May 4 11:17:48 bigfoot kernel: bonding: bond0: Warning: failed to get speed and duplex from eth0, assumed to be 100Mb/sec and Full. May 4 11:17:48 bigfoot kernel: bonding: bond0: enslaving eth0 as an active interface with an up link.

I have never had this problem before and quick googlage revealed that I am not alone. I came across this guy who had the same problem. He also links to the solution. Basically it seems Xen is causing the issue and to fix it you will need to edit /etc/xen/xend-config.sxp and force the network device to be used for network bridge in Xen:

(network-script 'network-bridge netdev=bond0')

Once I had that in place everything worked as advertised. Oh, and for thorough documentation check out Documentation included with kernel source. The file is called bonding.txt. Here is an online version of it.

Solaris Link Aggregation

somedude — Sun, 22 Mar 2009 11:19:44 +0000

Link aggregation takes a bunch of network interfaces and creates a big pipe out of them.

Aggregation also provides redundancy. If all interfaces but one go down, the server will remain connected to the network.

Before starting make sure that:

interfaces to be aggregated are of the following type: xge, e1000g, and bge
interfaces to be aggregated are not plumbed
they run in full duplex mode at the same speeds
eeprom’s local-mac-address? variable is set to true

The following will create aggr1 interface with bge1 as one of its members:

bash-3.00# dladm create-aggr -d bge1 1

Next plumb the aggregate interface, configure an IP address on it and bring it up:

bash-3.00# ifconfig aggr1 plumb 192.168.1.5 netmask 255.255.255.0 up

At this point you can list aggregations:

bash-3.00# dladm show-aggr key: 1 (0x0001) policy: L4 address: 0:3:ba:56:7f:ba (auto) device address speed duplex link state bge1 0:3:ba:56:7f:ba 0 Mbps unknown down standby

Now add bge0 as a second member of aggr1 aggregation interface and list aggregate interfaces:

bash-3.00# dladm add-aggr -d bge0 1 bash-3.00# dladm show-aggr key: 1 (0x0001) policy: L4 address: 0:3:ba:56:7f:ba (auto) device address speed duplex link state bge1 0:3:ba:56:7f:ba 0 Mbps unknown down standby bge0 0:3:ba:56:7f:b9 1000 Mbps full up attached

To keep the configuration persistent across reboots, create /etc/hostname.aggr1 with appropriate content and remove any hostname.* files pertaining to the interfaces that are now members of aggr1.

To have link aggregation working properly you need to have the switch to which server is connected to properly configured with LACP.

Another thing to consider is load balancing policy for outgoing traffic. You can load balance on layers 2,3 and 4. Load balancing policy can be changed using dladm command. Here is a quick example that will modify load balancing policy to combination of L3 and L4:

bash-3.00# dladm modify-aggr -P L3,L4 1 bash-3.00# dladm show-aggr -L key: 1 (0x0001) policy: L3,L4 address: 0:3:ba:56:7f:ba (auto) LACP mode: off LACP timer: short device activity timeout aggregatable sync coll dist defaulted expired bge1 passive short yes no no no no no bge0 passive short yes no no no no no

And finally, command that will allow you to see utilisation of individual links within aggregation. Note the %ipkts column, I did not have LACP turned on on the switch at that time:

bash-3.00# dladm show-aggr -s key: 1 ipackets rbytes opackets obytes %ipkts %opkts Total 2723785 2287233197 1481682 710633551 bge1 618712 115674760 870443 636559150 22.7 58.7 bge0 2105073 2171558437 611239 74074401 77.3 41.3

Basic IPMP

somedude — Mon, 26 Jan 2009 18:52:16 +0000

Finally, I got tired of remembering which network interfaces is configured on my Netra test box. So I do not have to remember which interface to plug cable into I configured IPMP on the box. IPMP provides link redundancy among multiple network interfaces in multipathing group. IPMP is not meant to be full fledged load balancing solution, though it will spread outgoing traffic across the interfaces.

I have put my two hme interfaces into a multipathing group. The group has a failover IP address assigned to it. Initially this address will be assigned to hme0. If hme0 fails, the address will automatically move to the other interface in the failover group.

First I edited /etc/hostname.hme0:

unreal-hme0 netmask + broadcast + deprecated -failover group unrealgrp1 up addif unreal netmask + broadcast + failover up

This configures physical hme0 with IP address of 192.168.11.6, which will not fail over and puts hme0 in unrealgrp1 multipathing group. It will additionally configure a virtual IP address of 192.168.11.5 which will failover when hme0 link goes down. Deprecated means the IP address 192.168.11.5 will not be used as source address for any outgoing packets.

Then I edited /etc/hostname.hme1:

unreal-hme1 netmask + broadcast + deprecated -failover group unrealgrp1 up

Similarly, hme1 will be configured with IP address of 192.168.11.7 and as a member of unrealgrp1 multipathing group. Again, 192.168.11.7 is marked as deprecated so it will not be used for outgoing packets. Finally I made sure my hosts file is correct:

bash-3.00# cat /etc/hosts 127.0.0.1 localhost 192.168.11.5 unreal loghost 192.168.11.6 unreal-hme0 192.168.11.7 unreal-hme1

And here is the result:

bash-3.00# ifconfig -a lo0: flags=2001000849 mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 hme0: flags=9040843 mtu 1500 index 2 inet 192.168.11.6 netmask ffffff00 broadcast 192.168.11.255 groupname unrealgrp1 ether 8:0:20:d9:ac:c hme0:1: flags=1000843 mtu 1500 index 2 inet 192.168.11.5 netmask ffffff00 broadcast 192.168.11.255 hme1: flags=19040803 mtu 1500 index 3 inet 192.168.11.7 netmask ffffff00 broadcast 192.168.11.255 groupname unrealgrp1 ether 8:0:20:d9:ac:d

Essentially IP address 192.168.11.5 “floats” among interfaces. If I were to unplug hme0, 192.168.11.5 would fail over to hme1. Failure is detected on link loss. There are some tunable parameters in /etc/default/mpathd.

This is all I needed. Of course, there is much more to IPMP: you can setup test system, that your system will test for reachability, detection of interfaces missing on boot, etc. Sun has much more info on it here.

Forcing network speed and duplex in Open Boot prompt

somedude — Sun, 03 Aug 2008 20:04:31 +0000

I had a Sun box attached to a switch I had not control over and I needed to jumpstart it. Unfortunately, I was getting nowhere. After crosschecking everything in my setup the only thing I did not try was to force speed and duplex on the Sun box to see if it would start booting.

I wondered if it would be possible to do so in Open Boot prompt. So I fired up the most logical man page in the given situation – the boot man page. Sure enough it is possible to force speed and duplex from ok prompt:

{1} ok boot net:speed=100,duplex=full - install

And off it went happily booting…

Determining ethernet MAC address from Open Boot prompt

somedude — Tue, 03 Jun 2008 19:02:54 +0000

Today I had a need to jumpstart SunFire V240 using network interface in one of its PCI slots. In order to do that I need to find the MAC address of that interface, preferably with least amount of hassle. There was no Solaris installed and all I had was ok prompt. So I took a quick look at device aliases:

{1} ok devalias xnet2 /pci@1d,700000/pci@1/SUNW,hme@0,1:dhcp, xnet1 /pci@1e,600000/pci@3/SUNW,hme@0,1:dhcp, xnet /pci@1e,600000/pci@2/SUNW,hme@0,1:dhcp, net3 /pci@1d,700000/network@2,1 net2 /pci@1d,700000/network@2 net1 /pci@1f,700000/network@2,1 net /pci@1f,700000/network@2 cdrom /pci@1e,600000/ide@d/cdrom@0,0:f ide /pci@1e,600000/ide@d disk3 /pci@1c,600000/scsi@2/disk@3,0 disk2 /pci@1c,600000/scsi@2/disk@2,0 disk1 /pci@1c,600000/scsi@2/disk@1,0 disk0 /pci@1c,600000/scsi@2/disk@0,0 disk /pci@1c,600000/scsi@2/disk@0,0 scsi /pci@1c,600000/scsi@2 sc-control /pci@1e,600000/isa@7/rmc-comm@0,3e8 ttyb /pci@1e,600000/isa@7/serial@0,2e8 ttya /pci@1e,600000/isa@7/serial@0,3f8 name aliases

Starting with first net* alias I did:

{1} ok cd /pci@1d,700000 {1} ok ls f00f8dac network@1 f00b3274 network@2,1 f00acbf4 network@2 {1} ok cd network@1 {1} ok ls {1} ok pwd /pci@1d,700000/network@1 {1} ok .properties assigned-addresses 82000810 00000000 00600000 00000000 00200000 82000830 00000000 00100000 00000000 00100000 d-fru-len 00000000 d-fru-off 0000e800 d-fru-dev eeprom s-fru-len 00000800 s-fru-off 0000e000 s-fru-dev eeprom compatible pci108e,abba.11 pci108e,abba pciclass,020000 pciclass,0200 reg 00000800 00000000 00000000 00000000 00000000 02000810 00000000 00000000 00000000 00200000 02000830 00000000 00000000 00000000 00100000 address-bits 00000030 max-frame-size 00004000 device_type network name network local-mac-address 00 03 ba 4d b3 3f version Sun PCI Gigaswift Base-X FCode 2.6 01/09/27 phy-type pcs board-model 501-5524 model SUNW,pci-ce fcode-rom-offset 00000000 66mhz-capable fast-back-to-back devsel-speed 00000002 latency-timer 00000040 cache-line-size 00000010 max-latency 00000040 min-grant 00000040 interrupts 00000001 00000002 00000003 00000004 class-code 00020000 revision-id 00000011 device-id 0000abba vendor-id 0000108e

The MAC address of the network interface is shown in local-mac-address field above. All it took then was following boot command to get jumpstart going:

{1} ok boot /pci@1d,700000/network@1 -v - install nowin

NFS4 Invalid inbound domain name

somedude — Mon, 03 Mar 2008 18:51:35 +0000

It seems that starting with Solaris 06/07 nfs4_domain is required in sysidcfg file, otherwise jumpstart will go interactive. You can force a value for example nfs4_domain=example.net or you can set it to be dynamic. In that case the value will be derived from the name service in use. Solaris 10 has nfsmapid daemon that maps numeric UID/GID to a string in format user@example.net.

If there is a domain mismatch between NFS4 client and server, the client will see files on the server owned by nobody. On the server syslog might log something like this:

Mar 3 15:13:14 ultra /usr/lib/nfs/nfsmapid[275]: [ID 300081 daemon.error] valid_domain: Invalid inbound domain name example.net..

In my case there was a typo in /etc/resolv.conf file at the end of domain entry. The entry contained trailing dot. This Sun document has all the useful info that might help troubleshooting similar problems with nfsmapid.