unixpowered.com

Finally, I got tired of remembering which network interfaces is configured on my Netra test box. So I do not have to remember which interface to plug cable into I configured IPMP on the box. IPMP provides link redundancy among multiple network interfaces in multipathing group. IPMP is not meant to be full fledged load balancing solution, though it will spread outgoing traffic across the interfaces.

I have put my two hme interfaces into a multipathing group. The group has a failover IP address assigned to it. Initially this address will be assigned to hme0. If hme0 fails, the address will automatically move to the other interface in the failover group.

First I edited /etc/hostname.hme0:

unreal-hme0 netmask + broadcast + deprecated -failover group unrealgrp1 up addif unreal netmask + broadcast + failover up

This configures physical hme0 with IP address of 192.168.11.6, which will not fail over and puts hme0 in unrealgrp1 multipathing group. It will additionally configure a virtual IP address of 192.168.11.5 which will failover when hme0 link goes down. Deprecated means the IP address 192.168.11.5 will not be used as source address for any outgoing packets.

Then I edited /etc/hostname.hme1:

unreal-hme1 netmask + broadcast + deprecated -failover group unrealgrp1 up

Similarly, hme1 will be configured with IP address of 192.168.11.7 and as a member of unrealgrp1 multipathing group. Again, 192.168.11.7 is marked as deprecated so it will not be used for outgoing packets. Finally I made sure my hosts file is correct:

bash-3.00# cat /etc/hosts
127.0.0.1 localhost
192.168.11.5 unreal loghost
192.168.11.6 unreal-hme0
192.168.11.7 unreal-hme1

And here is the result:

bash-3.00# ifconfig -a
lo0: flags=2001000849 mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
hme0: flags=9040843 mtu 1500 index 2
inet 192.168.11.6 netmask ffffff00 broadcast 192.168.11.255
groupname unrealgrp1
ether 8:0:20:d9:ac:c
hme0:1: flags=1000843 mtu 1500 index 2
inet 192.168.11.5 netmask ffffff00 broadcast 192.168.11.255
hme1: flags=19040803 mtu 1500 index 3
inet 192.168.11.7 netmask ffffff00 broadcast 192.168.11.255
groupname unrealgrp1
ether 8:0:20:d9:ac:d

Essentially IP address 192.168.11.5 “floats” among interfaces. If I were to unplug hme0, 192.168.11.5 would fail over to hme1. Failure is detected on link loss. There are some tunable parameters in /etc/default/mpathd.

This is all I needed. Of course, there is much more to IPMP: you can setup test system, that your system will test for reachability, detection of interfaces missing on boot, etc. Sun has much more info on it here. Continue Reading

Create simple striped pool:
zpool create [pool_name] [device] [device] ...
zpool create datapool c5t433127A900011C370000C00003210000d0 c5t433127B4001031250000900000540000d0

Create mirrored pool:
zpool create [pool_name] mirror [device] [device] ...
zpool create datapool mirror c5t433127A900011C370000C00003210000d0 c5t433127B4001031250000900000540000d0

Create Raid-Z pool:
zpool create [pool_name] raidz [device] [device] [device] ...
zpool create datapool raidz c5t433127A900011C370000C00003210000d0 c5t433127B4001031250000900000540000d0 c5t439257C4000019250000900000540000d0

Transform simple pool to a mirror:
zpool create [pool_name] [device]
zpool attach [pool_name] [existing_device] [new_device]
zpool create datapool c5t433127A900011C370000C00003210000d0
zpool attach datapool c5t433127A900011C370000C00003210000d0 c5t433127B4001031250000900000540000d0

Expand simple pool:
zpool create [pool_name] [device]
zpool add [pool_name] [new_device]
zpool create datapool c5t433127A900011C370000C00003210000d0
zpool add datapool c5t433127B4001031250000900000540000d0

Expand mirrored pool by attaching additional mirror:
zpool add [pool_name] mirror [new_device] [new_device]
zpool add datapool mirror c5t433127A900011C370000C00003460000d0 c5t433127B400011C370000C00003410000d0

Replace device in a pool:
zpool replace [pool_name] [old_device] [new_device]
zpool replace datapool c5t433127A900011C370000C00003410000d0 c5t433127B4001031250000900000540000d0

Destroy pool:
zpool destroy [pool_name]
zpool destroy datapool

Set pool mountpoint:
zfs set mountpoint=/path [pool_name]
zfs set mountpoint=/export/zfs datapool

Display configured pools:
zpool list
zpool list

Display pool status info:
zpool status [-v] [pool_name]
zpool status -v datapool

Display pool I/O statistics:
zpool iostat [pool_name]
zpool iostat datapool

Display pool command history:
zpool history [pool_name]
zpool history datapool

Export a pool:
zpool export [pool_name]
zpool export datapool

Import a pool:
zpool import [pool_name]
zpool import datapool

Create a filesystem:
zfs create [pool_name]/[fs_name]
zfs create datapool/filesystem

Destroy a filesystem:
zfs destroy [pool_name]/[fs_name]
zfs destroy datapool/filesystem

Rename a filesystem:
zfs rename [pool_name]/[fs_name] [pool_name]/[fs_name]
zfs rename datapool/filesystem datapool/newfilesystem

Move a filesystem:
zfs rename [pool_name]/[fs_name] [pool_name]/[fs_name]/[fs_name]
zfs rename datapool/filesystem datapool/users/filesystem

Display properties of a filesystem:
zfs get all [pool_name]/[fs_name]
zfs get all datapool/filesystem

Make a snapshot:
zfs snapshot [pool_name]/[fs_name]@[time]
zfs snapshot datapool/filesystem@friday

Roll back filesystem to its snapshot:
zfs rollback [pool_name]/[fs_name]@[time]
zfs rollback datapool/filesystem@friday

Clone a filesystem:
zfs snapshot [pool_name]/[fs_name]@[time]
zfs clone [pool_name]/[fs_name]@[time] [pool_name]/[fs_name]
zfs snapshot datapool/filesystem@today
zfs clone datapool/filesystem@today datapool/filesystemclone

Backup filesystem to a file:
zfs send [pool_name]/[fs_name] > /path/to/file
zfs send datapool/filesystem@friday > /tmp/filesystem.bkp

Restore filesystem from a file:
zfs receive [pool_name]/[fs_name] < /path/to/file
zfs receive datapool/restoredfilesystem < /tmp/filesystem.bkp

Create ZFS volume:
zfs create -V [size] [pool_name]/[vol_name]
zfs create -V 100mb datapool/zvolume
newfs /dev/zvol/dsk/datapool/zvolume Continue Reading

You have a SVM mirror and you need to grow it. Online. All steps are straight forward except for the last one. So, in a nutshell:

1. Grow sizes of both submirror LUN’s on SAN
2. Detach the first submirror metadevice
3. Clear the submirror metadevice
4. Make the system see new LUN size
5. Recreate the submirror metadevice, attach it to the mirror and let it sync
6. Repeat the above for the second submirror
7. Not so obvious step
8. Grow the filesystem

You have a mirror metadevice d200 consisting of submirrors d201 and d202. Both submirrors are sliced in such way that all space is allocated in slice 0. At this point both LUN’s should be already grown, but you need to make the system to see the size increase. Here is what d201, one of the submirrors of d200, should look like:

bash-3.00# metastat d201
d201: Submirror of d200
State: Okay
Size: 2064384 blocks (1008 MB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
/dev/dsk/c5t511714A510011C370000C00003410000d0s0 0 No Okay Yes

First, detach d201 from its parent and clear it:

bash-3.00# metadetach d200 d201
bash-3.00# metaclear d201

The next step is likely to alter the LUN slice configuration, so make sure you have it noted.

You need to run format command to get the system to see the grown LUN. So, run format then select type and then select Autoconfigure. At this point format should be reporting the new LUN size. Make sure your slice configuration is correct and that all the new space is tacked onto the end of correct slice. In this case, it’s slice 0.

Now recreate d201, attach it to d200 and let the mirror sync. When checking status of d200 note that overall mirror size is still the same but the size of d201 increased:

bash-3.00# metastat d200
d200: Mirror
Submirror 0: d202
State: Okay
Submirror 1: d201
State: Okay
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 2064384 blocks (1008 MB)

d202: Submirror of d200
State: Okay
Size: 2064384 blocks (1008 MB)
Stripe 0:
Device                                             Start Block  Dbase        State Reloc Hot Spare
/dev/dsk/c5t511714A510011C370000C00003460000d0s0          0     No            Okay   Yes

d201: Submirror of d200
State: Okay
Size: 4128768 blocks (2.0 GB)
Stripe 0:
Device                                             Start Block  Dbase        State Reloc Hot Spare
/dev/dsk/c5t511714A510011C370000C00003410000d0s0          0     No            Okay   Yes


Now repeat the same process with d202. After you are done you will have both submirrors bigger, but the size of overall mirror will still be the same:

bash-3.00# metastat d200
d200: Mirror
Submirror 0: d201
State: Okay
Submirror 1: d202
State: Okay
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 2064384 blocks (1008 MB)

d201: Submirror of d200
State: Okay
Size: 4128768 blocks (2.0 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
/dev/dsk/c5t511714A510011C370000C00003410000d0s0 0 No Okay Yes

d202: Submirror of d200
State: Okay
Size: 4161536 blocks (2.0 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
/dev/dsk/c5t511714A510011C370000C00003460000d0s0 0 No Okay Yes

Now comes the “Not so obvious” step. In order to increase overall mirror size you need to run metattach on the mirror itself:

bash-3.00# metattach d200
bash-3.00# metastat d200
d200: Mirror
Submirror 0: d201
State: Okay
Submirror 1: d202
State: Okay
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 4128768 blocks (2.0 GB)

This will grow the size of the entire mirror to the size of submirror devices. Now you can grow the filesystem on it without unmounting it. Keep in mind that for the duration of growfs command there will be no writes to the volume.

bash-3.00# growfs -M /export/home/users /dev/md/rdsk/d200
/dev/md/rdsk/d200: 4128768 sectors in 126 cylinders of 128 tracks, 256 sectors
2016.0MB in 63 cyl groups (2 c/g, 32.00MB/g, 15040 i/g)
super-block backups (for fsck -F ufs -o b=#) at:
32, 65824, 131616, 197408, 263200, 328992, 394784, 460576, 526368, 592160,
3487008, 3552800, 3618592, 3684384, 3750176, 3815968, 3881760, 3947552,
4013344, 4079136

That’s all there is to it. The “Not so obvious” step is mentioned in the metattach man page. But unless you do this stuff every day, this very useful piece of information can be easily forgotten. Continue Reading

SP on Sun Fire V40z can be configured so that you can access system console over the network as you would on a UltraSPARC machine with Net Management port. Here is a quick way to get started using V40z and RedHat. Before starting, connect up SP network interface to the network.

Now, edit /etc/inittab and add the following line:

co:12345:respawn:/sbin/agetty -t 60 ttyS0 9600 vt100

This will spawn agetty in runlevels 12345 on serial port 1 with 9600 baud rate and vt100 emulation.

Next you need to edit /etc/grub.conf and comment out splashimage line so boot menu gets rendered properly. Then add following two lines:

serial --unit=0 --speed=9600
terminal --timeout=10 console serial

This will initialize serial port 1 after GRUB startup. If you want to use serial port 2 you would set –unit=1. The terminal we will be using are console and serial in that order, with timeout of 10 seconds. Terminal gets selected depending on where keystroke is detected first, before timeout runs out. If timeout expires, first terminal specified is used.

Finally, append following to the kernel line: console=tty0 console=ttyS0,9600n8. So it will end up looking something like this:

kernel /vmlinuz-2.6.9-67.ELsmp ro root=/dev/VolGroup00/LogVol00 rhgb quiet console=tty0 console=ttyS0,9600n8

Now, you need to add serial port device name to /etc/securetty. This file specifies devices where root can log in. Just append ttyS0 (serial port 1) to the end of the file.

Time to reboot and go to BIOS’s Advanced Settings. Select Console Redirection to Serial Port A and verify you have correct baud rate selected. Reboot the server for all changes to take effect.

At this point SP might not have an IP address assigned, so assign it one using V40z’s front panel. Once you configured SP with IP address, subnet mask and default gateway, ssh to the IP address, from local subnet, using the following:

bash-3.00# ssh setup@IP

You will be asked to setup SP usernames and passwords. When you are done, ssh back to the SP using one of the usernames you have set up, and disable and then re-enable Serial Over LAN:

localhost $ platform set console -s platform
localhost $ platform set console -s sp -e -S 9600

After the SP has been re-enabled, it might be a good idea to set up command prompt so you know which server you are logged into:

localhost $ sp set hostname ultra-sp

Now you can access the console using:

ultra-sp $ platform console

After you connect to the console you can get help by pressing CTRL+E followed by c and ?.Here is sample output:

----
ultra-sp $ platform console
[Enter `^Ec?' for help]
Red Hat Enterprise Linux release 4
Kernel 2.4.21-3.EL on an i686

ultra login:
[help]
.    disconnect                        ;    move to another console
a    attach read/write                 b    send broadcast message
c    toggle flow control               d    down a console
e    change escape sequence            f    force attach read/write
g    group info                        i    information dump
L    toggle logging on/off             l?   break sequence list
l0   send break per config file        l1-9 send specific break sequence
m    display the message of the day    o    (re)open the tty and log file
p    replay the last 60 lines          r    replay the last 20 lines
s    spy read only                     u    show host status
v    show version info                 w    who is on this console
x    show console baud info            z    suspend the connection
|    attach local command              ?    print this message
<cr> ignore/abort command              ^R   replay the last line
ooo send character by octal code
----

At this point you should have a usable network console. You might want to make additional setup changes to the SP to fit your environment.

The first time I issued platform console command in SP I got this error:

console: connect: 59372@console: Connection refused

Rebooting SP using sp reboot fixed the issue for me. Continue Reading

I was configuring MPxIO on Sun Fire V490. After issuing stmsboot -e command and rebooting the server, internal disks were under MPxIO’s control.

This was not something I wanted and I needed to disable it. In order to do so I had to edit /kernel/drv/fp.conf file and tell Solaris not to enable MPxIO on internal disks. So I added the following line at the end of fp.conf file:

name="fp" parent="/pci@8,600000/SUNW,qlc@4" port=0 mpxio-disable="yes";

This line tells Solaris to disable MPxIO on port 0 for all devices whose parent device is /pci@8,600000/SUNW,qlc@4. Of course, similar line should be added for all HBA’s and ports you do not want to have under MPxIO’s control. You can get the parent device from your /var/adm/messages file or from device links pointing to the internal disks. Here is a partial list of device links:
<—————–SNIP—————->
lrwxrwxrwx 1 root other 70 Jun 12 11:50 c1t0d0s0 -> ../../devices/pci@8,600000/SUNW,qlc@4/fp@0,0/ssd@w2100002037c3e2ef,0:a
lrwxrwxrwx 1 root other 70 Jun 12 11:50 c1t0d0s1 -> ../../devices/pci@8,600000/SUNW,qlc@4/fp@0,0/ssd@w2100002037c3e2ef,0:b
lrwxrwxrwx 1 root other 70 Jun 12 11:50 c1t0d0s2 -> ../../devices/pci@8,600000/SUNW,qlc@4/fp@0,0/ssd@w2100002037c3e2ef,0:c
lrwxrwxrwx 1 root other 70 Jun 12 11:50 c1t0d0s3 -> ../../devices/pci@8,600000/SUNW,qlc@4/fp@0,0/ssd@w2100002037c3e2ef,0:d
lrwxrwxrwx 1 root other 70 Jun 12 11:50 c1t0d0s4 -> ../../devices/pci@8,600000/SUNW,qlc@4/fp@0,0/ssd@w2100002037c3e2ef,0:e
lrwxrwxrwx 1 root other 70 Jun 12 11:50 c1t0d0s5 -> ../../devices/pci@8,600000/SUNW,qlc@4/fp@0,0/ssd@w2100002037c3e2ef,0:f
lrwxrwxrwx 1 root other 70 Jun 12 11:50 c1t0d0s6 -> ../../devices/pci@8,600000/SUNW,qlc@4/fp@0,0/ssd@w2100002037c3e2ef,0:g
lrwxrwxrwx 1 root other 70 Jun 12 11:50 c1t0d0s7 -> ../../devices/pci@8,600000/SUNW,qlc@4/fp@0,0/ssd@w2100002037c3e2ef,0:h
<—————–SNIP—————-> Continue Reading