unixpowered.com

There are about 487359 documents on the Internet about how to mirror root disk in Solaris. So, here is 487360th.

The assumptions are following: the first disk has Solaris already installed, root slice is slice 1, and the disks are identical with the same size and geometry. If they have different cylinder, head, sector count or different size you will have to fiddle with sizing slices more.

The first step is to recreate the same slice arrangement on the second disk:

bash-3.00# prtvtoc /dev/rdsk/c1t0d0s2 | fmthard -s - /dev/rdsk/c1t1d0s2
fmthard: New volume table of contents now in place.

You can check both disks have the same VTOC using prtvtoc command

bash-3.00# prtvtoc /dev/rdsk/c1t0d0s2
* /dev/rdsk/c1t0d0s2 partition map
*
* Dimensions:
* 512 bytes/sector
* 424 sectors/track
* 24 tracks/cylinder
* 10176 sectors/cylinder
* 14089 cylinders
* 14087 accessible cylinders
*
* Flags:
* 1: unmountable
* 10: read-only
*
* First Sector Last
* Partition Tag Flags Sector Count Sector Mount Directory
0 3 01 0 4100928 4100927
1 2 00 4100928 20484288 24585215
2 5 00 0 143349312 143349311
3 8 00 24585216 118672512 143257727
7 0 00 143257728 91584 143349311

Now we have to create state database replicas on slice 7. We will be adding two replicas to each slice:

bash-3.00# metadb -a -f -c3 /dev/dsk/c1t0d0s7
bash-3.00# metadb -a -f -c3 /dev/dsk/c1t1d0s7

Database replicas are crucial part of SVM. Here is some important information about how they work, how many you need, etc.

Since the database replicas are in place we can start creating metadevices. The following commands will create metadevice d31 from slice c1t0d0s3, and metadevice d32 from slice c1t1d0s3. Then we create mirror d30 with d31 attached as a submirror. Finally we will attach submirror d32 to mirror d30. Once d32 is attached, the mirror d30 will automatically start syncing.

bash-3.00# metainit -f d31 1 1 c1t0d0s3
d31: Concat/Stripe is setup
bash-3.00# metainit -f d32 1 1 c1t1d0s3
d32: Concat/Stripe is setup
bash-3.00# metainit d30 -m d31
d30: Mirror is setup
bash-3.00# metattach d30 d32
d30: submirror d32 is attached

The procedure is the same for all other mirrors you might want to create. Root filesystem is slightly different. First you will have to create your submirrors. Then you will have to attach submirror with existing root filesystem, in this case d11, to the new mirror metadevice d10. Then you will have to run metaroot command. It will alter / entry in /etc/vfstab. Finally, you flush the filesystem using lockfs command and reboot.

bash-3.00# metainit -f d11 1 1 c1t0d0s1
d31: Concat/Stripe is setup
bash-3.00# metainit -f d12 1 1 c1t1d0s1
d32: Concat/Stripe is setup
bash-3.00# metainit d10 -m d11
d30: Mirror is setup
bash-3.00# metaroot d10
bash-3.00# lockfs -fa
bash-3.00# reboot

When the system reboots, you can attach the second submirror to d10 as follows:

bash-3.00# metattach d10 d12

You can check the sync progress using metastat command. Once all mirrors are synced up the next step is to configure the new swap metadevice, in my case d0, to be crash dump device. This is done using dumpadm command:

bash-3.00# dumpadm
Dump content: kernel pages
Dump device: /dev/dsk/c1t0d0s0 (dedicated)
Savecore directory: /var/crash/ultra
Savecore enabled: yes
bash-3.00# dumpadm -d /dev/md/dsk/d0

The final step is to modify PROM. First we need to find out which two physical devices c1t0d0 and c1t1d0 refer to:

bash-3.00# ls -l /dev/dsk/c1t0d0s1
lrwxrwxrwx 1 root root 43 Mar 4 14:38 /dev/dsk/c1t0d0s1 -> ../../devices/pci@1c,600000/scsi@2/sd@0,0:b
bash-3.00# ls -l /dev/dsk/c1t1d0s1
lrwxrwxrwx 1 root root 43 Mar 4 14:38 /dev/dsk/c1t1d0s1 -> ../../devices/pci@1c,600000/scsi@2/sd@1,0:b

The physical device path is everything starting from /pci…. Please make a note of sd towards the end of the device string. When creating device aliases below, sd will have to be changed to disk.

Now we create two device aliases called root and backup_root. Then we set boot-device to be root and backup_root. The :b refers to slice 1(root) on that particular disk.

bash-3.00# eeprom "use-nvramrc?=true"
bash-3.00# eeprom "nvramrc=devalias root /pci@1c,600000/scsi@2/disk@0,0 devalias backup_root /pci@1c,600000/scsi@2/disk@1,0"
bash-3.00# eeprom "boot-device=root:b backup_root:b net"

Now we can test that the system boot from both root and backup_root devices.

There is one more optional step. Adding two kernel tunables to /etc/system file. The first one is md_mirror:md_resync_bufsz which will speed up mirror resync. The second one is md:mirrored_root_flag. When this flag is enabled the system will boot even if less than majority of database replicas is available. Personally, I do not use the second tunable. More on these can be found in Solaris Tunable Parameters Reference Guide. Continue Reading

I had a need for a system that can boot two different versions of Solaris from two different disks. Both disks were on target 0 when Solaris was installed on them. But when I moved one of the disks to be target 1, obviously there was going to be a problem with booting from that disk.

There is a simple way to get the disk booting from the new target. Here are the steps to take:

1. Boot the system off a DVD or CD or jumpstart server in single user mode
2. Mount the root filesystem on the disk in question and edit vfstab to reflect the new controller/target setup

The last step is to regenerate /etc/path_to_inst and device links in /dev. Searching the internet for some unrelated info I found out devfsadm has an undocumented -p switch that recreates path_to_inst file. The -r switch specifies location of root filesystem.

bash-3.00# devfsadm -r /mnt -p /mnt/etc/path_to_inst

Now you can reboot and the drive should be bootable again. Continue Reading

The other day after mirroring root disk I needed to test new devalias I made for backup root disk to see if the system would boot from it. Since my servers are headless and I was not “inclined” to console into it I wondered if it was somehow possible to reboot the system and have it boot from alternate boot device.

Knowing it’s possible to do reboot — -rv and such, I fired up the reboot man page. It turns out one can reboot system using:

root@ultra# reboot -- backup_root

This will cause the system too boot from backup_root device alias. Of course, the device alias had to be valid, which it was. Otherwise, I would end up forcing myself to console to the server.

You can also pass in the boot command some flags. For example to boot system from backup_root with -rv boot flags you would do:

root@ultra# reboot -- backup_root -rv Continue Reading

Some useful Sun Cluster commands

Shut down a resource group:
scswitch -F -g [RESOURCE_GROUP_NAME]

Bring up a resource group:
scswitch -Z -g [RESOURCE_GROUP_NAME]

Move failover resource group to node_name:
scswitch -z -g [RESOURCE_GROUP_NAME] -h [NODE_NAME]

Restart resource group:
scswitch -R -h [NODE_NAME] -g [RESOURCE_GROUP_NAME]

Evacuate all resources from node_name:
scswitch -S -h [NODE_NAME]

Disable resource:
scswitch -n -j [RESOURCE]

Enable resource:
scswitch -e -j [RESOURCE]

Clear STOP_FAILED on resource:
scswitch -c -j [RESOURCE] -h [NODE_NAME] -f STOP_FAILED

Disable resource’s fault monitor:
scswitch -n -M -j [RESOURCE]

Enable resource’s fault monitor:
scswitch -e -M -j [RESOURCE]

Enable resource’s fault monitor:
scswitch -e -M -j [RESOURCE]

Lists currently configured DID’s:
scdidadm -L

Put a new device under cluster control:
scgdevs

Displays status of the cluster, resources, resource groups, etc.:
scstat

Display useful setup info about cluster nodes, cluster transport, disksets, etc.:
scconf -p -v Continue Reading

The following scenario assumes two mirrored disks, with two state database replicas located on slice 7 of both disks. High level steps for this are as follows:

1. determine failed disk
2. detach failed submirrors
3. clear failed submirror metadevices and database replicas from failed disk
4. unconfigure the failed disk and replace it
5. configure the new disk and recreate VTOC
6. add new database replicas
7. recreate the submirrors and reattach them to the respective mirrors

This is the current /etc/vfstab:

bash-3.00# more /etc/vfstab
#device device mount FS fsck mount mount
#to mount to fsck point type pass at boot options
#
fd - /dev/fd fd - no -
/proc - /proc proc - no -
/dev/md/dsk/d0 - - swap - no -
/dev/md/dsk/d10 /dev/md/rdsk/d10 / ufs 1 no logging
/dev/md/dsk/d30 /dev/md/rdsk/d30 /export/home ufs 2 yes logging
/devices - /devices devfs - no -
ctfs - /system/contract ctfs - no -
objfs - /system/object objfs - no -
swap - /tmp tmpfs - yes -

From here on I will use d0 and its submirrors as an example. d0 consists of d1 and d2. d2 is on the failed disk.

d0: Mirror
Submirror 0: d1
State: Okay
Submirror 1: d2
State: Needs maintenance
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 4100928 blocks (2.0 GB)
d1: Submirror of d0
State: Okay
Size: 4100928 blocks (2.0 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t0d0s0 0 No Okay Yes
d2: Submirror of d0
State: Needs maintenance
Invoke: metareplace d0 c1t1d0s0 <new device>
Size: 4100928 blocks (2.0 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t1d0s0 0 No Maintenance Yes

First we detach d2. The same has to be repeated for d32 and d12:

bash-3.00# metadetach -f d0 d2
d0: submirror d2 is detached

We need to clear d2. Again, the same is repeated for d32 and d12:

bash-3.00# metaclear d2
d2: Concat/Stripe is cleared

Now we delete database replicas from the failed disk. It’s also very important to make sure we have at least half of state database replicas available before we start removing them from the failed disk. Here is a Sun document that explains Majority Consensus Algorithm Solaris Volume Manager uses. You can determine number and location of the replicas using metadb -i command.

bash-3.00# metadb -d c1t1d0s7

Now we can unconfigure the failed disk using cfgadm, replace it and configure the new disk:

bash-3.00# cfgadm -al
Ap_Id Type Receptacle Occupant Condition
c0 scsi-bus connected configured unknown
c0::dsk/c0t0d0 CD-ROM connected configured unknown
c1 scsi-bus connected configured unknown
c1::dsk/c1t0d0 disk connected configured unknown
c1::dsk/c1t1d0 disk connected configured unknown
c2 scsi-bus connected unconfigured unknown
usb0/1 unknown empty unconfigured ok
usb0/2 unknown empty unconfigured ok
bash-3.00# cfgadm -c unconfigure c1::dsk/c1t1d
bash-3.00# cfgadm -c configure c1::dsk/c1t1d0

Now we replicate VTOC from the good disk:

bash-3.00# prtvtoc /dev/rdsk/c1t0d0s2 | fmthard -s - /dev/rdsk/c1t1d0s2

Add database replicas to the new disk:

bash-3.00# metadb -a -c2 c1t1d0s7

Finally, we can recreate failed submirrors and attach them to their respective mirrors and let them sync up. Again, the same is applies for d32 and d12:

bash-3.00# metainit d2 1 1 c1t1d0s0
d2: Concat/Stripe is setup
bash-3.00# metattach d0 d2
d0: submirror d2 is attached

Few notes: This setup contains total of 4 state database replicas. During a disk failure half of the replicas will be gone. If the server gets rebooted for whatever reason, it will not come up in multiuser mode. If you have less than half of the replicas, the system will panic. For more info on all that check out docs.sun.com.

When using cfgadm to unconfigure disk, there can be no resources using that disk. Otherwise, unconfigure will fail. Quite possibly swap metadevice is set to be dedicated dump device. To view or change dedicated dump device settings use dumpadm command. Continue Reading