unixpowered.com

There are two way to create a new container: create one from scratch, which takes a little while or you can clone an existing container. Cloning is quite faster than the actual creation so it is handy to have a “gold” master container which is used for cloning. Another reason for having a “gold” master container is the fact that the container from which you are making a clone has to be halted during cloning. I did this on Solaris 10 8/07 release. Cloning consists of the following:

1. Export configuration of an existing container or create a new configuration from scratch
2. Customize the exported configuration for the new container
3. Import the new container configuration
4. Clone the “gold” master container
5. Configure the new container

First you need configuration for the new container. You can either make one from scratch or export one from existing container and customize it.
The following command will export the configuration from “gold” master container and save it into /tmp/mx1.cfg file:

bash-3.00# zonecfg -z gold export -f /tmp/mx1.cfg

Now you can edit the configuration. Word of caution, importing seems to be a little flaky. For example, if an existing container has bootargs variable set, you will need to put quotes around them in the config file. So, if my bootargs are set to -m verbose, the string will have to be enclosed in “”. Otherwise the new container configuration will not be imported properly.

Once you have the configuration ready you can import it:
bash-3.00# zonecfg -z mx1 -f /tmp/mx1.cfg

Now you can perform actual clone operation:

bash-3.00# zoneadm -z mx1 clone -m copy gold
Cloning zonepath /export/home/zones/gold...

This should not take very long. Once the operation is done you can see that the new container has been installed:

bash-3.00# zoneadm list -cv
ID NAME             STATUS     PATH                           BRAND    IP
0 global           running    /                              native   shared
- mx1              installed  /export/home/zones/mx1         native   shared
- gold             installed  /export/home/zones/gold        native   shared

At this point you can boot the new container. You will be taken through initial configuration of the container. This initial container configuration can be pain if you are rolling out a lot of containers. You can simplify the process by creating a sysidcfg file and putting it into /etc directory of the container before performing the first boot.

In my case I would put the following sysidcfg into /export/home/zones/mx1/root/etc:

system_locale=C
terminal=vt100
network_interface=primary {
hostname=mx1
}
security_policy=NONE
name_service=DNS {
domain_name=example.com
name_server=192.168.1.1
search=example.com
}
nfs4_domain=dynamic
timezone=US/Central
root_password=n2GSHfh

The nice thing about cloning is the fact that it’s fairly quick. So if you hose up a container, you can simply run

bash-3.00# zoneadm -z mx1 uninstall
Are you sure you want to uninstall zone mx1 (y/[n])? y

followed by

bash-3.00# zonecfg -z mx1 delete
Are you sure you want to delete zone mx1 (y/[n])? y

and start all over again. Continue Reading

I needed to build a print server and I had to use HP Jetadmin. I also wanted to use the Containers technology for the job. Here are few things I found out in the process. This was done on Solaris 10 08/07 and Jetadmin version E.10.34.

I did a little bit of searching, and all I found were conflicting reports: it should work, it does not work, it works, but not really…

First of all the container that will host Jetadmin has to be full-root, as Jetadmin install will put some files into /usr/spool. I suppose it would be possible to work around it and use sparse-root, even though it might be a little messy.

The problem I ran into was while creating print queues using hppi tool. During the process mknod command is called to create printer device file in /dev directory of the container. You can not use mknod inside a container and creation of the device file will fail. Sure enough, greping for mknod in Jetadmin’s admin directory yields the following:

bash-3.00$ grep mknod *
addptrtoq:         mknod $DFILE c 13 2

Closer look at addptrtoq reveals the following:

HPNPMODEL=$HPNP/sh/hpnp.model
#DFILE=$HPNP/etc/$SPOOLNAME
DFILE=/dev/$SPOOLNAME
if [ ! -c $DFILE ]
then
mknod $DFILE c 13 2
fi
chmod 666 $DFILE

So to get around the problem, first you manually have to create the special file in the global zone inside /dev directory of the print server container. Then you can proceed with installing the printqueue using hppi tool.

bash-3.00# cd /export/home/zones/prtsvr/dev/
bash-3.00# mknod laser c 13 2
bash-3.00# chmod 666 laser

Now you should be able to add a printer queue without any problems. So Jetadmin works just fine inside Solaris 10 Container. For me, anyways… Continue Reading

Solaris Containers allow partitioning of a physical server into virtual servers. Containers do not provide virtualization in sense of Xen or VMware they are more similar to jails. Zone is a container without resource control.

Solaris instance running in a non-global zone shares parts of its filesystem with the global zone. There is only one kernel running – in the global zone. The kernel handles the physical machine on behalf of non-global zones running on the system. As far as non-global zones are concerned, they appear as separate machines with their own services running, etc. Non-global zones can not “see” each other. They can not see what is going on in the global zone. The global zone, however, can see what is going on inside the non-global zones.

Setting up a zone is fairly trivial. There are two types: full and sparse zones. Here is a quick rundown of a sparse zone setup on Solaris 10 08/07:

1. Change system-wide scheduler to FSS
2. Create a non-global zone
3. Install the zone
4. Boot the created zone

First you should set the default scheduler on the system to be Fair Share Scheduler. This will allow you to assign CPU shares to individual zones. This will also prevent a zone from monopolizing CPU:

bash-3.00# dispadmin -d FSS
bash-3.00# dispadmin -d
FSS (Fair Share)

You will need to reboot in order for system to start using FSS. Note that default scheduler in Solaris 10 is TS. If you want to change scheduler without reboot, in addition to using dispadmin, you could try something like this:

bash-3.00# priocntl -s -c FSS -i class TS
bash-3.00# priocntl -s -c FSS -i pid 1

The first command will move processes from TS class to FSS class and the second command will move init process to FSS class.

To see if there are any zones installed on the system you can use zoneadm command inside the global zone:

bash-3.00# zoneadm list -v

To start seting up a zone use zonecfg command:

bash-3.00# zonecfg -z ns1
ns1: No such zone configured
Use 'create' to begin configuring a new zone.
zonecfg:ns1>

Continuing with the interactive zonecfg session:

zonecfg:ns1> create
zonecfg:ns1> set zonepath=/export/home/zones/ns1
zonecfg:ns1> set autoboot=true
zonecfg:ns1> set bootargs="-m verbose"
zonecfg:ns1> set scheduling-class=FSS
zonecfg:ns1> set cpu-shares=5
zonecfg:ns1> add attr
zonecfg:ns1:attr> set name=comment
zonecfg:ns1:attr> set type=string
zonecfg:ns1:attr> set value="DNS Server"
zonecfg:ns1:attr> end

Remember, nothing is set in stone until you issue commit at the end of the zonecfg session. The above will start creating new zone called ns1. The zone will be installed in /export/home/zones/ns1. It will be booted automatically when the global zone is booted. Next you can add some boot arguments and scheduling class for the zone. If scheduling class is not defined it will be inherited from the global zone. Since you already set the system scheduling class to FSS, this entry is optional. You can also add a name attribute so you know what the purpose of the zone is.

Next, you can cap the memory usage for the zone:

zonecfg:ns1> add capped-memory
zonecfg:ns1:capped-memory> set physical=512M
zonecfg:ns1:capped-memory> set swap=1024M
zonecfg:ns1:capped-memory> end

The physical keyword specifies how much physical memory the zone is allowed to consume. Total swap consumed by user processes and tmpfs mounts inside the non-global zone is set by swap attribute.

Finally add a network interface:

zonecfg:ns1> add net
zonecfg:ns1:net> set physical=aggr1
zonecfg:ns1:net> set address=10.1.1.1
zonecfg:ns1:net> end

In this case, the physical interface for the zone will be aggr1 (aggregate interface in the global zone) with IP address of 10.1.1.1.

Now you can review the settings configured and then commit them:

zonecfg:ns1> info
zonename: ns1
zonepath: /export/home/zones/ns1
brand: native
autoboot: true
bootargs:
pool:
limitpriv:
scheduling-class: FSS
ip-type: shared
inherit-pkg-dir:
dir: /lib
inherit-pkg-dir:
dir: /platform
inherit-pkg-dir:
dir: /sbin
inherit-pkg-dir:
dir: /usr
net:
address: 10.1.1.1
physical: aggr1
capped-memory:
physical: 512M
[swap: 1G]
attr:
name: comment
type: string
value: "DNS Server"
rctl:
name: zone.cpu-shares
value: (priv=privileged,limit=5,action=none)
rctl:
name: zone.max-swap
value: (priv=privileged,limit=1073741824,action=deny)
zonecfg:ns1> commit
zonecfg:ns1> exit

At this point if you list zones on the system you will see something similar:

bash-3.00# zoneadm list -vc
ID NAME STATUS PATH BRAND IP
0 global running / native shared
- ns1 configured /export/home/zones/ns1 native shared

Now you can proceed with installing the zone. The install might take a little while depending on the type of the zone you are installing and the size of the global zone:

bash-3.00# zoneadm -z ns1 install
Preparing to install zone .
Creating list of files to copy from the global zone.
Copying files to the zone.
Initializing zone product registry.
Determining zone package initialization order.
Preparing to initialize packages on the zone.
Initialized packages on zone.
Zone is initialized.
The file contains a log of the zone installation.
bash-3.00#

It might be a good idea to set cpu-shares on the global zone so it’s not rendered unusable by a non-global zone going awol. This will require reboot:

bash-3.00# zonecfg -z global
zonecfg:global> set cpu-shares=10
zonecfg:global> commit
zonecfg:global> exit

Or in addition to the above you can use prctl command to avoid reboot:

bash-3.00# prctl -n zone.cpu-shares -v 10 -r -i zone global
bash-3.00# prctl -n zone.cpu-shares -i zone global
zone: 0: global
NAME PRIVILEGE VALUE FLAG ACTION RECIPIENT
zone.cpu-shares
privileged 10 - none -
system 65.5K max none

Now you can boot the zone and log into it:

bash-3.00# zoneadm -z ns1 boot
bash-3.00# zlogin -C ns1

That’s pretty much it as far as simple setup is concerned. Few links of interest:

Setting zone-wide resource controls
System Administration Guide: Solaris Containers-Resource Management and Solaris Zones
Fair Share Scheduler Overview
How to Set FSS Shares in the Global Zone Using the prctl Command
How to control CPU usage in Global Zone
resource_controls man page
Zone Resource Controls
Zone Resource Control in Solaris 10 08/07 OS
Jeff Victor’s blog – New Zones Features
Short thread on setting memory limits
Solaris 10 Scheduling

Setting Zone-Wide Resource ControlsS Continue Reading

Starting with Solaris 9 there is a very handy tool called logadm that makes management of any log files a breeze. Syslog and messages files, among others, are managed by logadm which is called from root’s crontab.
Logadm reads /etc/logadm.conf file to figure out what it needs to do. By default there are following entries in logadm.conf:

/var/log/syslog -C 8 -P 'Wed Apr 8 02:10:22 2009' -a 'kill -HUP `cat /var/run/syslog.pid`'
/var/adm/messages -C 4 -P 'Fri Apr 10 02:10:15 2009' -a 'kill -HUP `cat /var/run/syslog.pid`'
/var/cron/log -c -s 512k -t /var/cron/olog
/var/lp/logs/lpsched -C 2 -N -t '$file.$N'
/var/fm/fmd/errlog -M '/usr/sbin/fmadm -q rotate errlog && mv /var/fm/fmd/errlog.0- $nfile' -N -s 2m
smf_logs -C 8 -s 1m /var/svc/log/*.log
/var/adm/pacct -C 0 -N -a '/usr/lib/acct/accton pacct' -g adm -m 664 -o adm -p never
/var/log/pool/poold -N -a 'pkill -HUP poold; true' -s 512k

Logadm provides -w switch which will write an entry into logadm.conf file that reflects current command line arguments. Of course logadm.conf can be edited using text editor, if that is the preferred method. If that’s the case, -V option can validate syntax of logadm.conf for you. Another handy option is -n which will cause logadm to do a dry run without actually performing the log rotation.

Other useful switches are:

• -b and -a which allow specification of pre and post rotation commands to execute
• -e sends error messages to a specific address instead of sending it to the owner of the crontab
• -r removes entry from logadm.conf for a specific log file
• -o sets different owner for the new log file from the original
• -g sets different group for the new log file from the original
• -m sets different permissions for the new log file from the original

For the whole story on logadm check out logadm man page.

Continue Reading

Link aggregation takes a bunch of network interfaces and creates a big pipe out of them.

Aggregation also provides redundancy. If all interfaces but one go down, the server will remain connected to the network.

Before starting make sure that:

• interfaces to be aggregated are of the following type: xge, e1000g, and bge
• interfaces to be aggregated are not plumbed
• they run in full duplex mode at the same speeds
• eeprom’s local-mac-address? variable is set to true

The following will create aggr1 interface with bge1 as one of its members:

bash-3.00# dladm create-aggr -d bge1 1

Next plumb the aggregate interface, configure an IP address on it and bring it up:

bash-3.00# ifconfig aggr1 plumb 192.168.1.5 netmask 255.255.255.0 up

At this point you can list aggregations:

bash-3.00# dladm show-aggr
key: 1 (0x0001) policy: L4      address: 0:3:ba:56:7f:ba (auto)
           device       address                 speed           duplex  link    state
           bge1         0:3:ba:56:7f:ba   0     Mbps    unknown down    standby

Now add bge0 as a second member of aggr1 aggregation interface and list aggregate interfaces:

bash-3.00# dladm add-aggr -d bge0 1
bash-3.00# dladm show-aggr
key: 1 (0x0001) policy: L4 address: 0:3:ba:56:7f:ba (auto)
           device       address                 speed           duplex  link    state
           bge1         0:3:ba:56:7f:ba   0     Mbps    unknown down    standby
           bge0         0:3:ba:56:7f:b9   1000  Mbps    full    up      attached

To keep the configuration persistent across reboots, create /etc/hostname.aggr1 with appropriate content and remove any hostname.* files pertaining to the interfaces that are now members of aggr1.

To have link aggregation working properly you need to have the switch to which server is connected to properly configured with LACP.

Another thing to consider is load balancing policy for outgoing traffic. You can load balance on layers 2,3 and 4. Load balancing policy can be changed using dladm command. Here is a quick example that will modify load balancing policy to combination of L3 and L4:

bash-3.00# dladm modify-aggr -P L3,L4 1
bash-3.00# dladm show-aggr -L
key: 1 (0x0001) policy: L3,L4 address: 0:3:ba:56:7f:ba (auto)
                LACP mode: off  LACP timer: short
    device    activity timeout aggregatable sync  coll dist defaulted expired
    bge1      passive  short   yes          no    no   no   no        no
    bge0      passive  short   yes          no    no   no   no        no

And finally, command that will allow you to see utilisation of individual links within aggregation. Note the %ipkts column, I did not have LACP turned on on the switch at that time:

bash-3.00# dladm show-aggr -s
key: 1  ipackets  rbytes      opackets   obytes          %ipkts %opkts
           Total        2723785   2287233197  1481682   710633551
           bge1 618712    115674760   870443    636559150       22.7    58.7
           bge0 2105073   2171558437  611239    74074401        77.3    41.3

More info on link aggregation is here. Continue Reading