Very basic SELinux troubleshooting
SELinux has been around for a while in RedHat. SELinux is Mandatory Access Control mechanism. Starting with RedHat 6, the installer automatically sets SELinux to enforcing mode.
When troubleshooting something SELinux is one more thing to keep in mind.
If you are fixing something and you booted with SELinux disabled, all files created since you have disabled it will not have SELinux context (fcontext). This will cause filesystem relabelling, when you turn SELinux back on. This can take a long time and you will lose fcontexts unless you have added them to the policy database.
If you do decide to go this route, you can disable SELinux by passing selinux=0 to init or edit /etc/sysconfig/selinux config file and the reboot.
When troubleshooting, it’s probably better to use getenforce and setenforce commands:
[root@ultra opt]# getenforce
Enforcing
To change SELinux status use setenforce:
[root@ultra opt]# setenforce 0
[root@ultra opt]# getenforce
Permissive
One thing that I forget sometimes is difference between cp and mv commands with respect to SELinux. Moving file preserves fcontext, whereas copying does not, unless you use -a option.
Then there are different booleans that can be read and set using getsebool and setsebool. The key thing to remember is that unless you supply -P option to setsebool, the change will not survive reboot.
If you are suspecting problems with fcontexts, you can use chcon and semanage tools. Using chcon changes context on a file or directory, but the context is not added to policy database, so it will not survive reboot.
[root@ultra opt]# chcon --reference /var/www/html /www
This is handy if you want to quickly test out fcontext. The command applies the same fcontext from /var/www/html to /www.
To make fcontext stick across reboots you have to do something like:
[root@ultra opt]# semanage -a -t public_content_t '/www(/.*)?'
You will need to substitute desired fcontext in place of public_content_t.
Then there is setroubleshootd, which along with sealert can help you figure out what’s happening. The log file /var/log/messages will contain SELinux messages that setroubleshootd intercepts when it’s running, giving you sealert command to run to see in detail what SELinux violation occurred.
That would be it in a very basic nutshell.
One more thing to remember, if you did something like:
[root@ultra opt]# semanage -a -t private_content_t '/www/stuff(/.*)?'
[root@ultra opt]# semanage -a -t public_content_t '/www(/.*)?'
Then during next filesystem relabel /www/stuff will have public_content_t fcontext!
In: centos, linux, linux tips, redhat, security
OpenBoot: All CPUs failed or disabled
I guess every day you learn something new. This incident happened on Sun Fire V480R. The server was running for running for ages but for one reason or another it had to be rebooted.
So after the diags ran, it came back with this:
Sun Fire 480R, No Keyboard
Copyright 1998-2003 Sun Microsystems, Inc. All rights reserved.
OpenBoot 4.10.7, 4096 MB memory installed, Serial #55408554.
Ethernet address 0:3:ba:4d:77:aa, Host ID: 834d77aa.
FATAL: All CPUs failed or disabled.
Hmm, interesting. This was the first time I have seen this error. Admittedly, it was a little bit stressful, because the system had to come back up, never mind the fact that the hardware is at a completely different location.
So, pretty much the only viable options was to make server boot somehow. Let’s see what had been ASR disabled:
{3} ok .asr
ASR Disablement Status
Component: Status
CPU/Memory: Enabled
IO-Bridge5: Enabled
IO-Bridge8: Enabled
IO-Bridge9: Enabled
GPTwo Slots: Enabled
Onboard FCAL: Enabled
Onboard Net1: Enabled
Onboard Net0: Enabled
Onboard IDE: Enabled
PCI Slots: Enabled
All seems OK to me. Let’s see if disabling and enabling CPU/Memory banks will do the trick:
{3} ok asr-disable cpu0
{3} ok asr-disable cpu1
{3} ok asr-disable cpu2
{3} ok asr-disable cpu3
{3} ok .asr
ASR Disablement Status
Component: Status
CPU0: Disabled
Memory Bank0: Enabled
Memory Bank1: Enabled
Memory Bank2: Enabled
Memory Bank3: Enabled
CPU1/Memory: Disabled
Memory Bank0: Enabled
Memory Bank1: Enabled
Memory Bank2: Enabled
Memory Bank3: Enabled
CPU2: Disabled
Memory Bank0: Enabled
Memory Bank1: Enabled
Memory Bank2: Enabled
Memory Bank3: Enabled
CPU3: Disabled
Memory Bank0: Enabled
Memory Bank1: Enabled
Memory Bank2: Enabled
Memory Bank3: Enabled
IO-Bridge5: Enabled
IO-Bridge8: Enabled
IO-Bridge9: Enabled
GPTwo Slots: Enabled
Onboard FCAL: Enabled
Onboard Net1: Enabled
Onboard Net0: Enabled
Onboard IDE: Enabled
PCI Slots: Enabled
{3} ok asr-enable cpu0
{3} ok asr-enable cpu1
{3} ok asr-enable cpu2
{3} ok asr-enable cpu3
{3} ok reset-all
After reset, the system came back again with the same error, saying all CPU’s were failed or disabled. So the whole enable,disable procedure was repeated again. Except this time, the system was powered off and then back on.
This time it booted happily. Maybe simple poweroff and poweron would suffice. Anyways, there is some good information right here.
In: openboot, sun hardware
How to configure Solaris 10 to drive HP MSL 6000
I have done this a long time ago so there might be something missing as this comes from sketchy notes that I still have. The point of this exercise is to get Solaris 10 to use MSL 6000 fibre channel library for backups using HP Data Protector 5.5. I have done this on Sun Fire V240 servers.
The library had HP Ultrium drives installed. First you have to make sure the tape drives are defined in /kernel/drv/st.conf file. Here is the tape-config list I got somewhere on the net, I think:
tape-config-list =
"HP Ultrium 1-SCSI", "HP Ultrium 1-SCSI", "LTO-data",
"DEC DLT2000", "Digital DLT2000", "DLT2k-data",
"Quantum DLT4000","Quantum DLT4000", "DLT4k-data",
"QUANTUM DLT7000", "Quantum DLT7000", "DLT7k-data",
"QUANTUM DLT8000", "Quantum DLT8000", "DLT8k-data",
"COMPAQ SuperDLT1","Compaq SuperDLT","SDLT-data",
"COMPAQ SDLT320", "Compaq SuperDLT 2", "SDLT320-data",
"HP Ultrium","HP Ultrium","ULTRIUM",
"HP C9264CB-VS80","HP DLT vs80 DLTloader","HP_data1",
"QUANTUM SuperDLT1", "QUANTUM SuperDLT", "SDLT-data",
"TANDBERGSuperDLT1", "TANDBERG SuperDLT", "SDLT-data",
"STK 9840", "STK 9840", "CLASS_9840";
DLT2k-data = 1,0x38,0,0x8639,4,0x17,0x18,0x80,0x81,3;
DLT4k-data = 1,0x38,0,0x8639,4,0x17,0x18,0x80,0x81,3;
DLT7k-data = 1,0x38,0,0x8639,4,0x82,0x83,0x84,0x85,3;
DLT8k-data = 1,0x77,0,0x1D639,4,0x84,0x85,0x88,0x89,3;
ULTRIUM = 1,0x36,0,0x8639,4,0x00,0x00,0x00,0x42,3;
SDLT-data = 1,0x79,0,0x8639,4,0x90,0x91,0x90,0x91,3;
CLASS_9840 = 1,0x78,0,0x1d679,1,0x00,0;
Data Protector installation included sst driver for Solaris along with its configuration file sst.conf. When you install Data Protector client those two files should be in /opt/omni/spt directory.
The driver needs to be copied to /usr/kernel/drv/sparcv9 directory and the config file to /usr/kernel/drv directory. In sst.conf you need to define WWN of the tape library:
name="sst" parent="fp" target=6 lun=0 fc-port-wwn="1000d0e004b2562d";
Somewhere along the way you will need to inform Solaris that a new driver was installed:
bash-3.00# add_drv sst
Next, /etc/devlink.tab needs to be configured for the library’s robotic arm:
type=ddi_pseudo;name=sst;minor=character rsst\A1
Make sure you use spaces, in the above entry. If I remember right, all you need to do is reconfigure reboot:
bash-3.00# reboot -- -r
When the system comes up, you should see something like this:
bash-3.00# ls -l /devices/pci@1e,600000/QLGC,qla@2/fp@0,0/
drwxr-xr-x 2 root sys 512 Oct 14 16:14 sgen@w1000d0e004b2562d,0
crw------- 1 root sys 151, 0 Oct 20 14:54 sgen@w1000d0e004b2562d,0:changer
drwxr-xr-x 2 root sys 512 Oct 18 17:14 sst@w1000d0e004b2562d,0
crw------- 1 root sys 319, 0 Oct 18 17:25 sst@w1000d0e004b2562d,0:character
drwxr-xr-x 2 root sys 512 Oct 14 16:14 st@w1000d0e004b2562d,1
crw-rw-rw- 1 root sys 33, 26 Oct 20 14:54 st@w1000d0e004b2562d,1:
crw-rw-rw- 1 root sys 33, 90 Oct 20 14:54 st@w1000d0e004b2562d,1:b
crw-rw-rw- 1 root sys 33, 94 Oct 20 14:54 st@w1000d0e004b2562d,1:bn
crw-rw-rw- 1 root sys 33, 26 Oct 20 14:54 st@w1000d0e004b2562d,1:c
crw-rw-rw- 1 root sys 33, 90 Oct 20 14:54 st@w1000d0e004b2562d,1:cb
crw-rw-rw- 1 root sys 33, 94 Oct 20 14:54 st@w1000d0e004b2562d,1:cbn
crw-rw-rw- 1 root sys 33, 30 Oct 20 14:54 st@w1000d0e004b2562d,1:cn
crw-rw-rw- 1 root sys 33, 18 Oct 20 14:54 st@w1000d0e004b2562d,1:h
crw-rw-rw- 1 root sys 33, 82 Oct 20 14:54 st@w1000d0e004b2562d,1:hb
crw-rw-rw- 1 root sys 33, 86 Oct 20 14:54 st@w1000d0e004b2562d,1:hbn
crw-rw-rw- 1 root sys 33, 22 Oct 20 14:54 st@w1000d0e004b2562d,1:hn
crw-rw-rw- 1 root sys 33, 2 Oct 20 14:54 st@w1000d0e004b2562d,1:l
crw-rw-rw- 1 root sys 33, 66 Oct 20 14:54 st@w1000d0e004b2562d,1:lb
crw-rw-rw- 1 root sys 33, 70 Oct 20 14:54 st@w1000d0e004b2562d,1:lbn
crw-rw-rw- 1 root sys 33, 6 Oct 20 14:54 st@w1000d0e004b2562d,1:ln
crw-rw-rw- 1 root sys 33, 10 Oct 20 14:54 st@w1000d0e004b2562d,1:m
crw-rw-rw- 1 root sys 33, 74 Oct 20 14:54 st@w1000d0e004b2562d,1:mb
crw-rw-rw- 1 root sys 33, 78 Oct 20 14:54 st@w1000d0e004b2562d,1:mbn
crw-rw-rw- 1 root sys 33, 14 Oct 20 14:54 st@w1000d0e004b2562d,1:mn
crw-rw-rw- 1 root sys 33, 30 Oct 20 14:54 st@w1000d0e004b2562d,1:n
crw-rw-rw- 1 root sys 33, 26 Oct 20 14:54 st@w1000d0e004b2562d,1:u
crw-rw-rw- 1 root sys 33, 90 Oct 20 14:54 st@w1000d0e004b2562d,1:ub
crw-rw-rw- 1 root sys 33, 94 Oct 20 14:54 st@w1000d0e004b2562d,1:ubn
crw-rw-rw- 1 root sys 33, 30 Oct 20 14:54 st@w1000d0e004b2562d,1:un
In the above, the Qlogic adapter and its port is the one that is zoned for the tape library.
One last thing is to create device file for the autochanger:
bash-3.00# ls -l /dev/rsst*
lrwxrwxrwx 1 root root 74 Oct 18 17:18 /dev/rsst8 -> /devices/pci@1e,600000/QLGC,qla@2/fp@0,0/sst@w1000d0e004b2562d,0:character
Like I mentioned, this might be incomplete as I had spotty notes. So YMMV.
In: fibre channel, hp data protector, hp msl, solaris, solaris tips
Viewing and changing speed and duplex in Solaris
Depending on version of Solaris and the hardware it’s running on, you will either use ndd utility or use kstat or dladm commands to change speed and duplex. Remember that for example, e1000g driver will let you manipulate settings using dladm in Solaris 10, but you will not be able to do so using ndd. This is also true among different Solaris versions. For example, dladm is present only in Solaris 10.
First, let’s see what the current settings are. To do that you can use kstat or ndd commands. The following output is truncated to include only info we are interested in.
bash-3.00# kstat -m e1000g -i 0
module: e1000g instance: 0
name: mac class: net
adv_cap_1000fdx 1
adv_cap_1000hdx 0
adv_cap_100fdx 1
adv_cap_100hdx 1
adv_cap_10fdx 1
adv_cap_10hdx 1
adv_cap_asmpause 1
adv_cap_autoneg 1
Setting these variables on or off basically tells the driver if it should advertise corresponding capability. So setting adv_cap_100hdx to 0 will cause Solaris to stop advertising 100 half duplex capability.
If you are using hme driver you will need to use ndd command to view current setting:
bash-3.00# ndd -get /dev/hme link_speed
The “disadvantage” here is that you have to remember the name of the setting you are trying to query. If you are running Solaris 10 with GLDv3 (project Nemo) driver you can use dladm command:
bash-3.0# dladm show-dev
e1000g0 link: up speed: 1000 Mbps duplex: full
Now let see how to change the settings. You can immediately disable advertisement of 100 half-duplex capability using ndd:
bash-3.00# ndd -set /dev/hme adv_cap_100hdx 0
Similarly you can do the same using dladm:
bash-3.00# dladm set-linkprop -p adv_1000hdx_cap=0 e1000g0
Note that in neither case will changes persist across reboot. To make the changes stick after a reboot you can edit /etc/system file and add approriate entries for your hardware:
set eri:adv_100hdx_cap=0
Another option is to edit driver’s .conf file in /kernel/drv directory. For example, /kernel/drv/e1000g.conf
Next option is to create an rc script in /etc/rc3.d directory that contains individual ndd or dladm commands.
One last option, which only work in Solaris 10 is described in dladm man page. You can create SMF manifest. This is straight from dladm man page:
<service_bundle type='manifest'name='apply_linkprop'> <service name='network/apply_linkprop' type='service' version='1'> <instance name='default' enabled='true'> <dependency name='dlmgmtd' grouping='require_all' restart_ov='none' type='service'> <service_fmri value='svc:/network/datalink-management:default' /> </dependency> <exec_method type='method' name='stop' exec=':true' timeout_seconds='3' /> <property_group name='startd' type='framework'> <propval name='duration' type='astring' value='transient' /> </property_group> </instance> <stability value='Evolving' /> </service> </service_bundle>
Store the manifest in /lib/svc/manifest/network. Then create method in /lib/svc/method that contains appropriate commands, for example:
dladm set-linkprop -p adv_1000hdx_cap=0 e1000g0
I think this last option is the most elegant as it nicely integrates with the system.
In: networking, solaris, solaris tips
Removing file or directory via inode number
While back I was doing some work with SVM. Everything went smoothly, but after I was done I experienced some bizarre issue. I am not entirely sure how it came to be. I needed to remove a temporary mountpoint called /tmpmnt:
bash-3.00# ls -l
lrwxrwxrwx 1 root root 9 Nov 29 2010 bin -> ./usr/bin
drwxr-xr-x 8 root sys 512 Nov 29 2010 boot
drwxr-xr-x 3 root nobody 512 Nov 30 2010 cdrom
drwxr-xr-x 18 root sys 4096 Jun 8 10:04 dev
drwxr-xr-x 2 root sys 512 Jun 8 2011 devices
drwxr-xr-x 88 root sys 4608 Jun 8 10:04 etc
drwxr-xr-x 3 root sys 512 Nov 30 2010 export
dr-xr-xr-x 1 root root 1 Jun 8 10:04 home
drwxr-xr-x 18 root sys 512 Nov 29 2010 kernel
drwxr-xr-x 8 root bin 5632 Nov 29 2010 lib
drwx------ 2 root root 8192 Nov 29 2010 lost+found
drwxr-xr-x 2 root sys 512 Nov 29 2010 mnt
dr-xr-xr-x 1 root root 1 Jun 8 10:04 net
drwxr-xr-x 5 root sys 512 Nov 30 2010 opt
drwxr-xr-x 5 root sys 512 Nov 29 2010 platform
dr-xr-xr-x 40 root root 128448 Jun 8 10:09 proc
drwxr-xr-x 2 root sys 1024 Nov 29 2010 sbin
drwxr-xr-x 4 root root 512 Nov 29 2010 system
drwxrwxrwt 4 root sys 256 Jun 8 10:09 tmp
drwxr-xr-x 2 root root 512 Jun 8 10:09 tmpmnt
drwxr-xr-x 40 root sys 1024 Nov 29 2010 usr
drwxr-xr-x 45 root sys 1024 Nov 29 2010 var
dr-xr-xr-x 6 root root 512 Jun 8 10:04 vol
But when I tried to do rm -Rf on the directory I got No such file or directory:
bash-3.00# rm -Rf ./tmpmnt
./tmpmnt: No such file or directory
I tried mv-ing the file and who knows what else, but I still could not get rid of that mountpoint. I figured maybe something has gone wrong with the filesystem. However, fsck returned with no problems. So, now what?
It happens that find command has this nice parameter called -inum. I could do find for the inode number of the directory and remove it with -exec parameter. First I got the inode number:
bash-3.00# ls -li
1747 lrwxrwxrwx 1 root root 9 Nov 29 2010 bin -> ./usr/bin
66 drwxr-xr-x 8 root sys 512 Nov 29 2010 boot
327353 drwxr-xr-x 3 root nobody 512 Nov 30 2010 cdrom
1805 drwxr-xr-x 18 root sys 4096 Jun 8 10:04 dev
3568 drwxr-xr-x 2 root sys 512 Jun 8 2011 devices
226 drwxr-xr-x 88 root sys 4608 Jun 8 10:04 etc
4 drwxr-xr-x 3 root sys 512 Nov 30 2010 export
14782 dr-xr-xr-x 1 root root 1 Jun 8 10:04 home
3087 drwxr-xr-x 18 root sys 512 Nov 29 2010 kernel
1918 drwxr-xr-x 8 root bin 5632 Nov 29 2010 lib
3 drwx------ 2 root root 8192 Nov 29 2010 lost+found
1926 drwxr-xr-x 2 root sys 512 Nov 29 2010 mnt
327217 dr-xr-xr-x 1 root root 1 Jun 8 10:04 net
1927 drwxr-xr-x 5 root sys 512 Nov 30 2010 opt
51 drwxr-xr-x 5 root sys 512 Nov 29 2010 platform
1928 dr-xr-xr-x 40 root root 128448 Jun 8 10:10 proc
1929 drwxr-xr-x 2 root sys 1024 Nov 29 2010 sbin
1934 drwxr-xr-x 4 root root 512 Nov 29 2010 system
1937 drwxrwxrwt 4 root sys 298 Jun 8 10:10 tmp
326783 drwxr-xr-x 2 root root 512 Jun 8 10:09 tmpmnt
31 drwxr-xr-x 40 root sys 1024 Nov 29 2010 usr
6 drwxr-xr-x 45 root sys 1024 Nov 29 2010 var
327300 dr-xr-xr-x 6 root root 512 Jun 8 10:04 vol
Then came the actual command to remove the file:
bash-3.00# find . -inum 326783 -mount -exec rm -Rf {} \;
bash-3.00# ls -l
lrwxrwxrwx 1 root root 9 Nov 29 2010 bin -> ./usr/bin
drwxr-xr-x 8 root sys 512 Nov 29 2010 boot
drwxr-xr-x 3 root nobody 512 Nov 30 2010 cdrom
drwxr-xr-x 18 root sys 4096 Jun 8 10:04 dev
drwxr-xr-x 2 root sys 512 Jun 8 2011 devices
drwxr-xr-x 88 root sys 4608 Jun 8 10:04 etc
drwxr-xr-x 3 root sys 512 Nov 30 2010 export
dr-xr-xr-x 1 root root 1 Jun 8 10:04 home
drwxr-xr-x 18 root sys 512 Nov 29 2010 kernel
drwxr-xr-x 8 root bin 5632 Nov 29 2010 lib
drwx------ 2 root root 8192 Nov 29 2010 lost+found
drwxr-xr-x 2 root sys 512 Nov 29 2010 mnt
dr-xr-xr-x 1 root root 1 Jun 8 10:04 net
drwxr-xr-x 5 root sys 512 Nov 30 2010 opt
drwxr-xr-x 5 root sys 512 Nov 29 2010 platform
dr-xr-xr-x 40 root root 128448 Jun 8 10:38 proc
drwxr-xr-x 2 root sys 1024 Nov 29 2010 sbin
drwxr-xr-x 4 root root 512 Nov 29 2010 system
drwxrwxrwt 4 root sys 338 Jun 8 10:38 tmp
drwxr-xr-x 40 root sys 1024 Nov 29 2010 usr
drwxr-xr-x 45 root sys 1024 Nov 29 2010 var
dr-xr-xr-x 6 root root 512 Jun 8 10:04 vol
As you can see the directory was gone. I ran fsck again, just in case and again I got no errors back.
In: centos, linux, linux tips, redhat, solaris, solaris tips
