Sunday, September 29, 2013

Device eth0 Doesn't Seem to be Present, Delaying Initialization

One of the most common problem that we face for a new installed or replaced NIC card is "device eth0 does not seem to be present, delaying initialization". You may also face this problem is you copy/move a VMware guest OS from one server to another.

Problem: device eth0 does not seem to be present, delaying initialization

Cause:

  • Newly installed or replaced NIC Card
  • Copy/Move a virtual hard disk to another guest OS with different NIC MAC address

For RHEL/CentOS:

First of all you must edit the network settings in network configuration file (which one you required, e.g. ifcfg-eth0 or ifcfg-eth1) and remove or edit with new NIC Mac Address:

[root@localhost ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
#HWADDR=00:0c:29:c6:ac:09  Old MAC Address
HWADDR=00:0c:29:46:b1:d7   # New MAC Address 

Resolution:

Temporary Solution:
Check /sys/class/net/ directory where you can see a symbolic link to NIC number

[root@localhost net]# cd /sys/class/net
[root@localhost net]# ll
lrwxrwxrwx 1 root root 0 Sep 29  2013 eth0 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:01.0/net/eth0
lrwxrwxrwx 1 root root 0 Sep 29  2013 eth1 -> ../../devices/pci0000:16/0000:00:11.0/0000:06:01.0/net/eth1
lrwxrwxrwx 1 root root 0 Sep 29  2013 lo -> ../../devices/virtual/net/lo

[root@localhost net]# ip link set dev eth1 name eth0 (assuming that old NIC was eth0 and new one is detected as eth1)

This will solve the problem for the time being. But you have to keep in mind that this is a temporary solution. Whenever you reboot the pc/server all the settings will go back to its original settings.


Persistent Solution:

Edit 70-persistent-net.rules file, disable the old entry and add a new entry for new NIC MAC Address:


[root@localhost net]# vim /etc/udev/rules.d/70-persistent-net.rules 
# PCI device 0x1022:0x2000 (pcnet32)
#SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:0c:29:c6:ac:09", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"   //Old one. Add # to disable.

# PCI device 0x1022:0x2000 (pcnet32) (custom name provided by external tool).
#Add this entry for your new NIC
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:0c:29:46:b1:d7", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"
[root@localhost net]# 

Cheers!!!


*Thanks to suarez & google

Sunday, September 22, 2013

Apache Error [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)

One fine morning I have discovered that my Apache (httpd) service is down and can't be started :)

Step 1: Checking httpd Service Status:
#/etc/init.d/httpd status
httpd dead but pid file exists

Step 2: Search httpd processes and kill those
#ps -ef | grep httpd

# killall -9 httpd

Step 3: Remove httpd lock file if exist
# rm -f /var/lock/subsys/httpd

Step4: Restart Apache/httpd Service
# service httpd restart
Stop httpd    [FAILED]
Start httpd   [FAILED]


Need to dig down more.

Step 5: Check httpd error log file
# grep suexec /var/log/httpd/error_log
[Sun Sep 22 10:32:06 2013] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Sun Sep 22 10:33:51 2013] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Sun Sep 22 10:38:03 2013] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Sun Sep 22 10:45:53 2013] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)

Step 6: Check httpd nss error log file
#tail -300 /var/log/httpd/nss_error_log
[Sun Sep 22 04:03:27 2013] [error] Certificate not verified: 'Server-Cert'
[Sun Sep 22 04:03:27 2013] [error] SSL Library Error: -8181 Certificate has expired
[Sun Sep 22 04:03:27 2013] [error] Unable to verify certificate 'Server-Cert'. Add "NSSEnforceValidCerts off" to nss.conf so the server can start until the problem can be resolved.
[Sun Sep 22 10:38:03 2013] [error] Certificate not verified: 'Server-Cert'
[Sun Sep 22 10:38:03 2013] [error] SSL Library Error: -8181 Certificate has expired
[Sun Sep 22 10:38:03 2013] [error] Unable to verify certificate 'Server-Cert'. Add "NSSEnforceValidCerts off" to nss.conf so the server can start until the problem can be resolved.
[Sun Sep 22 10:45:53 2013] [error] Certificate not verified: 'Server-Cert'
[Sun Sep 22 10:45:53 2013] [error] SSL Library Error: -8181 Certificate has expired

[Sun Sep 22 10:45:53 2013] [error] Unable to verify certificate 'Server-Cert'. Add "NSSEnforceValidCerts off" to nss.conf so the server can start until the problem can be resolved.


So, the root cause of this issue is in a module called nss. mod_nss is an alternative to mod_ssl; it does a few things that mod_ssl doesn't do, and can run along side mod_ssl.

Step 7: Remove nss.conf file from httpd conf.d directory
#cd /etc/httpd/conf.d
#mv nss.conf nss.conf.orig
#/etc/init.d/httpd start
Start httpd   [OK]

Saturday, September 7, 2013

Install ntop on Linux Environment (RHEL/CentOS)


What is ntop

ntop is a network and traffic analyzer that provides a wealth of information on various networking hosts and protocols. ntop is primarily accessed via a built-in web interface.
Following instructions are tested on 32/64 bit versions only:
a) RHEL Linux 5.x
b) CentOS Linux 5.x


Download latest ntop


Visit ntop project to grab latest version. You can use wget to grab the same, enter:
#cd /opt
#wget http://freshmeat.net/redir/ntop/7279/url_tgz/ntop-3.3.6.tar.gz
Untar tar ball, enter:
#tar -zxvf ntop-3.3.6.tar.gz

Configure and Compile ntop under RHEL/CentOS

You must have RRDTool installed. You also need to install libpcap, enter:
# yum install libpcap-devel libpcap
Type the following commands to compile and install ntop:
#cd ntop
#./autogen.sh
Just type make to compile ntop:
#make
Just type make install to install ntop:
#make install
#make install-data-as

Create ntop user

Type the following command to run ntop as ntop user, enter:
#useradd -M -s /sbin/nologin -r ntop

Setup directory permissions

Next, you need to setup directory permissions, enter:
#chown ntop:root /usr/local/var/ntop/
#chown ntop:ntop /usr/local/share/ntop/

Setup ntop user admin password

Type the following command to set ntop admin password, enter:
#ntop -A
Sample output:
Mon Jul 28 03:38:34 2008  NOTE: Interface merge enabled by default
Mon Jul 28 03:38:34 2008  Initializing gdbm databases
ntop startup - waiting for user response!
Please enter the password for the admin user:
Please enter the password again:
Mon Jul 28 03:38:42 2008  Admin user password has been set

Start ntop

Type the following command to start ntop:
# /usr/local/bin/ntop -d -L -u ntop -P /usr/local/var/ntop --skip-version-check --use-syslog=daemon
Sample output:
Mon Jul 28 03:42:19 2008  NOTE: Interface merge enabled by default
Mon Jul 28 03:42:19 2008  Initializing gdbm databases
If you have multiple interface (eth0, eth1 and so on), start ntop as follows:
# /usr/local/bin/ntop -i "eth0,eth1" -d -L -u ntop -P /usr/local/var/ntop --skip-version-check --use-syslog=daemon
Where,
  • -i “eth0,eth1″ : Specifies the network interface or interfaces to be used by ntop for network monitoring. Here you are monitoring eth0 and eth1.
  • -d : Run ntop as a daemon.
  • -L : Send all log messages to the system log (/var/log/messages) instead of screen.
  • -u ntop : Start ntop as ntop user
  • -P /usr/local/var/ntop : Specify where ntop stores database files. You may need to backup database as part of your disaster recovery program.
  • –skip-version-check : By default, ntop accesses a remote file to periodically check if the most current version is running. This option disables that check.
  • –use-syslog=daemon : Use syslog daemon.

How do I view ntop stats?

By default ntop listen on 3000 port. You can view ntop stats by visiting following url:
http://localhost:3000/
OR
http://server-ip:3000/
Open port 3000 using iptables
Open /etc/sysconfig/iptables file, enter:
# vi /etc/sysconfig/iptables
Append following code before final REJECT line:
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 3000 -j ACCEPT
Save and close the file. Restart firewall:
# service iptables restart

How do I view ntop stats without opening port 3000?

Setup simple tunnel using ssh, enter the following on your local UNIX / Linux desktop system:
$ ssh -L 3000:localhost:3000 -N -f user@server.yourcorp.com
Now open browser and type the following command:
http://localhost:3000/

How do I start ntop on boot?

Open /etc/rc.local file, enter:
# vi /etc/rc.local
Append the following line:
/usr/local/bin/ntop -i "eth0,eth1" -d -L -u ntop -P /usr/local/var/ntop --skip-version-check --use-syslog=daemon
Save and close the file.

How do I stop ntop?

Use web interface to shutdown ntop, or use normal kill / killall command:
# killall ntop

Cheers!!

Configure MRTG Server in Linux Environment

What is MRTG:
The Multi Router Traffic Grapher (MRTG) is a powerful open source tool to monitor the traffic load on network links. MRTG generates HTML pages containing PNG images which provide a LIVE visual representation of this traffic. MRTG is written in perl and works on Unix/Linux as well as Windows and even Netware systems. MRTG is free software licensed under the Gnu GPL.
Go to http://oss.oetiker.ch/mrtg for all the details about MRTG and download.

Step # 1 : Make sure snmp server installed
Please note that snmpd configuration does not require using mrtg with remote network devices such as Routers and switches. If you just want mrtg graphs for router or switch then please refer to step# 4 (as all these devices comes preconfigured with snmpd software).
Run rpm command query option to find out whether snmp service is installed or not:
# rpm -qa | grep snmp
If snmp installed then please go to step # 2; otherwise you need to install them using following steps (login as a root user):

(a) Visit rpmfind.net to get snmp server and utility rpms’s. If you are a fedora, centos user then use yum command as follows to install it:
# yum install net-snmp-utils net-snmp
(b) If you are RHEL subscriber then use up2date command as follows to install:
#up2date -v -i net-snmp-utils net-snmp
Step # 2 : Determine if snmp server is running or not
Run ‘ps’ command to see if snmp server is running or not:
# ps -aux | grep snmp
Output:
root 5512 0.0 2.3 5872 3012 pts/0 S 22:04 0:00 /usr/sbin/snmpd
Alternatively, you can try any of the following two commands as well:
# lsof -i :199
Output:
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
snmpd 5512 root 4u IPv4 34432 TCP *:smux (LISTEN)
OR try out netstat command:
# netstat -natv | grep ‘:199′
Output:
tcp 0 0 0.0.0.0:199 0.0.0.0:* LISTEN
If you found service is running or listing on port 199 then please see step #3; otherwise start service using following command:
# service snmpd start
Make sure snmpd service starts automatically if the system reboots (add snmpd service):
# chkconfig –add snmpd
Open snmpd.conf file to add/edit community string:
#vi /etc/snmp/snmpd.conf
Add the below line in the file, save and exit:
ROCOMMUNITY didar15
Step3: Making snmp to walk :)
Before doing smnpwalk, login to the device you want to monitor e.g. switch, router, linux box, windows2003 server and add the same community string that you have entered few minutes ago in snmpd.conf file.

# snmpwalk -v 1 -c didar(community name) a.b.c.d(device ip e.g. router, switch)
Step4: Making document root to store mrtg graphs/html pages
#cd /var/www/html/
#mkdir -p mrtg
Step5: Creating configuration File
#cfgmaker –global ‘Workdir: /var/www/html/mrtg’ –global ‘Options[_] : bits,growright’ –output /etc/mrtg/sw.cfg didar15@A.B.C.D
syntax: cfgmaker –global ‘Workdir: ’ –global ‘Options[_] : bits,growright’ –output @
Step6: Creating Index File
#indexmaker –output= /var/www/html/mrtg/index.html /etc/mrtg/sw.cfg
Step7: Set Language Support
#env LANG=C /usr/bin/mrtg /etc/mrtg/sw.cfg
Step8: Adding a cron job to update the graph in every five minute
#crontab -e (Add the following line at the end of the file)
*/5 **** /usr/bin/mrtg /etc/mrtg/sw.cfg –logging /var/log/mrtg.log
Cheers!!

Friday, May 3, 2013

SSL Certificate Integration with Tomcat

Objective:
Lets assume our domain is: example.com.bd
When user requests http://example.com.bd , it will automatically redirected to https://example.com.bd

Tested Environment:
OS: RHEL5.4 2.6.18-164.el5PAE
Tomcat: Version jakarta-tomcat-4.1.20 - jakarta-tomcat-4.1.24
Supported Command: keytool

Work Steps:
1. CSR (Code Signing Request) & Keystore file Generation
2. Certificate Installation
3. Tomcat Configuration
4. Restart Tomcat Web Service


1. CSR (Code Signing Request) & Keystore file Generation
#cd /cert
#keytool -genkey -keyalg RSA -keysize 2048 -keystore /cert/example.com.bd.keystore
Enter keystore password: 123987

You will prompt some questions regarding your domain. You can use to get your domain details


What is your first and last name?
[Unknown]: example.com.bd
What is the name of your organizational unit?
[Unknown]: IT/Technology
What is the name of your organization?
[Unknown]: Example Holdings Pvt. Ltd.
What is the name of your City or Locality?
[Unknown]: 1, South Gulshan Avenue
What is the name of your State or Province?
[Unknown]: Dhaka
What is the two-letter country code for this unit?
[Unknown]: BD
Is CN=example.com.bd, OU=IT/Technology, O=Example Holdings Pvt. Ltd., 1, South Gulshan Avenue, ST=Dhaka, C=BD correct?
[no]: yes

Enter key password for
(RETURN if same as keystore password): 123987

#keytool -certreq -keyalg RSA -file /cert/example.com.bd.csr -keystore /cert/example.com.bd.keystore
Enter keystore password: 123987


2.Certificate Installation

There are different SSL certificate issuer e.g. verisign, COMODO etc.
You will receive following certificates according to your CSR file:

AddTrustExternalCARoot.crt
UTNAddTrustServerCA.crt or e.g. COMODOHigh-AssuranceSecureServerCA.crt or www.verisign.com/CPS Incorp.by Ref. LIABILITY LTD.(c)97 VeriSign.crt
example_com_bd.crt

Execute following commands from the directory where existing csr and keystore files are located:
#cd /cert
#keytool -import -trustcacerts -alias AddTrustExternalCARoot -file AddTrustExternalCARoot.crt -keystore example.com.bd.keystore

#keytool -import -trustcacerts -alias UTNAddTrust -file UTNAddTrustServerCA.crt -keystore example.com.bd.keystore


OR

#keytool -import -trustcacerts -alias COMODOHigh-AssuranceSecureServerCA -file COMODOHigh-AssuranceSecureServerCA.crt -keystore example.com.bd.keystore

#keytool -import -trustcacerts -alias example.com.bd -file example_com_bd.crt -keystore example.com.bd.keystore

#keytool -import -trustcacerts -file example_com_bd.crt -keystore example.com.bd.keystore



3. Tomcat Configuration
Edit your tomcat configuration file
#vim $TOMCAT_HOME/conf/server.xml


4. Restart Tomcat

**Gratitude to

Wednesday, January 9, 2013

Add New Disk Space to a Virtual Machine in Linux - in VMware

Among many FAQ; very familiar question in VM is how to add new disks. Here are the plain and simple steps to do that:

Objective: Add new disk space in a virtual machine in Linux.

Step 1: Open/Edit virtual machine settings:
Select your virtual machine, and press “Edit virtual machine settings’
Open/Edit Virtual Machine Settings

Step 2: Add new hardware:
From Hardware section click "Add" > "Next"
Add Hardware

Step 3: Select new Hard Disk:
From Hardware type select Hard Disk and click "Next"
Add Hardware

Step 4: Create the Disk Type:
Recommended Virtual Disk Type should be SCSI (don't know why :-P). Select the Disk type from available options:
Disk Type

Step 5: Set Disk Capacity & Options:
Now set the capacity you want to create. One of the nice features of VMWare is that you don’t have to allocate all of the disk space when you create it. So if you create a 20GB disk it won't take all spaces right away (unless you check the box), the disk will grow as your virtual machine needs it. Lastly, you can split the files into 2GB sizes, while this isn’t necessary, it just makes all the disks much easier to manage and move around. In example we are adding 20GB.
Disk Capacity

Step 6: Specify VM Disk File:
VM Disk File

Step 7: Ensure newly add Disk exists in Hardware Option:
HDD Added

Step 8: Boot & Power On the Virtual Machine.

Step 9: Create new Partition:
Login as root. In Linux the first SCSI drive is sda, the second sdb, the third sdc, etc. since this was the second SCSI drive we added in the VM, the device is known as /dev/sdb

[root@localhost ~]# fdisk /dev/sdb
Press m for help
To create a new partition, press n
This is going to be a primary partition so press p
Partition number is 1
Enter first Cylinder Number:
Enter Last Cylinder number: +20480M (as because our disk size is 20GB)
Press w for write and save the partition table.

Step 10: Format the partition:
Now we have to format the partition with new file system. We’ve decided to use ext3 file system for newly added disk.

[root@localhost ~]# mkfs -t ext3 /dev/sdb1
Above command changes the file system type to ext3 for /dev/sdb1 partition (-t fstype) (man mkfs)

Step 11: Creating Mount Point for new partition:
Create a new directory named app under /
[root@localhost ~]# mkdir -p /app
[root@localhost ~]# mount -t ext3 /dev/sdb1 /app


check whether the partition is properly mounted by running the following command
[root@localhost ~]# df -k

Step 12: Edit fstab for permanent entry:
mounting wont be permanent unless you add a fstab entry
/dev/sdb1 /app ext3 defaults 1 1

ENJOY :-)

Restore Archived Log into VMware Aria Operations for Logs (formerly known as vRealize Log Insight - vRLI)

As we cannot keep all logs in searchable space in vRLI production system due to performance and slowness issue, it is always recommended to ...