Device eth0 Doesn't Seem to be Present, Delaying Initialization

One of the most common problem that we face for a new installed or replaced NIC card is "device eth0 does not seem to be present, delaying initialization". You may also face this problem is you copy/move a VMware guest OS from one server to another.

Problem: device eth0 does not seem to be present, delaying initialization

Cause:

• Newly installed or replaced NIC Card
• Copy/Move a virtual hard disk to another guest OS with different NIC MAC address

For RHEL/CentOS:

First of all you must edit the network settings in network configuration file (which one you required, e.g. ifcfg-eth0 or ifcfg-eth1) and remove or edit with new NIC Mac Address:

[root@localhost ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0

#HWADDR=00:0c:29:c6:ac:09  Old MAC Address

HWADDR=00:0c:29:46:b1:d7   # New MAC Address 

Resolution:

Temporary Solution:

Check /sys/class/net/ directory where you can see a symbolic link to NIC number

[root@localhost net]# cd /sys/class/net

[root@localhost net]# ll

lrwxrwxrwx 1 root root 0 Sep 29  2013 eth0 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:01.0/net/eth0

lrwxrwxrwx 1 root root 0 Sep 29  2013 eth1 -> ../../devices/pci0000:16/0000:00:11.0/0000:06:01.0/net/eth1

lrwxrwxrwx 1 root root 0 Sep 29  2013 lo -> ../../devices/virtual/net/lo

[root@localhost net]# ip link set dev eth1 name eth0 (assuming that old NIC was eth0 and new one is detected as eth1)

This will solve the problem for the time being. But you have to keep in mind that this is a temporary solution. Whenever you reboot the pc/server all the settings will go back to its original settings.

Persistent Solution:

Edit 70-persistent-net.rules file, disable the old entry and add a new entry for new NIC MAC Address:

[root@localhost net]# vim /etc/udev/rules.d/70-persistent-net.rules 

# PCI device 0x1022:0x2000 (pcnet32)

#SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:0c:29:c6:ac:09", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"   //Old one. Add # to disable.

# PCI device 0x1022:0x2000 (pcnet32) (custom name provided by external tool).

#Add this entry for your new NIC

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:0c:29:46:b1:d7", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

[root@localhost net]# 

Cheers!!!

*Thanks to suarez & google

Apache Error [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)

One fine morning I have discovered that my Apache (httpd) service is down and can't be started :)

Step 1: Checking httpd Service Status:
#/etc/init.d/httpd status
httpd dead but pid file exists

Step 2: Search httpd processes and kill those
#ps -ef | grep httpd

# killall -9 httpd

Step 3: Remove httpd lock file if exist
# rm -f /var/lock/subsys/httpd

Step4: Restart Apache/httpd Service
# service httpd restart
Stop httpd    [FAILED]
Start httpd   [FAILED]

Need to dig down more.

Step 5: Check httpd error log file

# grep suexec /var/log/httpd/error_log
[Sun Sep 22 10:32:06 2013] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Sun Sep 22 10:33:51 2013] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Sun Sep 22 10:38:03 2013] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Sun Sep 22 10:45:53 2013] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)

Step 6: Check httpd nss error log file
#tail -300 /var/log/httpd/nss_error_log
[Sun Sep 22 04:03:27 2013] [error] Certificate not verified: 'Server-Cert'
[Sun Sep 22 04:03:27 2013] [error] SSL Library Error: -8181 Certificate has expired
[Sun Sep 22 04:03:27 2013] [error] Unable to verify certificate 'Server-Cert'. Add "NSSEnforceValidCerts off" to nss.conf so the server can start until the problem can be resolved.
[Sun Sep 22 10:38:03 2013] [error] Certificate not verified: 'Server-Cert'
[Sun Sep 22 10:38:03 2013] [error] SSL Library Error: -8181 Certificate has expired
[Sun Sep 22 10:38:03 2013] [error] Unable to verify certificate 'Server-Cert'. Add "NSSEnforceValidCerts off" to nss.conf so the server can start until the problem can be resolved.
[Sun Sep 22 10:45:53 2013] [error] Certificate not verified: 'Server-Cert'
[Sun Sep 22 10:45:53 2013] [error] SSL Library Error: -8181 Certificate has expired

[Sun Sep 22 10:45:53 2013] [error] Unable to verify certificate 'Server-Cert'. Add "NSSEnforceValidCerts off" to nss.conf so the server can start until the problem can be resolved.

So, the root cause of this issue is in a module called nss. mod_nss is an alternative to mod_ssl; it does a few things that mod_ssl doesn't do, and can run along side mod_ssl.

Step 7: Remove nss.conf file from httpd conf.d directory
#cd /etc/httpd/conf.d
#mv nss.conf nss.conf.orig
#/etc/init.d/httpd start
Start httpd   [OK]

Install ntop on Linux Environment (RHEL/CentOS)

What is ntop

ntop is a network and traffic analyzer that provides a wealth of information on various networking hosts and protocols. ntop is primarily accessed via a built-in web interface.

Following instructions are tested on 32/64 bit versions only:

a) RHEL Linux 5.x

b) CentOS Linux 5.x

Download latest ntop

Visit ntop project to grab latest version. You can use wget to grab the same, enter:
#cd /opt
#wget http://freshmeat.net/redir/ntop/7279/url_tgz/ntop-3.3.6.tar.gz

Untar tar ball, enter:
#tar -zxvf ntop-3.3.6.tar.gz

Configure and Compile ntop under RHEL/CentOS

You must have RRDTool installed. You also need to install libpcap, enter:
# yum install libpcap-devel libpcap

Type the following commands to compile and install ntop:
#cd ntop
#./autogen.sh

Just type make to compile ntop:
#make

Just type make install to install ntop:
#make install
#make install-data-as

Create ntop user

Type the following command to run ntop as ntop user, enter:
#useradd -M -s /sbin/nologin -r ntop

Setup directory permissions

Next, you need to setup directory permissions, enter:
#chown ntop:root /usr/local/var/ntop/
#chown ntop:ntop /usr/local/share/ntop/

Setup ntop user admin password

Type the following command to set ntop admin password, enter:
#ntop -A

Sample output:

Mon Jul 28 03:38:34 2008  NOTE: Interface merge enabled by default
Mon Jul 28 03:38:34 2008  Initializing gdbm databases
ntop startup - waiting for user response!
Please enter the password for the admin user:
Please enter the password again:
Mon Jul 28 03:38:42 2008  Admin user password has been set

Start ntop

Type the following command to start ntop:
# /usr/local/bin/ntop -d -L -u ntop -P /usr/local/var/ntop --skip-version-check --use-syslog=daemon
Sample output:

Mon Jul 28 03:42:19 2008  NOTE: Interface merge enabled by default
Mon Jul 28 03:42:19 2008  Initializing gdbm databases

If you have multiple interface (eth0, eth1 and so on), start ntop as follows:
# /usr/local/bin/ntop -i "eth0,eth1" -d -L -u ntop -P /usr/local/var/ntop --skip-version-check --use-syslog=daemon
Where,

• -i “eth0,eth1″ : Specifies the network interface or interfaces to be used by ntop for network monitoring. Here you are monitoring eth0 and eth1.
• -d : Run ntop as a daemon.
• -L : Send all log messages to the system log (/var/log/messages) instead of screen.
• -u ntop : Start ntop as ntop user
• -P /usr/local/var/ntop : Specify where ntop stores database files. You may need to backup database as part of your disaster recovery program.
• –skip-version-check : By default, ntop accesses a remote file to periodically check if the most current version is running. This option disables that check.
• –use-syslog=daemon : Use syslog daemon.

How do I view ntop stats?

By default ntop listen on 3000 port. You can view ntop stats by visiting following url:
http://localhost:3000/
OR
http://server-ip:3000/

Screenshot: ntop Global TCP/UDP Protocol Distribution Graphs

Open port 3000 using iptables

Open /etc/sysconfig/iptables file, enter:
# vi /etc/sysconfig/iptables
Append following code before final REJECT line:
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 3000 -j ACCEPT
Save and close the file. Restart firewall:
# service iptables restart

How do I view ntop stats without opening port 3000?

Setup simple tunnel using ssh, enter the following on your local UNIX / Linux desktop system:
$ ssh -L 3000:localhost:3000 -N -f user@server.yourcorp.com
Now open browser and type the following command:
http://localhost:3000/

How do I start ntop on boot?

Open /etc/rc.local file, enter:
# vi /etc/rc.local
Append the following line:
/usr/local/bin/ntop -i "eth0,eth1" -d -L -u ntop -P /usr/local/var/ntop --skip-version-check --use-syslog=daemon
Save and close the file.

How do I stop ntop?

Use web interface to shutdown ntop, or use normal kill / killall command:
# killall ntop

Cheers!!

Configure MRTG Server in Linux Environment

What is MRTG:

The Multi Router Traffic Grapher (MRTG) is a powerful open source tool to monitor the traffic load on network links. MRTG generates HTML pages containing PNG images which provide a LIVE visual representation of this traffic. MRTG is written in perl and works on Unix/Linux as well as Windows and even Netware systems. MRTG is free software licensed under the Gnu GPL.

Go to http://oss.oetiker.ch/mrtg for all the details about MRTG and download.

Step # 1 : Make sure snmp server installed
Please note that snmpd configuration does not require using mrtg with remote network devices such as Routers and switches. If you just want mrtg graphs for router or switch then please refer to step# 4 (as all these devices comes preconfigured with snmpd software).

Run rpm command query option to find out whether snmp service is installed or not:

# rpm -qa | grep snmp

If snmp installed then please go to step # 2; otherwise you need to install them using following steps (login as a root user):

(a) Visit rpmfind.net to get snmp server and utility rpms’s. If you are a fedora, centos user then use yum command as follows to install it:

# yum install net-snmp-utils net-snmp

(b) If you are RHEL subscriber then use up2date command as follows to install:

#up2date -v -i net-snmp-utils net-snmp

Step # 2 : Determine if snmp server is running or not
Run ‘ps’ command to see if snmp server is running or not:

# ps -aux | grep snmp
Output:
root 5512 0.0 2.3 5872 3012 pts/0 S 22:04 0:00 /usr/sbin/snmpd

Alternatively, you can try any of the following two commands as well:

# lsof -i :199
Output:
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
snmpd 5512 root 4u IPv4 34432 TCP *:smux (LISTEN)

OR try out netstat command:

# netstat -natv | grep ‘:199′
Output:
tcp 0 0 0.0.0.0:199 0.0.0.0:* LISTEN

If you found service is running or listing on port 199 then please see step #3; otherwise start service using following command:
# service snmpd start

Make sure snmpd service starts automatically if the system reboots (add snmpd service):
# chkconfig –add snmpd

Open snmpd.conf file to add/edit community string:
#vi /etc/snmp/snmpd.conf

Add the below line in the file, save and exit:
ROCOMMUNITY didar15

Step3: Making snmp to walk 

Before doing smnpwalk, login to the device you want to monitor e.g. switch, router, linux box, windows2003 server and add the same community string that you have entered few minutes ago in snmpd.conf file.

# snmpwalk -v 1 -c didar(community name) a.b.c.d(device ip e.g. router, switch)

Step4: Making document root to store mrtg graphs/html pages
#cd /var/www/html/
#mkdir -p mrtg

Step5: Creating configuration File
#cfgmaker –global ‘Workdir: /var/www/html/mrtg’ –global ‘Options[_] : bits,growright’ –output /etc/mrtg/sw.cfg didar15@A.B.C.D

syntax: cfgmaker –global ‘Workdir: ’ –global ‘Options[_] : bits,growright’ –output @

Step6: Creating Index File
#indexmaker –output= /var/www/html/mrtg/index.html /etc/mrtg/sw.cfg

Step7: Set Language Support
#env LANG=C /usr/bin/mrtg /etc/mrtg/sw.cfg

Step8: Adding a cron job to update the graph in every five minute
#crontab -e (Add the following line at the end of the file)

*/5 **** /usr/bin/mrtg /etc/mrtg/sw.cfg –logging /var/log/mrtg.log

Cheers!!