Encountering eth0 NIC SerDES Link is Down

I was noticing this error on my HS22 Blade log files occasionally and on one occasion the NFS which was relying on the ethernet connection got disconnected and hang when the load is exceedingly high. The problem is that it is very hard to reproduce the problem as it is quite random

My Server is using the Broadcom chipset bnx2 and my version of my CentOS is 5.4 or kernel  version is 2.6.18-164.el5

After a bit of searching, this particular Red Hat Bugzilla (https://bugzilla.redhat.com/show_bug.cgi?id=520888) reflects the problem and workaround very well. I encourage you to take a closer look. If you are not planning to upgrade your RHEL or CentOS to 5.6 ( http://rhn.redhat.com/errata/RHSA-2011-0017.html ) and above yet, you may want to consider the workaround as mentioned in the bugzilla



 From Comments 14

"Configuring IRQ SMP affinity has no effect on some devices that use message signalled interrupts (MSI) with no MSI per-vector masking capability. Examples of such devices include Broadcom NetXtreme Ethernet devices that use the bnx2 driver. 

If you need to configure IRQ affinity for such a device, disable MSI by creating a file in /etc/modprobe.d/ containing the following line: 

options bnx2 disable_msi=1 

Alternatively, you can disable MSI completely using the kernel boot parameter pci=nomsi. (BZ#432451)

" http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5.4/html/Technical_Notes/Known_Issues-kernel.html

To Check whether you are still having issues, you can use the command

# dmesg |grep bnx2

I guess the best way is to update your broadcom drivers. For latest update on this "NIC SerDES Link is Down", see my writeup on Upgrading of Broadcom Drivers to resolve eth0 NIC SerDES Link is Down