Compiling Chelseio IWARP Drivers (2.8.0.0) on Chelsio T4 Cards on CentOS 5

The below is a subset of the Chelsio 2.8.0.0 ReadMe

The Chelsio Unified Wire software has been developed to run on 64-bit Linux based platforms. Following is the list of Drivers/Software and supported Linux distributions. Here is a subset of the README.

The OS I used was CentOS 5.8

|########################|#####################################################|
|   Linux Distribution   |                Driver/Software                      |
|########################|#####################################################|
|RHEL5.8,2.6.18-308.el5  |NIC/TOE,vNIC,iWARP,WD-UDP*,WD-TOE*,iSCSI Target*,    |
|                        |Bonding,IPv6,Bypass*,Sniffer & Tracer                |
|                        |UM(Agent,Client),UDP-SO,Filtering,TM                 |
|------------------------|-----------------------------------------------------|
|RHEL5.9,2.6.18-348.el5  |NIC/TOE*,vNIC*,iWARP*,WD-UDP*,WD-TOE*,iSCSI Target*, |
|                        |Bonding*,IPv6*,Bypass*,Sniffer & Tracer*,UDP-SO*,    |
|                        |Filtering*,TM*                                       |
|------------------------|-----------------------------------------------------|
|RHEL6.3,                |NIC/TOE,vNIC,iWARP,WD-UDP,WD-TOE*,iSCSI Target*,     |
|2.6.32-279.el6          |iSCSI Initiator*,FCoE Initiator*,                    |
|                        |Bonding,IPv6,Bypass*,Sniffer & Tracer,UDP-SO,        |
|                        |UM(Agent,Client,WebGUI),Filtering,TM                 |
|------------------------|-----------------------------------------------------|
|RHEL6.4,                |NIC/TOE,vNIC,iWARP,WD-UDP,WD-TOE,iSCSI Target,       |
|2.6.32-358.el6          |iSCSI Initiator,FCoE Initiator,Bonding,IPv6,Bypass,  |
|                        |Sniffer & Tracer,UDP-SO,UM(Agent,Client,WebGUI),     |
|                        |Filtering,TM,uBoot(DUD)                              |
|------------------------|-----------------------------------------------------|

Strangely, I was not able to compile with 3.5.1. It seems that the compat-rdma on 3.5.1 is having issues with CentOS 5.8. See Failed to build compat-rdma RPM when compiling OFED 3.5.1 on CentOS 5.8

I tried with OFED 1.5.4.1, but errors occurred as well. But compiling OFED 1.5.3.2 works well and Chelsio T420-BCH was able to compile nicely with OFED 1.5.3.2. To download OFED 1.5.3.2, do visit the OFED Downloads Site

---

Part 1

To compile from source

 i.  Download the tarball ChelsioUwire-x.x.x.x.tar.gz

ii. Untar the tarball
  
[root@host]# tar zxvfm ChelsioUwire-x.x.x.x.tar.gz
 
iii. Change your current working directory to Chelsio Unified Wire package
     directory. Build the source:

[root@host]# make
 
iv. Install the drivers, tools and libraries:
  
[root@host]# make install
 
v. The default configuration tuning option is Unified Wire.
   The configuration tuning can be selected using the following commands:

[root@host]# make CONF=(T5/T4 Configuration)
[root@host]# make CONF=(T5/T4 Configuration install)

(where T5/T4 Configuration is
UNIFIED_WIRE, HIGH_CAPACITY_TOE, HIGH_CAPACITY_RDMA, LOW_LATENCY, UDP_OFFLOAD, T5_WIRE_DIRECT_LATENCY)

---

Part 2  - Installing Individual Drivers

i. To build and install iWARP driver against outbox OFED:
[root@host]# make iwarp
[root@host]# make iwarp_install

---

Part 3a - Loading IWARP Drivers

Manually  Load  Drivers
To load the iWARP driver we need to load the NIC driver & core RDMA drivers first:
[root@host]# modprobe cxgb4
[root@host]# modprobe iw_cxgb4
[root@host]# modprobe rdma_ucm

Part 3b - Automatic IWARP Drivers
To load the Chelsio iWARP drivers automatically, add this additional lines to /etc/modprobe.conf

options iw_cxgb4 peer2peer=1
install cxgb4 /sbin/modprobe -i cxgb4; /sbin/modprobe -f iw_cxgb4; /sbin/modprobe rdma_ucm
alias eth1 cxgb4 # assuming eth1 is used by the Chelsio interface

---

Finally Reboot the system to load the new modules

References:

1. Chelsio 2.8.0.0 ReadMe