Wednesday, September 11, 2013

Mellanox OFED 2.0-3.0.0 Linux Driver is now available.

Mellanox OFED 2.0-3.0.0 Linux Driver is now available.
MLNX_OFED 2.0-3.0.0 main features:
  1. SR-IOV for InfiniBand Ethernet         
  2. RoCE Over SR-IOV
  3. Port base QoS for both Ethernet and InfiniBand
  4. eIpoIB
  5. Connect-IB™
  6. Low Latency Socket
  7. 56GbE
  8. XRC
  9. Dynamically Connected Transport ( DCT)  - alpha level
  10. Flow Steering

MLNX_OFED 2.0-3.0.0 Download and Manual Page
MLNX_OFED 2.0-3.0.0 Release notes

 Differences between MLNX_OFED 1.5.3 and MLNX_OFED 2.0:
  1. MLNX_OFED 2.0 is based on Kernel 3.7 which is a huge leap forward from the 2.6.30 Kernel that the 1.5.x is based on.
  2. MLNX_OFED 2.0 supports the operating systems that are based on newer Kernels such as RHEL 6.x and SLES 11 SPx. MLNX_OFED 1.5.x will continue to support OS’s based on older kernels such as RHEL 5.x and SLES 10 SPx.
  3. MLNX_OFED 2.0 includes new functionality, 1.5.x does not.

Full Support Matrix for Both MLNX_OFED Branches 

For more information, do read
  1. Online version of Product and Support Updates Notification

Sunday, September 8, 2013

TORQUE Security Advisory - 6 September 2013


Vulnerability: A non-privileged user who can run jobs or login to a node running pbs_server or pbs_mom can submit an arbitrary job to the cluster; that job can run as root. The user can submit a command directly to a pbs_mom daemon to queue and run a job. A malicious user could use this vulnerability to remotely execute code as root on the cluster.

Versions Affected: All versions of TORQUE

Mitigating Factors:
- The user must be logged in on a node that is already legitimately able to contact pbs_mom daemons or submit jobs.
- If a user submits a job via this defect and pbs_server is running, pbs_server will kill the job unless job syncing is disabled. It may take up to 45 seconds for pbs_server to kill the job.
- There are no known instances of this vulnerability being exploited.

Remedy: All TORQUE users should patch their systems using the following instructions:

For 2.5.x versions of TORQUE:
----------------------------
1. Download the patch file:
     $ wget http://www.adaptivecomputing.com/torquepatch/fix_mom_priv_2.5.patch
2. Run the patch command in the root directory of the TORQUE source tree:
     $ patch -p1 < fix_mom_priv_2.5.patch
3. Recompile TORQUE:
     $ make
4. Install TORQUE:
     $ sudo make install
5. Restart pbs_mom (pbs_server is not affected)


For 4.x versions of TORQUE:
----------------------------
1. Download the patch file:
     $ wget http://www.adaptivecomputing.com/torquepatch/fix_mom_priv.patch
2. Run the patch command in the root directory of the TORQUE source tree:
     $ patch -p1 < fix_mom_priv.patch
3. Recompile TORQUE:
     $ make
4. Install TORQUE:
     $ sudo make install
5. Restart pbs_mom (pbs_server is not affected)

What the Patch Does: The patch checks that the connection to the pbs_mom daemon is coming from a privileged port. This follows the security model that only privileged users should be able to submit arbitrary jobs.

Attribution: This vulnerability was discovered by John Fitzpatrick of MWR InfoSecurity. Matt Ezell of Oak Ridge National Laboratory assisted in creating the patch. We thank these individuals for helping to improve TORQUE.

Thursday, September 5, 2013

Encountering __intel_sse2_strlen when compiling OpenMPI with Intel Compilers

I was compiling OpenMPI 1.6.x and Intel Compiler XE 2013, I used the following configuration found in

#./configure --prefix=/usr/local CC=icc CXX=icpc F77=ifort FC=ifort
Do specify CC=icc CXX=icpc F77=ifort FC=ifort as if you miss one of the parameters, it will take the default. For example, if you do not specify CXX=icpc, it will adopt CXX=g++
but when I run
# make -j8

I got this error:
..../openmpi-1.6.2/ompi/contrib/vt/vt/tools/opari/tool/../../../util/util.c:45: 
undefined reference to `__intel_sse2_strlen'
..../openmpi-1.6.2/ompi/contrib/vt/vt/tools/opari/tool/../../../util/util.c:61: 
undefined reference to `__intel_sse2_strlen'
..../openmpi-1.6.2/ompi/contrib/vt/vt/tools/opari/tool/../../../util/util.c:46: 
undefined reference to `__intel_sse2_strlen'
..../openmpi-1.6.2/ompi/contrib/vt/vt/tools/opari/tool/../../../util/util.c:46: 
undefined reference to `__intel_sse2_strlen' 
......
......

The solution is to do the following:

Firstly, make sure you compile with the following
# source /usr/local/intel/composerxe/bin/compilervars.sh
Do it again
#./configure --prefix=/usr/local CC=icc CXX=icpc F77=ifort FC=ifort
Run the make again
# make -j8
# make install

Wednesday, September 4, 2013

error while loading shared libraries: libcurand.so.5.5: cannot open shared object file: No such file or directory

If encountered this error:

....pmemd.cuda.MPI: Error while loading shared libraries:
 libcurand.so.5.5: cannot open shared object file: No such file or directory

It is due to the the LD_LIBRARY_PATH not referring to the CUDA Toolkits libraries. To solve the issue, just update your pathing in your .bashrc
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-5.5/lib64

You may also want to update /etc/ld.so.conf.d/ directories
# touch /etc/ld.so.conf.d/cuda.conf

/usr/local/cuda-5.5/lib64
/usr/local/cuda-5.5/lib


Tuesday, September 3, 2013

Error installing X11 libraries

If you have error installing X11 libraries for CentOS 5 or CentOS 6, you may want to check whether you have already installed the correct group. To do a grouplist,

 Do  yum grouplist
# yum grouplist 
Installed Groups:
   Administration Tools
   Authoring and Publishing
   Development Libraries
   Development Tools
   Editors
   GNOME Desktop Environment
   Graphics
   Legacy Network Server
   Legacy Software Development
   Legacy Software Support
   Mail Server
   MySQL Database
   Network Servers
   Office/Productivity
   OpenFabrics Enterprise Distribution
   Printing Support
   Server Configuration Tools
   System Tools
   X Software Development
   X Window System
Available Groups:
   Cluster Storage
   Clustering
   DNS Name Server
   Engineering and Scientific
   FTP Server
   GNOME Software Development
   Games and Entertainment
   Graphical Internet
   Java Development
   KDE (K Desktop Environment)
   KDE Software Development
   KVM
   News Server
   PostgreSQL Database
   Sound and Video
   Text-based Internet
   Web Server
   Windows File Server
   Xen

Do a yum groupinstall "X Software Development"
 # yum groupinstall "X Software Development"