Troubleshooting Gaussian Calculation

For installing Gaussian TCP Linda, you may want to take a look  UNIX Binary Gaussian 09 Revision A.02 Installation instructions which has been updated recently.

"Troubleshooting Gaussian calculations" is a good resource that helps explain some of the errors and issues faced when running Gaussian.

Optimizing OpenFOAM Productivity in Clusters - Article from Desktop Engineering

This is an interesting article from Desktop Engineering "Optimizing OpenFOAM Productivity in Clusters". This article explore the performance gained when the HPC system is configured properly, there is a lot of productivity and gain in performance.

Encountering Unauthorised Request for Qmgr command

Taken from the Torque Users Group http://www.clusterresources.com/pipermail/torqueusers/2006-March/003408.html


If you encounter "qmgr obj=que svr=default: Unauthorized Request" whenever you type the qmgr command, it is due to /var/spool/pbs/server_name not in sync with the name associated with eth0. To mitigate the problem,

1. server_name must match the first no-localhost name in /etc/hosts
2. You can check your hostname on your Linux Box, by typing hostname

Quick way to update user information for fingering

You can use the usermod command to update comments field or "Name" field

# usermod -c "User1 Name" user1

Alternatively, you can edit directly at the /etc/passwd 5th column

user1:x:524:100:User1 Name:/home/user1:/bin/bash

You can verify by fingering

# finger user1

Default ulimit setting in torque overide ulimit setting

Even though you may have setup the ulimit correctly as suggested by Blog entry "Checking that the ulimit is correct when login as ssh", your job may still encounter the error is you are using torque resource manager.
This is because the default setting in /etc/init.d/pbs_mom is set too low and overide the ulimit setting, the default setting at /etc/init.d/pbs_mom should be changed to

ulimit -l unlimited
ulimit -c unlimited
ulimit -n 32768
#ulimit -n 20000
#ulimit -i 20000

Checking that the ulimit is correct when login as ssh

To check that whether your ulimit work properly when you ssh, you should use the below command. To set ulimit, you may want to check on blog entry Encountering Segmentation Fault, Bus Error or No output

$ ssh node00 ulimit -l

To check that ulimits is set correctly for your mpirun

$ mpirun -np 8 bash -c ulimit -l

The 2 commands output should be similar.

Green Revolution Cooling (www.grcooling.com)

AT SC10, I saw quite a unique and disruptive solution for cooling HPC stuff. It won SC 2009 disruptive technology of the year.

Green Revolution (GR) Cooling method of cooling datacentre is very revolutionary and innovative. According to Green Revolution Cooling, datacentre are cooled by submerging servers in dielectric fluid coolant (GreenDEF coolant).

According to its website, GR Cooling is the first to develop a complete rack and server rack solution using this method at a feasible up-front costs and high performance.

For more information, read the GR Cooling website.

Undergraduate Petascale Education Program

Petascale Education Program seeks to promote understanding and petascale computing among undergraduate and faculty. On the site, you can fund useful materials, undergraduate petascale modules etc....

Bootable Cluster CD

Bootable Cluster CD......Cluster in your pocket....

The BCCD was created to facilitate instruction of parallel computing aspects and paradigms......The BCCD image addresses this problem by providing a non-destructive overlay way to run a full-fledged parallel computing environment on just about any workstation-class system.....

Take a look

Defunct but useful ClusterWorld Magazine

ClusterWorld Magazine, now defunct as it merge with Linux Magazine. The last edition is June 2005. But nonetheless, it has interesting information on MPI which is still applicable today.

Open CL Programming Webinar Series

This is a series of webinars where AMD experts discuss and answer questions about data parallel computing on GPU leveraging on the OpenCL(tm) architecture. These webinars will include beginning and advanced tracks offered at varying times

1. OpenCL Programming Webinar Series (http://developer.amd.com/) OR
2. OpenCL Programming Webinar Series (http://www.eventsvc.com/)

Graph Editor for Open-Source Computer Algebra System Sage

Graph editor part of the open-source computer algebra system Sage. Will need HTML 5

Installing lammps using Intel Compilers, OpenMPI and FFTW (Update)

This is an update of  Blog Entry Installing lammps using Intel Compilers, OpenMPI and FFTW

The steps are still relevant and workable, but if you are doing a lot of customisation pathing or you just simply wish to tweak the parameters in the configuration file, here is my sample script. I got my information from the Message threads of LAMMPS http://lammps.sandia.gov/threads/msg12768.html

Here are my settings, amending the $SOURCE/lammps-30Mar10/src/MAKE/Makefile.Linux

# linux = RedHat Linux box, Intel icc, MPICH2, FFTW

SHELL = /bin/sh
# ---------------------------------------------------------------------
# compiler/linker settings
# specify flags and libraries needed for your compiler

CC = mpicc
CCFLAGS = -g -O -I/usr/local/fftw/include -DFFT_FFTW
DEPFLAGS = -M
LINK = mpicc
LINKFLAGS = -O
USRLIB = -lfftw -lmpi
LIB = -lstdc++ -lpthread
ARCHIVE = ar
ARFLAGS = -rc
SIZE = size

# ---------------------------------------------------------------------

# LAMMPS-specific settings
# specify settings for LAMMPS features you will use
# LAMMPS ifdef options, see doc/Section_start.html

LMP_INC = -DLAMMPS_GZIP
# MPI library, can be src/STUBS dummy lib
# INC = path for mpi.h, MPI compiler settings
# PATH = path for MPI library
# LIB = name of MPI library:wq

MPI_INC = -DMPICH_IGNORE_CXX_SEEK
MPI_PATH = -L/usr/mpi/intel/lib # where I store my openmpi binary and lib
MPI_LIB = -lmpi -lpthread

# FFT library, can be -DFFT_NONE if not using PPPM from KSPACE package
# INC = -DFFT_FFTW, -DFFT_INTEL, -DFFT_NONE, etc, FFT compiler settings
# PATH = path for FFT library
# LIB = name of FFT library

FFT_INC = -DFFT_FFTW
FFT_PATH = -L/usr/local/fftw/lib # I've put my fftw at /usr/local/fftw
FFT_LIB = -lfftw

.............

You can add further flags at CCFLAGS. See my other blog entry

1. Understanding -DOMPI_SKIP_MPICXX Flag for compiler/linker settings
2. Understanding -DOMPI_IGNORE_CXX_SEEK Flag for compiler/linker settings
   

Copy to remote machines with symbolic links intact

If you are copying directories with hard links and symbolic links to another remote servers intact. Once of the best method is to use rsync

# rsync -lH -rva gromacs remote_server:/usr/local

where l - copy symlinks as symlinks, H - preserve hard links, r - recursive, v - verbose, a - archive

Compiling Infiniband or OpenIB with OpenMPI and Intel Compilers on CentOS

Building on the Blog Entry "Useful Information to Compile Infinifband with OpenMPI", here is a more detailed writeup

I'm assuming you have compiled the Intel Compilers and Unzip the OpenMPI Package. Some of these information can be found on simple Ethernet-based Building OpenMPI with Intel Compiler (Ver 2) .
Don't configure and make yet for the OpenMPI Package.


Firstly to compile with OpenIB support,

#./configure --prefix=/usr/mpi/intel/ \ 
CC=icc CXX=icpc F77=ifort FC=ifort \
--with-openib \ 
--with-openib-libdir=/usr/lib64/

# make all install 

To test whether you have compiled and installed correctly, you can run the "ompi_info" command and look for components for your networks.

# ompi_info | grep openib

You should get something like this depending on your component

MCA btl: openib (MCA v2.0, API v2.0, Component v1.4.1)

You are in good shape.

For more information, see OpenMPI, see

1. 26. How do I build Open MPI with support for Open IB (Infiniband), mVAPI (Infiniband), GM (Myrinet), and/or MX (Myrinet)?

Understanding -DOMPI_SKIP_MPICXX Flag for compiler/linker settings

When I was compiling LAMMPS, I come across this Flag when preparing the compiler/linker settings
CCFLAGS =       -g -O -I/usr/local/fftw/include -DFFT_FFTW -DOMPI_SKIP_MPICXX -DOMPI_IGNORE_CXX_SEEK

What do -DFFT_FFTW -DOMPI_SKIP_MPICXX really mean? The answer is nicely given Rolf Vandevaart "Speeding Up C++ Compiles"

In a summary, from the blog

The issue is that if you are compiling C++ code, but you are not using the C++ bindings, the code can be slow to compile.  The workaround is to use the -DOMPI_SKIP_MPICXX flag which tells MPI not to include the C++ header files

And here are the results.  Not quite a 50% speedup, but close

Understanding -DOMPI_IGNORE_CXX_SEEK Flag for compiler/linker settings

When I was compiling LAMMPS, I come across this Flag when preparing the compiler/linker settings


CCFLAGS =  -g -O -I/usr/local/fftw/include -DFFT_FFTW -DOMPI_SKIP_MPICXX -DOMPI_IGNORE_CXX_SEEK
 
What do -DOMPI_IGNORE_CXX_SEEK really seek to solve. It happens on openmpi-1.2 with Intel. This is due to BOTH stdio.h and the MPI C++ interface use SEEK_SET, SEEK_CUR, SEEK_END.
 
For more inforamtion. see this FAQs from NCSA
 I got an error message: SEEK_SET is #defined but must not be for the C++ binding of MPI. What does that mean?