Installing lammps using Intel Compilers, OpenMPI and FFTW (Update)

This is an update of  Blog Entry Installing lammps using Intel Compilers, OpenMPI and FFTW

The steps are still relevant and workable, but if you are doing a lot of customisation pathing or you just simply wish to tweak the parameters in the configuration file, here is my sample script. I got my information from the Message threads of LAMMPS http://lammps.sandia.gov/threads/msg12768.html

Here are my settings, amending the $SOURCE/lammps-30Mar10/src/MAKE/Makefile.Linux

# linux = RedHat Linux box, Intel icc, MPICH2, FFTW

SHELL = /bin/sh
# ---------------------------------------------------------------------
# compiler/linker settings
# specify flags and libraries needed for your compiler

CC = mpicc
CCFLAGS = -g -O -I/usr/local/fftw/include -DFFT_FFTW
DEPFLAGS = -M
LINK = mpicc
LINKFLAGS = -O
USRLIB = -lfftw -lmpi
LIB = -lstdc++ -lpthread
ARCHIVE = ar
ARFLAGS = -rc
SIZE = size

# ---------------------------------------------------------------------

# LAMMPS-specific settings
# specify settings for LAMMPS features you will use
# LAMMPS ifdef options, see doc/Section_start.html

LMP_INC = -DLAMMPS_GZIP
# MPI library, can be src/STUBS dummy lib
# INC = path for mpi.h, MPI compiler settings
# PATH = path for MPI library
# LIB = name of MPI library:wq

MPI_INC = -DMPICH_IGNORE_CXX_SEEK
MPI_PATH = -L/usr/mpi/intel/lib # where I store my openmpi binary and lib
MPI_LIB = -lmpi -lpthread

# FFT library, can be -DFFT_NONE if not using PPPM from KSPACE package
# INC = -DFFT_FFTW, -DFFT_INTEL, -DFFT_NONE, etc, FFT compiler settings
# PATH = path for FFT library
# LIB = name of FFT library

FFT_INC = -DFFT_FFTW
FFT_PATH = -L/usr/local/fftw/lib # I've put my fftw at /usr/local/fftw
FFT_LIB = -lfftw

.............

You can add further flags at CCFLAGS. See my other blog entry

1. Understanding -DOMPI_SKIP_MPICXX Flag for compiler/linker settings
2. Understanding -DOMPI_IGNORE_CXX_SEEK Flag for compiler/linker settings
   

Copy to remote machines with symbolic links intact

If you are copying directories with hard links and symbolic links to another remote servers intact. Once of the best method is to use rsync

# rsync -lH -rva gromacs remote_server:/usr/local

where l - copy symlinks as symlinks, H - preserve hard links, r - recursive, v - verbose, a - archive

Compiling Infiniband or OpenIB with OpenMPI and Intel Compilers on CentOS

Building on the Blog Entry "Useful Information to Compile Infinifband with OpenMPI", here is a more detailed writeup

I'm assuming you have compiled the Intel Compilers and Unzip the OpenMPI Package. Some of these information can be found on simple Ethernet-based Building OpenMPI with Intel Compiler (Ver 2) .
Don't configure and make yet for the OpenMPI Package.


Firstly to compile with OpenIB support,

#./configure --prefix=/usr/mpi/intel/ \ 
CC=icc CXX=icpc F77=ifort FC=ifort \
--with-openib \ 
--with-openib-libdir=/usr/lib64/

# make all install 

To test whether you have compiled and installed correctly, you can run the "ompi_info" command and look for components for your networks.

# ompi_info | grep openib

You should get something like this depending on your component

MCA btl: openib (MCA v2.0, API v2.0, Component v1.4.1)

You are in good shape.

For more information, see OpenMPI, see

1. 26. How do I build Open MPI with support for Open IB (Infiniband), mVAPI (Infiniband), GM (Myrinet), and/or MX (Myrinet)?

Understanding -DOMPI_SKIP_MPICXX Flag for compiler/linker settings

When I was compiling LAMMPS, I come across this Flag when preparing the compiler/linker settings
CCFLAGS =       -g -O -I/usr/local/fftw/include -DFFT_FFTW -DOMPI_SKIP_MPICXX -DOMPI_IGNORE_CXX_SEEK

What do -DFFT_FFTW -DOMPI_SKIP_MPICXX really mean? The answer is nicely given Rolf Vandevaart "Speeding Up C++ Compiles"

In a summary, from the blog

The issue is that if you are compiling C++ code, but you are not using the C++ bindings, the code can be slow to compile.  The workaround is to use the -DOMPI_SKIP_MPICXX flag which tells MPI not to include the C++ header files

And here are the results.  Not quite a 50% speedup, but close

Understanding -DOMPI_IGNORE_CXX_SEEK Flag for compiler/linker settings

When I was compiling LAMMPS, I come across this Flag when preparing the compiler/linker settings


CCFLAGS =  -g -O -I/usr/local/fftw/include -DFFT_FFTW -DOMPI_SKIP_MPICXX -DOMPI_IGNORE_CXX_SEEK
 
What do -DOMPI_IGNORE_CXX_SEEK really seek to solve. It happens on openmpi-1.2 with Intel. This is due to BOTH stdio.h and the MPI C++ interface use SEEK_SET, SEEK_CUR, SEEK_END.
 
For more inforamtion. see this FAQs from NCSA
 I got an error message: SEEK_SET is #defined but must not be for the C++ binding of MPI. What does that mean?