Memory Consumption in Linux

This web article from http://emilics.com talks about memory consumption by a process in Linux. As Linux has a sophisticated  memory management system, measuring may not be that simple.

Inside the article, it mention VSZ (Virtual Memory Size) and Demand Paging, RSS (Resident Set Size) and Shared Libraries, PSS (Proportional Set Size) and more.

Do read the article from Memory Consumption in Linux

Failed to build compat-rdma RPM when compiling OFED 3.5.1 on CentOS 5.8

I was compiling OFED 3.5.1 on CentOS 5.8. I have got this error.

make[3]: *** [/var/tmp/OFED_topdir/BUILD/compat-rdma-3.5/compat/main.o] Error 1
make[2]: *** [/var/tmp/OFED_topdir/BUILD/compat-rdma-3.5/compat] Error 2
make[1]: *** [_module_/var/tmp/OFED_topdir/BUILD/compat-rdma-3.5] Error 2
make[1]: Leaving directory `/usr/src/kernels/2.6.18-308.el5-x86_64'
make: *** [kernel] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.99798 (%build)


RPM build errors:
    user rupertd does not exist - using root
    user rupertd does not exist - using root
    Bad exit status from /var/tmp/rpm-tmp.99798 (%build)

I tried to rebuild the SRPMS directly by using the commands, but it will fail the rpmbuild. For more information on rpmbuild command, you can take a look at Rebuild a Source RPM

# rpmbuild --rebuild ~OFED_HOME/SRPMS/compat-rdma-3.5-OFED.3.5.1.src.rpm

I finally fixed my OFED by installing 1.5.3.2

Compiling R-3.0.2 on CentOS 5

Do note that R-3.0.2 requires a higher version of gfortran 4.4.4 and above. To help ensure that the gfortran44 is installed, do check that the gnu44 are installed. For more information how to install, see Installing GNU 4.4 of C, C++ and gfortran for CentOS 5

Compiling R-3.0.2

# tar -zxvf 3.0.2.tar.gz 
# ./configure --prefix=/usr/local/R-3.0.2/ CC=gcc44 CXX=g++44 F77=gfortran44 FC=gfortran44
# make
# make install

GPFS NSD Nodes stuck in Arbitrating Mode

One of our GPFS NSD Nodes are forever stuck in arbitrating nodes. One of the symptoms that was noticeable was that the users was able to log-in but unable to do a "ls" of their own directories. You can get a quick deduction by looking at one of the NSD Nodes. For this kind of issues, do a mmdiag --waiters first. There are limited articles on this

# mmdiag --waiters 

.....
.....
0x7FB0C0013D10 waiting 27176.264845756 seconds, SharedHashTabFetchHandlerThread: 
on ThCond 0x1C0000F9B78 (0x1C0000F9B78) (TokenCondvar), reason 'wait for SubToken to become stable'

References:

1. IZ17622: GPFS DEADLOCK WAITING FOR SUBTOKEN TO BECOME STABLE CAUSES HANG
2. GPFS File System Deadlock

For more information on the resolution, see  GPFS NSD Nodes stuck in Arbitrating Mode (Linux Cluster)

Convergence of Supercomputing and Extreme Big Data on the TSUBAME Supercomputer Exascale

I Saw this presentation by Satoshi Matsuoka, from the Tokyo Institute of Technology which shows how  Tokyo Institute of Technology converge HPC and Big Data. Do note that the presentation (pdf) is about 24MB.

Here is the links: http://www.opensfs.org/wp-content/uploads/2013/09/Matsuoka_TokyoTech_Tsubame.pdf