Sunday, December 29, 2013

Memory Consumption in Linux

This web article from http://emilics.com talks about memory consumption by a process in Linux. As Linux has a sophisticated  memory management system, measuring may not be that simple.

Inside the article, it mention VSZ (Virtual Memory Size) and Demand Paging, RSS (Resident Set Size) and Shared Libraries, PSS (Proportional Set Size) and more.

Do read the article from Memory Consumption in Linux

Thursday, December 26, 2013

Failed to build compat-rdma RPM when compiling OFED 3.5.1 on CentOS 5.8

I was compiling OFED 3.5.1 on CentOS 5.8. I have got this error.
make[3]: *** [/var/tmp/OFED_topdir/BUILD/compat-rdma-3.5/compat/main.o] Error 1
make[2]: *** [/var/tmp/OFED_topdir/BUILD/compat-rdma-3.5/compat] Error 2
make[1]: *** [_module_/var/tmp/OFED_topdir/BUILD/compat-rdma-3.5] Error 2
make[1]: Leaving directory `/usr/src/kernels/2.6.18-308.el5-x86_64'
make: *** [kernel] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.99798 (%build)


RPM build errors:
    user rupertd does not exist - using root
    user rupertd does not exist - using root
    Bad exit status from /var/tmp/rpm-tmp.99798 (%build)

I tried to rebuild the SRPMS directly by using the commands, but it will fail the rpmbuild. For more information on rpmbuild command, you can take a look at Rebuild a Source RPM
# rpmbuild --rebuild ~OFED_HOME/SRPMS/compat-rdma-3.5-OFED.3.5.1.src.rpm

I finally fixed my OFED by installing 1.5.3.2

Monday, December 23, 2013

Compiling R-3.0.2 on CentOS 5

Do note that R-3.0.2 requires a higher version of gfortran 4.4.4 and above. To help ensure that the gfortran44 is installed, do check that the gnu44 are installed. For more information how to install, see Installing GNU 4.4 of C, C++ and gfortran for CentOS 5

Compiling R-3.0.2
# tar -zxvf 3.0.2.tar.gz 
# ./configure --prefix=/usr/local/R-3.0.2/ CC=gcc44 CXX=g++44 F77=gfortran44 FC=gfortran44
# make
# make install

Monday, December 16, 2013

GPFS NSD Nodes stuck in Arbitrating Mode

One of our GPFS NSD Nodes are forever stuck in arbitrating nodes. One of the symptoms that was noticeable was that the users was able to log-in but unable to do a "ls" of their own directories. You can get a quick deduction by looking at one of the NSD Nodes. For this kind of issues, do a mmdiag --waiters first. There are limited articles on this
# mmdiag --waiters 

.....
.....
0x7FB0C0013D10 waiting 27176.264845756 seconds, SharedHashTabFetchHandlerThread: 
on ThCond 0x1C0000F9B78 (0x1C0000F9B78) (TokenCondvar), reason 'wait for SubToken to become stable'
References:
  1. IZ17622: GPFS DEADLOCK WAITING FOR SUBTOKEN TO BECOME STABLE CAUSES HANG
  2. GPFS File System Deadlock

For more information on the resolution, see  GPFS NSD Nodes stuck in Arbitrating Mode (Linux Cluster)

Saturday, December 14, 2013

Convergence of Supercomputing and Extreme Big Data on the TSUBAME Supercomputer Exascale

I Saw this presentation by Satoshi Matsuoka, from the Tokyo Institute of Technology which shows how  Tokyo Institute of Technology converge HPC and Big Data. Do note that the presentation (pdf) is about 24MB.

Here is the links: http://www.opensfs.org/wp-content/uploads/2013/09/Matsuoka_TokyoTech_Tsubame.pdf

Thursday, December 5, 2013

Error when installing xCAT 2.8.2

If you have error such as those below,
Error: Package: xCAT-2.8.2-snap201307222333.x86_64 (xcat-2-core)
Requires: conserver-xcat
Error: Package: xCAT-2.8.2-snap201307222333.x86_64 (xcat-2-core)
Requires: syslinux-xcat
Error: Package: xCAT-2.8.2-snap201307222333.x86_64 (xcat-2-core)
Requires: elilo-xcat
Error: Package: xCAT-2.8.2-snap201307222333.x86_64 (xcat-2-core)
Requires: ipmitool-xcat >= 1.8.9
Error: Package: 1:xCAT-genesis-scripts-x86_64-2.8.2-snap201307222333.noarch (xcat-2-core)
Requires: xCAT-genesis-base-x86_64
Error: Package: xCAT-2.8.2-snap201307222333.x86_64 (xcat-2-core)
Requires: xnba-undi
Go to http://sourceforge.net/projects/xcat/files/yum/xcat-dep/rh6/x86_64/ and find the respective rpms that fit the package name

Wednesday, December 4, 2013

Coprocessors Programming Codebook and Parallel Universe Magazine By Intel


Intel has put a 16-pages code book titled - Application Optimization for Intel® Processors and Coprocessors Programming CodeBook. Explore some of the tools, techniques, and models for creating highly parallelized code optimized for Intel® Core™, Intel® Xeon® processors, or the Intel® Xeon Phi™ coprocessor.


Download the latest issue of The Parallel Universe magazine and explore the implications of real-time everything as high-performance computing (HPC) moves out of the science lab and onto Main Street. Check out the features in OpenMP* 4.0, discover what’s new in Intel® Cluster Studio XE 2013 SP1, learn techniques for ID’ing and correcting HPC application bottlenecks, and more

Tuesday, December 3, 2013

Compiling SQLite 3.8.1 on CentOS 5

Taken from Beyond Linux® From Scratch - Version 2013-11-30
SQLite 3.8.1
  1. Download (HTTP): http://sqlite.org/2013/sqlite-autoconf-3080100.tar.gz
  2. Download size: 1.9 MB
SQLite 3.8.1 Documents
  1. Download (HTTP): http://sqlite.org/2013/sqlite-doc-3080100.zip
  2. Download size: 4.1 MB
Compiling SQLite 3.8.1
# ./configure --prefix=/usr/local/sqlite-3.8.1 --disable-static        \
CFLAGS="-g -O2 -DSQLITE_ENABLE_FTS3=1 \
-DSQLITE_ENABLE_COLUMN_METADATA=1     \
-DSQLITE_ENABLE_UNLOCK_NOTIFY=1       \
-DSQLITE_SECURE_DELETE=1" &&
# make
# make install

Compiling SQLite Documents
# install -v -m755 -d /usr/local/sqlite-3.8.1/share/doc/sqlite-3.8.1 &&
# cp -v -R sqlite-doc-3080100/* /usr/local/sqlite-3.8.1/share/doc/sqlite-3.8.1
For more information: