IBM Deep Computing for High Performance Computing (HPC) Site

Good Resource from IBM Deep Computing

1. IBM Deep Computing for High Performance Computing (HPC)
2. Good Bi-Monthly Article on IBM HPC "IBM HPC Insights"
3. IBM-HPC FaceBook Links http://www.facebook.com/pages/IBM-HPC/283405154323

Useful Information on R

R is a free software environment for statistical computing and graphics. It compiles and runs on a wide variety of UNIX platforms, Windows and MacOS.

1. R CRAN Mirror
2. R Installation and Administration

Some useful package management instructions

1. R CMD INSTALL -l /path/to/library pkg1 pkg2 ...
2. R CMD REMOVE -l /path/to/library pkg1 pkg2 ...
3. 
   R CMD check -l libdir --install=check:pkgname.log pkgname

For more information, see 6 Add-on packages

More understanding of ulimit

Adding on to blog entries Checking that the ulimit is correct when login as ssh

The stack is nothing but a memory storage which is used by programs to hold intermediate results and subroutine return information. The default size of stack is 10240 KB. You may check it by executing "ulimit -a" command.

The stack reclaim the memory when a process exits. If you allocate more memory in stack, but the memory is not available then it may crash due to stack overflow.


If the stack size is set to too small value and this is not enough for a program then it will fail.

ERROR unable to connect to '/var/run/libvirt/libvirt-sock': Connection refused

Please ensure all these steps are done first. We can then isolate the error more accurately. The 1st few steps are taken from HowtoForge Virtualization With KVM On A CentOS 5.2 Server



Step 1: Checking the Virtualisation Technology on AMD and Intel

Step 2: Install KVM and virtinst

# yum install  kvm kmod-kvm qemu libvirt python-virtinst

Step 3: Reboot the System

# shutdown -r now

Step 4: Check that the KVM kernel Module is loaded

# lsmod grep kvm

kvm_intel     50452      0
kvm           109264     1 kvm_intel

(If you use Intel VT CPU, you should see the above, if you are using AMD-V processor, you should see kvm_amd)

When installing KVM and its related components on CentOS 5.x , you might encounter the error when you type the command "# virt-install"

ERROR unable to connect to '/var/run/libvirt/libvirt-sock': Connection refused
Traceback (most recent call last):
   File "/usr/sbin/virt-install", line 889, in ?
      main()
   File "/usr/sbin/virt-install", line 628, in main
      conn = cli.getConnection(options.connect)
   File "/usr/lib/python2.4/site-packages/virtinst/cli.py", line 126, in getConnction
      conn = libvirt.open(connect)
   File "/usr/lib64/python2.4/site-packages/libvirt.py", line 159, in open
     if ret is None:raise libvirtError('virConnectOpen() failed')
libvirtError: unable to connect to '/var/run/libvirt/libvirt-sock': Connection refused

To solve this, edit /etc/libvirt/libvirtd.conf and

# vim /etc/libvirt/libvirtd.conf

uncommented out the listen_tls = 0 line

listen_tls = 0

Start the libvirtd Daemon

# service libvirtd start

You should able to eliminate the error

MOAB and storing of statistics with ODBC

This documentation ias a useful "How-To" to show how to set up and configure Moab to connect to a MySQL database using the MySQL ODBC driver.

For more information, do look at MOAB Workload Manager, 22.2 ODBC

Checking the Virtualisation Technology on AMD and Intel

You may want to check whether your hardware supports Virtualisation for Intel and AMD chipsets.

# egrep "flags.*:.*(svm|vmx)" /proc/cpuinfo

If there is output, your hardware supports virtualisation.

Other related Topics includes

1. Determining if Intel Virtualization Technology or AMD Virtualization is enabled in the BIOS without rebooting
2. Detecting Intel VT and AMD-V 
   
   

Dynamic server provisioning with xCAT and TORQUE

I have found a good article "Dynamic server provisioning with xCAT and TORQUE" from IBM Developers Network > Open Source > Technical Library

"This article describes a solution for building a dynamically provisioned high-performance computing (HPC) cluster system using the Extreme Cloud Administration Toolkit (xCAT) and Tera-scale Open-source Resource and QUEue Manager (TORQUE) open source packages. The xCAT is a leading solution for dynamically provisioning compute, storage, and network resources. TORQUE is a workload and resource-management system that manages batch jobs and compute nodes and schedules the execution of those jobs."

Getting Yum to install from selected repository only

Working from Using Yum to local install without checking Repository, you can use the same principle to install from selected repository while disabling the rest.

# yum --disablerepo=* --enablerepo=dag install (packages)

where dag and update are repositories. If you are not sure the repository names to use, go to /etc/yum.repos.d/xxx.repo. The first line [.....] is the repository names

Interesting Whitepaper - "Achieving MPI Performance with Fabrics Collectives Offload"

This white paper "Achieving MPI Performance with Fabrics Collectives Offload" written by Voltaire scientist namely Avid Cohen, Yiftah Sharhar, Yaron Haviv and Asaf Wachtel is really a good reading for those who are trying to figure out how to improve MPI from a fabric point of view.

Happy reading.

IBM Redbook - Configuration and Tuning GPFS for Digital Media Environments

An Interesing Red-Book named "Configuration and Tuning GPFS for Digital Media Environments "especially if you , abeit a long thick e-book with 272 pages.

Happy Reading!