Virtualization Solutions for High-Performance Computing

There is an interesting talk held in Stockholm, Sweden which taken from High-Performance Big Data (HiBD)

1. Opportunities and Challenges in Designing High-Performance MPI and Big Data Libraries on Virtualized InfiniBand Clusters

2015 Stanford HPC Conference Video Gallery

The 2015 Standford HPC Conference Video Gallery. Check it out!

Using iptables to allow compute nodes to access public network

Have written a tutorial on Using iptables to allow compute nodes to access public network on LinuxCluster. Do read it.

Lowest Latency Switch in the World ?

Gnodal GS-Series is registering an impressive record for ulta-low latency switches. Their enterprise switch GS7200 has the following records. The URL can be found at GS7200 Features and Benefits

1. High Port Count 72 SFP+ port, 1U top-of-rack Ethernet switch enables network consolidation at minimal cost
   
2. Ultra-low Latency Sub 150ns port-to-port with extremely low jitter, ideal for latency sensitive applications
   
3. High Throughput 1.5 Tb/s non-blocking throughput and forwarding capacity exceeding 1 Bpps. Ideal for bandwidth hungry, data-intensive applications in High Performance Data Centers
   
4. Minimized TCO Typical power consumption of 1.6W per port substantially reduces operating costs. Reliability and serviceability are ensured with hot-swappable, redundant PSUs and fans

IBM HPC Management Suite for Cloud

Do look at IBM Management Suite for Cloud . According to the site, the list of comprehensive tools include

1. Provisions bare metal high performance compute clusters for technical computing and analysis workloads.
2. Consolidates the infrastructure for efficient sharing of HPC resources.
3. Accesses the HPC infrastructure through an on-demand, self-service web portal optimized for HPC users and administrators.
4. Achieves rapid image deployment and resource management using diskless provisioning.
5. Centralizes user and energy management, usage metering and accounting.

Training Materials for HPC, MPI, Blue-Gene, IBM POWER and more by Livermore Computing Center

Livermore Computing Center has provided a series of very useful online Training Manual information for HPC users and administrator. See Livermore Computing Center High Performance Computing Training

The topics includes

1. Introduction To Parallel Computing
2. Introduction to Livermore Computing Resources
3. Moab
4. Using the Dawn BGP System
5. BG/L Tutorials
6. Using ASC Purple
7. IBM POWER Systems Overview
8. Linux Clusters Overview
9. Message Passing Interface (MPI)
10. Advanced MPI
11. POSIX Threads Programming
12. OpenMP
13. TotalView Debugger
14. ASC Tri-Lab Code Development Tools Workshop
15. IBM HPC Toolkit
16. A Holistic Approach Towards Automatic Performance Analysis and Tuning
17. Performance Analysis Tools and Topics for LC'S IBM SP Systems
18. MPI Performance Topics 
19. Performance Analysis Tools 
20. Red Storm / XT3 Workshop (LLNL internal only)
21. Using Scientific Python
22. Scientific Python II
23. Using Python Numeric
24. MPI Parallel Programming In Python

High Performance Cluster for Biomedical Research using 10GB Ethernet iWARP Fabric

This white paper from Intel is on a large research institute that has achieve 36 TeraFLOPS with at least 84% efficiency using HPL benchmark on a cluster of 4032 cores using 10GB Ethernet iWARP Fabric and ARISTA 10GB switches

For more information, see  High Performance Cluster for Biomedical Research using 10GB Ethernet iWARP Fabric

Singapore university and IBM cooperate on cloud

Interesting Article on a Singapore University and IBM collaboration on Cloud.
HPC Cloud to be exact

See Singapore university and IBM cooperate on cloud (Future Gov Asia Pacific)

IBM Deep Computing for High Performance Computing (HPC) Site

Good Resource from IBM Deep Computing

1. IBM Deep Computing for High Performance Computing (HPC)
2. Good Bi-Monthly Article on IBM HPC "IBM HPC Insights"
3. IBM-HPC FaceBook Links http://www.facebook.com/pages/IBM-HPC/283405154323

Optimizing OpenFOAM Productivity in Clusters - Article from Desktop Engineering

This is an interesting article from Desktop Engineering "Optimizing OpenFOAM Productivity in Clusters". This article explore the performance gained when the HPC system is configured properly, there is a lot of productivity and gain in performance.

Undergraduate Petascale Education Program

Petascale Education Program seeks to promote understanding and petascale computing among undergraduate and faculty. On the site, you can fund useful materials, undergraduate petascale modules etc....

Bootable Cluster CD

Bootable Cluster CD......Cluster in your pocket....

The BCCD was created to facilitate instruction of parallel computing aspects and paradigms......The BCCD image addresses this problem by providing a non-destructive overlay way to run a full-fledged parallel computing environment on just about any workstation-class system.....

Take a look

Advancing the Power of Visualization.

This is an interview by HPwire with Steve Briggs, HPCD’s SVA product marketing manager on Visualisation from HP point of view. Interesting information

Advancing the Power of Visualization –Coming Soon to Linux Clusters: 100 Million Pixels and More

Moving HPC Applications to Cloud - The Practitioner Prospective

This is a very good summarise presentation by Victoria Livschitz, CEO of Grid Dynamics on some of the issues and challenges we will face when we unify Cloud and HPC into HPC-Cloud.

Read this: Moving HPC Applications to Cloud - The Practitioner Prospective

MPIRun and " You may set your LD_LIBRARY_PATH to have the location of the shared libraries ...... " issues

The Scenario:
I encountered this error while executing an mpirun. Do a "pbsnodes -l" and everything seems is online. I thought my $LD_LIBRARY_PATH was giving the issues. But after some exhaustive check, I've realise that communication to one of our nodes was having issues. Here's are the steps I took to solve the issue

--------------------------------------------------------------------------
A daemon (pid 16704) died unexpectedly with status 127 while attempting to launch so we are aborting.

There may be more information reported by the environment (see above).

This may be because the daemon was unable to find all the needed shared libraries on the remote node. You may set your LD_LIBRARY_PATH to have the  location of the shared libraries on the remote nodes and this will automatically be forwarded to the remote nodes.
--------------------------------------------------------------------------

The Error seems like it is due to LD_LIBRARY_PATH, but it may or may not.

Step 1: Check whether it is a LD_LIBRARY_PATH Issue for your head and compute node
First thing first, you should try to check whether you have the pathing of your LD_LIBRARY_PATH is blank or filled with the correct information for your head node and compute node.

$ echo $LD_LIBRARY_PATH
$/usr/local/lib:/opt/intel/Compiler/11.1/069/lib/intel64 .....

If everything looks normal. Proceed to step 2


Step 2: Check whether the mpirun can be executed cleanly.

$ mpirun -np 32 -hostfile hostfilename openmpi-with-intel-hello-world

where

1. hostfilename contains all the compute node host name
2. openmpi-with-intel-hello-world is the compiled mpi program

Step 3: If the error still remains.....
Modify the hostfilename and insert 1 compute node at a time and compile the  mpirun. You should be able to quickly identify that the problem is not $LD_LIBRARY_PATH but a problematic compute node

n01
n02
...

. In my situation, my problem was due to a broken ssh-generated-key and despite my torque showing all nodes as healthy

Torque Error - Address already in use (98) in scan_for_exiting, cannot bind to port 464 in client_to_svr - too many retries

pbs_mom;Svr;pbs_mom;LOG_ERROR:: Address already in use (98) in scan_for_exiting, cannot bind to port 464 in client_to_svr - too many retries

One cause for this is very high traffic on the network not allowing the mom and the server to communicate properly. One common case are job scripts that incessantly run qstat. You will be surprise that sometimes users input some of these qstat scripts that cause the error

Multi-core Strategies: MPI and OpenMP

Multi-core Strategies: MPI and OpenMP is an interesting article on MPI and OpenMP from HPCommunity.org

MAUI: Cannot send request to server ...... (server may not be running)

ERROR: cannot send request to server ................42559 (server may not be running)
ERROR: cannot request service (status)

A few assumptions for this blog entry. You are using Torque and MAUI only for the resource manager and scheduler respectively.

Several things to check.

1. Make sure your /etc/hostname reflect the correct entries for all the head and compute nodes
2. For the Torque configuration, do follow the configuration seen in the blog entry Setting up Torque Server on xCAT 2.x. Pay particular attendtion to Step 2 and 3
3. For the MAUI configuration, do follow the configuration seen in the blog entry MAUI Installation in xCAT 2.x
4. Check the logs for MAUI at /usr/local/maui/log

After following Pointer 1-4, you are still unable to solve, you can get some hints if you can go to

# cd /opt/maui/sbin/

and type

./maui

In my particular situation, I encounter server must be started on host 'xxx.xxx.xxx.xxx.xxx' (currently on 'xxx') which is my short-name. I resolve it quickly by going to /usr/local/maui/maui.cfg and edit the

SERVERHOST = xxxx

10GE Switches as viable consideration for HPC Interconnect

Take a look at this whitepapers on the tremendous inroad Ethernet has made to contest IB for the HPCC market

1. 10 GE Fabric Delivers Consistent High Performance for Computing Clusters at Sandia National Labs (Results from Sandia CBench benchmark tests demonstrate higher throughput using compared to statically routed InfiniBand)
2. Are we there yet? 10Gb Ethernet for HPC
3. Experiences with NFS over IB and iWARP RDMA

Compiling with MPI Parallel Library

This is a good site for MPICH Parallel Library Compilation from Duke University