Using qmove to move a batch job for Torque or OpenPBS

If you wish to remove a job from the queue in which it resides and instantiate the job in another queue, you should use the qmove commands. The commands is as simple as

# qmove queue_name job_id

For example

# qmove pulasan_queue 1234

For more information, remember to look at qmove man pages

Commonly used qstat options

Commonly used Qstat Options

Options	Description
qstat -i	Display jobs that are non-running in alternative format
qstat -r	Display jobs that are running
qstat -n	In addition to basic information, it also provide information of nodes allocated to the job listed.
qstat -u users(s)	Display jobs of a user or users
qstat -Q	Status of queues
qstat -Q -f	Full status of queues in the alternative format
qstat -q	Status of queues in the alternative format
qstat -B	Batch server status
qstat -B -f	Full batch server status including configuration

This information can also be found at  Commonly used qstat options

Using type to check how each command is interpreted

If you are hoping to check how each command is interpreted, then use the BASH BuiltIn command TYPE

To print all the places that contain an executable name you are looking for, use "type -a", you will bring out the alias and the path of the icc. In this case, there is no aliase so the alias did not surface up in the command line

$ type -a icc
icc is /opt/intel/composerxe-2011.2.137/bin/intel64/icc

To check the type of file the executable is categorised ie alias, keyword, function, builtin, file,you can use the command

$ type -t icc
file

For more usage of type, see the man page.

High Performance Data Transfers on TCP/IP

This writeup is a summary of the excellent article from Pittburgh Supercomputing Centre “Enabling High Performance Data Transfers“

For more information, see  High Performance Data Transfers on TCP/IP

Speeding up multiple ssh connections with ControlMaster

I'm assuming you are using OpenSSH 4.

If you are make multiple connections to the same server, you can enables the sharing of multiple sessions over a single network connections. In other words, the additional sessions will try to reuse the master instance's connection rather than initiating new ones.

Step 1: Create a config file in your ~/.ssh directory. Make sure the permission is readable and writable by the owner only ie permission of 600

Step 2: Add the following lines

Host *
   ControlMaster auto
   ControlPath ~/.ssh/master-%r@%h:%p

ControlMaster auto Tries to start a master if there is no existing connection or it will use an existing master connection. ControlPath is the location socket for the ssh processes to communicate among themselves. The %r, %h and %p are replaced with your user name, the host to which you're connecting and the port number—only ssh sessions from the same user to the same host on the same port can or should share a TCP connection, so each group of multiplexed ssh processes needs a separate socket.

Step 3a: To test the configuration, start an ssh session and keep it connected, you should see something like

...........
debug1: setting up multiplex master socket
debug1: channel 0: new [client-session]
...........

Step 3b: Launch another ssh connection to a the same server with the same userid

....................
debug1: auto-mux: Trying existing master
...................

Much of the materials come from  Speed Up Multiple SSH Connections to the Same Server (Linux Journal).

Tuning TCP for Linux 2.4 and Linux 2.6

I have read an interesting article on TCP Tuning for the Linux 2.4 and 2.6 from (Advanced Networking, Pitsburgh Supercomputing Centre). From the article

"NB: Recent versions of Linux (version 2.6.17 and later) have full autotuning with 4 MB maximum buffer sizes. Except in some rare cases, manual tuning is unlikely to substantially improve the performance of these kernels over most network paths, and is not generally recommended
 
All system parameters can be read or set by accessing special files in the /proc file system. E.g.:
 

cat /proc/sys/net/ipv4/tcp_moderate_rcvbuf

If the parameter tcp_moderate_rcvbuf is present and has value 1 then autotuning is in effect. With autotuning, the receiver buffer size (and TCP window size) is dynamically updated (autotuned) for each connection. (Sender side autotuning has been present and unconditionally enabled for many years now)."

Displaying defaults for useradd

If you are take a look at the defaults that useradd will use for  new logins, you can use the command

# useradd -D

Output

GROUP=100
HOME=/home
INACTIVE=-1
EXPIRE=
SHELL=/bin/bash
SKEL=/etc/skel
CREATE_MAIL_SPOOL=yes

For more information, see Default values used by useradd command

Installing Adobe Flash Plugin on CentOS 5 and 6

Step 1: Install Yum Repository from Adobe

(i) For i386

# rpm -ivh http://linuxdownload.adobe.com
/adobe-release/adobe-release-i386-1.0-1.noarch.rpm

# rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-adobe-linux

(ii) For x86-64

# rpm -ivh http://linuxdownload.adobe.com/
adobe-release/adobe-release-x86_64-1.0-1.noarch.rpm

# rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-adobe-linux

Step 2: Update the repositories

# yum update

Step 3: Installing Adobe Player -

CentOS 5

#  yum install flash-plugin nspluginwrapper curl

CentOS 6

# yum install flash-plugin nspluginwrapper alsa-plugins-pulseaudio libcurl

Lowest Latency Switch in the World ?

Gnodal GS-Series is registering an impressive record for ulta-low latency switches. Their enterprise switch GS7200 has the following records. The URL can be found at GS7200 Features and Benefits

1. High Port Count 72 SFP+ port, 1U top-of-rack Ethernet switch enables network consolidation at minimal cost
   
2. Ultra-low Latency Sub 150ns port-to-port with extremely low jitter, ideal for latency sensitive applications
   
3. High Throughput 1.5 Tb/s non-blocking throughput and forwarding capacity exceeding 1 Bpps. Ideal for bandwidth hungry, data-intensive applications in High Performance Data Centers
   
4. Minimized TCO Typical power consumption of 1.6W per port substantially reduces operating costs. Reliability and serviceability are ensured with hot-swappable, redundant PSUs and fans

Installing Grace (xmgrace) on CentOS 5 and 6

For further information on what is Grace ( xmgrace ) and some notes during installation. Do read the blog entry

1. Grace plotting tool for X Window System 
2. Compiling Grace: checking for a Motif >= 1002 compatible API... no

For installing in a nutshell on CentOS 5 and CentOS 6. 

./configure --enable-grace-home=/opt/grace \
--with-extra-incpath=/usr/local/include:/opt/include \
--with-extra-ldpath=/usr/local/lib:/opt/lib
--prefix=/usr/local

--enable-grace-home=DIR      define Grace home dir [PREFIX/grace]

--with-extra-incpath=PATH    define extra include path (dir1:dir2:...) [none]

--with-extra-ldpath=PATH     define extra ld path (dir1:dir2:...) [none]

Compiling,

make

Testing

make tests

Installation

make install

Making links

make links

Adaptive Computing Engine Frame and Moab Demo

 Adaptive Computing Engine Frame and Moab Demo (YouTube (unlisted))

1. Batch Job Submission and Session Management  http://www.youtube.com/watch?v=i3Z5_Tz9Llk 
2. Multiple Desktop Request  http://www.youtube.com/watch?v=HUW1A7zwOW
3.  Multi-User and Multi-Session Use http://www.youtube.com/watch?v=glUgAhmhXp8 
4. Session Sharing http://www.youtube.com/watch?v=Cvcpgv71qYo 
5. Direct App Launch http://www.youtube.com/watch?v=WzKwlXTJ-WI 
6. DCV Demonstration, Multi-User Graphic Assignment http://www.youtube.com/watch?v=c1t9BBPtRoU

Module - Software Environment Management

For Academic Environment, this tool is especially useful to dynamically modified user environment via modulefiles

A good description of the module tool can be taken from the Sourceforge - Environment Modules Project.

"The Environment Modules package provides for the dynamic modification of a user's environment via modulefiles.

Each modulefile contains the information needed to configure the shell for an application. Once the Modules package is initialized, the environment can be modified on a per-module basis using the module command which interprets modulefiles. Typically modulefiles instruct the module command to alter or set shell environment variables such as PATH, MANPATH, etc. modulefiles may be shared by many users on a system and users may have their own collection to supplement or replace the shared modulefiles.

Modules can be loaded and unloaded dynamically and atomically, in an clean fashion. All popular shells are supported, including bash, ksh, zsh, sh, csh, tcsh, as well as some scripting languages such as perl.

Modules are useful in managing different versions of applications. Modules can also be bundled into metamodules that will load an entire suite of different applications."
More can be written on this.

1. The Modules Environment at the ITP Cluster

Understand parameters in /etc/fstab

Do look at the article form "Understand parameters in /etc/fstab" (linuxcluster.wordpress.com). Just a more indepth understanding of /etc/fstab parameters

Using qmgr to remove a queue attribute

To clear the queue attribute in qmgr, you should use the command "unset". For example, you wish to clear acl_users attribute

# qmgr -c "unset queue default_queue acl_users"

where -c = execute single command and exit qmgr.

You will notice that the attribute has been removed by verifying with this command

# qmgr -c "p s"

Using shutdown -h command to do a scheduled shutdown

If you wish to use the Linux command to remind users of scheduled maintenance and halts the system at 10:00am, you can use the command

# shutdown -h 10:00 "Scheduled Maintenance. Expected downtime at 10am"

Alternatively, you may wish to specify a relative downtime. If you wish to shutdown the system 20min when the command is executed

# shutdown -h +20 "We need to power down to do an emergency repair"