Friday, May 31, 2013

Installing Mellanox ConnectX® EN 10GbE Drivers for VMware® ESX 5.x Server

If you have Mellanox Technologies MT27500 Family [ConnectX-3] 10G Ethernet Card, it may not be automatically detected by VMware ESX 5.x Hypervisor. You have to install the driver manually into Vmware 5.x

Step 1: At VMware ESX 5.x Hypervisor,
  • Click the F2 button
  • Select "Troubleshoot Options"
  • "Enable ESXi Shell" and "Enable SSH"
Step 2: Download the VMware ESXi 5.0 Driver for Mellanox ConnectX Ethernet Adapters

Step 3: Unzip the mlx4_en-mlnx-1.6.1.2-offline_bundle-471530.zip

Step 4: Read the README file
VMware uses a file package called a VIB (VMware Installation Bundle) as the  mechanism for installing or upgrading software packages on an ESX server.

The file may be installed directly on an ESX server from the command line, or through the VMware Update Manager (VUM).

Step 5:  For New Installation (From README and modified)

For new installs, you should perform the following steps:

Step 5a: Copy the VIB to the ESX server.  Technically, you can place the file anywhere that is accessible to the ESX console shell, but for these instructions, we'll assume the location is in '/tmp'.

Here's an example of using the Linux 'scp' utility to copy the file from a local system to an ESX server located at 10.10.10.10:
# scp net-mlx4-en-1.6.1.2-1OEM.500.0.0.406165.x86_64.vib root@10.10.10.10:/tmp

Step 5b: Issue the following command (full path to the VIB must be specified):

# esxcli software vib install -v {VIBFILE}

In the example above, this would be:
# esxcli software vib install -v 
/tmp/net-mlx4-en-1.6.1.2-1OEM.500.0.0.406165.x86_64.vib

Wednesday, May 29, 2013

Platform LSF – Administration and Accounting Commands

I thought I list out some useful commands that can be used for for viewing a cluster using an LSF Cluster. Please read the manual for more in-depth information. Taken from Platform LSF 8.3 Quick References.
Administration and Accounting commands
lsadmin LSF administrative tool to control the operation of the LIM and RES daemons in an LSF cluster, lsadmin help shows all subcommands
lsfinstall Install LSF using install.config input file
lsfrefresh Restart the LSF daemons on all hosts in the local cluster
lsfshutdown Shut down the LSF daemons on all hosts in the local cluster
lsfstartup Start the LSF daemons on all hosts in the local cluster
badmin LSF administrative tool to control the operation of the LSF Batch system including sbatchd, mbatchd, hosts and queues, badmin help shows all subcommands
bconf Changes LSF configuration in active memory
bladmin Displays the current LSF version number, cluster name and master host name

Other References:
  1. Platform LSF – Monitoring jobs and tasks
  2. Platform LSF – Administration and Accounting Commands
  3. Platform LSF - View information about cluster
  4. Platform LSF - Submitting and Controlling jobs 

Platform LSF - View information about cluster

I thought I list out some useful commands that can be used for for viewing a cluster using an LSF Cluster. Please read the manual for more in-depth information. Taken from Platform LSF 8.3 Quick References.

bhosts Display hosts and static and dynamic resources
bmgroup Displays Information about host groups and compute units
blimits Displays Information about resource allocation limits of running jobs
bparams Displays Information about tunable batch system parameter
bqueues Displays Information about batch queue
busers Displays Information about users and user groups
lshosts Displays hosts and their static resource Information
lsid Displays the current LSF version number, cluster name and master host name
lsinfo Displays load sharing configuration information
lsload Displays dynamic load indices for hosts


Other References:
  1. Platform LSF – Monitoring jobs and tasks
  2. Platform LSF – Administration and Accounting Commands
  3. Platform LSF - View information about cluster
  4. Platform LSF - Submitting and Controlling jobs 

Monday, May 27, 2013

rpm packages for latex2html

If you are not using yum which you should be but relying on good old rpm to unpack the package for whatever reasons, if you wish to install latex2html you may wish to take note of the dependency packages. Quite a bit eh? Most of the packages can be found in the original CD install of the RH or CentOS CD

latex2html-2008-4.el6.noarch.rpm 
texlive-texmf-dvips-2007-38.el6.noarch.rpm
netpbm-10.47.05-11.el6.i686.rpm 
texlive-texmf-errata-2007-7.1.el6.noarch.rpm
netpbm-10.47.05-11.el6.x86_64.rpm 
texlive-texmf-errata-dvips-2007-7.1.el6.noarch.rpm
netpbm-progs-10.47.05-11.el6.x86_64.rpm 
texlive-texmf-errata-fonts-2007-7.1.el6.noarch.rpm
psutils-1.17-34.el6.x86_64.rpm 
texlive-texmf-errata-latex-2007-7.1.el6.noarch.rpm
texlive-2007-57.el6_2.x86_64.rpm 
texlive-texmf-fonts-2007-38.el6.noarch.rpm
texlive-dvips-2007-57.el6_2.x86_64.rpm 
texlive-texmf-latex-2007-38.el6.noarch.rpm
texlive-latex-2007-57.el6_2.x86_64.rpm 
texlive-utils-2007-57.el6_2.x86_64.rpm
texlive-texmf-2007-38.el6.noarch.rpm

Friday, May 24, 2013

Installing and upgrading to VSphere 5.1

I was upgradingfrom VCentre 4.1 to VCentre 5.1. I found this online video particular useful in quickly get to start of upgrading VCentre



The blog of the youtuber can be found at Plain Visualization

Thursday, May 23, 2013

Installing perl-ncurses for pbstop

pbstop is a  monitoring utility for OpenPBS or Torque. When installing pbstop, you will need the packages perl-nurses

Make sure you have EPEL repository and the perl-ncurses look like this.
# yum install perl-Curses

Wednesday, May 22, 2013

Compiling Cufflinks

Reference from http://cufflinks.cbcb.umd.edu/tutorial.html
Cufflinks assembles transcripts, estimates their abundances, and tests for differential expression and regulation in RNA-Seq samples. It accepts aligned RNA-Seq reads and assembles the alignments into a parsimonious set of transcripts. Cufflinks then estimates the relative abundances of these transcripts based on how many reads support each one, taking into account biases in library preparation protocols. Preliminary Compilation
  1. Compiling and Installing Boost 1.53
  2. Compiling SamTools
  3. Compiling Eigen
Step 1: Download Cufflinks Download Cufflinks Step 2: Compile Cufflinks
# tar -zxvf cufflinks-2.1.1.tar.gz
# cd cufflinks-2.1.1
#  ./configure --prefix=/usr/local/cufflinks/ \
--with-boost=/usr/local/boost \ 
--with-eigen=/usr/local/include
# make
# make install

Step 3: Testing the installation Download the test data
# /usr/local/cufflinks/bin/cufflinks ./test_data.sam

You should see the following output:
[bam_header_read] EOF marker is absent. The input is probably truncated.
[bam_header_read] invalid BAM binary header (this is not a BAM file).
File ./test_data.sam doesn't appear to be a valid BAM file, trying SAM...
[13:23:15] Inspecting reads and determining fragment length distribution.
> Processed 1 loci.                            [*************************] 100%
Warning: Using default Gaussian distribution due to insufficient paired-end reads in open ranges.  
It is recommended that correct paramaters (--frag-len-mean and --frag-len-std-dev) be provided.
> Map Properties:
>       Total Map Mass: 102.50
>       Read Type: 75bp x 75bp
>       Fragment Length Distribution: Truncated Gaussian (default)
>                     Estimated Mean: 200
>                  Estimated Std Dev: 80
[13:23:15] Assembling transcripts and estimating abundances.
> Processed 1 loci.                            [*************************] 100%



Verify that the file transcripts.gtf is in the current directory and looks like this (your file will have GTF attributes, omitted here for clarity)
test_chromosome Cufflinks       exon    53      250     1000    +       . 
test_chromosome Cufflinks       exon    351     400     1000    +       . 
test_chromosome Cufflinks       exon    501     550     1000    +       .

Tuesday, May 21, 2013

Compiling Eigen

Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms.. The current version is 3.1.3

To use the Eigen libraries, just untar and enter the source the directory
# tar -xvf eigen-3.1.3.tar

Copy the whole Eigen directory found in “eigen-eigen-2249f9c22fe8″ folder to the /usr/local/include directory
# cd eigen-eigen-2249f9c22fe8/
# cp -Rv Eigen /usr/local/include

Saturday, May 18, 2013

Dr. Eng Lim Goh on Designing SGI Altix UV

This is a refreshing technical design view on the design view of SGI Altix UV

 

Friday, May 17, 2013

SGI® DMF™ ZeroWatt™ Disk Solution

In this demonstration of SGI® DMF™ ZeroWatt™ disk solution, Dr. Eng Lim Goh, SGI CTO, discusses a function of SGI DMF software to reduce costs and power consumption in an exascale (Big Data) storage datacenter.


Wednesday, May 15, 2013

Redhat and CentOS Update for httpd

 Redhat Update for httpd

Description
Red Hat has issued an update for httpd. This fixes multiple vulnerabilities, which can be exploited by malicious people to conduct cross-site scripting attacks and compromise a vulnerable system.

1) The "do_rewritelog()" function (modules/mappers/mod_rewrite.c) does not properly handle certain escape sequences when writing to the log file and can be exploited by sending a specially crafted HTTP request.

Successful exploitation of this vulnerability may allow execution of arbitrary commands but requires the user to view the log file in a terminal emulator.


Original AdvisoryRHSA-2013:0815-1:
http://rhn.redhat.com/errata/RHSA-2013-0815.html

Red Hat:
https://bugzilla.redhat.com/show_bug.cgi?id=953729

Sunday, May 12, 2013

Compiling SamTools

Taken from Cufflinks – Getting Started

For more information, see Compiling SamTools


SAM (Sequence Alignment/Map) format is a generic format for storing large nucleotide sequence alignments. SAM Tools provide various utilities for manipulating alignments in the SAM format, including sorting, merging, indexing and generating alignments in a per-position format.

SAMTools (0.1.19)
Step 1: – Download SAMTools

Step 2: – Unpack and compile the SAMTools
# tar -zxvf samtools-0.1.19
# make

Step 3: – Copy the compiled libraries to /usr/local/lib
# cp $SAMTOOLS_HOME/libbam.a /usr/local/lib

Step 4: Copy headers to /usr/local/include
# mkdir /usr/local/include/bam
# cp $SAMTOOLS_HOME/*.h /usr/local/include/bam

Friday, May 10, 2013

Compiling and Installing Boost 1.53

Referenced from Cufflinks – Get started.



Step 1 – Download Boost from Boost Website

Step 2 – Unpack the boost
# tar -zxvf boost_1_53_0.tar.gz

Step 3 – Run Bootstrap.sh
# ./bootstrap.sh
Building Boost.Build engine with toolset gcc...
.....

Step 4 – Run the binary b2
# ./b2 install --prefix=/usr/local/boost
It will take a while……

Step 5: Test that the compilation is working.
Compile the test case.
# cd $BOOST_ROOT/boost_1_53_0/tools/build/v2/example/hello

# $BOOST_ROOT/b2
You should see a binary file “hello” at
# $BOOST_ROOT/boost_1_53_0/tools/build/v2/example/hello/bin/gcc-4.4.6/debug

For more information, see Compiling and Installing Boost 1.53

Monday, May 6, 2013

Enable and Disable Quota Management for GPFS

Taken from GPFS Administration and Programming Reference – Enabling and disabling GPFS quota management
To enable GPFS quota management on an existing GPFS file system
  1. Unmount the file system everywhere.
  2. Run the mmchfs -Q yes command. This command automatically activates quota enforcement whenever the file system is mounted.
  3. Remount the file system, activating the new quota files. All subsequent mounts follow the new quota setting.
  4. Compile inode and disk block statistics using the mmcheckquota command. The values obtained can be used to establish realistic quota values when issuing the mmedquota command.
  5. Issue the mmedquota command to explicitly set quota values for users, groups, or filesets.
Once GPFS quota management has been enabled, you may establish quota values by:
  1. Setting default quotas for all new users, groups of users, or filesets.
  2. Explicitly establishing or changing quotas for users, groups of users, or filesets.
  3. Using the gpfs_quotactl() subroutine.
To disable quota management:
  1. Unmount the file system everywhere.
  2. Run the mmchfs -Q no command.
  3. Remount the file system, deactivating the quota files. All subsequent mounts obey the new quota setting.
To enable GPFS quota management on a new GPFS file system:
  1. Run  mmcrfs -Q yes command. This option automatically activates quota enforcement whenever the file system is mounted.
  2. Mount the file system.
  3. Issue the mmedquota command to explicitly set quota values for users, groups, or filesets. See Explicitly establishing and changing quotas.

Thursday, May 2, 2013

Deleting Nodes from a GPFS Cluster

Taken from Deleting Nodes from a GPFS Cluster (IBM)

Step 1: Shutdown the Nodes before deleting
On the NSD Node
# mmshutdown -N node01

mmshutdown -N node01 
Wed May  1 01:09:51 SGT 2013: mmshutdown: Starting force unmount of GPFS file systems
Wed May  1 01:09:56 SGT 2013: mmshutdown: Shutting down GPFS daemons
node01:  Shutting down!
node01:  'shutdown' command about to kill process 10682
node01:  Unloading modules from /lib/modules/2.6.32-220.el6.x86_64/extra
node01:  Unloading module mmfs26
node01:  Unloading module mmfslinux
node01:  Unloading module tracedev
Wed May  1 01:10:04 SGT 2013: mmshutdown: Finished


Step 2: Deleting a Node
# mmdelnode -N node01

Verifying GPFS is stopped on all affected nodes ...
mmdelnode: Command successfully completed
mmdelnode: Propagating the cluster configuration data to all
affected nodes.  This is an asynchronous process.


Step 3: Confirm that the nodes has been deleted
# mmlscluster


Step 4: If you are deleting the client permanently, check and update the license file.
# mmlslicense

Summary information
---------------------
Number of nodes defined in the cluster:                         20
Number of nodes with server license designation:                 3
Number of nodes with client license designation:                17
Number of nodes still requiring server license designation:      0
Number of nodes still requiring client license designation:      0