Quick Listing of users who have current login session

If you wish to have quicklisting of users who are logging on to your servers, you can use the command "users". If a user is running multiple session, they will appear multiple time.

# users
root root user1 user2 user3 user3

There are more comprehensive tools like who and finger. Will write in future blog entries.

Using host command as an alternative to nslookup

host is a simple utility for performing DNS lookups. It is normally used to convert names to IP addresses and vice versa. When no arguments or options are given, host prints a short summary of its command line arguments and options.

Common basic Usages

Using host command to check resolving DNS Servers

# host www.google.com.sg

www.google.com.sg has address 173.194.38.159
www.google.com.sg has address 173.194.38.151
www.google.com.sg has address 173.194.38.152
www.google.com.sg has IPv6 address 2404:6800:4003:802::1018

Using host command with "-a" to display a query of type ANY

# host -a google.com.sg

Trying "google.com.sg"
;; ->>HEADER<<- 51686="51686" br="br" id:="id:" noerror="noerror" opcode:="opcode:" query="query" status:="status:">;; flags: qr rd ra; QUERY: 1, ANSWER: 14, AUTHORITY: 4, ADDITIONAL: 3

;; QUESTION SECTION:
;google.com.sg.                 IN      ANY

ANSWER SECTION:
google.com.sg.          177     IN      TXT     "v=spf1 -all"
google.com.sg.          86277   IN      SOA     ns1.google.com. dns-admin.google.com. 2012032600 21600 3600 1209600 300
google.com.sg.          177     IN      AAAA    2404:6800:4003:801::101f
google.com.sg.          177     IN      A       74.125.235.63
google.com.sg.          177     IN      A       74.125.235.55
google.com.sg.          177     IN      A       74.125.235.56
google.com.sg.          10677   IN      MX      10 google.com.s9b1.psmtp.com.
google.com.sg.          10677   IN      MX      10 google.com.s9b2.psmtp.com.
google.com.sg.          10677   IN      MX      10 google.com.s9a1.psmtp.com.
google.com.sg.          10677   IN      MX      10 google.com.s9a2.psmtp.com.
google.com.sg.          345477  IN      NS      ns2.google.com.
google.com.sg.          345477  IN      NS      ns3.google.com.
google.com.sg.          345477  IN      NS      ns1.google.com.
google.com.sg.          345477  IN      NS      ns4.google.com.

;; AUTHORITY SECTION:
google.com.sg.          345477  IN      NS      ns2.google.com.
google.com.sg.          345477  IN      NS      ns3.google.com.
google.com.sg.          345477  IN      NS      ns1.google.com.
google.com.sg.          345477  IN      NS      ns4.google.com.

;; ADDITIONAL SECTION:
ns1.google.com.         188413  IN      A       216.239.32.10
ns2.google.com.         188413  IN      A       216.239.34.10
ns3.google.com.         188413  IN      A       216.239.36.10

Using host -t parameter to select the query type

# host -t MX google.com.sg

google.com.sg mail is handled by 10 google.com.s9b2.psmtp.com.
google.com.sg mail is handled by 10 google.com.s9a1.psmtp.com.
google.com.sg mail is handled by 10 google.com.s9a2.psmtp.com.
google.com.sg mail is handled by 10 google.com.s9b1.psmtp.com.

Using free to see memory and cache usage

The utility free is a useful tool for Linux to help to analyse the memory usage

To get the output below, I issued a command

# free -mt

where
-m => Display amount in  megabytes
-t => Add a Total row at the bottom
-s N => Run continuously and update the display  every N seconds

             total       used       free     shared    buffers     cached
Mem:          7871       5995       1875          0        310       3729
-/+ buffers/cache:       1956       5914
Swap:        10047        283       9764
Total:       17919       6279      11640

How do we interpret the results. I would like to credit the article from Determining free memory on Linux for a simple but very good explanation on memory.Here is what I have gathered from the read-up.

Basically, Linux caches blocks from the disk in memory to make data reading as efficient as possible. The buffer cache will shrink to accommodate the increased memory needs.

The actual free memory is column free (1875) + Buffers (310) + Cached (3729). So it is more than we think

Compiling and Installing Boost C++ Libraries on CentOS

Boost C++ libraries provides free peer-reviewed portable C++ source libraries. Boost libraries are intended to be widely useful, and usable across a broad spectrum of applications.

Easy Build and Install (Taken from Getting Started on Unix Variants)

$ cd path/to/boost_1_52_0
$ ./bootstrap.sh --help

Select your configuration options and invoke ./bootstrap.sh again without the --help option. Unless you have write permission in your system's /usr/local/ directory, you'll probably want to at least use

$ ./bootstrap.sh --prefix=/usr/local/boost
$ ./b2 install

This will leave Boost binaries in the lib/ subdirectory of your installation prefix. You will also find a copy of the Boost headers in the include/ subdirectory of the installation prefix, so you can henceforth use that directory as an #include path in place of the Boost root directory.

JELLYFISH - Fast, Parallel k-mer Counting for DNA

What is Jellyfish - Fast, Parallel k-mer Counting for DNA?
(Taken from Jellyfish Site)
JELLYFISH is a tool for fast, memory-efficient counting of k-mers in DNA. A k-mer is a substring of length k, and counting the occurrences of all such substrings is a central step in many analyses of DNA sequence. JELLYFISH can count k-mers using an order of magnitude less memory and an order of magnitude faster than other k-mer counting packages by using an efficient encoding of a hash table and by exploiting the "compare-and-swap" CPU instruction to increase parallelism.

JELLYFISH is a command-line program that reads FASTA and multi-FASTA files containing DNA sequences. It outputs its k-mer counts in an binary format, which can be translated into a human-readable text format using the "jellyfish dump" command. See the documentation below for more details.


Requirements:

JELLYFISH runs on 64-bit Intel-compatible processors running Linux or FreeBSD (including Intel Macs). It requires GNU GCC to compile.


Download (current version 1.1.6.):
http://www.cbcb.umd.edu/software/jellyfish/jellyfish-1.1.6.tar.gz


Installation:

# ./configure --prefix=/usr/local/jellyfish
# make
# make install

Testing- Test 1

# make check

... 
...
====================
All 19 tests passed
(1 test was not run)
====================
...
...

All tests should pass and 1 test should be skipped (big.sh). Running
'make check' will use about 50MB of disk space and will use every CPUs
found on the machine. On our test machine with 32 cores, it takes a
few minutes to run.

Testing -Test 2

# make check BIG=1

....
....
PASS: tests/generate_sequence.sh
PASS: tests/serial_hashing.sh
PASS: tests/parallel_hashing.sh
PASS: tests/serial_direct_indexing.sh
PASS: tests/parallel_direct_indexing.sh
....
....

Basic Installation of Quake - Package to correct substitution sequencing errors in experiments with deep coverage

What is Quake?
(Taken from Quake Site)

Quake is a package to correct substitution sequencing errors in experiments with deep coverage (e.g. >15X), specifically intended for Illumina sequencing reads. Quake adopts the k-mer error correction framework, first introduced by the EULER genome assembly package. Unlike EULER and similar progams, Quake utilizes a robust mixture model of erroneous and genuine k-mer distributions to determine where errors are located. Then Quake uses read quality values and learns the nucleotide to nucleotide error rates to determine what types of errors are most likely. This leads to more corrections and greater accuracy, especially with respect to avoiding mis-corrections, which create false sequence unsimilar to anything in the original genome sequence from which the read was taken.

Setting up is quite straight-forward, just untar in an appropriate directory.

# tar -zxvf quake-0.3.4.tar.gz
# cd Quake\src

Edit the Makefile if you are using Linux (Link CFLAGS to Boost Directory). Boot Software can be downloaded at Boost C++ Libraries

CC=g++
CFLAGS=-O3 -fopenmp -I/usr/local/boost/include/boost -I.
LDFLAGS=-L. -lgzstream -lz
.....
.....

To complete the installation, do a make at the src

 # make

You should see executable in the src

IBM Mobile Solution

IBM Mobile Solution based on the Worklight and Mobile Foundation Platform is making waves for Enterprise Mobile Solutioning. Here are some news excerpt

1. IBM Interactive has already been named a leader in building engaging mobile apps. The IBM Mobile press release, in conjunction with the communications and assets below, continues to build out our mobile story and reinforce IBM as a leader in the mobile space:
2. Forbes article (IBM Takes Mobile To The Road With Tools & Strategy) illustrates how the IBM mobile portfolio offers a complete solution set. Positive press and analyst reactions serve to further validate the strength of our offerings.
3. The Wall Street Journal published an IBM Op-Ad entitled, "The Mobile World is Open for Business." The piece established IBM's mobile leadership and highlighted examples of companies using our products and services.

Timestamp for BASH history

If you run to "record" a time-stamp history on your BASH command so that you can do a better tracing of what was run at certain time. It is quite easy to implement.

$ echo 'export HISTTIMEFORMAT="%h/%d - %H:%M:%S "' >> ~/.bashrc

$ source .bashrc

$ history

  ....
  ....
  997  Nov/20 - 13:46:11 vim .bashrc
  998  Nov/20 - 13:46:17 source .bashrc
  999  Nov/20 - 13:46:26 ls
 1000  Nov/20 - 13:46:36 exit
 1001  Nov/20 - 13:46:50 ls
 1002  Nov/20 - 13:46:54 ssh server-c00
 1003  Nov/20 - 13:47:15 ls
 1004  Nov/20 - 13:47:19 ls -al
 1005  Nov/20 - 13:47:27 history
 1006  Nov/20 - 13:56:58 history
 1007  Nov/20 - 13:57:02 history|more
 1008  Nov/20 - 13:58:47 history|more

Graphical Interface to manage runlevels - ntsysv

If you like GUI outlay for chkconfig, you may want to use this utility ntsysv which you can modify very utility. Installation could not be easier.

# yum install ntsysv

================================================================================
 Package            Arch            Version                 Repository     Size
================================================================================
Updating:
 ntsysv             x86_64          1.3.49.3-2.el6          base           29 k
Updating for dependencies:
 chkconfig          x86_64          1.3.49.3-2.el6          base          159 k

Transaction Summary
================================================================================
Upgrade       2 Package(s)

Total download size: 188 k
Is this ok [y/N]:y 

Compiling and Installing GAP System for Computational Discrete Algebra

GAP is a system for computational discrete algebra, with particular emphasis on Computational Group Theory. GAP provides a programming language, a library of thousands of functions implementing algebraic algorithms written in the GAP language as well as large data libraries of algebraic The objects. The information is taken from Compilation Step 1: Download the GAP Software Download the GAP Software at http://www.gap-system.org/Releases/index.html Current version at point of writing is 4.5.6

# tar -zxvf gap4r5p6_2012_11_04-18_46.tar.gz

# cd gap4r5p6

Step 2: Configure and Install (Default Installation)

#./configure

# make

Step 3: Optional Installation - GMP packages. If you wish to use the GAP internal GMP Packges, then the version of GMP bundled with this GAP will be used. This is the default.

# ./configure --with-gmp=yes|no|system|"path"

Step 4: Optional Installation - Readline Editing for better command-line Editing. If the argument you supply is yes, then GAP will look in standard locations for a Readline installed on your system. Or you can specify a path to a Readline installation.

# ./configure --with-readline=yes|no|"path"

For more information, See INSTALL when you unpacked the GAP.

Apache Server Setting Mistakes Can Aid Hackers

This article from Michael J. Schwartz on InfomationWeek titled Apache Server Setting Mistakes Can Aid Hackers

According to a study of 10 million websites released last week, more than 2,000 sites -- including big-name businesses such as Cisco, Ford and Staples -- have left the status pages for their Apache servers visible, which could give attackers information that would help them penetrate corporate networks.
.......
.......
According to Apache documentation, the Apache mod_status module "allows a server administrator to find out how well their server is performing," via an HTML page that delivers up-to-date server statistics. "It is basically an HTML page that displays the number of [processes] working, status of each request, IP addresses that are visiting the site, pages that are being queried and things like that. All good," said Cid in a related blog post. 

"However, this feature can also have security implications if you leave it wide open to the world. Anyone would be able to see who is visiting the site, the URLs and sometimes even find hidden -- obscure -- admin panels or files that should not be visible to the outside," he said. "That can help attackers easily find more information about these environments and use them for more complex attacks." 

......
......
For more information, I encourage to read the full article on Apache Server Setting Mistakes Can Aid Hackers

Installing and Configuring Environment Modules on CentOS 6

This tutorial is very similar to Installing and Configuring Environment Modules on CentOS 5 and the steps are very similar for CentOS 6 except that the tcl/tk 8.5.x used in CentOS repository does not have tclConfig.sh which is needed when you compile the Modules packages. I used 8.4.x which is similar to the version used in the CentOS 5 repository. You can use more more updated version of tcl.

See Installing and Configuring Environment Modules on CentOS 6.

Further Information

1. Installing and Configuring Environment Modules on CentOS 5
2. Usage of Environment Modules on CentOS and in Cluster

Usage of Environment Modules on CentOS and in Cluster

This is the 2nd Part continuation of Installing and Configuring Environment Modules on CentOS 5


In the write-up, Usage of Environment Modules on CentOS and in Cluster, Basic Usage of Module commands like module avail, module load, module unload and module switch.

Installing and Configuring Environment Modules on CentOS 5

Here a short tutorial on how to install and configure Environmental Modules on CentOS. See
Installing and Configuring Environment Modules on CentOS 5 for more details.
In the short tutorial,  I have document steps how to

1. Tools of managing User Environment in Linux
2. Unpacking, Installing and Configure Environment Modules. The Environment Modules Package can be taken from Environment Modules Project
3. Creating a sample Module File for Intel Compilers

Tools of managing User Environment in Linux

As far as I know there are 2 tools used to manage the environment.

1. One tool is SoftEnv Manual

• Softenv is a system used to build the user's environment. Each user has a ".soft" file in which they specify groups of applications that they're interested in. Softenv reads a central database when necessary to update the user's PATH, MANPATH and other variables. This version of SoftEnv is currently being used at MCS. 
• Current version 1.6.2. The last time it was updated is 12 March 2007
• Download Site - Sotftenv

2. The other tool is Modules - Software Environment Management

• The Environment Modules package provides for the dynamic modification of a user's environment via modulefiles.
  
  Each modulefile contains the information needed to configure the shell for an application. Once the Modules package is initialized, the environment can be modified on a per-module basis using the module command which interprets modulefiles. Typically modulefiles instruct the module command to alter or set shell environment variables such as PATH, MANPATH, etc. modulefiles may be shared by many users on a system and users may have their own collection to supplement or replace the shared modulefiles. 
• Current Version modules-3.2.9c. The last update was 19th Nov 2011
• Download Site - Environment Modules

Further Information

• Module vs SoftEnv
• Installing and Configuring Environment Modules on CentOS 5
• Usage of Environment Modules on CentOS and in Cluster