Removing old package for Linux Mint

To remove the orphaned dependencies using apt-get

# apt-get autoremove

To clean old versions of packages using apt-get

# apt-get autoclean

To remove all packages files from the acahe using apt-get
# apt-get clean

Installing Java Runtime Environment (JRE) from the command line on Linux Mint

If you are planning to install JRE and the web browser plugin, you will need to do the following:

# apt-get install sun-java6-jre sun-java6-plugin sun-java6-fonts

Prevent non-users from login into the system

Sometimes as administrator, you may need to some immediate repair work and need to prevent users from logging in immediately, you can easily do that by doing the following:

# touch /etc/nologin

If the file exists, only the root can gain accessnow

pbs node manipulation Revision

It has been a long time since I used torque. I though I pen down some common commands I will used

• pbsnodes -l (List node names and their state. If no state is specified, only nodes in the DOWN, OFFLINE, or UNKNOWN states are listed. Specifying a state string acts as an output filter. Valid state strings are "active", "all", "busy", "down", "free", "offline", "unknown", and "up". )
  
• pbsnodes -o (Add the OFFLINE state. This is different from being marked DOWN. OFFLINE prevents new jobs from running on the specified nodes. This gives the administrator a tool to hold a node out of service without changing anything else. The OFFLINE state will never be set or cleared automatically by pbs_server; it is purely for the manager or operator.)
  
• pbsnodes -c (Clear OFFLINE from listed nodes)
  
• pbsnodes -a (All attributes of a node or all nodes are listed. This is the default if no flag is given)
  

libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations.

We are using OpenFarics for our HPC. According to the FAQ, "With OpenFabrics (and therefore the openib BTL component), you need to set the available locked memory to a large number (or better yet, unlimited) -- the defaults with most Linux installations are usually too low for most HPC applications that utilize OpenFabrics"

libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
This will severely limit memory registrations.

For the solution, look here at http://www.open-mpi.org/faq/?category=openfabrics#ib-locked-pages
Or you can also look at my blog entry Encountering Segmentation Fault, Bus Error or No output

Looking at Xorg High CPU Usage Issue for netbook

If you are suffering from excessive slowness from your netbook or your old and slow notebook, you may want to tweak the CPU frequency governor to performance mode instead of the general mode. How do you do that? If you are using Linux Mint, you have to install a utilities called cpufrequtils

Step 1: Install CPU Frequency Utilities
# apt-get install cpufrequtils

Step 2: Query the CPU frequency governors
# cpfreq-info

Step 3: Force the CPU to use performance mode
# cpufreq-set -g performance

If this does not resolve the issues, you really have to look at other possibilities that is impacting your Xorg. Sometimes it could be your video drivers not the CPU governor issues....

Encountering Segmentation Fault, Bus Error or No output

When you encounter some of the issues like Segmentation Fault, Bus Error or No output during your serial or parallel run, it is could be due that the user limits are insufficient

Firstly do a ulimit -a and check the actual limits. Here is a sample output
# ulimit -a
===================================
core file size                 (blocks, -c) 0


data seg size                 (kbytes, -d) unlimited
scheduling priority         (-e) 0
file size                         (blocks, -f) unlimited
pending signals              (-i) 24574
max locked memory      (kbytes, -l) 64
max memory size          (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size                       (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority           (-r) 0
stack size                     (kbytes, -s) 10240
cpu time                       (seconds, -t) unlimited
max user processes      (-u) 1024
virtual memory             (kbytes, -v) unlimited
file locks                      (-x) unlimited
====================================
To solve the issues, you have to look at important parameters like data size, stack size, max memory size, and virtual memory. The parameters should be either unlimited or as big as your total RAM. 


# vim /etc/security/limits.conf
==================
* soft memlock unlimited
* hard memlock unlimited
* soft stack 4194303
* hard stack unlimited
==================

Log-off and Login again ad type # ulimit -a You should see that parameter has been fixed

Conversely, You can also use the configuration file to restrict memory and processes. See blog entry Using /etc/security/limits.conf to restrict processes and memory

Reference:

• Turbomole FAQ

Using Top and renice

When you are running a server that have more than 1 user competing for the limited number of resources, how do you prioitise the run manually?

I use top to check who is hogging the resources and I use the command

# renice +5 -u peter
# renice + 5 -p 8765

It helps alot. But I prefer the htop to influence process directly. See Using htop to influence processes directly

Using /etc/security/limits.conf to restrict processes and memory

The configuration file at /etc/security/limits.conf help in settings limits to processes or logins oe more. This helps in preventing a rogue applications in consuming too much processes that might cause the servers to crash.

* soft nproc 400

* hard nproc 500

(If you hit 400 processes, the system will warn you. If you hit 500, the system will block you) 

Alternatively if you do not wish to set a limit to the number of processes that can be run on your servers due to the particular application, you can also

* soft nproc unlimited

* hard nproc unlimited

You do not need to reboot to effect the settings, just relogin and the users will be subjected to the /etc/security/limits.conf configuration file

User friendly command line shell - Fish

fish is a user friendly command line shell for UNIX-like operating systems such as linux.

Some of the cool features of fish. It uses:

1. Uses fresh sprinkling of colour to indicate files, directory, prompt
2. Tab Completion for your commands. After typing the first few letters of the command, you can tab and suggested commands are made available
3. You can pick out command history by typing a few letter and clicking the "up" key instead of cycling through the whole history
4. A html document is made available when yo type fish --help.

To install fish at Fedora

# yum install fish

# fish

Similarly for Linux Mint

# apt-get install fish

# fish

Learning MOAB - Using showq

showq displays information about queued jobs including active, eligible, blocked, and/or recently completed jobs. Do note that the -c, -i, and -r flags can be used  Administrator only

Note 1: Display blocked jobs only
> showq -b

Note 2: Display details about recently jobs (by MOAB Admin)
# showq -c

Note 3: Display local and full resource manager job ids. If specified with the -i option, will display job reservation time. (If using MOAB
# show -vi

Note 4: Display jobs associated with specified constraint. Valid constraint include user, group, account, class and qos
> showq -w user=john

Note 5: Display extended details about active (running jobs)
# show -r
------------------------------------------------------
S - Job State. Either "R" for Running or "S" for Starting.
PAR - Partition in which job is running.
EFFIC - CPU efficiency of job.
XFACTOR - Current expansion factor of job, where XFactor = (QueueTime + WallClockLimit) / WallClockLimit
Q - Quality Of Service specified for job.
USERNAME - User owning job.
GROUP - Primary group of job owner.
MHOST - Master Host running primary task of job.
PROC  - Number of processors being used by the job.
--------------------------------------------------------

Learning MOAB. What is workload?

When you are doing High Performance Computing or Cloud, the concept of workload is very important. Basically there are 4 kinds of workloads (Taken and summarise from MOAB Admin Guide 5.2)

1. Batch Workload - With a Batch Job, the job is submitted to a job queue, and is run somewhere on the cluster as resources becomes available
2. Interactive Workload - Requestors are interested in immediate response and are generally waiting for interactive request to be executed before going on to other activities. To manage interactive jobs, the focus is usually on setting aside resources to guarantee immediate execution or at least a minimal wait time for interactive jobs
3. Calendar Workload - Calendar workload must be executed at a particular time and possibly in a regular periodic manner. There must be guarantee of resource availability at the needed time to allow calendar jobs to run as required.
4.  Service Workload - Service workload processes externally generated transaction requests while a scheduler or resource allocation mechanism provides the distributed service with needed resources to meet target backlog or response targets to the service. Examples are web farms etc. 

Installing VirtualBox on Fedora 11

Step 1: You can either install by downloading and installing the rpm directly of use the Fedora repo. There are repo for RHEL and OpenSuSE as well

Step 1a: If you wish to download and install the rpm directly, type the following
# wget
http://download.virtualbox.org/virtualbox/3.0.8/VirtualBox-3.0.8_53138_fedora11-1.i586.rpm
# yum localinstall VirtualBox-3.0.8_53138_fedora11-1.i586.rpm

Step 1b: A better method in my opinion will be by using the repository method
Create a virtualbox.repo file at /etc/yum.repos.d/
# touch /etc/yum.repos.d/virtualbox.repo
# vim virtualbox.repo
-------------------------------
[virtualbox]
name=VirtualBox
baseurl=http://download.virtualbox.org/virtualbox/rpm/fedora/$releasever
enabled=1
gpgcheck=0
---------------------------------
# yum check-update


Step 2: Run setup file for Virtualbox
# /etc/init.d/vboxdrv setup


Step 3: Add yourself as a user in the vboxusers group at /etc/group 
# usermod -G vboxusers myserid


Step 4: USB Fix for VirtualBox
# vim /etc/group
----------------------------------------------
# create a new group and add your name to it
usb:x:502:myserid
----------------------------------------------

# vim /etc/fstab
----------------------------------------------
none /sys/bus/usb/drivers usbfs devgid=502,devmode=664 0 0
----------------------------------------------

Step 5: Run VirtualBox
# VirtualBox


Reference:

• USB Fix for VirtualBox on Fedora 8 Host

Changing Compilers from Intel to PGI for OpenMPI

Extending from Building OpenMPI with Intel Compiler (Ver 2). Assuming you are using Intel Compiler as the default compiler.

Step 1: If you wish to change to another compiler like PGI (in this example) from the Intel Compiler,
You can issue the commands
# export CC=pgcc
# export CXX=pgCC
# export F77=pgf77
# export FC=pgf90

To check that the openmpi is using the PGI compiler
# /usr/local/openmpi/bin/mpicc --showme
# /usr/local/openmpi/bin/mpiCC --showme
# /usr/local/openmpi/bin/mpif77 --showme
# /usr/local/openmpi/bin/mpif90 --showme


Step 2: If you wish to "return" to the default
# unset CC
# unset CXX
# unset F77
# unset FC


To check that the openmpi is using the PGI compiler
# /usr/local/openmpi/bin/mpicc --showme
# /usr/local/openmpi/bin/mpiCC --showme
# /usr/local/openmpi/bin/mpif77 --showme
# /usr/local/openmpi/bin/mpif90 --showme

eyeos - The Open Source Cloud Web's Desktop

According to the eysos website,

eyeos is an open source web desktop following the cloud computing concept. It is mainly written in PHP, XML, and Javascript. It acts as a platform for web applications written using the eyeos Toolkit. It includes a Desktop environment with 67 applications and system utilities. It is accessible by portable devices via its mobile front end. eyeos lets you upload your files and work with them no matter where you are. It contains applications like Word Processor, Address Book, PDF reader, and many more developed by the community

Intel Data Center Manager Software

Intel® Data Center Manager (Intel® DCM) SDK provides power and thermal monitoring and management for servers, racks and groups of servers in data centers. Management Console Vendors (ISVs) and System Integrators (SIs) can integrate Intel® DCM into their console or command-line applications and provide high value power management features to IT organizations


Intel® DCM features include:

• Built-in policy based intelligent heuristics engine that can maintain group power capping while dynamically adapting to changing server loads and minimizing performance impact of workloads
• Uses Intel® Intelligent Power Node Manager for node power and thermal management
• Designed as an SDK to integrate into existing management software products by means of a web service application programming interface (WSDL API)

Video on Overview of Intel Data Center Manager

Free ebook - High Performance Computing for Dummies

This special edition eBook "High Performance Computing for Dummies" from Sun and AMD shares details on real-world uses of HPC, explains the different types of HPC, guides you on how to choose between different suppliers, and provides benchmarks and guidelines you can use to get your system up and running.


Happy Reading!

Integrated Development Environment for C, C++, Python and GTK

Anjuta is a versatile Integrated Development Environment (IDE) for C and C++ on GNU/Linux.

Anjuta also works well with Glade fro building Gnome Apps GTK. Do note that there is also support for Java and Python.

The application wizards is a wonderful feature as it guide you along to setting up all the files for the project.

One limiting feature is the lack of quality documentation.

To install for Debian distro derivatives
# apt-get install anjuta


To install for Red-Hat Distro Derivatives
# yum install anjuta

My Thoughts on what is Cloud Computing? (Part 1)

Cloud Computing is gaining a lot of publicity. I have attended a few myself. But up to now even if you gather the experts on this, even the definition is abstract. Maybe one of the best way is to ask a question and try to answer it

So what is Cloud Computing?

• Cloud Computing can be referred to as IT as a service. In the same way, a utility company delivers electricity for the end users for their specific and unique consumption, the challenge is whether Cloud Computing can deliver cheaper, faster, superior user experiences.

What are the essential components of Cloud?

• Service Catalogue - From my limited understanding, Service catalogue is the standardised IT offering you are offering to the customers. Maybe it is just some virtualised infrastructure for development, maybe it is a development environment like Hadoop, or even maybe it is even a Windows environment inside for an application requirement....
• Automation - By automation, what I meant was the ability to auto-provisioned by users for the standardised environment they require. In other words, there is some sort like a self-service portal where the users can select their choices from. Another essential atribute of automation comes in the form of workload analysis. The ability of the Cloud Computing infrastructure to meet the demand based on the workload analysis
• Virtualisation - Virtualisation is very essential as many image format of the computing environment or the service catalogue can be kept efficiently. It is also a mean where the image can be deployed very quickly and the environment consistently kept for the particular user. Bare-metal provisioning is not impossible for the Cloud Environment, it is just less efficient and more time consuming as the nodes have to be reformatted with the OS and the image. Of course you can dual-boot or triple-boot. If your number of images are small or have very standardised user base, it is still manageable.

What are the catalysts pushing industry to Cloud Computing?

• I think the current IT infrastructure limitation will gets more and more glaring as the explosion of services and data are harder to manage in a distributed environment. On average, about 85% of a compute resource remain idle. This is definitely not the best use of resources.
• Another is the amount of budget spent on maintaining instead of adding new capabilities is another "push" factor. With a consolidated environment, cost will definitely be more managed.
• I guess the last catalyst will be is the rising perception of the users. IT services is going to be just a utility. IT is so pervasive now like electricity. If you check in a hotel and there is no Internet, it is perceived as very out-of-touch. Rising expectation will definitely put demands on IT resources and yet the need to be flexible to provision with peak and trough of computing demands. This is the impetus given to the Cloud

Installing Presto Plugin for yum to boost speed

You can install the Presto Plugin for yum to download the changes from previous download which will reduce download time

# yum install yum-presto

Making yum use the fastest mirror

To make sure yum plugin download packages from the fastest mirror, you can install the following plugins

# yum install yum-fastestmirror

Restarting X server with keyboard short-cut on Fedora 11

If you need to restart X Server quickly and do not wish to use the commandline, you can use CRTL-ALT-BACKSPACE.

If you use the keyboard short-cut and it does not restart the X11, you may want to check the settings on

1. the System > Preferences > Keyboard. 
2. Select the Layout Tab and click the "Layout Options...." Button. 
3. Finally, click the item "Key sequence to kill the X Server" and check the item
   

 

And that's it.....

Enable Automatic Login for GNOME on Fedora 11

For debian derivative linux, you may want to view Enable Automatic Login for GNOME on Linux Mint,

For Fedora 11, there is a slightly different way to configure if you need automatic login. Fedora 11 uses the xml schematic to configure.

# vim /etc/gdm/gdm.schemas

 

You can tweaked the xml schematic from the screen shot above to enable automatic login

Enable Automatic Login for GNOME on Linux Mint

If you wish to speed up your boot-time and totally not concerned with security, you may want to enable automatic login.

For Linux Mint, my favourite Linux desktop, you can use the command line to enable automatic login.

# vim /etc/gdm/gdm.conf

-----------------------------------------

[daemon]

......

AutomaticLoginEnable=true
AutomaticLogin=(your userid to login)

-----------------------------------------

Alternatively, you can use the gdmsetup to configure. See pix below.

Installing chrome on Fedora 11

Installing Chrome is not too difficult on Fedora 11. Assuming you are using yum, you can create a new repository file called "chromium.repo" and input the following contents into the file

[chromium]
name=Chromium Test Packages
baseurl=http://spot.fedorapeople.org/chromium/F11/
enabled=1
gpgcheck=0

After saving the file, you should run the following
# yum update
# yum install chromium


Alternatively, you can install by downloadin the rpm from http://spot.fedorapeople.org/chromium/
and run a
# rpm -ivh (chromium package)

Rocky Mountain Supercomputing Centers (RMSC) - Supercomputer platform as a Service

Cloud Computing is evolving so quickly. Usually HPC and Enterprise Computing are usually very seperate with distinct characeristics and usually it does not converge so easily.

Cloud Computing which is the natural evolution of virtualisation and consolidation and usually comes from the Enterprise Computing paradigm is  finally closing the gap of this convergence.

One important development can be seen from Rocky Mountain Supercomputing Centers (RMSC) may be one of the first HPC Cloud being developed with the concept of supercomputing on the demand concept or Supercomputer platform as a service (SPaaS). See Rocky Mountain Supercomputing Centers (RMSC) offering.

Very thought provoking

Video of IBM Cloud Computing Community

IBM has setup a IBM Cloud Channel via Youtube. In the Cloud Channel, you have a collection of customers testimonials and how IBM is shaping business strategy via Cloud.

Good site to know more about IBM Cloudburst offering.