Sunday, June 28, 2015

Resolving unreach or unavail nodes in OpenLava-3.0

After configuring OpenLava-3.0 using the tar ball and following the instruction according to the OpenLava – Getting Started Guide After fixing OpenLava with LM is Down Error Messages for OpenLava-3.0, you may errors

HOST_NAME          STATUS       JL/U    MAX  NJOBS    RUN  SSUSP  USUSP    RSV
compute-c00     unreach              -     16      0      0      0      0      0
headnode-h00     ok              -     16      0      0      0      0      0

Suggestions:
  1. Check your permission where openlava-3.0 reside. Make sure the HeadNode and ComputeNode has the user and group openlava and openlava have permission on the folder
    drwxr-xr-x. 10 openlava openlava 4096 Jun 26 00:32 openlava-3.0
  2. Install pdsh. See Installing pdsh to issue commands to a group of nodes in parallel in CentOS on all the compute nodes and use pdcp to copy /etc/passwd /etc/shadow /etc/group to all the nodes
    # pdcp -a /etc/passwd /etc
    # pdcp -a /etc/shadow /etc
    # pdcp -a /etc/group /etc
  3. Make sure your /etc/hosts reflect the short hostname of the cluster both in the HeadNode and ComputeNode. Refrain from putting 2 hostnames per line.
  4. Check your firewalls settings. Make sure the ports 6322:6325 are opened.
  5. Ensure your NTP are synchronized across the clients and HeadNode with the designated NTP Server. If the NTP

Thursday, June 25, 2015

LM is Down Error Messages for OpenLava-3.0

After configuring OpenLava-3.0 using the tar ball and following the instruction according to the OpenLava – Getting Started Guide

I was encountering errors like
# lsid
openlava project 3.0, June 25 2015
ls_getclustername(): LIM is down; try later

Debugging:
# service openlava stop

# vim /usr/local/openlave-3.0/etc/lsf.conf

# /usr/local/openlava-3.0/sbin/lim -2

Solution (Check first):
Check that the
# hostname -s
# hostname -f

In your /etc/hosts, you may want to change to something like this. It solved my issues
127.0.0.1   headnode-h00 localhost

Friday, May 29, 2015

Inappropriate ioctl for device MSG=cannot create job file for Torque

I encountered this error on the cluster.

qsub: submit error (PBS_Server System error: 
Inappropriate ioctl for device MSG=cannot create job file 
/var/spool/torque/server_priv/jobs/497741.headnode-h00.cluster.com 
(28 - No space left on device)) 

I did a df -h and notice that there is still space. But when I did a df -i, I've noticed all the spaces iUSE% is almost 100% used up. To bring up the

# find / -xdev -printf '%h\n' | sort | uniq -c | sort -k 1 -n


References:
  1. Find where inodes are being used

Thursday, May 21, 2015

Beware of Trojanized version of Putty SSH client distributed in the Wild

Summary
Reports of a trojanized version of Opensource SSH PUTTY client is found to the distributed in the wild.

Attacks
 According to the report, if appear to occur in the following manner
  1. The victim performs a search for PuTTY on a search engine.
  2. The search engine provides multiple results for PuTTY. Instead of selecting the official home page for PuTTY, the victim unknowingly selects a compromised website.
  3. The compromised website redirects the user several times, ultimately connecting them to an IP address in the United Arab Emirates. This site provides the user with the fake version of PuTTY to download.

Mitigation
  1. Always ensure that you only download the software from the authors/publisher official homepage.
  2. Check the Software’s “About Information”. According to the report, the malicious version will show this. 
 References:


  1. http://www.net-security.org/malware_news.php?id=3041 
  2. http://www.symantec.com/connect/blogs/check-your-sources-trojanized-open-source-ssh-software-used-steal-information

Friday, May 15, 2015

Buffer Overflow vulnerability within the QEMU system emulator

Red Hat Product Security is now aware of a 'buffer overflow' vulnerability within the QEMU system emulator, which is widely installed and used for virtualization purposes on Linux systems. QEMU is also used by Red Hat’s cloud and virtualization products.

The vulnerability is known as VENOM and is assigned the identifier CVE-2015-3456.

This vulnerability affects the Floppy Disk Controller (FDC) emulation implemented in QEMU and could cause VM guests to crash the host's hypervisor and potentially facilitate arbitrary code execution on the host via guests. Even if the guest does not explicitly enable an FDC, all x86 and x86_64 guests are vulnerable.


For more more detailed information, do take a look at Redhat Security Blog: VENOM, don't get bitten 

  1. https://access.redhat.com/articles/1444903 
  2. https://securityblog.redhat.com/2015/05/13/venom-dont-get-bitten/