Thursday, January 10, 2013

Killing a SSH User Shell Session

As Administrator, when you do a notice that you will some of your users have idle and active ssh session. The idle ssh session could be be due to the hanged ssh session. So the question is how to remove the individual session without killing the active and genuine session.

First thing first, do

# w 

You may get some information like this
USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU WHAT 

.....
user1   pts/31   :24.0            08Oct12 22days  0.05s  0.05s -bash
user2   pts/24   :30              02Jan13  2days  0.66s  0.66s -bash
user3   pts/55   :17              12Nov12 59days  0.01s  0.01s -bash
.....

To get process id (PID) of the idle session, do the command
# ps -aux |grep 'pts/31'

Warning: bad syntax, perhaps a bogus '-'? See /usr/share/doc/procps-3.2.7/FAQ
root     27552  0.0  0.0  61172   776 pts/1    S+   00:41   0:00 grep pts/31
546      30050  0.0  0.0  64188  1516 pts/31   Ss+   2012   0:00 -bash

Kill the Process
# kill -9 30050 

The idle ssh session has been removed. You can verify with the command "w"

Wednesday, January 9, 2013

Disabling CPU speed scaling in CentOS 5

If you are using CentOS, you may notice that with the cpuspeed daemon, you will find that somehow you do not get the full CPU Frequency as specified in the CPU model.

To check whether you have discrepancy, do fire the command
# grep -E '^model name|^cpu MHz' /proc/cpuinfo

model name      :        Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
cpu MHz         : 1200.000
model name      :        Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
cpu MHz         : 1200.000
model name      :        Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
cpu MHz         : 1200.000
model name      :        Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
cpu MHz         : 1200.000
model name      :        Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
cpu MHz         : 1200.000
model name      :        Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
cpu MHz         : 1200.000
model name      :        Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
cpu MHz         : 1200.000
model name      :        Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
cpu MHz         : 1200.000
model name      :        Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
cpu MHz         : 1200.000
model name      :        Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
cpu MHz         : 1200.000
model name      :        Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
cpu MHz         : 1200.000
model name      :        Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
cpu MHz         : 1200.000

You can stop CPU throttling by disabling the cpuspeed daemon
# service cpuspeed stop

But if you do a closer check, you realise that if you have dual CPU socket, the disabling of the daemon seems to only impact 1 CPU socket..... Notice the figures below.....Hmmmm

model name      :        Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
cpu MHz         : 2501.000
model name      :        Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
cpu MHz         : 2501.000
model name      :        Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
cpu MHz         : 2501.000
model name      :        Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
cpu MHz         : 2501.000
model name      :        Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
cpu MHz         : 2501.000
model name      :        Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
cpu MHz         : 2501.000
model name      :        Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
cpu MHz         : 1200.000
model name      :        Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
cpu MHz         : 1200.000
model name      :        Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
cpu MHz         : 1200.000
model name      :        Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
cpu MHz         : 1200.000
model name      :        Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
cpu MHz         : 1200.000
model name      :        Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
cpu MHz         : 1200.000

Monday, January 7, 2013

Errors when running doing ib testing with ib_write_lat

I was doing a ib test using  perftest package which has simple tests for benchmarking IB bandwidth and latency. the 2 simple packages are ib_write_bw and ib_write_lat. 

On the Server side, I launched
# ib_write_lat -a

------------------------------------------------------------------
                    RDMA_Write Latency Test
 Number of qps   : 1
 Connection type : RC
 Mtu             : 2048B
 Link type       : IB
 Max inline data : 400B
 rdma_cm QPs     : OFF
 Data ex. method : Ethernet
------------------------------------------------------------------
 local address: LID 0x03 QPN 0x0065 PSN 0xba11f4 RKey 0x003900 VAddr 0x002ab9bad9600

On the Client Side,
# ib_write_lat -a 192.168.5.1

The errors are as followed
Conflicting CPU frequency values detected: 1200.000000 != 2501.000000
 2       1000          inf            inf          inf
Conflicting CPU frequency values detected: 1200.000000 != 2501.000000
 4       1000          inf            inf          inf
Conflicting CPU frequency values detected: 1200.000000 != 2501.000000
 8       1000          inf            inf          inf
Conflicting CPU frequency values detected: 1200.000000 != 2501.000000
 16      1000          inf            inf          inf
Conflicting CPU frequency values detected: 1200.000000 != 2501.000000
 32      1000          inf            inf          inf

To solve the issues, use "-F" option while running the tests. The flag will ignore "Conflicting CPU frequency" errors. Although there will still be error messages, but with "-F", you will also see the results at least

A better solution is to disable the cpuspeed if you are on CentOS. For more information see blog entry

Friday, January 4, 2013

Performance Tuning of Latency-Sensitive Workloads in Vmware vSphere VMs

This article Best Practices for Performance Tuning of Latency-Sensitive Workloads in vSphere VMs (pdf). This article covers and recommends the tuning of differnt layers of an application's environment for latency-sensitive workloads. A word of caution in the article, is that the recommendations in the paper the tweaking may hurt the performance that are tolerant of higher latency

Wednesday, January 2, 2013

Multiprotocol Performance Test of VMware EX 3.5 on NetApp Storage Systems

This paper "Performance Report: Multiprotocol Performance Test of VMware® ESX 3.5 on NetApp Storage Systems" by NetApp and Vmware was written in 2008 is a interesting comparison with 3 protocols, FC, iSCSI and NFS. The variation between the protocols is about maximum 9%. With the advent of the new Vmmare VSphere, 10G with TOE etc, the performance gap may be close that in 2008.

The summary of the pro and cons as mentioned in the article is summarised at Multiprotocol Performance Test of VMware EX 3.5 on NetApp Storage Systems