Ganglia – error collecting data (127.0.0.1:8652): fsockopen error: Connection refused

If you are having issues after installing Ganglia according to the Blog  Installing and Configuring Ganglia on CentOS 5.8, and if you are facing errors like Ganglia – error collecting data (127.0.0.1:8652): fsockopen error: Connection refused 

First thing do take a look at whether you have the necessary ganglia and rrdcached userid and group in your /etc/passwd and /etc/shadow. You should have something lines like thie

/etc/passwd

..... 
rrdcached:x:101:445:rrdcached:/var/rrdtool/rrdcached:/sbin/nologin
ganglia:x:102:446:Ganglia Monitoring System:/var/lib/ganglia:/sbin/nologin

At /etc/group

rrdcached:x:445:
ganglia:x:446:

Make sure your /var/lib/ganglia permission is

drwxr-xr-x 4 ganglia ganglia 4096 Aug 19 10:47 rrds

Or you can change permission if your permission is not correct

# chown -Rv ganglia.ganglia rrds

Error when yum install ganglia on CentOS 5.8

 I was doing a yum install of Ganglia on CentOS 5.8,

# yum install php rrdtool ganglia ganglia-gmetad ganglia-gmond ganglia-web httpd

I got the error as below. 

ganglia-web-3.1.7-3.el5.rf.x86_64 from rpmforge has depsolving problems
  --> Missing Dependency: php-gd is needed by package ganglia-web-3.1.7-3.el5.rf                                                              .x86_64 (rpmforge)
rrdtool-1.4.7-1.el5.rf.x86_64 from rpmforge has depsolving problems
  --> Missing Dependency: libdbi.so.0()(64bit) is needed by package rrdtool-1.4.                                                              7-1.el5.rf.x86_64 (rpmforge)
ganglia-web-3.1.7-3.el5.rf.x86_64 from rpmforge has depsolving problems
  --> Missing Dependency: php is needed by package ganglia-web-3.1.7-3.el5.rf.x8                                                              6_64 (rpmforge)
Error: Missing Dependency: php is needed by package ganglia-web-3.1.7-3.el5.rf.x                                                              86_64 (rpmforge)
Error: Missing Dependency: libdbi.so.0()(64bit) is needed by package rrdtool-1.4                                                              .7-1.el5.rf.x86_64 (rpmforge)
Error: Missing Dependency: php-gd is needed by package ganglia-web-3.1.7-3.el5.r                                                              f.x86_64 (rpmforge)
 You could try using --skip-broken to work around the problem
 You could try running: package-cleanup --problems
                        package-cleanup --dupes
                        rpm -Va --nofiles --nodigest

To avoid the error during the installation, see Installing and Configuring Ganglia on CentOS 5.8
for more information.

Step 1, install the libdbi.so.0(64bits) package

1. Make sure you have the RPM Repositories installed. For more information, see Useful Repositories for CentOS 5
2. Make sure installed the libdbi-0.8.1-2.1.x86_64.rpm for CentOS 5.9 be installed on the CentOS 5.8. Apparently, there is no conflict or dependency issues. See RPM Resource libdbi.so.0()(64bit)

   # wget 
   ftp://rpmfind.net/linux/centos/5.9/os/x86_64/CentOS/libdbi-0.8.1-2.1.x86_64.rpm

   
   

   # rpm -ivh libdbi-0.8.1-2.1.x86_64.rpm

3. Install PHP 5.4. See Installing PHP 5.4 on CentOS 5
4. Finally do the 

# yum install php rrdtool ganglia ganglia-gmetad ganglia-gmond ganglia-web httpd

5. For the rest of the steps do look at Installing and Configuring Ganglia on CentOS 5.8

Debugging gmond issue quickly

If you need to debug gmond issues quickly, use the command

# /usr/sbin/gmond --debug=9

loaded module: core_metrics
loaded module: cpu_module
loaded module: disk_module
loaded module: load_module
loaded module: mem_module
loaded module: net_module
loaded module: proc_module
loaded module: sys_module
loaded module: multicpu_module
udp_recv_channel mcast_join=NULL mcast_if=NULL port=8649 bind=NULL
tcp_accept_channel bind=NULL port=8649
Unable to create tcp_accept_channel. Exiting.

For more information and example see: 

1. Ganglia Node unable to update Gmetad Node 
2. Gmond dead but subsys locked for ganglia monitoring daemon

Ganglia Node unable to update Gmetad Node

I was using CentOS 5.5 for the Ganglia Head Node and CentOS 5.7 for the Ganglia node. I followed the blog entry Installing and configuring Ganglia on CentOS 5.x , but somehow the Ganglia Head Node did not register the compute node and there is no error when I used the command

# service gmond restart

Even when I turn on the Ganglia debugging as seen in Gmond dead but subsys locked for ganglia monitoring daemon

The only hint of an issue is to go to the /usr/sbin directory and use the command

# /usr/sbin/gmond --debug=1

You will get this output

Unable to create tcp_accept_channel. Exiting

Solution

Actually I realise the version of the Ganglia package and the Ganglia package are not of the same version. My Ganglia  Head Node is v3.0.7. But my Compute Node is v3.1.7, I have to downgrade my Ganglia Node to v3.0.7

First remove the newer version

# yum remove ganglia-gmond

Install the older version

# yum install ganglia-gmond-3.0.7

Removing a node from the Ganglia Web Frontend

According to the Ganglia_Readme, there is not easy way to remove a single dead node from the list from the ganglia web front-end. To flush the dead node from the record by restarting the the gmetad and gmond processes, you have to add the line at /etc/gmond.conf

globals { 
host_dmax = 3600 
}

The hosts will be removed from host tables when they haven't been heard from in 3600 seconds. See "man gmond.conf" for details.

Gmond dead but subsys locked for ganglia monitoring daemon

Following the tutorial instructions from Installing and configuring Ganglia on CentOS 5.x ,I've installed ganglia and ganglia-gmond. However when I attempt to start the gmond service

# service gmond start

Somehow the service refused to start. When I try another way to start the service

# /etc/init.d/gmond start

I notice the error, and the service do not start

gmond dead but subsys locked

To Solve the Issue
Step 1: Turn the Debug Level at /etc/gmond.conf to 100 found at the "globals" section

globals {
daemonize = yes
setuid = yes
user = ganglia
debug_level = 100
max_udp_msg_len = 1472
mute = no
deaf = no
host_dmax = 0 /*secs */
cleanup_threshold = 300 /*secs */
gexec = no
}

Step 2: Start the Service again.We should be able to see the error that caused "gmond dead but subsys locked". In my case, it is due to somehow my passwd file does not contain ganglia user. After fixing this missing user group, I was able to restart the ganglia-gmond daemon sucessfully.

For additional Information  on /etc/gmond.conf, you may want to use man pages or go to http://linux.die.net/man/5/gmond.conf

Error: Missing Dependency: librrd.so.2()(64bit) is needed by package ganglia-gmetad (epel)

This is the continuation of Blog Entry Error: Missing Dependency: rrdtool = 1.2.27-3.el5 is needed by package rrdtool-perl

1. When I'm installing Ganglia on my CentOS, I've did the following:
   

   # yum install rrdtool ganglia ganglia-gmetad ganglia-gmond ganglia-web httpd php

   However, instead I got the followings "Error: Missing Dependency: rrdtool = 1.2.27-3.el5 is needed by package rrdtool-perl"
   
2. To resolve it, similar to the earlier Blog Entry, I install rrdtool first. But when I use the command
   

   # yum install ganglia ganglia-gmetad ganglia-gmond ganglia-web httpd php

   There is still error "Error: Missing Dependency: librrd.so.2()(64bit) is needed by package ganglia-gmetad (epel)...."
3. To resolve the issue, you have to manually get rid of the latest rrdtool (x86-64) installed by EPEL and install the rrdtool-1.2.27-3.el5.x86_64
   

   # yum remove rrdtool-1.3.8-2.el5.rf.x86_64

   
   

   # yum install rrdtool-1.2.27-3.el5.x86_64

4. Install the Ganglia and its related tools
   

   # yum install ganglia ganglia-gmetad ganglia-gmond ganglia-web httpd php

Using Ganglia Status Client - gstat

Ganglia Status Client (gstat) connects with a Ganglia Monitoring Daemon (gmond) and output. It is a clean command line tool

To see all the gstat Options
# gstat --help

Shows CPU utilization for running jobs
# gstat -al1

For more inforamtion See Ganglia Toolkit

Brief Technical Overview and Installation of Ganglia on Solaris

Tutorial from SUN on how to install Ganglia on Solaris "Brief Technical Overview and Installation of Ganglia on Solaris"

Error: Missing Dependency: rrdtool = 1.2.27-3.el5 is needed by package rrdtool-perl

When I'm installing Ganglia on my CentOS, I've did the following:
# yum install rrdtool ganglia ganglia-gmetad ganglia-gmond ganglia-web httpd php

However, instead I got the followings
# Error: Missing Dependency: rrdtool = 1.2.27-3.el5 is needed by package rrdtool-perl

To workaround this issue, I've installed in stages. I'm assuming you have installed RpmForge Repository

# yum install rrdtool-perl
# yum install ganglia ganglia-gmetad ganglia-gmond ganglia-web httpd php

ok. It works.

Installing Ganglia on Standalone or Cluster

Good Readup on How to Install Ganglia on Clusters or Standalone Machine. Taken from I do linux blog, an excellent resource.

1. Ganglia Cluster Monitoring Made Easy by Idolinux.blogspot.com
2. Ganglia Howto by IBM
3. Ganglia Documentation by SourceForge.Net