CentOS集群上安装Ganglia-3.6.1监控

1、安装依赖

1、建议使用超级用户安装

yum -y install apr-devel apr-util check-devel cairo-devel pango-devel libxml2-devel \
rpmbuild glib2-devel dbus-devel freetype-devel fontconfig-devel gcc-c++ expat-devel \
python-devel libXrender-devel pcre pcre-devel

2、关闭selinux

#vi /etc/selinux/config

把SELINUX=enforcing改成SELINUX=disable,重启机器。

2、安装expat依赖

#mkdir /home/ganglia

#cd /home/ganglia

#wget http://jaist.dl.sourceforge.net/project/expat/expat/2.1.0/expat-2.1.0.tar.gz

#tar -xf expat-2.1.0.tar.gz

#cd expat-2.1.0

#./configure --prefix=/usr/local/expat

#make

#make install

对于64位操作系统，需要手动的拷贝下动态链接库到lib64下

mkdir /usr/local/expat/lib64

#cp -a /usr/local/expat/lib/* /usr/local/expat/lib64/

3、安装confuse

#cd /home/ganglia

#wget ftp://ftp.icm.edu.pl/vol/rzm1/linux-fedora-secondary/development/source/SRPMS/libconfuse-2.6-1.fc9.src.rpm

#tar -xf confuse-2.7.tar.gz

#cd confuse-2.7

#./configure CFLAGS=-fPIC --disable-nls --prefix=/usr/local/confuse

#make

#make install

64bit机器需要拷贝动态链接库：

#mkdir -p /usr/local/confuse/lib64

#cp -a -f /usr/local/confuse/lib/* /usr/local/confuse/lib64/

4、安装ganglia

#cd /home/ganglia

#wget http://jaist.dl.sourceforge.net/project/ganglia/ganglia%20monitoring%20core/3.6.1/ganglia-3.6.1.tar.gz

#tar -xf ganglia-3.6.1.tar.gz

#cd ganglia-3.6.1

#./configure --with-gmetad --enable-gexec --with-libconfuse=/usr/local/confuse --with-libexpat=/usr/local/expat --prefix=/usr/local/ganglia --sysconfdir=/etc/ganglia

#make

#make install

5、服务端配置（Gmetad节点）

#mkdir -p /var/lib/ganglia/rrds

#mkdir -p /var/lib/ganglia/dwoo

#chown -R root:root /var/lib/ganglia

配置一个数据源

vi /etc/ganglia/gmetad.conf,修改如下信息：

data_source "Hadoop" 192.168.1.108:8649

setuid_username "root"

说明：这里的 " hadoop " 表示的是集群的名称， 192.168.1.108:8649 是Gmeta服务所在的IP和监听端口。
添加自启动脚本

#cp -f gmetad/gmetad.init /etc/init.d/gmetad

#cp -f /usr/local/ganglia/sbin/gmetad /usr/sbin/gmetad

#chkconfig --add gmetad

启动gmetad服务

#service gmetad start

看见Starting GANGLIA gmetad: [ OK ]就代表运行正常了。通过telnet localhost 8651验证gmetad是否正常

6、客户端配置（gmond节点）

本机安装如下：

#cp -f gmond/gmond.init /etc/init.d/gmond

#cp -f /usr/local/ganglia/sbin/gmond /usr/sbin/gmond

#chkconfig --add gmond

#gmond --default_config > /etc/ganglia/gmond.conf

对于生成的默认配置文件需要做适当的修改

globals {

daemonize = yes

setuid = yes

user = root /*运行Ganglia的用户*/

debug_level = 0

max_udp_msg_len = 1472

mute = no

deaf = no

host_dmax = 120 /*secs */

cleanup_threshold = 300 /*secs */

gexec = no

send_metadata_interval = 15 /*发送数据的时间间隔*/

}

cluster {

name = "hadoop" /*集群名称*/

owner = "root" /*运行Ganglia的用户*/

latlong = "unspecified"

url = "unspecified"

}

udp_send_channel {

# mcast_join = 239.2.11.71 /*注释掉组播*/

host = 192.168.1.108/*发送给安装gmetad的机器的IP*/

port = 8649

ttl = 1

}

udp_recv_channel { #接受UDP包配置

# mcast_join = 239.2.11.71

port = 8649

# bind = 239.2.11.71

}

其中name是将要在服务端进行的分组，是服务端的数据源。接下来开启服务

#service gmond start

看见Starting GANGLIA gmetad: [ OK ]代表启动成功。如果有失败，可以讲gmond.conf中的debug从0改为100，看更多的日志，然后进行排查。

7、服务端的WEB配置

1、PHP程序需要依赖Apache来运行，因此需要安装如下依赖

# yum -y install php httpd

# service httpd start //启动httpd 服务

2、下载最新版本的gweb（https://sourceforge.net/projects/ganglia/files/gweb/）本示例当前最新版本为3.6.2

# wget http://jaist.dl.sourceforge.net/project/ganglia/ganglia-web/3.6.2/ganglia-web-3.6.2.tar.gz
# tar -zxvf ganglia-web-3.6.2.tar.gz
# cd ganglia-web-3.6.2

3、编译Makefile
# vim Makefile

可以查看一些用户默认配置：

##########################################################
# User configurables:
##########################################################
# Location where gweb should be installed to (excluding conf, dwoo dirs).
GDESTDIR = /usr/share/ganglia-webfrontend
# Location where default apache configuration should be installed to.
GCONFDIR = /etc/ganglia-web
# Gweb statedir (where conf dir and Dwoo templates dir are stored)
GWEB_STATEDIR = /var/lib/ganglia-web
# Gmetad rootdir (parent location of rrd folder)
GMETAD_ROOTDIR = /var/lib/ganglia
APACHE_USER = www-data
##########################################################

 
4、修改默认配置：
GDESTDIR = /var/www/html/ganglia
APACHE_USER = apache

注意：GDESTDIR 和 APACHE_USER 要与APACHE的配置文件（/etc/httpd/conf/httpd.conf）中的 DocumentRoot 、 User保持一致

#make install

8、F&Q

1、Ganglia访问失败
There was an error collecting ganglia data (127.0.0.1:8652): fsockopen error: Permission denied
解决办法：
需要关闭selinux：vi /etc/selinux/config，把SELINUX=enforcing改成SELINUX=disable；需要重启机器。可以使用命令setenforce 0来关闭selinux而不需要重启，刷新页面，即可访问。但此方法只是一权宜之计。要想永久修改selinux设置，还是要使用第一种方法。
重启httpd服务器即可看到效果

9、附录：

 1、监控Hbadoop和Hbase

Hadoop和 HBase社区一直使用它作为监控集群的业界标准方案。在maste上安装gmetad，在master、node、slave上安装gmond。为了监控Hadoop和HBase，需要对它们做一些配置。这里Hadoop使用的是2.2.0版本，HBase使用的是0.96.0这个版本。这里监控的分布式集群也是文章《Ubuntu和CentOS中分布式配置Hadoop-2.2.0》http://www.linuxidc.com/Linux/2014-01/95799.htm 和《CentOS分布式环境安装HBase-0.96.0》http://www.linuxidc.com/Linux/2014-01/95801.htm 中布置的hadoop和hbase。

为了能让ganglia监控hadoop，Hadoop2.2.0需要配置hadoop-2.2.0/etc/hadoop/目录下的 hadoop-metrics.properties和hadoop-metrics2.properties文件。其中，hadoop- metrics.properties配置如下：

# Configuration of the "dfs" context for null

dfs.class=org.apache.hadoop.metrics.spi.NullContext

# Configuration of the "dfs" context for file

#dfs.class=org.apache.hadoop.metrics.file.FileContext

#dfs.period=10

#dfs.fileName=/tmp/dfsmetrics.log

# Configuration of the "dfs" context for ganglia

# Pick one: Ganglia 3.0 (former) or Ganglia 3.1 (latter)

# dfs.class=org.apache.hadoop.metrics.ganglia.GangliaContext

# dfs.class=org.apache.hadoop.metrics.ganglia.GangliaContext31

# dfs.period=10

# dfs.servers=localhost:8649

# Configuration of the "mapred" context for null

mapred.class=org.apache.hadoop.metrics.spi.NullContext

# Configuration of the "mapred" context for file

#mapred.class=org.apache.hadoop.metrics.file.FileContext

#mapred.period=10

#mapred.fileName=/tmp/mrmetrics.log

# Configuration of the "mapred" context for ganglia

# Pick one: Ganglia 3.0 (former) or Ganglia 3.1 (latter)

# mapred.class=org.apache.hadoop.metrics.ganglia.GangliaContext

# mapred.class=org.apache.hadoop.metrics.ganglia.GangliaContext31

# mapred.period=10

# mapred.servers=localhost:8649

# Configuration of the "jvm" context for null

#jvm.class=org.apache.hadoop.metrics.spi.NullContext

# Configuration of the "jvm" context for file

#jvm.class=org.apache.hadoop.metrics.file.FileContext

#jvm.period=10

#jvm.fileName=/tmp/jvmmetrics.log

# Configuration of the "jvm" context for ganglia

# jvm.class=org.apache.hadoop.metrics.ganglia.GangliaContext

# jvm.class=org.apache.hadoop.metrics.ganglia.GangliaContext31

# jvm.period=10

# jvm.servers=localhost:8649

# Configuration of the "rpc" context for null

rpc.class=org.apache.hadoop.metrics.spi.NullContext

# Configuration of the "rpc" context for file

#rpc.class=org.apache.hadoop.metrics.file.FileContext

#rpc.period=10

#rpc.fileName=/tmp/rpcmetrics.log

# Configuration of the "rpc" context for ganglia

# rpc.class=org.apache.hadoop.metrics.ganglia.GangliaContext

# rpc.class=org.apache.hadoop.metrics.ganglia.GangliaContext31

# rpc.period=10

# rpc.servers=localhost:8649

# Configuration of the "ugi" context for null

ugi.class=org.apache.hadoop.metrics.spi.NullContext

# Configuration of the "ugi" context for file

#ugi.class=org.apache.hadoop.metrics.file.FileContext

#ugi.period=10

#ugi.fileName=/tmp/ugimetrics.log

# Configuration of the "ugi" context for ganglia

# ugi.class=org.apache.hadoop.metrics.ganglia.GangliaContext

# ugi.class=org.apache.hadoop.metrics.ganglia.GangliaContext31

# ugi.period=10

# ugi.servers=localhost:8649

hadoop-metrics2.properties文件配置如下：

#

# Licensed to the Apache Software Foundation (ASF) under one or more

# contributor license agreements. See the NOTICE file distributed with

# this work for additional information regarding copyright ownership.

# The ASF licenses this file to You under the Apache License, Version 2.0

# (the "License"); you may not use this file except in compliance with

# the License. You may obtain a copy of the License at

#

# http://www.apache.org/licenses/LICENSE-2.0

#

# Unless required by applicable law or agreed to in writing, software

# distributed under the License is distributed on an "AS IS" BASIS,

# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

# See the License for the specific language governing permissions and

# limitations under the License.

#

# syntax: [prefix].[source|sink].[instance].[options]

# See javadoc of package-info.java for org.apache.hadoop.metrics2 for details

##############一定要注释掉原来的这块，否则会监控不到#########

#*.sink.file.class=org.apache.hadoop.metrics2.sink.FileSink

# default sampling period, in seconds

#*.period=10

#############################################################

# The namenode-metrics.out will contain metrics from all context

#namenode.sink.file.filename=namenode-metrics.out

# Specifying a special sampling period for namenode:

#namenode.sink.*.period=8

#datanode.sink.file.filename=datanode-metrics.out

# the following example split metrics of different

# context to different sinks (in this case files)

#jobtracker.sink.file_jvm.context=jvm

#jobtracker.sink.file_jvm.filename=jobtracker-jvm-metrics.out

#jobtracker.sink.file_mapred.context=mapred

#jobtracker.sink.file_mapred.filename=jobtracker-mapred-metrics.out

#tasktracker.sink.file.filename=tasktracker-metrics.out

#maptask.sink.file.filename=maptask-metrics.out

#reducetask.sink.file.filename=reducetask-metrics.out

*.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31

*.sink.ganglia.period=10

*.sink.ganglia.slope=jvm.metrics.gcCount=zero,jvm.metrics.memHeapUsedM=both

*.sink.ganglia.dmax=jvm.metrics.threadsBlocked=70,jvm.metrics.memHeapUsedM=40

namenode.sink.ganglia.servers=master:8649

resourcemanager.sink.ganglia.servers=master:8649

datanode.sink.ganglia.servers=master:8649

nodemanager.sink.ganglia.servers=master:8649

maptask.sink.ganglia.servers=master:8649

reducetask.sink.ganglia.servers=master:8649

为了让ganglia监控hbase，需要配置hbase-0.96.0目录下的conf目录中的hadoop-metrics2-hbase.properties文件。内容如下：

# syntax: [prefix].[source|sink].[instance].[options]

# See javadoc of package-info.java for org.apache.hadoop.metrics2 for details

##############一定要注释掉原来的这块，否则会监控不到#########

#*.sink.file*.class=org.apache.hadoop.metrics2.sink.FileSink

# default sampling period

#*.period=10

#############################################################

# Below are some examples of sinks that could be used

# to monitor different hbase daemons.

# hbase.sink.file-all.class=org.apache.hadoop.metrics2.sink.FileSink

# hbase.sink.file-all.filename=all.metrics

# hbase.sink.file0.class=org.apache.hadoop.metrics2.sink.FileSink

# hbase.sink.file0.context=hmaster

# hbase.sink.file0.filename=master.metrics

# hbase.sink.file1.class=org.apache.hadoop.metrics2.sink.FileSink

# hbase.sink.file1.context=thrift-one

# hbase.sink.file1.filename=thrift-one.metrics

# hbase.sink.file2.class=org.apache.hadoop.metrics2.sink.FileSink

# hbase.sink.file2.context=thrift-two

# hbase.sink.file2.filename=thrift-one.metrics

# hbase.sink.file3.class=org.apache.hadoop.metrics2.sink.FileSink

# hbase.sink.file3.context=rest

# hbase.sink.file3.filename=rest.metrics

*.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31

*.sink.ganglia.period=10

hbase.sink.ganglia.period=10

hbase.sink.ganglia.servers=master:8649

2、监控mongoDB

1、rpm -ivh ganglia-gmond-modules-python-3.6.0-1.x86_64.rpm 

2、yum -y install python-simplejson

3、检查python模块是否被挂载

gmond -d10

4、在https://github.com/ganglia/gmond_python_modules地址下载mongoDB python插件

将如下两个文件拷贝到对应目录中

cp mongodb.conf /etc/ganglia/conf.d/mongodb.pyconf

cp mongodb.py /usr/lib64/ganglia/python_modules/mongodb.py

5、修改mongodb.pyconf

把value = "mongo --quiet --eval 'printjson(db.serverStatus())'"

和value = "mongo --quiet --eval 'printjson(rs.status())'"

把Value中的mongo替换成该文件的绝对路径，当然你也可以不必修改mongodb.pyconf，保证mongodb/bin在PATH环境变量里就可（mongodb启动不使用27017的还要加-port xxxxx）

6、修改mongodb.py，将import json改成import simplejson as json

7、重启gmond,service gmond restart

8、重启httpd,service httpd restart

3、监控Mysql或mariadb

1、yum install MySQL-python

2、cd /etc/ganlia/conf.d ; mv mysql.pyconf.disabled mysql.pyconf

3、vi mysql.pyconf 编辑用户名和密码

4、重启gmond

5、重启httpd