Chukwa 0.4.0 详细安装流程
1.安装好Hadoop,确保hadoop运行无误(我hadoop版本为0.20.2)
2.去官网上下载chukwa 0.4.0版本:
http://www.apache.org/dyn/closer.cgi/hadoop/chukwa/chukwa-0.4.0
3.下载后别急着分发到各个节点,因为代码中存在着一个bug,这个bug会影响后面hicc的显示,所以需要经过修改编译以后再将其分发到各个节点上比较好!修改编译具体过程如下:
(1)源代码中找到ViewStore.java
(package org.apache.hadoop.chukwa.datastore)
在函数JSONArray list(String uid) 中,大概210行左右修改:
FileStatus[] fstatus = fs.listStatus(viewFile);
if(fstatus!=null) {
for(int i=0;i<fstatus.length;i++) {
+ if(!fstatus[i].getPath().getName().endsWith(".view")) {
+ continue;
+ }
long size = fstatus[i].getLen();
FSDataInputStream viewStream = fs.open(fstatus[i].getPath());
byte[] buffer = new byte[(int)size];
(2)然后在$CHUKWA_HOME/输入ant,重新编译
会在$CHUKWA_HOME/build/目录下得到编译后的文件
(3)将$CHUKWA_HOME/build/chukwa-core-0.4.0.jar 拷贝到$CHUKWA_HOME,将$CHUKWA_HOME/build/chukwa.war拷贝到$CHUKWA_HOME/webapps当中
4.配置各个节点的环境变量,最好在~/.bash_profile当中配好,下面是我的配置:
export JAVA_HOME=/usr/java/jdk1.6.0_16
export JRE_HOME=/usr/java/jdk1.6.0_16/jre
export HADOOP_VERSION=0.20.2
export HADOOP_HOME=/home/hadoop/hadoop-${HADOOP_VERSION}
export CHUKWA_HOME="/home/zhangliuhang/chukwa/"
export ANT_HOME=/usr/ant/ant-1.7.0
export PATH=.:$PATH: $JAVA_HOME/bin:$HADOOP_HOME/bin:$CHUKWA_HOME/bin:$ANT_HOME/bin
export CLASSPATH=$CLASSPATH:$JAVA_HOME/lib:$JRE_HOME/lib:$ANT_HOME/lib:$HADOOP_HOME/hadoop-${HADOOP_VERSION}-core.jar:$HADOOP_HOME/lib/*:$HADOOP_HOME/lib/commons-logging-1.0.4.jar:$ANT_HOME/junit-3.8.1.jar:$HADOOP_HOME/ivy/ivy-2.0.0-rc2.jar:$HADOOP_HOME/lib/commons-logging-api-1.0.4.jar:$HADOOP_HOME/lib/log4j-1.2.15.jar:$HADOOP_HOME/lib/hadoop-lzo-0.4.8.jar
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib:/home/hadoop/hadoop-0.20.2/lib/native/Linux-amd64-64
export JAVA_LIBRARY_PATH=/home/hadoop/hadoop-0.20.2/lib/native/Linux-amd64-64
export javalibpath=/home/hadoop/hadoop-0.20.2/lib/native/Linux-amd64-64
5.配置$CHUKWA_HOME/conf/chukwa-env.sh文件,其中的我的配置如下:(我只列出了没有被我注释掉的)
export JAVA_HOME=/usr/java/jdk1.6.0_16
export HADOOP_HOME="/home/hadoop/hadoop-0.20.2"
export HADOOP_CONF_DIR="/home/hadoop/hadoop-0.20.2/conf/"
export HADOOP_JAR=${HADOOP_HOME}/hadoop-0.20.2-core.jar
export chukwaRecordsRepository="/chukwa/repos/"
export CHUKWA_DATA_DIR="${CHUKWA_HOME}/data"
export CHUKWA_IDENT_STRING=chukwa
export JAVA_PLATFORM= Linux-amd64-64
export JAVA_LIBRARY_PATH=${HADOOP_HOME}/lib/native/${JAVA_PLATFORM}
export JDBC_DRIVER=com.mysql.jdbc.Driver
export JDBC_URL_PREFIX=jdbc:mysql://
export CHUKWA_HICC_MIN_iMEM=300M
export CHUKWA_HICC_MAX_MEM=300M
6.将必要的库从$CHUKWA_HOME/及$CHUKWA_HOME/lib拷贝到$HADOOP_HOME/lib当中,我拷贝的库有:
Chukwa-agent-0.4.0.jar,
chukwa-hadoop-0.4.0-client.jar,
tools-0.4.0.jar,json.jar