ElasticSearch, Kibana, and Fluentd 的简单部署实例
参考：http://docs.fluentd.org/articles/free-alternative-to-splunk-by-fluentd
要求：JDK1.6及以上，ruby1.9.2及以上(最后附安装过程macos)
一、安装ElasticSearch：
1、下载：http://www.elasticsearch.org/download/，本人下载的是：elasticsearch-0.90.3.tar.gz包
$ tar zxvf elasticsearch-0.90.0.RC2.tar.gz $ cd elasticsearch-0.90.0.RC2/
Onceinstallationiscomplete,startElasticSearch.
$ ./bin/elasticsearch -f
2、安装Kibana(http://kibana.org/intro.html)
Extractyourarchive
OpenKibanaConfig.rbinyourfavoriteeditor
SetElasticsearch="localhost:9200"toyourElasticSearchserver
Run gem install bundler Run bundle install
运行
$ bundle exec ruby kibana.rb
3、fluentd的安装，参照：http://docs.fluentd.org/categories/installation
4、elasticsearchplugin的安装：https://github.com/uken/fluent-plugin-elasticsearch(运行时如有问题，请查阅：https://gist.github.com/y-matsuwitter/4951605)
如下：
#/usr/lib64/fluent/ruby/bin/fluent-geminstallfluent-plugin-mysql
#vi/etc/td-agent/td-agent.conf
file1.txt
#servicetd-agentrestart
Startingtd-agent:2013-02-1418:14:40+0900:fluent/supervisor.rb:187:rescueinmain_process:configerrorfile="/etc/td-agent/td-agent.conf"error="Unknownoutputplugin'mysql'.Run'gemsearch-rdfluent-plugin'tofindplugins"
[FAILED]
file2.txt
#/usr/lib64/fluent/ruby/bin/gemenvironment
~~~~~中略~~~~~~~~~~~~~
-GEMPATHS:
-/usr/local/rvm/gems/ruby-1.9.3-p327
-/usr/local/rvm/gems/ruby-1.9.3-p327@global
~~~~~中略~~~~~~~~~~~~~
file3.txt
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems
file4.txt
GEM_HOME="/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/"
GEM_PATH="/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/"
/usr/lib64/fluent/ruby/bin/fluent-geminstallfluent-plugin-mysql
-----------------------------------------------------
三个节点：192.168.1.4，192.168.1.5，192.168.1.6；
fluentd的配置，
192.168.1.4
********************************************************************
<source>
typehttp
port8888
</source>
<source>
typeforward
port24224
bind0.0.0.0
</source>
<source>
typetail
format/^*(?<level>[^]*)(?<type>[^]*)(?<time>[^]*[^]*)(?<message>.*)$/
path/data/cassandra/log/system.log
pos_file/root/logserver/fluentd/logpos/system.pos
tag192.168.1.4
</source>
<match192.**>
typecopy
<store>
typefile
path/root/logserver/fluentd/log/cassandra
</store>
<store>
typeelasticsearch
logstash_formattrue
flush_interval10s
include_tag_keytrue
tag_keyhost
</store>
</match>
<matchdebug.**>
typestdout
</match>
192.168.1.5/6
********************************************************************
<source>
typehttp
port8888
</source>
<source>
typetail
format/^*(?<level>[^]*)(?<type>[^]*)(?<time>[^]*[^]*)(?<message>.*)$/
path/data/cassandra/log/system.log
pos_file/root/logserver/fluentd/logpos/system.pos
tag192.168.1.5
</source>
<matchdebug.**>
typestdout
</match>
<match192.**>
typeforward
send_timeout60s
recover_wait10s
heartbeat_interval10s
phi_threshold8
hard_timeout60s
<server>
nameicecrown
host192.168.1.4
port24224
weight60
</server>
<secondary>
typefile
path/root/logserver/fluentd/forward-failed
</secondary>
</match>
********************************************************************
--------------ok-------------------------------------
安装相关资料
rubyinstall:
1、rvm:的安装（http://stackoverflow.com/questions/11677771/rvm-command-not-found-mac-ox）
curl -L https://get.rvm.io | bash -s -- --version latest
$ source ~/.rvm/scripts/rvm
then
$ type rvm | head -n 1
Iftheoutputis:
rvm is a function
Youmayneedtoadd"source~/.rvm/scripts/rvm"toyour~/.bash_profilefile
或者（thisworkedforme):
rm -rf ~/.rvm curl -L https://get.rvm.io | bash -s stable
uservminstallruby(http://misheska.com/blog/2013/06/16/using-rvm-to-manage-multiple-versions-of-ruby/)
$ rvm install 1.9.3
Youmaygetanerrormessagesaying“Therewasanerrorwhiletryingtoresolverubygemsversionfor‘latest’.Haltingtheinstallation.”Justruntheinstallagainlikesotofixtheissue:
$ rvm reinstall 1.9.3
VerifytheRVMinstallbyrunningthefollowingcommands:
$ rvm -h $ rvm list $ rvm use 1.9.3 $ rvm rubygems latest
ToensurethatthenewerRuby1.9.3isusedbydefaultinsteadofthesystem1.8.7version,runthefollowingcommand:
$ rvm use 1.9.3 --default
linux上的安装：http://tecadmin.net/how-to-install-ruby-2-0-0-on-centos-6-using-rvm/#