elasticsearch + logstash + kibana 强大的日志处理分析系统

 

一、设计流程:

1.数据导入redis队列 key=logstash

2.logstash 获取redis队列logstash里面的数据

3.logstash把收集到数据传入elasticsearch 里面的index = sitelogs

4.kibana 设置 index = sitelogs

5.打开kibana 查询日志

 

二、启动程序

./src/redis-server redis.conf

./bin/logstash -f config/log_redis_to_es.conf

./elasticsearch -d (不能用root权限、必须新建型的用户比如es)

./bin/kibana

 

三、核心配置

1.redis 代码设置

$redis = new Redis();

$redis->connect('127.0.0.1', 6379);

$redis->auth('zzs@888');

$expire_time = 3600;

$queue_key="logstash";

$value="wanghui is not a good man";

$redis->rPush($queue_key,json_encode($value,JSON_UNESCAPED_UNICODE));

2.logstash config配置

新建log_redis_to_es.conf

1 input {

stdin {}

redis {

host => '127.0.0.1'

data_type => 'list'

port => "6379"

password=>'zzs@888'

key => 'logstash'

type => 'redis-input'

}

}

output {

elasticsearch {

hosts => ["127.0.0.1:9200"]

index => "sitelogs"

}

stdout {

codec => rubydebug

}

}

3.kibana kibana.yml配置

底部加入

server.port: 5601

server.host: "114.55.116.177"

elasticsearch.url: "http://127.0.0.1:9200"

kibana.index: ".kibana"