Logstash部署

一、环境准备

1.1 拉取镜像

docker pull logstash:7.6.2

1.2 创建目录及文件

• 创建目录及文件

mkdir -p /mydata/logstash/config
mkdir -p /var/logs
touch /mydata/logstash/config/logstash.conf

• 编辑配置文件logstash.conf，内容如下

input {
  file{
    path => [ "/var/logs/**/*-debug.log" ]
    codec=>json
    start_position => "beginning"
    type => "debug"
  }
  file{
    path => [ "/var/logs/**/*-error.log" ]
    codec=>json
    start_position => "beginning"
    type => "error"
  }
  file{
    path => [ "/var/logs/**/*-record.log" ]
    codec=>json
    start_position => "beginning"
    type => "record"
  }
}

filter {
  date{
    match => [ "print_time" , "ISO8601" ]
    target => "@timestamp"
  }
  mutate{
    remove_field => ["@version","print_time","fields","host","log","prospector","tags"]
  }
  if [type] == "record" {
      json {
        source => "message"
        remove_field => ["message"]
      }
    }
}

output {
  elasticsearch {
    hosts => ["elasticsearch:9200"]
    index => "kkb-%{type}-%{+YYYY.MM.dd}"
  }
}

二、创建容器

• 创建指令

docker run -p 5045:5045 -p 9600:9600 --name logstash \
  --link elasticsearch \
  -e TZ="Asia/Shanghai" \
  -v /mydata/logstash/config/logstash.conf:/usr/share/logstash/config/logstash.conf \
  -v /var/logs:/var/logs \
  -d logstash:7.6.2 -f /usr/share/logstash/config/logstash.conf

• 查看容器日志

docker logs -f -t --tail 10 logstash

• 如果日志没有报错，且显示成功运行如下，说明部署成功

2021-05-20T19:24:57.650367856Z [2021-05-20T01:24:57,648][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}