Để tăng tốc độ tải Elasticsearch phiên bản 7.17.1, cần thiết lập proxy Docker

Cấu hình Mirror Docker

Chỉnh sửa tệp /etc/docker/daemon.json

Thêm các mirror sau:

{
  "registry-mirrors": [
    "https://docker.mirrors.ustc.edu.cn",
    "https://cr.console.aliyun.com/",
    "https://reg-mirror.qiniu.com",
    "https://6kx4zyno.mirror.aliyuncs.com",
    "https://dockerproxy.com"
  ]
}

Tải lại cấu hình Docker

sudo systemctl daemon-reload

Khởi động lại Docker

sudo systemctl restart docker

Tải Elasticsearch qua proxy

docker pull dockerproxy.com/library/elasticsearch:7.17.1

Tương tự với:

docker pull dockerproxy.com/library/kibana:7.17.1

docker pull dockerproxy.com/library/logstash:7.17.1

Tạo cấu trúc thư mục:

mkdir -p /elk/es/config /elk/es/data /elk/es/logs /elk/es/plugins/ik /elk/es/plugins/sql
chmod -R 777 /elk

Khởi động Elasticsearch:

docker run -d --name es -p 9200:9200 -p 9300:9300 \
-e "discovery.type=single-node" \
-v /elk/es/config:/usr/share/elasticsearch/config \
-v /elk/es/data:/usr/share/elasticsearch/data \
-v /elk/es/plugins:/usr/share/elasticsearch/plugins \
-v /elk/es/logs:/usr/share/elasticsearch/logs \
-e ES_JAVA_OPTS="-Xms512m -Xmx1g" \
elasticsearch:7.17.1

Tệp cấu hình elasticsearch.yml:

cluster.name: "docker-cluster"
network.host: 0.0.0.0

Khởi động Kibana:

docker run -d -p 5601:5601 \
-e ELASTICSEARCH_URL=http://192.168.0.176:9200 \
-e ELASTICSEARCH_HOSTS=[{"host":"192.168.0.176","port":"9200"}] \
dockerproxy.com/library/kibana:7.17.1

Cấu hình Docker Compose:

version: '3'
services:
  elasticsearch:
    image: dockerproxy.com/library/elasticsearch:7.17.1
    container_name: elasticsearch
    ports:
      - "9200:9200"
      - "9300:9300"
    volumes:
      - ./es/config:/usr/share/elasticsearch/config
      - ./es/data:/usr/share/elasticsearch/data
      - ./es/plugins:/usr/share/elasticsearch/plugins
    environment:
      ES_JAVA_OPTS: "-Xms512m -Xmx1g"
      discovery.type: single-node
    restart: always

  kibana:
    image: dockerproxy.com/library/kibana:7.17.1
    container_name: kibana
    ports:
      - "5601:5601"
    environment:
      ELASTICSEARCH_URL: http://elasticsearch:9200
    depends_on:
      - elasticsearch

Khởi động hệ thống:

docker-compose -f elk-stack.yaml up -d

Thiết lập Logstash với Kafka:

Tạo cấu hình logstash.conf:

input {
  kafka {
    bootstrap_servers => "192.168.0.173:9092"
    topics => ["events"]
    codec => json
  }
}

output {
  elasticsearch {
    hosts => ["192.168.0.173:9200"]
    index => "kafka-logs-%{+YYYY.MM.dd}"
  }
}

Khởi động Logstash qua Docker Compose:

version: '3'
services:
  logstash:
    image: dockerproxy.com/library/logstash:7.17.1
    container_name: logstash
    ports:
      - "9600:9600"
    volumes:
      - ./logstash/pipeline:/usr/share/logstash/pipeline
      - ./logstash/config:/usr/share/logstash/config
    depends_on:
      - elasticsearch