[运维]3.containerd无法使用fluentd的问题
一开始看的教程是使用dockerd的运行时配置fluentd,但是我使用的是containerd的容器运行时,这两个运行时存储日志的位置不一样,导致fluentd一开始什么日志都收集不到。
修改fluentd的deployment文件,更改Mounts挂载的位置。
containerd的挂载位置是/var/log/pods和/var/log/containers。
如下是我fluentd的挂载配置,其中还包括一个configMap的挂载,这个configMap是帮助fluentd解析containerd日志的。
volumeMounts:
- name: varlog
mountPath: /var/log
- name: varlogcontainers
mountPath: /var/log/containers
readOnly: true
- name: varlogpods
mountPath: /var/log/pods
readOnly: true
- name: fluentd-config
mountPath: /fluentd/etc/kubernetes.conf
subPath: kubernetes.conf
terminationGracePeriodSeconds: 30
volumes:
- name: varlog
hostPath:
path: /var/log
- name: varlogcontainers
hostPath:
path: /var/log/containers
- name: varlogpods
hostPath:
path: /var/log/pods
- name: fluentd-config
configMap:
name: fluentd-config
items:
- key: kubernetes.conf
path: kubernetes.conf
具体的configMap如下:
apiVersion: v1
kind: ConfigMap
metadata:
name: fluentd-config
namespace: kube-logging31
data:
kubernetes.conf: |
<source>
@type tail
@id in_tail_container_logs
path /var/log/containers/*.log
pos_file /var/log/fluentd-containers.log.pos
tag "#{ENV['FLUENT_CONTAINER_TAIL_TAG'] || 'kubernetes.*'}"
exclude_path "#{ENV['FLUENT_CONTAINER_TAIL_EXCLUDE_PATH'] || ''}"
read_from_head true
<parse>
@type regexp
expression /^(?<time>[^ ]+) (?<stream>stdout|stderr) (?<flags>[^ ]+) (?<message>.*)$/
time_format %Y-%m-%dT%H:%M:%S.%N%:z
</parse>
</source>
<source>
@type tail
@id in_tail_pod_logs
path /var/log/pods/**/*.log
pos_file /var/log/fluentd-pods.log.pos
tag kubernetes_pods.*
read_from_head true
<parse>
@type regexp
expression /^(?<time>[^ ]+) (?<stream>stdout|stderr) (?<flags>[^ ]+) (?<message>.*)$/
time_format %Y-%m-%dT%H:%M:%S.%N%:z
</parse>
</source>
<filter kubernetes.**>
@type kubernetes_metadata
kubernetes_url "https://kubernetes.default.svc"
bearer_token_file "/var/run/secrets/kubernetes.io/serviceaccount/token"
ca_file "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
cache_size 1000
cache_ttl 300
</filter>
<filter kubernetes.**>
@type concat
key log
partial_key logtag
partial_value P
separator ""
</filter>
<match kubernetes.**>
@type elasticsearch
host "#{ENV['FLUENT_ELASTICSEARCH_HOST'] || 'elasticsearch-svc.kube-logging31.svc.cluster.local'}"
port "#{ENV['FLUENT_ELASTICSEARCH_PORT'] || 9200}"
scheme "#{ENV['FLUENT_ELASTICSEARCH_SCHEME'] || 'http'}"
logstash_format true
logstash_prefix fluentd
include_tag_key true
type_name _doc
logstash_dateformat %Y.%m.%d
time_key @timestamp
time_key_format %Y-%m-%dT%H:%M:%S.%N%:z
reconnect_on_error true
reload_connections true
reload_on_failure true
request_timeout 2147483648
</match>
这个configmap还有问题,具体来讲识别不了kubernetes开头的数据标签。这个留到后面解决。