fluentd

Home

Fluentd Setup for Kubernetes

This document outlines the steps to set up Fluentd as a DaemonSet in a Kubernetes cluster to forward logs to an external syslog server.

Kubernetes Configuration

1. Create Fluentd YAML

First, create a file named fluentd.yaml with the following content. You can use vi or any other text editor.

vi fluentd.yaml

2. Fluentd DaemonSet YAML

Paste the following YAML configuration into fluentd.yaml:

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: fluentd
  namespace: kube-system
  labels:
    k8s-app: fluentd-logging
    version: v1

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: fluentd
rules:
- apiGroups:
  - ""
  resources:
  - pods
  - namespaces
  verbs:
  - get
  - list
  - watch

---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: fluentd
roleRef:
  kind: ClusterRole
  name: fluentd
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
  name: fluentd
  namespace: kube-system

---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluentd
  namespace: kube-system
  labels:
    k8s-app: fluentd-logging
    version: v1
spec:
  selector:
    matchLabels:
      k8s-app: fluentd-logging
  template:
    metadata:
      labels:
        k8s-app: fluentd-logging
        version: v1
    spec:
      serviceAccount: fluentd
      serviceAccountName: fluentd
      tolerations:
      - key: node-role.kubernetes.io/master
        effect: NoSchedule
      containers:
      - name: fluentd
        image: fluent/fluentd-kubernetes-daemonset:v1-debian-syslog
        env:
          - name:  SYSLOG_HOST
            value: "198.211.125.176" # Replace with your syslog server IP
          - name:  SYSLOG_PORT
            value: "514"
          - name:  SYSLOG_PROTOCOL
            value: "udp"
        resources:
          limits:
            memory: 200Mi
          requests:
            cpu: 100m
            memory: 200Mi
        volumeMounts:
        - name: varlog
          mountPath: /var/log
        - name: varlibdockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
      terminationGracePeriodSeconds: 30
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
      - name: varlibdockercontainers
        hostPath:
          path: /var/lib/docker/containers

Note: Ensure this YAML is correctly pasted into fluentd.yaml. The namespace: kube-system under ClusterRole.metadata has been removed as ClusterRoles are not namespaced. Remember to replace the placeholder SYSLOG_HOST IP address with your actual syslog server IP.

3. Apply the Configuration

Apply the YAML to your cluster:

kubectl apply -f fluentd.yaml

Syslog Server Setup (Linux Host)

These steps are to be performed on the machine that will act as the syslog server.

1. Install rsyslog

sudo apt-get update
sudo apt-get install -y rsyslog

(Added sudo and -y for non-interactive install)

2. Configure rsyslog

Edit the rsyslog configuration file:

sudo vi /etc/rsyslog.conf

(Added sudo)

Uncomment or add the following lines to enable UDP syslog reception. Edit the following section:

# provides UDP syslog reception
module(load="imudp")
input(type="imudp" port="514")
# Make sure these lines are uncommented or present
# module(load="imtcp") # if you want to use TCP
# input(type="imtcp" port="514") # if you want to use TCP

(Corrected the code block and added the input rule which is usually needed)

3. Restart rsyslog

sudo systemctl restart rsyslog

(Added sudo and used systemctl for modern systems)

4. Check Logs

Tail the syslog file to see incoming logs:

sudo tail -f /var/log/syslog

(Added sudo)

Home