Installing Fluentd: A Step-by-Step Guide

Now that you've been introduced to Fluentd, there's no need to let that information get stale. Let's get to work.

This guide provides step-by-step instructions on how to install Fluentd on different platforms. Choose the installation guide that suits your operating system.

For Windows or MacOS users, you can use the installation instructions from the source, Docker, or Kubernetes.

General prerequisites

Before installing Fluentd, you want to ensure that your system meets the following requirements:

• System Requirements: Sufficient CPU and memory to handle the expected data load.
• Permissions: Administrative privileges to install packages and modify system configurations.

Installation on Ubuntu (Debian-based systems)

Fluentd provides stable DEB packages under two distributions:

• fluent-package (maintained by the Fluentd project)
• calyptia-fluentd (maintained by Chronosphere).

Step 1: Install Fluentd via the Apt repository

Below are several commands for various system versions. Choose the one compactable with your system and run it.

For Fluent-package

• Ubuntu Noble (LTS): curl -fsSL https://toolbelt.treasuredata.com/sh/install-ubuntu-noble-fluent-package5-lts.sh | sh
• Ubuntu Jammy (LTS): curl -fsSL https://toolbelt.treasuredata.com/sh/install-ubuntu-jammy-fluent-package5-lts.sh | sh
• Ubuntu Focal (LTS): curl -fsSL https://toolbelt.treasuredata.com/sh/install-ubuntu-focal-fluent-package5-lts.sh | sh

For Calyptia-Fluentd

• Ubuntu Focal: curl -fsSL https://calyptia-fluentd.s3.us-east-2.amazonaws.com/calyptia-fluentd-1-ubuntu-focal.sh | sh
• Ubuntu Bionic: curl -fsSL https://calyptia-fluentd.s3.us-east-2.amazonaws.com/calyptia-fluentd-1-ubuntu-bionic.sh | sh

After running the command, you should get an output similar to this:

==============================
 fluent-package Installation Script 
==============================
This script requires superuser access to install apt packages.
You will be prompted for your password by sudo.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  7636  100  7636    0     0   191k      0 --:--:-- --:--:-- --:--:--  196k

Step 2: Launch Fluentd

The next step is to start the Fluentd service using systemd. The following command will help you do this and check the current status of Fluentd:

sudo systemctl start fluentd.service  
sudo systemctl status fluentd.service  

For Calyptia-Fluentd, replace fluentd.service with calyptia-fluentd.service.

You should see the following output:

Step 3: Post sample logs

The last step is to send a test log to verify the installation. You can do this using the command below:

curl -X POST -d 'json={"json":"message"}' http://localhost:8888/debug.test  
tail -n 1 /var/log/fluent/fluentd.log  

Your output should look similar to this:

2024-12-04 07:19:13.420429094 +0000 debug.test: {"json":"message"}

Install from source

This installation guide involves cloning the Fluentd GitHub repository, installing dependencies, and running Fluentd.

The following steps will guide you along the way:

Step 1: Download the source code

Get the latest source from the Fluentd GitHub repository:

git clone https://github.com/fluent/fluentd.git  
cd fluentd  

Step 2: Install dependencies

Ensure Ruby and required libraries are installed, then run the following command:

gem install bundler  
bundle install  

Step 3: Build and run Fluentd

bundle exec fluentd --setup ./fluent  
bundle exec fluentd -c ./fluent/fluent.conf  

Installation with Docker

Using Docker is a quick and isolated way to run Fluentd. This approach is suitable for all OS types as well.

The following steps will guide you in installing Fluentd using Docker:

Step 1: Pull the Fluentd image

The following command will pull the official Fluentd image from Docker Hub:

docker pull fluent/fluentd:v1.14-1  

You should get a similar output:

v1.14-1: Pulling from fluent/fluentd
6097bfa160c1: Pull complete 
fbc3d36e35e4: Pull complete 
26054f1c5f42: Pull complete 
ea86cadf5e05: Pull complete 
b05ff3fb7de5: Pull complete 
Digest: sha256:3e0bd37d74c95cff04a5a70ca4bb7470c6f807075cb914c1267c89f5801e1333
Status: Downloaded newer image for fluent/fluentd:v1.14-1
docker.io/fluent/fluentd:v1.14-1

Step 2: Run Fluentd

Run Fluentd using the command below:

docker run -d -p 8888:8888 -v $(pwd)/fluent.conf:/fluentd/etc/fluent.conf fluent/fluentd:v1.14-1  

Your output will be your Docker container ID:

4c8367720d3b8a2f5881a98493b16dfb4d440bcc3e70f9619b0e9a8b0e31d3a7

Step 3: Test the setup

To post a test log to Fluentd, run the following command:

curl -X POST -d 'json={"json":"message"}' http://localhost:8888/debug.test  
docker logs <container_id>  

You should get a similar output like this:

Installation on Kubernetes

This guide will walk you through installing Fluentd as a DaemonSet in Kubernetes.

Prerequisites

Before installing Fluentd on Kubernetes, ensure you have the following:

• Kubernetes cluster: A running Kubernetes cluster
• kubectl: Kubernetes command-line tool installed and configured
• Cluster access: Administrative access to the Kubernetes cluster
• Log destination: (Optional) A configured log destination like Elasticsearch, CloudWatch, or another log aggregation service

Step 1: Create RBAC resources

Fluentd requires specific permissions to collect logs from your Kubernetes cluster.

Apply the following RBAC (Role-Based Access Control) configuration:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: fluentd
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: fluentd
rules:
- apiGroups: [""]
  resources: ["pods", "nodes", "namespaces"]
  verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: fluentd
roleRef:
  kind: ClusterRole
  name: fluentd
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
  name: fluentd
  namespace: kube-system

Apply the configuration:

kubectl apply -f fluentd-rbac.yaml

Step 2: Create Fluentd ConfigMap

Configure Fluentd's log collection and forwarding behavior using a ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: fluentd-config
  namespace: kube-system
data:
  fluent.conf: |-
    <source>
      @type tail
      @id in_tail_container_logs
      path /var/log/containers/*.log
      pos_file /var/log/fluentd-containers.log.pos
      tag kubernetes.*
      read_from_head true
      <parse>
        @type json
        time_key time
        time_format %Y-%m-%dT%H:%M:%S.%NZ
      </parse>
    </source>

    <match **>
      @type stdout
    </match>

Apply the configuration:

kubectl apply -f fluentd-configmap.yaml

Step 3: Deploy Fluentd DaemonSet

Deploy Fluentd as a DaemonSet to ensure log collection on every node:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluentd
  namespace: kube-system
  labels:
    k8s-app: fluentd-logging
    version: v1
spec:
  selector:
    matchLabels:
      k8s-app: fluentd-logging
      version: v1
  template:
    metadata:
      labels:
        k8s-app: fluentd-logging
        version: v1
    spec:
      tolerations:
      - key: node-role.kubernetes.io/control-plane
        operator: Exists
        effect: NoSchedule
      - key: node-role.kubernetes.io/master
        operator: Exists
        effect: NoSchedule
      serviceAccount: fluentd
      serviceAccountName: fluentd
      containers:
      - name: fluentd
        image: fluent/fluentd-kubernetes-daemonset:v1.16-debian-elasticsearch8-1
        imagePullPolicy: IfNotPresent
        env:
        - name: K8S_NODE_NAME
          valueFrom:
            fieldRef:
              fieldPath: spec.nodeName
        resources:
          limits:
            memory: 512Mi
            cpu: 500m
          requests:
            cpu: 100m
            memory: 200Mi
        volumeMounts:
        - name: varlog
          mountPath: /var/log
          readOnly: true
        - name: varlibdockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
        - name: fluentd-config
          mountPath: /fluentd/etc/fluent.conf
          subPath: fluent.conf
        securityContext:
          privileged: true
      terminationGracePeriodSeconds: 30
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
      - name: varlibdockercontainers
        hostPath:
          path: /var/lib/docker/containers
      - name: fluentd-config
        configMap:
          name: fluentd-config

Apply the DaemonSet:

kubectl apply -f fluentd-daemonset.yaml

Step 4: Verify installation

Check the status of Fluentd pods:

kubectl get pods -n kube-system | grep fluentd

You should see an output similar to this:

fluentd-rj8t2                            1/1     Running   0              2m9s

View Fluentd logs:

kubectl logs -n kube-system -l k8s-app=fluentd-logging

Final thoughts

In this guide, we’ve walked through installing Fluentd across multiple environments—from native Ubuntu setups using fluent-package or calyptia-fluentd, to Docker containers and Kubernetes clusters. Each method ensures Fluentd is ready to collect, process, and route logs efficiently, regardless of your infrastructure.

Installation is the critical first step in building a unified logging layer. With Fluentd now running, you’re positioned to centralize logs from applications, services, and infrastructure into a cohesive pipeline.

Like this article? Sign up for our newsletter below and become one of over 1000 subscribers who stay informed on the latest developments in the world of DevOps. Subscribe now!