Send logs with Fluent Bit

Fluent Bit is a lightweight log forwarder built for edge and host-level collection. This page tails files under /var/log/, batches the records, and ships them to Rootprint’s OTLP endpoint with a Bearer token. Records land in the index your ingest token is scoped to: otel-logs-v0_9 in this guide. Run Fluent Bit on bare metal or in Docker — pick the tab below.

The tail input paths below assume a Linux host. Fluent Bit itself runs on macOS and Windows, but this page does not cover those platforms. Fluent Bit’s opentelemetry output reaches OTLP/HTTP parity in 2.x and later. Pin to a recent Fluent Bit release if you can.

Prerequisites

A running Rootprint instance and its base URL — you’ll substitute it for <your-rootprint>.
A Linux host. The Bare Metal tab uses systemd; the Docker tab uses Docker Compose.
An ingest API key scoped to your target index. In Settings → API keys, click Create ingest key, give it a name, and pick the index (otel-logs-v0_9 here; see Indexes for its schema). See API keys.

Bare Metal
Docker

Install Fluent Bit

Install the Fluent Bit package for your platform from the official downloads page. Per-distro instructions (Debian/Ubuntu apt, RHEL/Fedora dnf, Windows installer) are maintained upstream. Pin to a recent release so the opentelemetry output is mature.

Write the Fluent Bit config

Save the following at /etc/fluent-bit/fluent-bit.conf. Replace <your-rootprint-host> with the hostname of your Rootprint deployment, <port> with 443 (HTTPS) or 80 (HTTP) or whatever custom port you serve on, <your-ingest-token> with the API key you copied in step 1, and /var/log/myapp/*.log with the glob that matches your application’s log files.

[SERVICE]
    Flush         1
    Log_Level     info
    Daemon        Off
    HTTP_Server   Off

[INPUT]
    Name             tail
    Path             /var/log/myapp/*.log
    Tag              myapp.*
    Read_from_Head   False

[OUTPUT]
    Name                  opentelemetry
    Match                 *
    Host                  <your-rootprint-host>
    Port                  <port>
    Tls                   On
    Tls.verify            On
    Logs_uri              /v1/logs
    Header                Authorization Bearer <your-ingest-token>
    Logs_body_key         $log
    Log_response_payload  True

Read_from_Head False skips existing content on first start, so installing Fluent Bit against an existing log file does not replay everything that was already there. Flip it to True if you want a one-time backfill. Without a DB parameter (covered below), Fluent Bit re-applies Read_from_Head on every restart. Fine for the first install, worth revisiting once the setup is in production.

Grant read access to the log files

The Fluent Bit package installs a fluent-bit service user that may not be able to read files under /var/log/. Add it to the group that owns them, or sudo chmod a+r the log files (which loosens permissions for every user on the host).

Restart Fluent Bit

sudo systemctl restart fluent-bit
sudo systemctl status fluent-bit

The status output should show active (running) and the most recent log lines should not contain config-parse or output-startup errors.

Send a test log line

Create the application log directory if it doesn’t exist yet and append a single line:

sudo mkdir -p /var/log/myapp
echo "$(date -Iseconds) hello from fluent-bit" | sudo tee -a /var/log/myapp/test.log

Verify in Rootprint

Open Search, pick otel-logs-v0_9 from the index selector, and query for hello from fluent-bit. Records typically appear within 5–10 seconds. The OTLP path commits on Quickwit’s normal cadence, there is no commit=wait_for knob.

Run Fluent Bit as a container that bind-mounts the log files you want to tail. The config is the same one from the Bare Metal tab; the bind mount keeps the /var/log/myapp/*.log paths identical inside the container.

Write the Fluent Bit config

Save the same fluent-bit.conf shown in the Bare Metal tab next to your docker-compose.yml. The container reads it at /fluent-bit/etc/fluent-bit.conf (mounted in the next step). Replace <your-rootprint-host>, <port>, and <your-ingest-token>, and point the tail Path at the log path you bind-mount in.

Add the Fluent Bit service to your compose file

services:
  rootprint-fluentbit:
    image: fluent/fluent-bit:latest
    container_name: rootprint-fluentbit
    restart: unless-stopped
    volumes:
      - ./fluent-bit.conf:/fluent-bit/etc/fluent-bit.conf:ro
      - /var/log/myapp:/var/log/myapp:ro

The bind-mounted files keep their host ownership inside the container, so the Fluent Bit container user may not be able to read them. If logs don’t appear and Fluent Bit reports permission denied, run the service as user: root or loosen host permissions with sudo chmod a+r /var/log/myapp/*.log.

Start Fluent Bit

docker compose up -d rootprint-fluentbit

Send a test log line

Append a line to a file under the bind-mounted directory on the host:

sudo mkdir -p /var/log/myapp
echo "$(date -Iseconds) hello from fluent-bit" | sudo tee -a /var/log/myapp/test.log

Verify in Rootprint

Open Search, pick otel-logs-v0_9 from the index selector, and query for hello from fluent-bit. Records typically appear within 5–10 seconds.

Setting `service.name` and `host.name`

The minimum config above does not populate OTLP resource attributes. Fluent Bit’s opentelemetry output has no resource-attributes parameter. The upstream plugin treats every record-level field as an OTLP log-record attribute, not a resource attribute. Rows then show up under (unknown service) in the Rootprint service filter, and host.name is empty. To fix this, add a record_modifier filter that injects the fields. Insert this [FILTER] block between [INPUT] and [OUTPUT]:

[FILTER]
    Name           record_modifier
    Match          *
    Record         service.name myapp
    Record         host.name ${HOSTNAME}

Replace myapp with the value you want to see in the Rootprint service filter. ${HOSTNAME} is expanded from the Fluent Bit process environment; if you run Fluent Bit under systemd, set Environment=HOSTNAME=%H in the unit override or pass the value through. These fields land as OTLP log-record attributes rather than resource attributes. Rootprint’s service filter falls back to log-record service.name when the resource attribute is missing, so the filter populates correctly. If you need a true resource-attribute split (different services multiplexed through the same Fluent Bit instance), an OpenTelemetry Collector sidecar in front of Fluent Bit is the cleaner option.

Persisting tail offsets across restarts

By default the wizard config does not persist file offsets, so every Fluent Bit restart re-applies Read_from_Head. With False, that means lines written between stop and restart are skipped. For production, add a DB parameter to the [INPUT] block so Fluent Bit checkpoints offsets in SQLite:

[INPUT]
    Name             tail
    Path             /var/log/myapp/*.log
    Tag              myapp.*
    Read_from_Head   False
    DB               /var/lib/fluent-bit/myapp.db

The Fluent Bit package does not create /var/lib/fluent-bit/, so create it first or the tail input fails to initialize with cannot open database ...:

sudo mkdir -p /var/lib/fluent-bit

If your systemd unit drops privileges to a non-root user, also sudo chown -R fluent-bit:fluent-bit /var/lib/fluent-bit.

Troubleshooting

401 from Rootprint: the Authorization header is missing or malformed. Confirm the value on the Header line reads exactly Authorization Bearer <token>, with one space between Authorization and Bearer and one between Bearer and the token.
403 from Rootprint: the API key’s index scope does not match otel-logs-v0_9, or the key has been deleted. Mint a fresh ingest API key for otel-logs-v0_9 in Settings → API keys.
413 from a proxy in front of Rootprint: a single batch was too large on the wire. Rootprint itself sets no OTLP body-size limit, but a reverse proxy may. Lower the output batch size by setting Workers 1 and reducing Mem_Buf_Limit on the [INPUT] block, or split the upstream load across multiple [INPUT] instances.
415 from Rootprint: wrong output plugin selected. The Name line on the [OUTPUT] block must read opentelemetry, not http. The http plugin sends a content type Rootprint’s OTLP endpoint rejects.
Fluent Bit starts cleanly but no logs appear in Rootprint: Read_from_Head False skipped the existing file content. Either append a new line to trigger a read, or set Read_from_Head True for a one-time replay (Fluent Bit remembers the offset in DB after the first read).
permission denied reading /var/log/...: the Fluent Bit package installs a fluent-bit service user that owns the daemon. Either sudo chmod a+r the log files, or add fluent-bit to the group that owns them.
cannot open database /var/lib/fluent-bit/myapp.db / failed initialize input tail.0: you added a DB parameter (see Persisting tail offsets across restarts) but the directory does not exist. sudo mkdir -p /var/lib/fluent-bit and restart.
Records arrive but body is empty: confirm the Logs_body_key value matches the field the tail input emits. The tail input writes the raw line under the key log, so Logs_body_key $log is the correct mapping. If you inserted a parser earlier in the chain that renames or unwraps the field, point Logs_body_key at the new key.

Send logs from Docker: collect every container’s stdout via the Docker daemon, rather than tailing specific files.
Send logs from Kubernetes: run the Collector as a DaemonSet across a cluster.
OTLP reference: endpoint URL, response codes, body limits.
Indexes: the otel-logs-v0_9 schema, so you know what you can search.
Manage indexes: API key lifecycle and per-index permissions.

​Prerequisites

​Setting service.name and host.name

​Persisting tail offsets across restarts

​Troubleshooting

​Related

Prerequisites

Setting `service.name` and `host.name`

Persisting tail offsets across restarts

Troubleshooting

Related