Send logs with Fluent Bit

Fluent Bit tails log files and ships them to Rootprint over OTLP. Records land in the index your ingest API key is scoped to: otel-logs-v0_9 in this guide.

Prerequisites

A running Rootprint instance. Substitute its hostname for <your-rootprint-host> and its port for <port>.
An ingest API key scoped to otel-logs-v0_9. Create one in Settings → API keys with Create ingest key, then substitute it for <your-ingest-api-key>. See API keys.
A Linux host, with Docker Compose if you run Fluent Bit in a container.
Fluent Bit 3.1 or newer.

Write the config

Replace <your-rootprint-host>, <port> (443 for HTTPS), <your-ingest-api-key>, the path glob, and myapp with your service name. For plain HTTP, set tls to off.

service:
  flush: 1
  log_level: info

pipeline:
  inputs:
    - name: tail
      path: /var/log/myapp/*.log
      tag: myapp.*
      read_from_head: false
      db: /var/lib/fluent-bit/myapp.db
      processors:
        logs:
          - name: opentelemetry_envelope
          - name: content_modifier
            context: otel_resource_attributes
            action: upsert
            key: service.name
            value: myapp

  outputs:
    - name: opentelemetry
      match: "*"
      host: <your-rootprint-host>
      port: <port>
      tls: on
      tls.verify: on
      logs_uri: /v1/logs
      header: Authorization Bearer <your-ingest-api-key>
      logs_body_key: $log
      compress: gzip
      retry_limit: no_limits
      log_response_payload: true

Keep the YAML format. The classic .conf format has no processors, so it can’t set service.name.
read_from_head: false skips existing lines on first start. Set it to true for a one-time backfill.
db saves file offsets, so a restart resumes where Fluent Bit stopped instead of skipping lines.
retry_limit: no_limits keeps retrying a failed batch. The default drops it after one retry.

Run Fluent Bit

Bare Metal
Docker

Install Fluent Bit from the downloads page.

Save the config as /etc/fluent-bit/fluent-bit.yaml, and create the offsets directory:
sudo mkdir -p /var/lib/fluent-bit
If your systemd unit runs Fluent Bit as a non-root user, also chown the directory to that user.

The package’s systemd unit loads fluent-bit.conf. Point it at the YAML file:

sudo mkdir -p /etc/systemd/system/fluent-bit.service.d
printf '[Service]\nExecStart=\nExecStart=/opt/fluent-bit/bin/fluent-bit -c /etc/fluent-bit/fluent-bit.yaml\n' \
  | sudo tee /etc/systemd/system/fluent-bit.service.d/yaml.conf
sudo systemctl daemon-reload

Restart Fluent Bit. status should show active (running) with no config or output errors:
sudo systemctl restart fluent-bit sudo systemctl status fluent-bit

Save the config as fluent-bit.yaml next to your docker-compose.yml, and add this service. The bind mount keeps the /var/log/myapp path the same inside the container, and the named volume keeps the offsets database across restarts.

services:
  rootprint-fluentbit:
    image: fluent/fluent-bit:latest
    container_name: rootprint-fluentbit
    restart: unless-stopped
    # The image's default command loads fluent-bit.conf.
    command: ["-c", "/fluent-bit/etc/fluent-bit.yaml"]
    volumes:
      - ./fluent-bit.yaml:/fluent-bit/etc/fluent-bit.yaml:ro
      - /var/log/myapp:/var/log/myapp:ro
      - fluent-bit-db:/var/lib/fluent-bit

volumes:
  fluent-bit-db:

docker compose up -d rootprint-fluentbit

Send a test log line

sudo mkdir -p /var/log/myapp
echo "$(date -Iseconds) hello from fluent-bit" | sudo tee -a /var/log/myapp/test.log

Verify in Rootprint

Open Logs, pick otel-logs-v0_9 from the index selector, and search for hello from fluent-bit. Records typically appear within 5–10 seconds.

Set the service name and host name

Rootprint’s service_name column comes from the service.name resource attribute. The two processors on the tail input set it: opentelemetry_envelope wraps records in an OTLP resource, and content_modifier writes service.name into it. A record_modifier filter won’t work, because it sets a log-record attribute and leaves service_name empty. To add host.name, append another content_modifier to the same processors.logs list:

          - name: content_modifier
            context: otel_resource_attributes
            action: upsert
            key: host.name
            value: ${HOSTNAME}

systemd doesn’t set HOSTNAME, so add Environment=HOSTNAME=%H to the [Service] section of the drop-in. Every record from one tail input gets the same service.name. To ship several services, give each its own tail input and processors.

Troubleshooting

Records arrive with no service name: the processors are missing, sit under the output instead of the tail input, or Fluent Bit is older than 3.1.
Fluent Bit ignores the YAML config: the service still loads fluent-bit.conf. Add the systemd drop-in (bare metal) or the command override (Docker).
cannot open database /var/lib/fluent-bit/myapp.db and failed initialize input tail.0: the offsets directory doesn’t exist. Create it and restart.
413 from a proxy in front of Rootprint: lower batch_size on the opentelemetry output. The default is 1000 log records per flush.
401/403/413/415: see Response codes.

Send logs from Docker: collect every container’s stdout instead of specific files.
OTLP reference: endpoint, response codes, limits.
Indexes: the otel-logs-v0_9 schema.

Get Started

Send Logs

Traces

Search Logs

Manage Rootprint

Send logs with Fluent Bit

Prerequisites

Set the service name and host name

Troubleshooting

​Prerequisites

​Set the service name and host name

​Troubleshooting

​Related

Prerequisites

Set the service name and host name

Troubleshooting

Related