Send logs with Vector

Vector is a stand-alone log collector that reads files, applies transforms, and ships records to one or more sinks. This page tails files under /var/log/, shapes each line into the OTLP wire format with a remap transform, and ships the result to Rootprint’s OTLP endpoint with a Bearer token. Records land in the index your ingest token is scoped to: otel-logs-v0_9 in this guide. Run Vector on bare metal or in Docker — pick the tab below. Vector’s opentelemetry sink is currently in beta upstream: stable enough for production logs, but worth tracking the Vector changelog for breaking changes.

The file source paths below assume a Linux host. Vector itself runs on macOS and Windows, but this page does not cover those platforms.

Prerequisites

A running Rootprint instance and its base URL — you’ll substitute it for <your-rootprint>.
A Linux host. The Bare Metal tab uses systemd; the Docker tab uses Docker Compose.
An ingest API key scoped to your target index. In Settings → API keys, click Create ingest key, give it a name, and pick the index (otel-logs-v0_9 here; see Indexes for its schema). See API keys.

Bare Metal
Docker

Install Vector

Install the Vector package for your platform from the official installation page. Per-distro instructions (Debian/Ubuntu apt, RHEL/Fedora dnf, macOS, Windows) are maintained upstream.

Write the Vector config

Save the following at /etc/vector/vector.yaml. Replace <your-rootprint> with your Rootprint base URL, <your-ingest-token> with the API key you copied in step 1, /var/log/myapp/*.log with the glob that matches your application’s log files, and myapp with your service name.

Show vector.yaml

sources:
  app_logs:
    type: file
    include:
      - /var/log/myapp/*.log
    read_from: end

transforms:
  to_otlp:
    type: remap
    inputs: [app_logs]
    source: |
      .resourceLogs = [{
        "resource": {
          "attributes": [
            { "key": "service.name", "value": { "stringValue": "myapp" } },
            { "key": "host.name", "value": { "stringValue": get_hostname!() } }
          ]
        },
        "scopeLogs": [{
          "scope": { "name": "vector", "version": "" },
          "logRecords": [{
            "timeUnixNano": to_unix_timestamp!(now(), unit: "nanoseconds"),
            "observedTimeUnixNano": to_unix_timestamp!(now(), unit: "nanoseconds"),
            "severityNumber": 9,
            "severityText": "INFO",
            "body": { "stringValue": .message },
            "attributes": [
              { "key": "log.file.path", "value": { "stringValue": .file } }
            ],
            "traceId": "",
            "spanId": "",
            "flags": 0,
            "droppedAttributesCount": 0
          }]
        }]
      }]

      # Drop the original top-level fields so the sink sees only the OTLP envelope.
      del(.message)
      del(.file)
      del(.host)
      del(.source_type)
      del(.timestamp)

sinks:
  rootprint:
    type: opentelemetry
    inputs: [to_otlp]
    protocol:
      type: http
      uri: https://<your-rootprint>/v1/logs
      method: post
      encoding:
        codec: otlp
      compression: gzip
      request:
        headers:
          Authorization: "Bearer <your-ingest-token>"
      batch:
        timeout_secs: 1
        max_bytes: 8388608

read_from: end skips existing content on first start, so installing Vector against an existing log file does not replay everything that was already there. Flip it to beginning if you want a one-time backfill. batch.max_bytes is set to 8 MiB, a conservative size that stays well within typical reverse-proxy and upstream limits even after gzip variance.

Grant Vector read access to the log files

Vector runs as the vector user, which may not be able to read files under /var/log/. Add vector to the group that owns them (for example adm on Debian/Ubuntu):

sudo usermod -aG adm vector

Alternatively sudo chmod a+r the log files, which loosens permissions for every user on the host.

Restart Vector

sudo systemctl restart vector
sudo systemctl status vector

The status output should show active (running) and the most recent log lines should not contain config-parse or sink-startup errors.

Send a test log line

Create the application log directory if it doesn’t exist yet and append a single line:

sudo mkdir -p /var/log/myapp
echo "$(date -Iseconds) hello from vector" | sudo tee -a /var/log/myapp/test.log

Verify in Rootprint

Open Search, pick otel-logs-v0_9 from the index selector, and query for hello from vector. Records typically appear within 5–10 seconds. The OTLP path commits on Quickwit’s normal cadence, there is no commit=wait_for knob.

Run Vector as a container that bind-mounts the log files you want to tail. The config is the same one from the Bare Metal tab; the bind mount keeps the /var/log/myapp/*.log paths identical inside the container.

Write the Vector config

Save the same vector.yaml shown in the Bare Metal tab next to your docker-compose.yml. Replace <your-rootprint> and <your-ingest-token>, and point the file source include at the log path you bind-mount in the next step.

Add the Vector service to your compose file

services:
  rootprint-vector:
    image: timberio/vector:0.54.0-alpine
    container_name: rootprint-vector
    restart: unless-stopped
    volumes:
      - ./vector.yaml:/etc/vector/vector.yaml:ro
      - /var/log/myapp:/var/log/myapp:ro

The bind-mounted files keep their host ownership inside the container, so the vector container user may not be able to read them. If logs don’t appear and Vector reports permission denied, run the service as user: root or loosen host permissions with sudo chmod a+r /var/log/myapp/*.log.

Start Vector

docker compose up -d rootprint-vector

Send a test log line

Append a line to a file under the bind-mounted directory on the host:

sudo mkdir -p /var/log/myapp
echo "$(date -Iseconds) hello from vector" | sudo tee -a /var/log/myapp/test.log

Verify in Rootprint

Open Search, pick otel-logs-v0_9 from the index selector, and query for hello from vector. Records typically appear within 5–10 seconds.

What the remap does

The OTLP wire format is nested: a resourceLogs array, each entry containing a scopeLogs array, each of those containing a logRecords array. Vector’s file source emits a flat event with .message, .file, .host, .source_type, and .timestamp at the top level, so the remap transform builds the nested envelope itself and then deletes the original top-level fields. The opentelemetry sink rejects any event with stray fields outside the OTLP shape. Field-by-field:

service.name (resource attribute): lands in Quickwit’s service_name column. Replace the hard-coded myapp with whatever you want shown in the Rootprint service filter.
host.name (resource attribute): populated from get_hostname!() so it’s non-empty without any further config.
body.stringValue: the raw log line. Lands in body.message in otel-logs-v0_9. To parse structured fields out of the line (JSON, regex), insert another remap transform upstream of to_otlp and assign the parsed fields to attributes instead.
severityText and severityNumber: currently hard-coded to INFO / 9. Real configs should derive these from the log line; the simplest derivation is a remap that pattern-matches on the message and overrides both fields.
timeUnixNano and observedTimeUnixNano: both use the time Vector reads the line, not the timestamp embedded in the line itself. Use a regex or JSON parse to populate timeUnixNano from your log format if the read time is too coarse for your needs.

Troubleshooting

401 from Rootprint: the Authorization header is missing or malformed. Confirm the value under sinks.rootprint.protocol.request.headers.Authorization reads exactly Bearer <token>, with a single space between Bearer and the token.
403 from Rootprint: the API key’s index scope does not match otel-logs-v0_9, or the key has been deleted. Mint a fresh ingest API key for otel-logs-v0_9 in Settings → API keys.
413 from a proxy in front of Rootprint: a single batch was too large on the wire. Rootprint itself sets no OTLP body-size limit, but a reverse proxy may. Lower sinks.rootprint.protocol.batch.max_bytes.
415 from Rootprint: encoding is misconfigured. The encoding.codec value must be otlp; any other value sends a content type Rootprint rejects.
Vector starts cleanly but no logs appear in Rootprint: read_from: end skipped the existing file content. Either append a new line to trigger a read, or set read_from: beginning for a one-time replay (Vector remembers the offset across restarts after the first read).
permission denied reading /var/log/...: the Vector package installs a vector user that owns the service. Either sudo chmod a+r the log files, or add vector to the group that owns them.
Records arrive but body is empty: confirm the literal body.stringValue assignment in the to_otlp remap reads the same field the file source emits (default: .message). If you reordered the remap, make sure del(.message) runs after the assignment, not before.

Send logs from Docker: collect every container’s stdout via the Docker daemon, rather than tailing specific files.
Send logs from Kubernetes: run the Collector as a DaemonSet across a cluster.
OTLP reference: endpoint URL, response codes, body limits.
Indexes: the otel-logs-v0_9 schema, so you know what you can search.
Manage indexes: API key lifecycle and per-index permissions.

​Prerequisites

​What the remap does

​Troubleshooting

​Related

Prerequisites

What the remap does

Troubleshooting

Related