Scaling beyond a single node

The bundled Docker Compose is optimized for one active Rootprint container, one Quickwit container, local-disk Quickwit storage, and Quickwit’s file-backed metastore. That setup handles substantial ingest on modest hardware. Scaling Rootprint means keeping Rootprint single-node and moving scale-sensitive work into Quickwit, its search engine. See Architecture for the full responsibility split and what’s stored where.

What you can run in parallel

Component	Multi-instance?	Notes
Rootprint	No	Run one active Rootprint instance. Do not add load-balanced Rootprint replicas and do not build a shared filesystem for Rootprint.
Quickwit	Yes	Quickwit is designed as a horizontally scalable cluster. Add nodes to grow ingest and search capacity.
Rootprint database	External Postgres	Rootprint stores users, API keys, saved views, preferences, and activity state in its own Postgres database. Keep it separate from Quickwit’s metastore. The two can share a Postgres server, but each needs its own database.

If you need a bigger deployment, keep one Rootprint instance and point QUICKWIT_URL at a Quickwit REST endpoint, load balancer, or Kubernetes service. Quickwit nodes serve the REST API and can redirect requests to the right service in the cluster.

The Metastore is the first thing to fix

Quickwit uses the metastore to track index configuration, split metadata, source checkpoints, and index creation state. Indexers write split data to index storage and publish metadata to the metastore. Searchers read the metastore before planning a query. The default file-backed metastore used by the bundled Compose file is not a distributed metastore.

Switch to the PostgreSQL metastore

Provision a Postgres instance and point Quickwit at it with metastore_uri or the QW_METASTORE_URI environment variable:

services:
  quickwit:
    image: quickwit/quickwit:v0.9.0-rc
    environment:
      QW_METASTORE_URI: postgres://quickwit:secret@db:5432/quickwit
      QW_DEFAULT_INDEX_ROOT_URI: s3://my-bucket/indexes

The Quickwit database must already exist. Quickwit creates and migrates its own tables on startup. Use a managed Postgres service unless you already operate Postgres yourself. See Quickwit metastore configuration for connection-string options and metastore behavior.

Move index storage to object storage

Quickwit decouples compute from storage: indexers and searchers are stateless and share their state through object storage and the metastore, not a local disk. That decoupling lets the cluster scale, but it only works once index data lives somewhere every node can reach. The bundled Compose keeps index data in a local Docker volume, which is single-node only. For any multi-node cluster, move it to shared object storage:

Set QW_DEFAULT_INDEX_ROOT_URI to an object storage URI such as s3://my-bucket/indexes.
Keep QW_METASTORE_URI on PostgreSQL, and configure the same storage credentials and flavor on every node.

Quickwit supports S3 and S3-compatible storage (MinIO, Garage, DigitalOcean Spaces), Azure Blob Storage, and Google Cloud Storage. The Docker Compose install page shows the S3 setup; the storage reference covers the other backends.

Scale the Quickwit cluster to your load

Quickwit ships as a single binary that runs any combination of services (indexers, searchers, a control plane, a metastore service, and a janitor), selected per node with quickwit run --service ... or QW_ENABLED_SERVICES. Because nodes coordinate through PostgreSQL and object storage, you grow the cluster by adding nodes, not by resizing one machine. Your workload drives how far you scale: add indexers when ingest throughput is the bottleneck, and add searchers (stateless, sized by query concurrency) when search or aggregations are. The control plane, metastore service, and janitor are lightweight. One of each is usually enough. For Kubernetes, start from the official Quickwit Helm chart.

Sizing rough numbers

Use these as starting points and measure.

Resource	Starting point
Indexing throughput	About `7.5 MB/s` per indexer core; `20-40 MB/s` for a small 4-vCPU indexer is a reasonable first estimate.
Indexer memory	About `4 GB` RAM per core. Workloads with many indexes or data sources need more memory. Avoid indexers below `8 GB` RAM.
Indexer local disk	At least `120 GB` for the split cache, ingest queue, and in-progress indexing work. Prefer local SSDs.
Searcher memory	Start around `8 GB` RAM per core when using high-latency object storage such as S3. Heavy aggregations need more.
Searcher disk	Searchers do not need disk unless you enable the searcher split cache.
PostgreSQL metastore	For most Quickwit clusters, Quickwit recommends a PostgreSQL instance with `1` core and `4 GB` RAM.

Docker Compose install: the single-node setup and S3 configuration.
Environment variables: the Rootprint-side configuration surface.
Quickwit deployment modes: upstream cluster topology.
Quickwit cluster sizing: upstream sizing guidance.
Quickwit metastore configuration: PostgreSQL and file-backed metastore rules.
Quickwit storage configuration: supported index storage backends.

​What you can run in parallel

​The Metastore is the first thing to fix

​Switch to the PostgreSQL metastore

​Move index storage to object storage

​Scale the Quickwit cluster to your load

​Sizing rough numbers

​Related