LoginSign Up
← Back to all sparks
M

Meilisearch

DEVOPS
Velocity5.0

Open-source, fast, typo-tolerant search engine API.

www.meilisearch.com ↗

Meilisearch is trimming ~100ms sharding latency in prototype while shipping task-queue controls in stable.

searchshardingperformanceopen-sourceself-hosteddevtools
◆Current state
Meilisearch is running two active prototype branches in parallel: the reuse-http-client track targets sharded deployments by reusing the index-scheduler's HTTP client, cutting around 100ms per proxied task. The stable v1.53.1 released independently, shipping a configurable cap on simultaneous LMDB read transactions via MEILI_EXPERIMENTAL_TASK_QUEUE_MAX_READERS — a tuning knob for operators under heavy concurrent load. The 1.52.3 stable patched a crash by reverting a search speed optimization introduced in an earlier release.
◆Where it's heading
Sharding is becoming a first-class operational concern. The prototype branch pattern (public experimental tags before stable merges) is a deliberate quality gate: riskier changes land in a named prototype where the community can validate them before production inclusion. The operator-facing controls shipped in 1.53.1 — task queue reader cap, mini-dashboard bump — point toward a secondary track of infrastructure tuning features separate from core search capability.
◆Prediction
The reuse-http-client prototype will stabilize into a stable release once the batch of error-context fixes (rc.1 through rc.3) settle; the 100ms-per-task gain is a compelling merge for sharded clusters. The task queue reader cap env var suggests ongoing work to make LMDB concurrency more operator-tunable.

◆Recent moves

  1. 5d ago

    prototype-v1.50.0-reuse-http-client.3

    Adds diagnostic context to the CorruptedTaskQueue error message in the HTTP client reuse prototype — a debugging aid for operators hitting corrupted state, with no user-visible behavior change.

    View source ↗
  2. 5d ago

    Add error context for CorruptedTaskQueue in HTTP client reuse prototype

    Second iteration of the CorruptedTaskQueue error context fix in the reuse-http-client prototype — iterative debugging improvement with no functional change.

    View source ↗
  3. 5d ago

    Allow deletion of non-deserializable enqueued tasks

    Allows operators to delete enqueued tasks that cannot be deserialized — a practical fix for sharded deployments where corrupted or schema-mismatched tasks could block the queue indefinitely.

    View source ↗
  4. 6d ago

    Prototype: reuse HTTP client when proxying tasks (~100ms savings for sharded deployments)

    Prototype release that reuses the index-scheduler's HTTP client when proxying tasks in sharded setups, saving approximately 100ms per proxied task. This fits the trajectory of sharding becoming a production-grade, tunable deployment mode rather than an experimental one.

    View source ↗
  5. 14d ago

    prototype-v1.52.3-fix-crashes.6

    Switches to a temporary mimalloc repository in the crash-fix prototype branch — a build dependency swap with no user-facing impact.

    View source ↗
  6. 26d ago

    🖇️​ v1.53.1

    v1.53.1 ships MEILI_EXPERIMENTAL_TASK_QUEUE_MAX_READERS, an environment variable giving operators direct control over the number of simultaneous LMDB read transactions on the task queue. This is the kind of operational knob that matters at scale — pairing with the sharding prototype work underway, it continues the trend of exposing concurrency controls to operators rather than baking them as fixed constants.

    View source ↗