LoginSign Up
← Back to all sparks
V

vLLM

AI-ASSISTANTS
Velocity5.0

High-throughput inference and serving engine for large language models

docs.vllm.ai ↗

Only release candidates reach this feed, each carrying a single cherry-picked fix

llm-inferencerelease-candidatesrocmcudadisaggregated-servinghardware-support
◆Current state
vLLM is a high-throughput inference engine for large language models, but what this feed captures is exclusively its release-candidate tags. All five entries are rc builds spanning v0.24.0rc2 to v0.26.1rc0, and each body is a single commit subject: a ROCm test reference value, a prefill/decode KV load fix, embedding scaling under CUDA graphs, a flaky ARM CPU test. No stable release appears in the window at all.
◆Where it's heading
The visible signal is release engineering rather than product direction. Hardware breadth — ROCm, ARM CPU, CUDA graph capture — and disaggregated prefill/decode correctness are the recurring themes, consistent with an engine being hardened across accelerators rather than one gaining new capability. Because only rc tags are captured, the cadence here reflects patch traffic; the substantive release notes live on the stable tags this feed is missing.
◆Prediction
Expect further rc tags in the same shape. A confident read on vLLM's direction isn't possible until stable releases appear in this feed rather than candidates alone.

◆Recent moves

  1. 3d ago

    Release candidate fixes a ROCm correctness test reference

    A CI-only correction to a reference value in a ROCm wikitext correctness test. Nothing user-visible, and representative of what this feed actually captures.

    View source ↗
  2. 22d ago

    Release candidate fixes KV load lookahead in disaggregated serving

    Corrects async KV load lookahead handling for speculative decoding under prefill/decode disaggregation — a targeted fix in one of the engine's more intricate paths.

    View source ↗
  3. 22d ago

    Release candidate fixes embed scaling with CUDA graphs

    Fixes embedding scaling and CUDA graph capture in the Transformers modelling backend. Correctness work on one backend path rather than new capability.

    View source ↗
  4. 23d ago

    Release candidate fixes a flaky ARM CPU prefill test

    Resolves an uninitialized-value flake in a ShortConv prefill test on ARM. CI hygiene on the CPU backend.

    View source ↗
  5. 1mo ago

    Release candidate fixes prefill-decode with the DP supervisor

    A cherry-picked fix for prefill/decode disaggregation under the data-parallel supervisor. The same pattern as the rest of the window: one backported commit per tag.

    View source ↗