LoginSign Up
← Back to all sparks
O

OpenVINO

AI-ASSISTANTS
Velocity2.5

Toolkit for optimizing and deploying deep learning inference

docs.openvino.ai ↗

OpenVINO is chasing every new model release while quietly moving under llama.cpp.

inference-optimizationnpumodel-compressionspeculative-decodingllama-cppintel-hardware
◆Current state
OpenVINO ships a numbered release each quarter with hotfixes and automated version bumps in between. Each release refreshes the supported model list across CPU, GPU and NPU — SmolLM3, LFM2, Qwen3 variants, Gemma 4, FLUX.2, YOLO26 — and tracks Hugging Face Transformers releases closely, now through v5.5. The compression and decoding work is where the durable value sits: EAGLE-3 speculative decoding extended to both LLMs and VLMs, INT4 KV-cache compression on GPU, and lazy weight loading for IR and ONNX models to cut peak memory at initialisation.
◆Where it's heading
Intel is fighting on two fronts with this toolkit. One is model coverage, which is a treadmill — every quarter's release is judged on whether last month's models run. The other is distribution, and that is where the 2026.1.0 llama.cpp backend matters: rather than asking developers to adopt the OpenVINO API, it puts Intel silicon optimisation underneath a runtime they already use. The NPU work follows the same logic, with ahead-of-time on-device compilation that no longer waits on OEM driver updates.
◆Prediction
The llama.cpp backend is still labelled preview, so promoting it out of preview with a wider validated GGUF model list is the natural next step.

◆Recent moves

  1. 1d ago

    2026.3.0

    The quarterly model refresh plus the decoding work that makes it worth taking: EAGLE-3 speculative decoding now covers VLMs as well as LLMs, Top-K sampling joins continuous batching, and lazy weight loading picks a compilation path automatically to hold down peak memory at initialisation.

    View source ↗
  2. 1mo ago

    Hotfix: YOLO26 GPU compilation and NPU queue priority

    Two defects fixed — YOLO26 failing to compile on GPU, and an NPU shared command queue violating inference priorities. Corrective work on models the previous release had just added.

    View source ↗
  3. 2mo ago

    2026.2.0

    INT4 KV-cache compression on GPU is the substantive change for anyone serving long prompts, and custom extension loading lets developers register operations OpenVINO does not implement rather than waiting for a release. The Gemma 4 and Qwen3 additions are the usual quarterly refresh.

    View source ↗
  4. 3mo ago

    Automated version bump to 2026.1.2

    A version bump commit that reached the feed with its pull request template unfilled. No change content is available from this entry.

    View source ↗
  5. 4mo ago

    2026.1.0

    ⚡ SPARK

    The release where OpenVINO stopped requiring adoption of its own API. Shipping as a llama.cpp backend puts Intel CPU, GPU and NPU optimisation under a runtime developers already run, which is a different distribution strategy than the model-coverage treadmill the other releases are on.

    View source ↗
  6. 4mo ago

    Hotfix: single commit cherry-picked from master

    A one-commit hotfix on the 2026.1 line, cherry-picked from master. The entry body is the project's pull request template rather than a description of the fix.

    View source ↗