Transformers vs Gemini

Transformers is in a routine release rhythm: a minor release every two-to-three weeks adding new model families (Cohere2Moe, DeepSeek-V4, Laguna from Poolside, Parakeet, HRM-Text, OpenAI Privacy Filter), interleaved with patch releases that fix tokenizers, attention paths, and vendor-specific integration bugs (Qwen 3.5/3.6 FP8, Kimi-K2.5 tokenizer, Gemma4 device-map). Mixture-of-experts is the dominant architecture in this window — most newly added models are MoE variants.

The library is consolidating its position as the reference implementation for new model architectures: as soon as a vendor ships a frontier model, the corresponding transformers integration lands within days or weeks. MoE-with-novel-routing (sigmoid routers, expert-id hashing, hybrid attention) is becoming the default architectural assumption, and transformers is absorbing the variations without major API churn. The patch-release pattern — flash-attention paths, FP8 quantization fixes, tokenizer regressions — shows the maintenance load is concentrated at the integration edges, not the core.

The next minor release will almost certainly add another two-to-four MoE models on the current cadence, and the next patch release will land within a week to fix whatever quantization or tokenizer regression slipped through. Watch for a deeper refactor of the MoE routing abstractions if vendor architectures keep diverging — the current per-model branches are accumulating.