NVIDIA NeMo vs Transformers

Transformers ships every two to four weeks on a split rhythm: minors carry day-0 architecture support for newly released open-weight models, patches almost exclusively unblock downstream serving runtimes. The last six releases added Meta's Muse Glimmer, Thinking Machines' Inkling, the Kimi K2.5 family and MiMo-V2-Flash, while three separate patches existed mainly to keep vLLM in sync. v5.15.0 breaks that pattern by landing four flagged breaking changes at once, including making kernel selection opt-in for linear attention models.

The refactor visible across these releases is a consolidation onto shared attention and kernel dispatch: the T5 family moved onto ALL_ATTENTION_FUNCTIONS, every linear attention model was rewritten against one convolution standard, and Gemma 4's heterogeneous attention config was made explicit through per_layer_config. The release notes state outright that the kernels package will likely become a required dependency of transformers[torch]. Alongside that, the project is absorbing compatibility work on behalf of vLLM rather than its own direct users — weight remaps and attention-backend flags added specifically for the vLLM modelling backend.

Expect kernels to move from opt-in to a hard dependency of transformers[torch], with more model families migrated onto the shared attention backend path and the eager-only route treated as a fallback. Day-0 architecture additions continue at the current pace on every minor.