vLLM 0.28.0: Deep Kimi-K3 and DeepSeek V4 Gains, and a Painful bitsandbytes Split

The most widely deployed open-source inference engine just shipped a dense release: vLLM v0.28.0 landed on August 26, 2026 with 584 commits from 270 contributors (76 first-time). The changelog is long, but three pillars stand out: a comprehensive Kimi-K3 performance push, end-to-end DeepSeek V4 support, and breaking changes that will decide exactly when — and whether — your cluster upgrades this week.

The Kimi-K3 Push

The effort spans the whole stack: Decode Context Parallel (DCP) support, fused FlashKDA decode and prefill kernels, SiTU activation support for MegaMoE, GEMM-RS for sequence parallelism, combined all-gather kernels with a 1.5–3× kernel-level speedup, an adaptive speculative token budget delivering ~60% better DSpark TTFT, and an optional shared-expert sharding mode that saves ~17 GiB of memory per GPU. Kimi-K3 also now runs on AMD ROCm with the V2 model runner.

DeepSeek V4 Goes End-to-End

Sparse MLA now works through the entire lifecycle — plain decode, MTP, and DSpark speculative decoding — which is the foundation of practical throughput for DeepSeek’s sparse-attention architecture. It ships alongside AMD Quark NVFP4 support, reasoning-effort prompts and mappings, sparse top-k metadata kernel optimizations, narrowed eager CUDA graph regions, and ROCm enablement on gfx11 and gfx950.

Three Changes That Serve Everyone

Speculative decoding advances: DFlash2 brings local convolution and a candidate selector, DSpark gains confidence-scheduled verification, and async scheduling is now auto-enabled for draft models.
Tiered KV cache offloading to disk: a second memory tier with out-of-tree tier managers via module_path and built-in tiering metrics. Practically: longer contexts on the same HBM budget.
Model Runner V2 maturation: E/P/D disaggregation, weight offloading, multi-layer MTP KV cache support, and thinking_token_budget support.

Breaking Changes — Read Before Upgrading

bitsandbytes moved out of tree to a plugin — a default pip install vllm no longer includes quantization support via bitsandbytes.
Transformers bumped to 5.15.0 — pinned environments will need coordination.
The deprecated calculate_kv_scales runtime KV-scale calculation was removed, as was override_attention_dtype.
The good-news defaults riding along: max_num_batched_tokens doubles from 8192 to 16384, prefix caching is enabled by default for Mamba models, and the Blackwell CUDA graph capture default rises to 1024.

If your deployment leans on bitsandbytes quantization or assumes the old defaults, this is not a “bump and restart” upgrade — install the companion plugin and re-benchmark after the move.

New Model Support

A short but telling list: Meta’s Muse Glimmer, Ling 3.0 Flash with BF16, MTP and parser support plus an FP8 variant and hybrid MXFP4 routed experts, Dots3 NOTE native multimodal support, and Interns2mobius. For AMD hardware: Qwen3.8 enabled on ROCm and a fused CUDA post-conv MTP decode kernel for Qwen3.5 GDN.

What Operators Should Do

Install per your hardware: official wheels ship for CUDA 13.0 and CUDA 12.9, ROCm 7.2.2, and Docker images for every stack — including vllm/vllm-openai-cpu and XPU.
Re-test, don’t assume: changed defaults (max_num_batched_tokens, prefix caching for Mamba) can move your memory footprint and throughput profile, not just raw speed.
Plan for shared-expert sharding if you run Kimi-K3 — 17 GiB per GPU is the difference between a node that fits and one that doesn’t.
Watch the trend line: the release lands amid a real acceleration in engine support for new models (Kimi-K3, Hy4-preview, Ling 3.0). The window between a model’s launch and full vLLM support is visibly shrinking.