[ Category ]
All technical deep dives, system architectures, and AI breakdowns, ordered by date.
Self-Hosting & Open SourceShimmy is a standalone, lightweight LLM inference server written in pure Rust that leverages WebGPU compute shaders to run GGUF models across Vulkan, Metal, and DirectX 12 hardware with zero CUDA or Python dependencies.
Self-Hosting & Open SourcevLLM v0.28.0 (August 26, 2026) packs 584 commits from 270 contributors: Decode Context Parallel for Kimi-K3, end-to-end sparse MLA for DeepSeek V4, tiered KV cache offloading to disk, DFlash2 speculative decoding — and breaking changes, with bitsandbytes migrated out of tree.
Self-Hosting & Open SourceVersion 2.8.8, shipped 29 August 2026, adds in-place PDF text editing with live reflow and font matching, plus two new compression engines. The architecture is the real story: the self-hosted image has no application server and no upload endpoint, which changes what you are trusting — and makes the one component that does make network calls worth reading about.
Self-Hosting & Open SourcePangolin 1.22 puts an identity-aware proxy in front of OpenAI, Anthropic and Gemini as well as your own Ollama and vLLM servers, with virtual API keys and budget enforcement. It is also a clean case study in open-core: the gateway itself is AGPL-3, but the session logging is a 727-byte stub whose real 10 KB implementation sits under a proprietary licence header.