[vLLM Office Hours #58] – Intro to llm-d Semantic Classifier – October 1, 2026

Welcome to vLLM office hours! These bi-weekly sessions are your chance to stay current with the vLLM ecosystem, ask questions, and hear directly from contributors and power users.

This week’s special topic: Intro to llm-d Semantic Classifier.

vLLM project update from core maintainer Michael Goin, covering a new hardware backend from Tenstorrent, HiSparse for sparse-MLA decode, native tiered KV cache offloading, and a month of optimization work on Kimi K3. Plus two releases, v0.29 and v0.30, which turned Model Runner V2 on by default, generalized adaptive verification to all draft-model speculators, landed watermarking support, and sped up startup and scale-out.

Then our special topic with Christopher Nuland, Chief AI Architect at Red Hat: llm-d-sc, a lightweight classification microservice built for speed. Where KV-cache routing decides which pod already has your prefix warm, semantic classification decides which model tier a request belongs to in the first place. Christopher covers why it was split out as its own service, how it integrates with llm-d and the Praxis proxy, how it compares to vLLM Semantic Router and Switchyard, what it deliberately does not try to solve, and a look at domain-based routing from an in-flight NASA proof of concept.

Slides: https://docs.google.com/presentation/d/1ZnH1l1tiTTiOBmZ62gqBGn9-SBSBKfN8nhIuB22aYdw

Want to join the discussion live on Google Meet? Get a calendar invite by filling out this form: https://red.ht/office-hours

Timestamps:
00:00 Intro
02:09 vLLM project update
12:29 vLLM v0.29 and v0.30 releases
16:35 llm-d Semantic Classifier deep dive
42:50 Q&A