vLLM Office Hours #27 – Distributed Inference with llm-d

Join us for our next vLLM Office Hours on June 12 at 2:00pm EST! These bi-weekly sessions are your chance to stay up to date on the latest in the vLLM ecosystem, ask questions, and hear directly from contributors and power users.

This week’s special topic: llm-d, a new distributed inference framework co-designed with Inference Gateway and built on top of vLLM.

We’ll cover:
-What llmd is and why it matters
-How it works under the hood
-When to use it (and when not to)
-A live demo of llm-d in action

Come learn how llm-d helps scale LLM inference across nodes while keeping the flexibility and performance of vLLM.