0 views
Why do powerful GPUs sometimes sit idle during AI inference? 🚀 Nick Hill tells Chris Wright about the performance tricks vLLM uses, like speculative decoding and batching, to solve AI’s biggest bottleneck and boost throughput. Hear more of vLLM’s optimization secrets in the full Technically Speaking with Chris Wright episode!
#vllm #ai #AIOptimization #GPU #KVcache #LLM #RedHat
Date: July 16, 2025
![[vLLM Office Hours #26] Intro to torch.compile and how it works with vLLM](https://videos.sebae.net/wp-content/uploads/2025/05/hqdefault-386.jpg)
