How do enterprise tech leaders look past the immediate AI hype to place the right internal technology bets? Go inside the Office of the CTO at Red Hat to see exactly how we evaluate next-gen AI infrastructure and forecast where open source communities are heading next. In this episode of Technically Speaking, Red Hat CTO Chris Wright sits down with Steve Watt, VP of the Office of the CTO, to discuss how the industry is shifting from monolithic frontier models to composable intelligence and specialized hardware. Learn how open source projects are tackling everything from data privacy and semantic routing to running inference on modern x86 CPUs and orchestrating Slurm workloads inside Kubernetes.
Key topics from this episode:
00:00 – Intro: CTO perspective on outsourcing tasks vs. purpose in the age of AI
01:39 – Inside Red Hat’s playbook for tracking open source innovation
03:12 – Composable Intelligence: Vetting the shift from mixtures of experts to mixtures of models
05:11 – Why build vLLM Semantic Router: Internal cost, policy, and PII controls
08:19 – Internal R&D: Protecting proprietary model weights with Confidential Computing
12:01 – Evaluating self-managed multimodal workflows: vLLM-Omni & ComfyUI
14:38 – Optimizing power budgets: Leveraging x86 CPUs (Intel/AMD) for enterprise inference
17:54 – Inference-focused accelerators and the PyTorch OpenReg project
21:09 – Mass Open Cloud research: Running Slurm inside Kubernetes with Slinky operator
23:58 – Choosing our next agent bets: Benchmarking SkyRL, Monarch, & zero-trust security
Learn More:
Explore Red Hat’s emerging tech initiatives: https://www.redhat.com/en/technically-speaking
Deep dive into open source AI infrastructure: https://www.redhat.com/en/topics/ai
Follow us:
Chris Wright LinkedIn: https://www.linkedin.com/in/chris-wright-b733851/
Chris Wright Twitter/X: https://twitter.com/kernelcdub
Steve Watt LinkedIn: https://www.linkedin.com/in/wattsteve/
What is Technically Speaking?
Technically Speaking taps into emerging technology trends with insights from leading experts across the globe and Red Hat CTO Chris Wright. The series blends deep-dive discussions, tech updates, and creative short-form content, solidifying Red Hat’s role as a pioneer in technology innovation and open source thought leadership.
Want to participate? Leave us a comment if there’s a topic or a guest you’d like to see featured.
Watch More Technically Speaking:
YT playlist: https://www.youtube.com/playlist?list=PLbMP1JcGBmSGMPKkLoq6CGwwKDxl9VvBE
Subscribe to Red Hat’s YouTube channel: https://www.youtube.com/redhat/?sub_confirmation=1
#RedHat #EnterpriseAI #SemanticRouting #ConfidentialComputing #vLLM #OpenSource
