Random Samples: Towards Combinatorial Interpretability of Neural Computation [May 9, 2025]

Random Samples is a weekly seminar series that bridges the gap between cutting-edge AI research and real-world application. Designed for AI developers, data scientists, and researchers, each episode explores the latest advancements in AI and how they’re being used in production today.

This week’s topic: Towards Combinatorial Interpretability of Neural Computation

This session introduces a novel combinatorial approach to neural network interpretability, based on MIT CSAIL and IST Austria research. It focuses on relationships within network weights and biases to understand how neural networks compute logic, specifically through the Feature Channel Coding Hypothesis. This hypothesis reveals how networks compute Boolean expressions by mapping features to neuron combinations, forming "codes." Understanding these codes enables decoding network logic without retraining. The session will also discuss "code interference," a complexity-driven phenomenon revealing natural limitations. Attendees will gain a deeper understanding of neural network "thought," making this research critical for more interpretable, scalable, and trustworthy AI.

Paper: https://arxiv.org/abs/2504.08842
Blog: https://developers.redhat.com/articles/2025/04/22/how-neural-networks-might-actually-think

Subscribe to stay ahead of the curve with weekly deep dives into AI! New episodes drop every Friday.