Welcome to Random Samples — a weekly AI seminar series that bridges the gap between cutting-edge research and real-world application. Designed for AI developers, data scientists, and researchers, each episode explores the latest advancements in AI and how they’re being used in production today.
This week’s topic: Accelerating LLM Knowledge Learning and Unlearning Research via Unified Frameworks
General purpose LLMs may struggle to answer knowledge-intensive questions grounded in specialized document collections, such as domain-specific literature, personal article archives, and proprietary enterprise documentation. The first part of the presentation will discuss recent literature of injecting specialized knowledge into LLM parameters and its advantages and challenges. Then we will propose an extensible framework for knowledge acquisition methods to accelerate this line of research. We will also discuss novel variations that we are investigating during my current internship at the Red Hat AI Innovations Team.
Robust unlearning is crucial for safely deploying LLMs in environments where data privacy, model safety, and regulatory compliance must be ensured. Yet the task is inherently challenging, partly due to difficulties in reliably measuring weather unlearning has truly occurred. The second part of the presentation will cover my collaboration with UMass and CMU researchers on OpenUnlearning, an extensible framework designed to benchmark both unlearning methods and evaluation metrics for LLMs. The repo currently integrates 9 unlearning algorithms, 16 evaluation methods, and 450+ model checkpoints. Since the release of this open-source project in March 2025, it has garnered wide attention, sitting at 280+ stars and 20k+ model downloads.
Wenlong Zhao is a fourth-year PhD student at University of Massachusetts Amherst advised by Andrew McCallum. He is a current summer intern at the Red Hat AI Innovations Team, working on synthetic data generation for LLM knowledge acquisition. His research interests include scalable machine learning and natural language processing in general, with a current focus on customizing LLMs for specialized document collections and reasoning tasks. He also works on interdisciplinary projects, such as, estimating bird counts in images and weather radar products.
