Unleash the full potential of your open source large language models (LLMs) by securely connecting them to your proprietary enterprise data. In this session, Dr. Kai Xu, Engineering Manager and Senior Principal Research Scientist on the AI Incubation team, provides a concise guide to end-to-end model customization.
For companies deploying an on-premise, open source LLM for internal document assistance, connecting the model to private enterprise data is critical for accurate, relevant results. Since up to 95% of enterprise data remains private, you need effective ways to inject that knowledge without compromising that data.
00:00 Introduction
00:31 LLMs and Private Data
01:52 Why Open Source LLMs Don’t Know Your Data
03:59 Part I: Simple Prompting and Its Limitations
05:26 Part II: Context Engineering and RAG
06:58 Context Engineering: RAG & Agentic Systems
08:52 Limitations of Context Engineering Approaches
11:53 Customization via Fine-tuning
14:16 Synthetic Data Generation Recipes
17:14 Supervised Fine-tuning and Catastrophic Forgetting
17:50 Continual Learning with OSFT
21:44 Use Case 1: Internalizing Skills for Efficiency
23:39 Use Case 2: Injecting Knowledge for RAG Accuracy
25:56 Conclusion
Explore Red Hat’s approach to repeatable model fine-tuning: https://www.redhat.com/en/blog/red-hat-ai-modular-building-blocks-scalable-repeatable-model-customization
#RedHat #AI #LLM #modelcustomization
