System Design for the GenAI Era | Ai Heroes 2024 Talk

🎟️ the 2025 edition on 27 May: https://aiheroes.it/2025/tickets/ ⏩ Chapter: 00:00 Intro 02:50 Two problems to build my own chatbot 08:10 One solution: GROUNDING 09:03 RAG architecture 11:50 System Design For GenAI 16:55 How do you use your LLM 28:30 LIVE DEMO We all learnt the basic modules of system design. Relational database, Blob storage, API Gateway, and Distributed Logging are familiar concepts. However, the rise of Generative AI applications brings a new set of challenges and tools to solve them. In this talk, we will explore the new modules essential for developing Generative AI (GenAI) applications, addressing their unique challenges. We will begin by talking about why using Large Language Models (LLMs) is insufficient. Next, we'll introduce guardrails, which are crucial for sanitizing both the input and output of LLMs and diffusion models. We'll then cover prompt compression techniques, designed to reduce the cost of generation. Following this, we'll present prompt registries, which enable over-the-air A/B testing and updates of prompts, and the role of observability, highlighting tools such as openllmetry, a new standard based on opentelemetry. We'll also examine how to enhance LLM capabilities with function calls, allowing LLMs to interact as agents and communicate with other external systems. During the discussion, we will show code examples in Python. By the end of this talk, attendees will have a clear understanding of the main modules used in building GenAI applications and gain valuable insights into designing their own. 🎤 EMANUELE FABBIANI, Head of Ai & Professor 📍AI Heroes is the Italian Artificial Intelligence Conference for Developers, Engineers, Data Scientists & Product Managers. The New Workflow to Shape the Future of Business AI in the Next Generation Workforce Key Technologies, Applications and Trends ℹ️ inform: https://aiheroes.it/ ✅ Tweet:   / aiheroes_it   ✅ Connect:   / aiheroes-it   ✅ Read: https://t.me/aiheroes #GenAI #SystemDesign #AIDevelopment #LLMArchitecture #AIEngineering #PromptEngineering #OpenLLMetry #AIBuilders #TechTalk #AIHeroes2025