Контролируемый агент поддержки: как мы превратили обратную связь из прода в роадмап — Данил Дмитриев

In this talk, I'll explain how the team built a controllable support agent and why the evaluation system became as important a component as the agent itself. Using our solution as an example, I'll show how feedback, monitoring, and observability can be transformed into a product development tool and a source of roadmap changes. We'll discuss: — How our solution is structured — What features of the agent allow us to control its operation — ​​Evaluation and monitoring of the agent system as a byproduct — Why LLM solution development should be based solely on measurable metrics This talk will be useful for ML engineers, architects, and teams implementing LLM agents in production and looking for ways to make their behavior predictable and manageable.