AI-агент в продакшне: браузер, тулы и эвалы | Андрей Мелихов, DataLens, Yandex Cloud

We discussed creating an AI agent for a BI system with Andrey Melikhov, author of https://t.me/melikhov_dev and developer of DataLens: why the agent lives in the browser, how it calls tools, builds charts, works with data, verifies itself via an eval harness, and why simply adding LLM to production quickly turns into a major engineering challenge. We discussed: — Why ready-made frameworks are lacking; — How AI agents are structured in the browser; — SSE streaming, BFF, tool calling, and agent loop; — Problems with different models and inferences; — evals, LLM-as-a-judge, and the hardware user; — Trust in numbers in BI and verifiable charts; — Will GenBI replace analysts; — MCP, server agents, and future APIs; — How Claude Code, Codex, OpenCode, and other coding agents are changing development; — Why AI speeds up developers by 3-5 times, but the tasks only increase; — Security, permissions, sandboxing, and corporate restrictions; — AI outside of work: repairs, research, medicine, presentations, NotebookLM, and news bots; — Bus factor, spec-driven development, and context for agents. I write more about AI in the "Grinenko Pro" Telegram channel: https://t.me/devspotting, and you can discuss neural networks in the community chat — https://t.me/grinenko_pro TIMECODES 01:24 — Andrey Melikhov 03:30 — Why building an agent in 15 minutes doesn't work in production 05:08 — What are DataLens and GenBI 06:29 — Why you can't just feed all the data to a neural network 07:45 — What does "frontend" mean in Yandex 09:31 — Browser state, sessions, and the Responses API 13:38 — Different models, inferences, and tool calling issues 16:14 — First mistakes: Vercel AI SDK, OpenAI SDK, and your own layer 18:28 — Why you can't develop an agent without evals 20:33 — Eval harness, LLM judge, and iron User 24:52 — Agent Architecture: BFF, SSE, Tools, and Agent Loop 28:59 — How an Agent Builds Interactive Charts in Chat 31:06 — Query Classifier and Filtering Out Non-BI Questions 34:55 — Will GenBI Replace Analysts? 38:51 — Trust in Numbers: Why LLM Shouldn't Calculate Itself 41:38 — Tool Judge and Filter Checking 43:38 — Dialog Costs and Context Optimization 47:55 — MCP, API, and Agent Migration to the Server 51:14 — How AI Agents Change Developer Work 54:35 — Flow: Plan → Review → Implementation → Review with a Second Model 56:46 — Automation of Bug Reports and Tickets 59:07 — Duties, Logs, and Neural Networks as a Third Line of Support 01:04:46 — Security of coding agents: permissions, sandboxing, open source 01:09:49 — Native harnesses, universal agents, and benchmarks 01:14:51 — Accelerating development by 3-5 times 01:18:44 — AI outside of work: repairs, medicine, presentations 01:21:07 — NotebookLM, podcasts, and preparing for presentations 01:22:09 — AI/Frontend news bot 01:24:01 — What will happen to development in 1-2 years 01:25:26 — Bus factor, spec-driven development, and context for agents 01:30:28 — Final thoughts LINKS ON THE TOPIC: DataLens: https://datalens.ru/ OpenAI Responses API: https://developers.openai.com/api/doc... OpenAI Chat Completions: https://developers.openai.com/api/ref... LangGraph: https://docs.langchain.com/oss/python... Vercel AI SDK: https://ai-sdk.dev/docs/introduction Langfuse: https://langfuse.com/docs Playwright: https://playwright.dev/ NotebookLM: https://notebooklm.google/ Grinenko Pro Telegram Channel: https://t.me/devspotting Like, subscribe, and share this episode with others building AI agents. #AI #LLM #AIAgents #DataLens #GenBI #Frontend #ClaudeCode #Codex #MCP