153. LLM Inference with Bedrock

If you’re curious about building with LLMs, but you want to skip the hype and learn what it takes to ship something reliable in production, this episode is for you. We share our real-world experience building AI-powered apps and the gotchas you hit after the demo: tokens and cost, quotas and throttling, IAM and access friction, marketplace subscriptions, and structured outputs that do not break your JSON parser. We focus on Amazon Bedrock as AWS’s managed inference layer: how to get started with the current access model, how to choose models, how pricing works, and what to watch for in production. We also go deep on structured outputs: constrained decoding, schema design that improves output quality, and how to avoid “grammar compilation timed out”. 🧠 Sponsor Thanks to fourTheorem for powering AWS Bites. We help teams build cloud systems that are simple, scalable, and cost effective. Visit https://fourtheorem.com 🧭 Chapters 00:00 Intro 02:06 Definitions: LLM, inference, tokens 05:02 Real-world use cases 11:32 AI vs GenAI, Agents and workflows 16:51 Bedrock overview 23:26 Getting started with Bedrock 30:51 Pricing and cost control 35:13 Gotchas: quotas and access 39:24 Structured outputs and JSON 41:50 Wrap up 🔗 Resources fourTheorem: Bedrock structured outputs guide https://fourtheorem.com/amazon-bedroc... Amazon Bedrock https://aws.amazon.com/bedrock/ Bedrock docs https://docs.aws.amazon.com/bedrock/l... Bedrock pricing https://aws.amazon.com/bedrock/pricing/ Structured outputs https://docs.aws.amazon.com/bedrock/l... Cross-region inference https://docs.aws.amazon.com/bedrock/l... Quotas https://docs.aws.amazon.com/bedrock/l... Throttling help https://repost.aws/knowledge-center/b... Prompt caching https://docs.aws.amazon.com/bedrock/l... Troubleshooting error codes https://docs.aws.amazon.com/bedrock/l... 🎧 Listen Apple Podcasts https://podcasts.apple.com/us/podcast... Spotify https://open.spotify.com/show/3Lh7Pzq... RSS https://anchor.fm/s/6a3312a0/podcast/rss 💬 Questions or feedback? Comment here or reach us Eoin https://bsky.app/profile/eoin.sh |   / eoins   Luciano https://bsky.app/profile/loige.co |   / lucianomammino   #AWS #AmazonBedrock #LLM #GenAI #StructuredOutputs #AWSBites