Find the BEST RAG Strategy with Domain Specific Evals
Creating custom RAG chunking and embedding strategies with domain specific evaluation experiments Resources: Notebook Repo: https://github.com/ALucek/custom-rag-... Prior Chunking Breakdown: • The BEST Way to Chunk Text for RAG ChromaDB Repo: https://github.com/brandonstarxel/chu... ChromaDB Colab Notebook: https://colab.research.google.com/dri... ChromaDB Research Report: https://research.trychroma.com/evalua... Chapters: 00:00 - Why Measure Chunking & Embedding 03:44 - Creating a Custom Chunking Strategy 06:12 - Breaking Down Eval Metrics 08:00 - Metrics: Eval Dataset 09:17 - Metrics: Recall, Precision, IoU 13:27 - General Evals: Describing Test Set 14:32 - General Eval: Process & Running Test 17:58 - General Eval: Embedding Test 19:00 - Running Multiple Evals Across Strategies 20:36 - Multiple Evals: Interpreting Results 23:18 - Domain Specific Dataset Generation & Filtering 27:06 - Running Domain Specific Evals 29:32 - Final Thoughts #ai #machinelearning #datascience
Stop Prompt Engineering! Program Your LLMs with DSPy
Building with Claude API - Part 3 - Temperature
I Made 3 AI Tools That Solve Real Problems
Is RAG Still Needed? Choosing the Best Approach for LLMs
The BEST Way to Chunk Text for RAG
Improving RAG Retrieval by 60% with Fine-Tuned Embeddings
RAG Evaluation Metrics Explained: Context Precision, Recall, Relevancy & Faithfulness
Knowledge Graph or Vector Database… Which is Better?
Fine-Tuning Text Embeddings For Domain-specific Search (w/ Python)
Practical RAG - Choosing the Right Embedding Model, Chunking Strategy, and More
Do Reranking Models Actually Improve RAG?
The 5 Levels Of Text Splitting For Retrieval
RAG vs. CAG: Solving Knowledge Gaps in AI Models
Does Fine Tuning Embedding Models Improve RAG?
From Retrieval to Navigation: The New RAG Paradigm
How to build Multimodal Retrieval-Augmented Generation (RAG) with Gemini
The Best RAG Technique Yet? Anthropic’s Contextual Retrieval Explained!
What Do LLM Benchmarks Actually Tell Us? (+ How to Run Your Own)
What Is Disrupting GPS Over Europe?
