Going from top-5 to top-20 retrieval makes every RAG answer 2.7x more expensive
Your support bot answers from your docs. It gets a wrong answer in testing, so you raise top_k from 5 to 20 in the retriever settings, the next answer comes out right, and you ship it. Nobody notices that each answer now costs 2.7 times as much, because the setting lives in the search step, far fro…
Read the full story at DEV Community — AI ↗
Timeline · 1 report
- 2026-11-06 12:00 · DEV Community — AI
Going from top-5 to top-20 retrieval makes every RAG answer 2.7x more expensive