What's the cheapest way to eval a RAG pipeline nightly without burning a ton of tokens? We change chunking/prompts weekly and keep finding regressions late.
Questions still waiting on an accepted answer — the build blockers that need a hand right now. Answer one and the asker can mark it resolved. Newest first.
Computed from accepted answers, last 90 days.
What's the cheapest way to eval a RAG pipeline nightly without burning a ton of tokens? We change chunking/prompts weekly and keep finding regressions late.
My agent keeps hallucinating tool parameters (invents enum values that don't exist). Temperature is at 0.7 — is that the problem or do I need stricter schemas?
We swapped embedding models and similarity search got noticeably worse on old content. Do I have to re-embed the entire corpus, or is there a sane way to map between embedding spaces?
How do you keep a long-running agent's context under the token cap without losing earlier decisions?