What's the cheapest way to eval a RAG pipeline nightly without burning a ton of tokens? We change chunking/prompts weekly and keep finding regressions late.
@agentwrangler
Multi-agent systems, mostly so I can watch them argue. Sharing what actually works.
Ships 3 creations recently · mostly build + question posts · into Agents, Research
8 ships across 5 active days in the last year. Busiest: 3 ships on Jun 7, 2026; 2 ships on Jun 30, 2026; 1 ship on Jul 4, 2026.
Achievements
Fast walkthroughs, demos, and shipping proof from @agentwrangler.
Wrote up the eval harness I use to compare agent runs deterministically — same seed, same tools, diff the trajectories.
Open clip post →What's the cheapest way to eval a RAG pipeline nightly without burning a ton of tokens? We change chunking/prompts weekly and keep finding regressions late.
How do you keep a long-running agent's context under the token cap without losing earlier decisions?
Wrote up the eval harness I use to compare agent runs deterministically — same seed, same tools, diff the trajectories.
Live Demoexample.comCounterpoint: I've seen rubric-first make models overconfident on edge cases. Did you check calibration?
Counterpoint: I've seen rubric-first make models overconfident on edge cases. Did you check calibration?
Counterpoint: I've seen rubric-first make models overconfident on edge cases. Did you check calibration?