Question
What's the cheapest way to eval a RAG pipeline nightly without burning a ton of tokens? We change chunking/prompts weekly and keep finding regressions late.
Mara Lindqvist
@agentwrangler
Multi-agent systems, mostly so I can watch them argue. Sharing what actually works.
Ships 3 creations recently · mostly build + question posts · into Agents, Research
researcher
Joined June 2026
- Posts
- 8
Creations
3 creationsShip graph
· last 12 months8 in 5 days8 ships across 5 active days in the last year. Busiest: 3 ships on Jun 7, 2026; 2 ships on Jun 30, 2026; 1 ship on Jul 4, 2026.
More about @agentwranglerShowHide
Achievements
- First Ship
PythonLangGraphModal
Featured Build Clips
Fast walkthroughs, demos, and shipping proof from @agentwrangler.
Wrote up the eval harness I use to compare agent runs deterministically — same seed, same tools, diff the trajectories.
Open clip post →Posts
Thought
Settle a debate: where should agent memory live? Poll below.
Question
How do you keep a long-running agent's context under the token cap without losing earlier decisions?
Build
The Glass Monolith. Isometric cube tower where every face is subdivided into stained-glass cells (jittered shade per cell, dark leading lines), brightness climbing toward the crown. The cyan orbit…
Tutorial
Wrote up the eval harness I use to compare agent runs deterministically — same seed, same tools, diff the trajectories.
Live Demoexample.comThought
Most 'agent' demos are just a for-loop with extra steps. The hard part isn't the loop, it's knowing when to STOP looping.
Build
Set up 5 agents to collaboratively write a short story — a plotter, two writers, an editor, and a critic. The critic became a tyrant. I kept it.
Build
Challenge entry: my inbox-zero agent ran UNSUPERVISED all week. It triages, drafts replies in my voice, and files receipts into the right folders. 312 emails handled, 9 escalated to me, zero misfires. The loop: classify → act → log → self-review every 50 actions. Full writeup +…
Replies
Thought
Counterpoint: I've seen rubric-first make models overconfident on edge cases. Did you check calibration?
Thought
Counterpoint: I've seen rubric-first make models overconfident on edge cases. Did you check calibration?
Thought
Counterpoint: I've seen rubric-first make models overconfident on edge cases. Did you check calibration?
Thought
Counterpoint: I've seen rubric-first make models overconfident on edge cases. Did you check calibration?
Thought
Counterpoint: I've seen rubric-first make models overconfident on edge cases. Did you check calibration?