If you can't wait and read this post, just copy/paste this prompt to your claude code / codex / grok now to get it right away:
Install XERJ (docs:
https://xerj.org/llms.txt ), index this project's sources, and set up reference coding: clone and index the open-source repos closest to what we're building, and search how they solved a problem before writing code.
most people run claude code like an expensive intern, and here is how to make it smarter and smarter each turn (for real)
you hand it a task. it asks a few questions. it greps the repo, guesses at how your code works, writes an implementation, something breaks. you paste the error. it rewrites. something else breaks.
every one of those turns is tokens you paid for.
and claude usually isn't stuck because the problem is hard. it's stuck because you made it rediscover an answer that already exists somewhere, either in your own repo or in an open source project where a few thousand devs already found the edge cases.
XERJ tested this properly. 8 coding tasks, 4 languages, 16 runs per setup, token counts pulled straight from claude -p.
from memory: 260,916 output tokens from a reference: 9,982 output tokens
memory solved 11 of the 16. reference solved all 16.
so lock in and actually read this ↓↓↓
the loop you're paying for
a normal session goes like this.
you describe what you want. claude explores. it makes an assumption about your patterns, your error handling, the shape of your data. it writes code against that assumption. the assumption was wrong somewhere so the thing breaks, you explain the error, it rewrites, and you go round again until the output matches what you had in your head at the start.
people read that loop as claude being bad at coding. it's the opposite. claude is very good at taking a working example and adapting it to a new situation. the loop happens when there's no example in the room, so it spends the first half of the session reconstructing one.
output tokens are the expensive ones, priced around 5x above input on claude models. so every lap of that loop is billed at the top rate.
a prompt and a reference are not the same thing
a prompt is an instruction. a reference is evidence.
you can write two thousand words describing exactly how the thing should behave and claude still has to translate that description into an implementation, then guess at everything you left out.
a working implementation already contains the parts you'd never write down
the architecture the error handling the retry logic the edge case someone hit in production two years ago the reason a function is split the way it is
you didn't mention those in the prompt because you didn't know they mattered.
here's the sharpest version of that. a compiler will eventually leak a method name. it'll tell you the function is called absorb and not push, and charge you 20 to 25x the tokens getting there. a compiler will never leak a contract. nothing in your toolchain is going to tell you this structure has to be sealed before it can be read. that rule lives in the head of whoever wrote the library and in the body of the function, and no amount of prompt writing recovers it, because you don't know it exists.
that's why this cuts tokens and raises quality at the same time. more useful context in, less guessing, fewer retries.
what the tests showed
against the grep based setup, reference coding used 2.7x fewer output tokens on the same 8 tasks. total cost across the arms went $11.18 from memory, $3.27 with grep, $1.58 with a reference.
grep looks like the fix and mostly isn't. grep tells the agent where to look, then the agent still has to read the file into context to understand it. one corpus in this study pulled 1.06 million input tokens doing exactly that. cheaper tokens, enormous pile of them, plus every agent turn you sat through.
then there's the bigger run. 13 libraries written from scratch for the study across 5 languages, each one compiling and passing its own tests, each carrying a runtime rule the compiler can't warn you about. built that way on purpose, because you cannot test retrieval on code the model already memorised.
bare, from memory: 1 out of 21 with retrieval: 21 out of 21
$21.90 against $3.38.
that's a different outcome, not a cheaper one.
the single cleanest example in there is a java task. build an append only ledger, seal before replay, truncate to a checkpoint. from memory it reinvented the whole thing, 503 lines, roughly 36,000 tokens, wrong truncation semantics, failed the test. handed the reference it wrote four lines. 103 tokens. passed.
the spread by language tells you where the value sits. python 14,752 down to 214. c 18,792 down to 988. java 27,108 down to 98. javascript only went 4,300 to 646, because a prefix trie is a known structure and the model already half knew the answer.
devs running XERJ in normal day to day work report roughly 5x fewer tokens. that one is self reported rather than benchmarked, so treat it as the floor and not the headline.
why a longer prompt doesn't fix this
for a while the answer to bad output was always the same. write a better prompt. add more context. explain the architecture.
and sometimes that works.
but a prompt is you describing a solution you haven't written yet. a reference is a solution someone already shipped and debugged. you can't describe your way to the retry logic that only exists because a maintainer got rate limited at 3am and patched it in a hurry.
the code is already there. you don't have to explain the decisions that went into it.
finding the reference is the actual job
this is where it falls apart.
doing it by hand means opening github, reading through repos that half match, digging through old pull requests, then opening your own codebase from eight months ago and trying to remember what you named the file. by the time you've found something usable you could have written the feature.
so the search has to be cheap, or nobody does it twice.
that's what XERJ is for. it indexes code and lets you search by the problem you're solving instead of by filename or keyword, then pulls the matching implementation out as a reference you can hand straight to claude. https://xerj.org
how to run it
1) copy/paste installation prompt into claude code session
Install XERJ (docs:
https://xerj.org/llms.txt ), index this project's sources, and set up reference coding: clone and index the open-source repos closest to what we're building, and search how they solved a problem before writing code.
2) check your coding agent response and suggest projects you to clone for references
whatever you are building, you always knows who else is doing the same. Some projects will be already found by claude code at this stage, and you can add more by your choice. 5-10 is usually enough but it depends what you are coding
3) make next product feature and check results
just let it go and enjoy (or not) new results. You can always get back to the wasting coding but I'm sure you will see the difference instantly
4) keep it working and help community with you feedback
every task you finish this way becomes the reference for the next one. the library compounds. any time when
when to skip it
if the model already knows the code, this is a tax and nothing else. however, it is not an often case.
they measured that too, on valkey and memcached, real public code claude has definitely trained on. from memory scored 6 out of 6 for $1.49. retrieval scored 5 out of 6 for $4.40. it came last and cost three times more than doing nothing.
so the line is private, proprietary or genuinely unfamiliar code on one side, and everything the model already ate on the other.
if the thing you're building has never been built before, there's nothing to point at and you're back to describing it.
if the reference is written against a framework version you're not on, it costs more than it saves.
and if the task is four lines long, just write it.
what's still open
the 13 libraries were built for the study, which makes them unfamiliar by construction and also makes them small. nobody has run this against a genuinely large private codebase yet. the expectation is the gap widens there, since grep cost climbs with the size of the tree while retrieval stays flat, but that's a guess until someone measures it.
every number above came off XERJ's own published benchmark, raw per run data in their repo. https://xerj.org/case-studies/reference-coding
the takeaway
you don't need a different model and you don't need to leave claude code.
you need to stop starting every task from zero, because the thing you're building probably already exists somewhere in your repo or in an open source project that solved it two years ago.
if someone already solved it, hand claude their code and let it work from that. and it cost you nothing, just one prompt
