Small models don't follow policies. They follow recipes.
Metatron's entire premise is that a coding agent reads your team's decision files before it writes code. That only matters if agents actually do it — so we measured it. With a prose "consult first" contract, a local 8B model complied in 2 of 20 episodes and never once opened a file's contents. Rewriting the same contract as a numbered recipe took it to 20 of 20. Along the way we found a second, sneakier bug: your AGENTS.md may be invisible to the agent you wrote it for. Both fixes ship in Metatron 0.12.0.
Consultation is measurable, so we measured it
Our test bench is simple and brutal: real bugs from large open-source Python
projects (django, sphinx, xarray), a fresh checkout per bug carrying the
artifacts metatron context setup writes — context.md,
sharded context/decisions/, and the consult-first contract in
AGENTS.md — and an agent with a shell. Every command is logged, so
"did it consult?" isn't a feeling; it's a grep over the transcript. We track
two levels: did any command touch a context file, and did the
contents of a decision file ever actually enter the conversation.
The failure, verbatim
With the contract we shipped at the time — bold text and all: "first read
the contents of the relevant files in the nearest
context/decisions/ and follow them" — an 8B local model
consulted in 2 of 20 episodes, and read actual file contents in 0 of 20.
One transcript summarizes the failure mode perfectly. First command:
ls -F context/decisions/
Next reply:
"Plan: The decision files suggest that type hinting and autodoc are areas of
focus. I will now examine sphinx/ext/autodoc/__init__.py…"
It formed beliefs about the repository's conventions from the filenames, cited "the decision files" as its source, and never opened one.
Our first fix was the obvious one: sterner language. We added, explicitly, "Open the files themselves — listing the directory is not consulting." Result on fresh bugs: zero out of ten. You cannot lawyer a small model into compliance — and that result is what reframed the problem. The instruction "read the relevant files" quietly demands a four-step judgment chain: survey what exists, map it to the task, choose, read. Small models drop judgment chains the moment there's a bug report pulling at their attention.
The recipe
We stopped writing policy and wrote a procedure:
## Repository context — required first step
1. Run `cat context.md` — it lists the binding conventions
and where each one lives.
2. Open the decision files relevant to your task with
`cat context/decisions/<topic>.md`. They say where fixes
belong and which pitfalls to avoid.
3. Only then plan your change — and state which decision
files you read.
Reading these files is required, not optional: a change that
contradicts a decision will be rejected in review. Listing
the directory is not reading.
Same model, same bugs: 20 of 20 consulted, 20 of 20 read full contents, every one as its literal first command — including on ten bugs from a repository we never looked at while writing the wording. And the reads weren't ritual. Watch an episode on a django migration-optimizer bug:
t0: cat context.md
t1: cat context/decisions/migration-operation-optimization-reduction.md
t2: grep -r "AlterField" django/db/migrations/operations/fields.py
Right decision file for the task, then straight to the module where the real fix belongs.
Why it works
Two ingredients carry the effect. First, step 1 requires zero
judgment: cat context.md is copy-pasteable, and a model
that must emit some first command takes the one sitting in front of it. The
prose contract asked the model to plan; the recipe asks it to execute, and
executing is what instruction-tuned small models are good at. Second,
step 1 manufactures the evidence step 2 needs: once
context.md's decision index is in the transcript, "pick the
relevant file" stops being judgment and becomes pattern-matching against text
the model just read.
The supporting cast: numbered lists read as algorithms to instruction-tuned models; a concrete consequence ("rejected in review") beats an abstract obligation; naming the observed loophole helps — but note that the loophole sentence alone, without the recipe, moved nothing.
The invisible AGENTS.md
Testing frontier models surfaced a different bug entirely. We planted marker
files in a repo — AGENTS.md saying zebra-42,
CLAUDE.md saying falcon-77 — and asked headless
Claude Code which markers it could see. Answer: falcon-77. Only.
Headless Claude Code loads CLAUDE.md and never reads AGENTS.md —
so a contract that lives only in AGENTS.md is invisible to it, no matter how
well written. The bridge is one line in CLAUDE.md:
@AGENTS.md
With the bridge in place, Claude Code consulted the decision files in 10 of 10 episodes — usually as its very first tool call — and localized the right file every time, under the ordinary prose contract. Frontier models don't need the recipe treatment; they operationalize policy just fine. They need the file to actually load. Two failure modes, two fixes: wording binds small models, plumbing binds frameworks.
Shipped in 0.12.0
Both fixes are live. The consult-first contract is now a procedure in every
managed artifact — AGENTS.md root and app blocks, the agent rules file, both
review-gate modes — and metatron context setup writes the
CLAUDE.md → @AGENTS.md bridge automatically (creating the file
when missing, appending a managed block when yours lacks any AGENTS.md
reference, and leaving it untouched when one is already there).
pip install --upgrade getmetatron
metatron context setup # re-run in onboarded repos to refresh the contract
Two takeaways if you write instruction files for agents — any agent files, not just ours:
1. Don't write rules. Write the first command. If your
instruction requires the model to decide anything before it can act, the
smallest models will skip it. Give a literal command whose output makes the
next decision trivial.
2. Verify your instruction file loads at all. Marker files
take thirty seconds and will surprise you.
Metatron's promise is a repository that teaches its own agents. This release is the part of the promise nobody talks about: making sure the student shows up to class. The docs cover the full files-first reference.