Eliminating cruft from the codebase

Two posts back, I wrote about tackling the bloat in standing instructions for AI agents. Where we left it: the amorphous mass of commentary cruft was shifted elsewhere, not eliminated, and it was still growing. It lurked like a fatberg in the sewers beneath this blog.

Being something of a pedant with obsessive-compulsive tendencies, I couldn't let that outcome sit. I kept nagging and pestering Claude until we could figure out a more permanent solution. Seriously, I just couldn't let it go. If my AI agent had a mouth, it would scream.¹Note 1¹With apologies to Harlan Ellison.

So I'm happy to report that the file that logs the arguments behind the rules is now less than half the size it was. The decisions log is now 1,058 lines long. A week ago it was 2,447. The core documentation survived, and now there's a test to stop it growing back.

Previously on Be Useful

Recapping the saga, the instructions file that Claude reads at the start of every session had grown to 1,734 lines. This was bad because it was nibbling away at my precious tokens – 25,000 and counting – which hampered how much work could be done in a single session.

So what we did was cleave the file in two. The rules stayed in CLAUDE.md, held to 280 lines by a test. The arguments behind the rules moved over to docs/decisions.md, a log that Claude only opens when a rule points it there.

That kept every session cheap. So it was a victory, of sorts. But the log had no limit, and at the time of publication it was 2,260 lines and still growing. The obvious next step was a script to audit those files.

Counting the words

npm run measure:bloat reads the instructions file, the decisions log and every comment scattered across the codebase (which had a habit of parroting the log), then prints a report.

It checks how close the instructions file is to its 280-line limit. It lists the longest entries in the log. And it looks for repetition, by sliding an eight-word window along every document and flagging any comment that repeats a run of words from the log. A comment that copies an argument from the log is redundant and has to go.

But before the script gets round to doing any of that, it has to test itself. It takes 20 real words from the log and checks that it spots them, then feeds in a passage of made-up nonsense and checks that it spots nothing. A script that has stopped working and a codebase with nothing wrong in it would otherwise print the same clean report.

const N = 8; // words per shingle

function shingles(ws) {
  const set = new Set();
  for (let i = 0; i + N <= ws.length; i++)
    set.add(ws.slice(i, i + N).join(" "));
  return set;
}

// Lift 20 real words from the log — the detector must find them.
// Feed in nonsense — it must find nothing.
const lifted   = words(decText).slice(400, 420).join(" ");
const invented = "zephyr marmalade quantum lighthouse accordion tangerine velvet orbit";

const hit  = [...shingles(words(lifted))].filter((k) => decisions.has(k)).length;
const miss = [...shingles(words(invented))].filter((k) => decisions.has(k)).length;
// Self test passed: 13 copied shingles found, 0 invented.

With the script up and running, it found the same arguments written out multiple times, once in the log and again in comments across four files. Replacing each copy with a pointer to the log removed 42 lines.

It kept growing, though

Removing duplicates didn't stop the growth, unfortunately. A few days later, the log was 2,447 lines.

At the roughly four characters per token from the last post, that's about 34,000 tokens, more than the instructions file that started all this. Granted, decisions.md isn't going to decimate the token budget in the same manner as CLAUDE.md upfront, but when called upon it still takes a bite.

The script could measure the problem, evidently, but it couldn't say what the log should look like instead.

Time for a new rule

Going back to the fundamentals, the problem was written into the log's own first paragraph.

It described every entry as "a decision someone had to defend, kept in full". CLAUDE.md sent it everything that wouldn't fit in a few lines. So the log was where all the overflow went, with an instruction to keep the whole story and no limit on size.

So the rule had to be replaced with a new one, stating more carefully what an entry is for.

An entry states the decision, the reason and what would reopen it, in about 20 lines. How it got there … is history and lives in git.

Comments in the code got the same treatment, too. A comment is now a line or two. It says only what the code can't, such as a trap or a hidden constraint, or it points to the log entry that holds the argument.

The key realisation is that the whole story of each decision was already saved. Git, the version history behind the code, keeps every earlier state of every file. The log had been keeping a second copy of all of it, and it really didn't need to.

Documentation should preserve the information future collaborators need, not the information that explains every step by which the present was reached.

Making the cuts

Thus armed with the new rule, we reviewed the log and the code comments and cut them back. Across both, 4,370 lines came out and 1,359 went back in.

About 1,400 lines of that saving came from the log, about 1,600 from comments in the code, and a few dozen from tidying a test. Comments that described how the code used to behave were history too, so they went the same way. All 60 decisions survived the cut.

The median entry went from 34 lines to 16, and the longest from 127 to 41. In the code, the worst repetition of the log went from 73 matching runs of words to 11.

Did anything get lost? Shorter entries are only worth having if they still answer the questions. So I had Claude take the six entries that lost the most, write two questions for each from the full versions, and check the answers against the trimmed log.

Ten of the 12 came back complete. One kept its rule and lost the reason behind it. The last answer was missing because it no longer applied. The full entry was still explaining a workaround for page transitions that had since been removed from the site.

A limit for the log

Next, we carried over a guardrail from CLAUDE.md. The instructions file stayed small because a test enforced its limit of 280 lines. The log didn't have one, but now it does.

The build fails if the log goes past 1,058 lines, the length it was cut to, and that number is only allowed to come down. A new entry has to be justified by trimming away an old one.

The number itself is arbitrary. It's simply where the cut happened to land, and a limit that can only go down will one day push out a decision that still matters. Long entries caused the bloat, so a fairer test would cap each entry at about 25 lines and let the total grow with the number of decisions. That's next on the roadmap.

A second test checks that every comment pointing to a log entry points to one that exists, so trimming an entry can never leave the code pointing at nothing.

Lean and mean

The instructions file is at its 280-line limit and holding. The log is at its limit too, so the next decision that needs writing down will have to replace something.

Reading the log now costs about 13,000 tokens, down from an estimated 34,000 a week ago. The fatberg is gone.

A dynamic, gritty comic book splash panel of a heroic worker in a scuffed yellow hazmat suit hydro-blasting a monstrous anthropomorphic fatberg. The creature is an urban garbage golem: its face and body are formed from congealed grease, tangled white wet wipes, flattened soda cans, plastic forks for teeth, and discarded plastic bottles. Chunks of congealed lard, bottle caps, and greasy packaging fly off as a high-pressure water blast hits it. Deep black inking, intricate crosshatching, vintage halftone screentone dots, cracked Victorian sewer tiles covered in faded tags and grime.