My AI translation pipeline was 96% overhead

The component was working. That is exactly why I stopped looking at it.

Folio is the app I built to help translate my book into multiple languages. A small daemon spawns Claude Code once per chapter. Pass one seeds a translation with Sonnet, which on its own is worth very little: it exists to put something on the page. Pass two is the real work. Opus reviews that draft against the source and proposes fixes block by block, and it is what turns a rough machine translation into something worth a human reviewer's time.

Each pass carries three prompts. Book notes that apply to every language, language notes for the target, and the chapter itself. The first two never change across a run. That detail matters more than it sounds like it should.

The daemon runs on my laptop against a Claude Max plan rather than an API key, so the currency here is session window, not dollars. Translating one language, both passes, used to eat most of a five-hour session. I had accepted that as the price.

Then I got curious about whether it was the price.

Start with the number you already have

The daemon had been logging the answer the whole time. Cache writes and cache reads, every call, and nobody had ever added them up. So I added them up across the French pass one, 22 chapters plus metadata:

96% of every call was not the book.

It was scaffolding. Book notes, language notes, a glossary, chapter summaries. Re-sent in full with every chapter.

Two patterns in the raw log told most of the story before I read any code. Sixteen of the twenty-two calls showed an identical value on the read side, so a stable block genuinely was cached and being reused. Everything after that block, about 17,000 tokens, was written fresh every time. And five calls mid-run showed no read at all: one chapter had taken 525 seconds, long enough to push the next request past the cache's five-minute expiry, and 79,970 tokens got rewritten for nothing.

Byte-stable is not the same as cached

The daemon's own documentation described three layers. A cold Book card, a warm Edition card, a hot Chapter card, with the cold and warm layers "designed so they stay byte-stable across chapters."

They were. That was never the problem.

All three lived inside a single user message, and Claude Code places no cache breakpoint inside a user message. A prefix cache matches up to a breakpoint. With no breakpoint between the stable text and the chapter that changes every call, the stable text was written at full price every time and read back never.

I was paying full price to send the same instructions twenty-three times.

The translations were identical either way. Nothing in the output revealed it. The only evidence was a number nobody was reading.

Guess, measure, repeat

System text sits ahead of the user message, so the obvious hypothesis is to move the invariant notes into --append-system-prompt.

The useful discipline is testing that before building it. A probe spawns the CLI the same way the daemon does, varies one thing, and reads the token counts off the result. Each one costs a fraction of a cent. Three shapes, same content, and the number that matters is the second call, where a cache should be paying off:

--append-system-prompt lands the text after Claude Code's own breakpoint, so it gets rewritten every call. 24,137 on the first call, 24,138 on the second, which is the signature of text that is never read back. --system-prompt is a different instruction: it replaces the CLI's default system prompt rather than appending to it, which puts the text inside the readable prefix.

Two flags that read like synonyms. Byte-identical output from both. The mechanism underneath is the whole lesson. A cache write bills at 1.25x. A cache read bills at 0.10x.

Writing to the cache costs a premium. Reading from it costs almost nothing. If you are paying to put something in there, make sure the next call can actually get it back out.

I had been paying the premium every chapter and collecting on it never.

Reasoning got me a sensible candidate. Measuring told me which one to ship.

What else was in there

Claude Code inherits the working directory of whatever spawns it, and the daemon ran from inside the Folio repository. So every call carried Folio's own CLAUDE.md: 29 KB of engineering guardrails written for people building the app, shipped to a model translating French prose. That cost 16,111 tokens per chapter. Pointing the daemon at a fixed empty directory took it to 566. Fixed, not temporary: the working directory is itself part of the prompt.

Claude Code also sends definitions for its built-in tools at the top of every request. A translation job never touches a file. Removing them saved 12,195 tokens per call, and there is a trap in that one worth knowing. --allowedTools governs which tools may be used and leaves every schema in the prompt. Only --disallowedTools removes the definitions. Allow-listing and removing are different operations.

The ladder

No single change did this. Four did, and they are not close to equal. Same content, steady state, measured through the real code path:

The first two steps are worth 1.8x between them, and they are the ones that look like the obvious wins. The third is worth 13x on its own, because it is the only one that changes what kind of token you are buying: it converts a per-chapter write into a per-chapter read. The last row is a thousand tokens of whitespace per chapter, indentation added for human readability that no human was ever going to read.

Where it landed

Italian was the deliberate replication. Third language, one uninterrupted session, every optimization on, window read before pass one and after pass two with nothing else running. The 26x above is measured on identical content, which flatters it; across four genuinely different chapters the real figure is 10x, and that is the one I trust.

The tell is in the per-chapter numbers. Italian's cache writes matched German's chapter for chapter, 2,370 and 2,895 and 1,261 and 229, because the write is now exactly the source blocks and nothing else. Same book, different language, same cost, because the only thing I pay for now is the book.

At API rates the entire Italian book came to $5.12. Pass one took three to four minutes, pass two about eight and a half. A single French chapter had once taken 525 seconds on its own.

Spanish, the edition currently open for editing, went through all of this before any of it existed. I can't price that run, because I wasn't logging the expensive half yet. But the one old-path number I do have is pass one at 24% of a session, and both Italian passes together now cost 18%. The seeding pass alone used to cost more than the entire book does today.

Cheaper was the goal. Faster came with it.

A call carrying 20,000 fewer tokens is a call you wait less for, and it compounds across 22 chapters and two passes. Quality held too: Italian produced 187 review findings against French's 183, on settings that cost a fraction as much.

Same translations. 10x cheaper. Nothing traded.

None of this required new technology. It required looking at a number I already had, on a component I had already decided was finished.

That is the input side. Part two is the expensive half, because output bills at five times input, it is 85% of what this book costs, and I wasn't logging it at all.

That's the next article. Stay tuned →