How hard should the AI think?

Part one cut the input cost of my translation pipeline by 10x. Then I added one word to a log line and found out input was never the expensive half.

The agent makes two passes over every chapter. Pass one seeds a translation with Sonnet, the cheaper model. Pass two reviews that draft with Opus, the expensive one, and proposes fixes block by block.

It had been logging cache reads and cache writes since the day I built it. Not output. Output bills at five times the input rate, and none of it was being measured.

Adding out= to the log line was a two-minute change.

What was actually in the output

Here are the first two chapters to run on the optimized input path:

A chapter goes in at roughly 1,800 tokens of prose. German runs maybe 1.3x the length of English, so a faithful translation should come back at something like 2,400.

It was coming back at 10,000.

The other 7,000-odd tokens were the model thinking, billed as output at five times the input rate.

Model strength is two dials, not one. Which model, and how hard it thinks. I had been setting one and ignoring the other.

The default belongs to somebody else's job

Claude Code ships with its effort set to xhigh. That is the right default for Claude Code, which exists to do agentic software work: read a repo, hold a plan across twenty file edits, catch the consequence three steps out. That work rewards deliberation.

My agent was not doing that work. It was translating paragraph blocks from one language to another.

I had never passed an effort setting at all, so every chapter of three languages had run at the highest routine level available. Same 47-block passage, same model, four settings:

The translation itself came back at about 1,400 tokens at every single level. Identical length, identical block structure, correct every time.

At xhigh, 85% of what I was paying for was deliberation about a task that did not need it. One block differed enough to compare word for word: "Sie bleiben hängen" at xhigh against "Sie bleiben stecken" at low. Both correct. The cheap one is arguably the better German.

Medium and low cost the same within noise, so medium is free headroom. That became the seeding setting, and across the full German book it showed up exactly where the probe said it would:

1.3x is German being longer than English and nothing else. The thinking is gone.

One chapter broke the pattern. A gentle push drew 19,083 output tokens at medium, four times its input, nearly three minutes. Adaptive thinking decided that chapter was hard and spent accordingly. One in fifteen, and I left it alone. A model that spends more on the hard chapter is behaving correctly.

Where it gets harder

Pass one was easy to judge, because its output is a translation and a translation has a right length.

Pass two is not like that. Its output is a set of editorial findings, each carrying a full replacement block, so output legitimately scales with how many blocks the reviewer decides to touch. A run that produces more tokens might be thinking too much, or it might be finding more things. The token count cannot tell you which. And it runs on Opus, where those tokens cost the most.

Cutting it blind would have been guessing.

A test with known answers

So I planted defects.

Five of them, at known block indices, in a real machine-translated passage: the kinds of errors a seeding pass actually makes, each one unambiguous enough that catching it is a yes or a no. Then I ran the review at every effort level and checked what came back.

Every level caught every planted defect. Including the cheapest one, in twelve seconds, for a seventh of the output.

What separated the levels was the discretionary findings, the ones nobody planted. Those turned out to be unstable: a second run at high and medium produced 2 and 3 of them, inverting the order. Run-to-run variance was larger than the difference between the settings.

They were real findings at every level, not padding. A literal rendering that read as a calque of the English. A drift between formal and informal address. An idiom flattened into a plainer word, with a better one proposed in its place.

A test you know the answers to beats any amount of reasoning about which setting feels safer.

I chose high for review anyway, and the reason matters more than the choice. The test passage was a blog article, not literary prose, so the failure classes where depth might genuinely pay (a lost metaphor, a through-line that breaks across a chapter) never got tested. Headroom on the untested cases costs about four cents a chapter. I will try medium on a later book, once I have a harder passage to plant defects in.

What you cannot cut

The last question was whether cheaper seeding cost quality. If translating at medium left more defects behind, the reviewer should find more of them.

Three languages, same book, same chapters:

French ran at the expensive settings for both passes. German and Italian ran at the cheap ones. Italian came back level with French, and German came in below both.

Read across the rows, not down the columns. Ripple-effect draws six findings in every language. About-maddie draws one. Upart-003 draws none, three times.

The hard chapters are hard in every language. Finding counts track the source text, not the model settings. Nothing in three books suggests the cheaper seeding pass cost anything.

It also says where the work stops being compressible. Pass two's output is the findings, the findings track the chapter, and the findings are the product. Review sits at 14% of a session window, and no setting makes it much cheaper without making it worse.

When the expensive output is the thing you came for, there is nothing left to tune. You find that point by counting deliverables, not tokens.

Where both parts landed

Pass two is 85% of the bill, and that is the correct shape. It is the pass that does the work.

Part one was about paying for the same instructions over and over. Part two was about paying for deliberation that changed nothing. Both were invisible, both were measurable, and neither needed a new tool to find.

Same translations, in three languages, at a tenth the cost and a fraction of the wait. The only thing that changed is that I started looking at what I was buying.