Every metric said it worked

Part two ended on a claim I thought was finished.

Cheaper seeding hadn't cost quality, and I had the table to prove it: 183 review findings in French, 155 in German, 187 in Italian. French ran at the expensive settings. German and Italian ran at the cheap ones. Nothing in three books suggested the cheap pass cost anything.

Then I read the findings.

The run that every number called a success

Here is a German review pass. Eighteen findings on one chapter. It cost what it should have cost, took the time it should have taken, and correctly identified the chapter's most important defect: the seeding pass had left an English word untranslated in the scene the whole chapter turns on, and three findings were classed critical for exactly that.

It was also unusable.

hoechsten Huegel. groesste. Identitaet. liessen, for ließen. Thirteen of eighteen proposals had every umlaut and eszett replaced with an ASCII fallback. Accept that batch and you inject broken orthography into the baseline, three critical fixes included.

Nothing in the pipeline saw it. Not the token counts, not the timings, not the finding counts that part two had used as the quality signal. Eighteen findings is eighteen findings whether or not a German reader can use them.

Cost is a number the agent logs. Quality is something a person has to read.

French was fine. Same prompt, same model tier, same chapter, same afternoon.

The review prompt's output contract has a line about JSON safety: don't use typographic quotes, mixed Unicode and ASCII pairs break parsing. It is talking about the quotation marks around one field. The model generalized it to the whole response.

German has a standardized ASCII fallback. ae, oe, ue, ss are a real convention every German speaker recognizes. French has no equivalent. Pushed toward ASCII, German had somewhere to go and French did not.

The tell is that the same run emitted correct typographic quotation marks inside its own proposals. It was not avoiding Unicode. It was transliterating German, because German can be transliterated.

The instructions were the bug

Two rules failed, in opposite directions.

One contained the defect it existed to prevent. A chapter drew 28 findings where similar chapters drew ten or twelve. Twenty-two were the same fix: a closing quotation mark. German quotation marks are „text", low opener and high closer, and the seeding pass was producing „text followed by an ASCII straight quote, 33 times across the book.

Then I read the instruction meant to prevent it:

- Quotation marks: „..." (German style) not "..."

The closing mark in that rule is U+0022. The instruction meant to teach the correct pair was written with the wrong character, and the seeding pass was doing exactly what it was told. Twenty-four percent of that book's entire review queue was waste generated by one character in my own notes.

The rule contained the defect it existed to prevent. Nothing but reading it as data could have caught that.

The other never fired at all. French requires a non-breaking space before a colon, semicolon, exclamation point and question mark, and inside its quotation marks. My French review notes said so, as item thirteen in a numbered watch list at the bottom of the file.

The chapter contained zero non-breaking spaces and twenty-one violations. The review pass flagged none of them.

I moved the same rule into a locked block near the top, named the measured miss rate in the rule itself, and added one sentence saying a whitespace-only fix is expected and correct here. Next run, full book:

Same model, same defects, same words in the rule. The only thing that changed was where it sat and how it was framed.

What the numbers looked like afterward

I rebuilt three editions from scratch against revised notes and ran both passes again. If the new instructions were only making the reviewer chattier, the totals would go up.

Down across the board, which is the right direction. A better seeding pass leaves fewer defects for review to find.

The number I trusted was not the total. Part two established that finding counts track the source text rather than the settings, so chapters that drew almost nothing before should still draw almost nothing.

They did. The drop is concentrated in the chapters where the seeding pass used to leave work behind, which is what a real improvement looks like and what padding does not.

One chapter went the other way. In the rebuilt German, the chapter written in my daughter's voice drew 28 findings against a baseline of 16, and 22 were the quotation-mark fix. Her chapter is almost entirely dialogue, so it has more quotation marks than any other, so it had more instances of a defect that had always been there and had never once been flagged.

What no prompt could fix

The review pass adds a translator's note when the notes call for one. A note that exists only in the translation makes the translated chapter one block longer than its source, and the review pass pairs source and target by index. One extra block near the middle means every block after it gets reviewed against the wrong source. All three languages independently added a footnote on the same line, all three broke alignment, and nothing caught it until I compared two files by hand.

The same book carries <!-- size: --> hints on image blocks that the print exporter reads. The seeding prompt has a rule about preserving <!-- caption: --> comments and says nothing about the sibling comment beside it.

Four editions, none correct, across every book I had produced. Ninety-four hints restored by a repair script. Left alone, the translated print editions come back from the typesetter with differently sized images than the English one, months later, with nobody able to say why.

What it caught

The same pass caught things I would have shipped.

The seeding pass invented profanity in two of three languages. The source reads fear of "messing" up, with the scare quotes doing the work. Italian rendered it as an obscenity and then added a footnote explaining the obscenity. Spanish did the same with a different word. There is no profanity anywhere in this book. It reaches the reader in my twenty-year-old daughter's voice.

The review pass caught the Italian one and classed it critical. A human caught the Spanish one and left a note asking whether I had meant it.

Where this leaves the series

Part three is different in kind. There was no number to add up. The finding counts from part two measured what they measure, and they would have called a run of unusable German a clean success.

Everything above came from reading output, one chapter at a time, against a source I mostly cannot read. Then encoding what I learned back into the instructions, and running it again to see whether the instruction changed the behavior. Several times it did not, and the only way to know was to look again.

Cheaper is a measurement. Better is an iteration.

Eighteen commits, a hundred and two new tests, and four editions that now hold together.

Same pipeline. Same cost. The difference is that somebody read it.