Should we train our own model? We edited a text file instead.

Spanish, French and Italian are all Romance languages. Two cohorts of native readers had already corrected the Spanish and French translations of the book. Italian and German were next.

So the question sounded reasonable. Should we train our own model on what those reviewers taught us, so Italian starts closer to right?

In the article on what the reviewers fixed in the instructions, an agent sorted all 242 of their edits. That sort answered this question too.

What the AI actually knows about the book

Each chapter is translated on its own. The AI does not remember chapter three when it reaches chapter four. What it carries from one chapter to the next is the set of instructions it is handed, every time.

For Italian, that is three pieces of text:

The review pass reads the same brief with its own, longer Italian notes. Every editorial decision that reaches the AI, in any language, lives in that text.

So the model we were tempted to train already existed, as text that anyone could read line by line and edit by hand.

Why training would have taught the wrong things

Training learns from examples. The examples here were the cohorts' corrections, and they were not clean.

Fifteen percent of them reversed rules in the instructions, some of which the reviewers were right about. A few were approved with typos in them: tonterïa, recodado, Le Captaine, mon perspective. On some passages, reviewers went back and forth with changes, which showed where the language carried nuance a literal translation misses.

A model fitted to that would have absorbed the typos and the disagreements as if they were lessons. Editing the notes meant working through each suggestion, settling what the back-and-forth was about, and writing the decision into the notes with a reason next to it, so the translation reads more naturally.

Start over, or build on it?

There was a second question underneath. Italian and German had been translated before the French cohort's lessons existed, so nothing the French reviewers taught had reached either language. That left a chance to carry those lessons in before the Italian and German cohorts opened.

No human had touched Italian yet, so starting over looked clean, but it had a cost: 126 of the AI's own review findings on the Italian text had already been decided, and a reset would have discarded those decisions.

The worry about the alternative was that the AI, shown its earlier translation, would keep too much of it. That was testable: re-run three chapters with the new notes, against the existing text, and count what it wanted to change.

It changed almost only what the new notes told it to. So we re-ran the whole book that way.

Every other paragraph kept its earlier review.

One text for the people and the machine

The last piece was the house style, a short page of settled decisions for each language: Gramma is a name, a generic state park gets translated, and the Italian for the chapter title "Howdy" is Ciao.

The reviewers read it in a side panel while they work, and the AI gets the same text added to its instructions. When reviewer feedback changes a decision, that one text is updated, and every reviewer and the AI work from the same current version.

What you can do with this

Ask two questions first. What exactly is in the instructions your AI gets today? And who last read all of them?

If nobody can answer the second one, the improvement you are looking for is probably in that text, and it costs a careful read rather than a budget.

Before you fund training, read the instructions you already give the model.

The German and Italian reviewers started from those notes and that house style. The unit of improvement was never a model. It was one cohort's corrections, read, sorted, and turned into the next cohort's notes.