The French menu that taught us not to assume
Better Translate Team ·
A French menu came back unreadable, and the reason was not the model. It was that we
had told the phone to expect Japanese.
What the numbers looked like
Recall went up four times over. Confidence went from 0.38 to 0.98, which is the recogniser saying
it went from guessing to certain.
The part that surprised us
We assumed handing the recogniser a longer list of languages would cover it. We tested that too:
five languages at once was identical to two, on every menu in the set. Not better,
identical. The list is a hint applied to the whole image, not tried line by line, so a French line
under a Japanese-first list stays broken no matter how long the list gets.
Detection was free and strictly better. There was no menu where pinning won.
The knock-on bug
It got worse before it got better. The app also estimates how damaged a photograph is, to decide
how hard to think about it. That estimate was reading the language mismatch as a bad photograph, so
the cleanest scan in the whole set scored 0.59 damage and would have been sent to maximum effort
every time. One wrong assumption produced two wrong behaviours.
Detecting the language rather than declaring it fixed both.