The three ways machine translation fails, and the one that matters at dinner
The Better Translate team ·
Machine translation does not fail like a person failing. It fails fluently, in
grammatical sentences, with no signal that anything went wrong.
The research literature sorts serious translation errors into three kinds, and it is worth
knowing them, because they are not equally dangerous at a restaurant table.
Mistranslation
Essential content comes back inaccurate. A dish name is culturally specific, or it is a
shortened form of several words, and a system translating close to word by word produces something
confident and wrong. This is the failure everybody has seen and laughed at.
Hallucination
Content appears in the translation that was never in the original. Research on detecting these
notes they are, paradoxically, the easiest of the three for automatic metrics to catch, because the
output stops corresponding to the input at all.
Deletion, which is the one to worry about
Crucial content from the source is dropped. Critically, that includes the removal of
negations. Work revisiting negation in neural machine translation puts the problem
plainly: an error in translating negation may change the meaning of a sentence completely.
Consider what that means when the sentence is "this does not contain peanuts." A dropped
negation does not produce nonsense. It produces a fluent, confident sentence that means the exact
opposite of the truth, and nothing on screen looks wrong.
Why this is not an abstract risk
Guidance for people travelling with food allergies is consistent on this point: a mistake such
as rendering one nut as another can have life-threatening consequences, and allergen wording is not
an area where general-purpose machine translation belongs. Advice for allergic travellers has long
centred on translated chef cards prepared in advance, precisely because reading a translation live
off a phone is not the same as a reviewed one.
What we do about it
We do not claim to have solved this, because it is not solved. What we do instead:
- The original text stays on screen next to the translation, always. If a line looks wrong you
can point at the source and ask.
- Our terms say it outright rather than in small print: do not rely on this app for allergens,
medical instructions, dosage, or anything where an error causes harm. Confirm those with a
person.
- Menu items come back with the recogniser's confidence behind them, so a badly read line is
treated as a badly read line rather than presented as fact.
An app that told you it was safe for allergies would be a better sales pitch and a worse app.
Point at the original and ask. That is the whole trick, and it works in every language.