The latency budget of a translated conversation
Better Translate Team ·
If a conversation runs on gaps measured in hundreds of milliseconds, it is worth
knowing where the seconds go when you put a translator in the middle.
The budget
Two things on that list are worth pulling out.
Transcription is free, in latency terms
Speech becomes text on the device using Apple's on-device framework, so that step never touches
the network. It is also the step people assume needs a server. It does not, and keeping it local
takes a round trip out of every single turn as a side effect of a privacy decision.
The handover was the biggest line item
Practitioners working on bilateral speech translation describe a two second delay as disruptive
and four seconds as making a conversation feel broken. One-way conference interpreting gets three to
five seconds of tolerance because the audience is passive; a back-and-forth gets far less.
Unlocking a phone, turning it around, waiting while somebody reads and taking it back spends that
entire budget before a single word is translated. It is the largest, most reliable delay in the
whole interaction and it is pure interface.
Splitting the screen and rotating the far half removes it. Both halves are live, so the
translation lands in front of the person who needs it without anybody moving.
What we do not claim
The menu camera is a different animal. A photographed board goes through on-device text
recognition and then a model that reads the whole page at once, and in our own benchmark that took
between 94 and 1,029 seconds depending on configuration. Nobody should call that realtime, and we
do not.
Two features, two very different clocks. The conversation is built to keep up with people. The
camera is built to be right about a handwritten board at the back of a bar.