Universal translation on iPhone is a fantastically useful feature when traveling. Arthur C Clarke famously said that any sufficiently advanced technology is indistinguishable from magic, and this is one capability that still feels that way to me.
I’m in Buenos Aires at the moment, with extremely limited Spanish, and it’s incredible to me that I can point my iPhone camera at any Spanish text and have it instantly translated into English. But there is one big limitation that requires an extremely clunky workaround …
Apple introduced Live Text OCR in iOS 15.
With iOS 15 in the Camera app, look for the new Live Text button (viewfinder square with three lines). Bottom right corner when in portrait orientation or bottom left in landscape. Look for the yellow viewfinder rectangle to appear as the camera notices text as you move your camera around. When you have the yellow viewfinder surrounding the text you want, tap the Live Text button to be able to select, copy, look up, translate, and share recognized text.
You can also highlight text in apps, select the More option and then select Translate.
However, there’s one big limitation: not all apps let you select text. For example, in Maps, I can see the names of streets, but I can’t select them as text in order to check the pronunciation to give the address to a taxi driver. So this last week I’ve been doing this a lot:
- Pick a place I want to visit
- Look up the address in Maps*
- Take a screengrab
- Open the Photos app
- Select the street name
- Open the Translate app
- Paste it in
*Taxi drivers here aren’t interested in the actual address, they want either the street name and block number, or the intersection of two streets, so I often need to know how to pronounce two streets.
Similarly, there’s a local app here that lists all the milongas (tango dance events) and lets you see all the details, but there’s no way to select the text. So again, I have to use the same clunky process to translate from the screengrab.
What I’d love to see Apple introduce is a system-wide OCR capability, so that translation is directly available from any app. Anytime text is on the screen, there should be a way to select it for OCR.
Some thought would need to be given to a gesture for this which wouldn’t hijack the existing app interface. For example, iOS couldn’t use long-press on non-selectable text as in Maps, for example, that’s how you drop a pin. Perhaps a double-tap?
Is this a feature you’d like to see? Please take our poll, and share your thoughts in the comments.
FTC: We use income earning auto affiliate links. More.
Comments