Facebook improves its AI that generates photo descriptions on iOS for visually impaired users

Facebook today announced a major update to its Automatic Alt Text (AAT) system, which automatically generates photo descriptions for people with visual impairments. Based on AI, the system can now recognize 10 times more objects and scenes in photos shared on Facebook and also Instagram, providing detailed descriptions for screen readers like Apple’s VoiceOver.

According to Facebook, the artificial intelligence technology developed by the company can now identify activities, landmarks, types of animals, and various other details in the photos. This will make it easier for blind or visually impaired people to understand the context of a photo shared on social networks.

Instead of just describing what’s in the photo, the updated AAT system can also specify the composition of the image, such as saying that there are two people in the center of the photo and three others in the sides instead of just saying that there are five people in the photo.

Or, instead of simply describing a lovely landscape with “May be a house and a mountain,” we can highlight that the mountain is the primary object in a scene based on how large it appears in comparison with the house at its base.

Taken together, these advancements help users who are blind or visually impaired better understand what’s in photos posted by their family and friends — and in their own photos — by providing more (and more detailed) information.

As pointed out by CNET, the new AAT system will be available on Facebook for photos in News Feed, Profiles, and Groups, while on Instagram the generated descriptions will be available for photos in Feed, Explore, and Profile. The feature is compatible with 45 different languages.

To access the detailed photo descriptions on iOS, users must use the new custom actions through VoiceOver. On Android, you just have to long-press the image. The update is still being rolled out for users.