Artificial Intelligence Will Dictate Photo Captions for iOS Users

The Menlo Park-based social networking giant is training its artificial intelligence to turn into ‘Photo Guide’ for visually impaired and blind people.

The feature, which is rolling out today, exclusively for Facebook users on iPhone will interpret the picture’s description using ‘Automatic Alternative Text’ tool powered by artificial intelligence. The tool will recognize objects and faces. It will effectively work on ‘TEXT to SPEECH’ conversion engine.

The ‘Deep Neural Networks’ used in the system will identify specific objects in the picture, from pizza and ice cream to scooters and cars. It can efficiently select a particular characteristic of an individual in the photograph, including eyeglasses, beards, and smiles. It can also analyze the picture in a better sense, interpreting that an image depicts snow, waves of the ocean or the sun. The ‘text-to-speech’ tool will read the descriptions aloud after a comprehensive study.

VoiceOver, the built-in screen reader for iPhone should be turned on in order to allow the Facebook’s artificial intelligence tool to read the descriptions. Automatic Alternative Text is only available in English and for iOS users for now, but Facebook is looking forward to making it available to Android users and Facebook website users very soon.

Automatic Alternative Text. Image Credit: Facebook

The descriptions via this ‘Text-to-Speech’ tool are confined initially to a strict vocabulary of restricted 100 words, which will prevent the AI tool from providing excessive details. It includes words like selfies, baby, car, shoes, and dessert. The automated voice, for instance, will describe that the picture contains a pizza, but it won’t add that it contains jalapenos, olives, and pepperoni as toppings.

Mark Zuckerberg’s team is taking slow and careful steps with ‘Automatic Alternative Text’. They want to avoid causing an error, which offends the audience. The search engine giant Google had to issue a heavy apology due to one such mistake. The company learned the risks involved with using such technology when its Photos app’s image recognition tool offended a black couple by labeling them ‘Gorillas’.

The social networking giant hopes to polish the technology. This is the reason they have provided precise descriptions and answers to questions, which a user might ask about a certain photograph.

Facebook is trying to make sure that nearly 300M visually challenged individuals globally remain fascinated in its website as the number of photographs appearing on the service are increasing regularly. Over 2 billion pictures are posted on Facebook and its subsidiaries, which include WhatsApp, Messenger, and Instagram.

So far, individuals were only getting notified about the pictures posted on the website without any description or elaboration, but now it’s changing.

What do you think? Will it work or end up becoming subject to heavy criticism like Microsoft’s AI Chatbot ‘Tay’ or Google’s Image Recognition Tool?