We face communication barriers every day. Switching back and forth between apps and screens to translate shouldn’t be another one. We’ve heard your feedback, and have worked with the Android team to make translating text, chats, and other app content a whole lot easier.

Beginning this week, you’ll be able to translate in 90 languages right from within some of your favorite apps like TripAdvisor, WhatsApp and LinkedIn.
Translating a TripAdvisor review from Portuguese
Composing a WhatsApp message in Russian 

This update works on any device running the newest version of Android’s operating system (Android 6.0, Marshmallow). To get started, you first need to have the Translate app downloaded on your Android phone. From there, just go to an app, like TripAdvisor or LinkedIn, and highlight and select the text you want to translate. This feature is already enabled in apps that use Android text selection behavior. Developers who created custom text selection behavior can also easily add the new feature.

More than 500 million people translate over 100 billion words a day on Google Translate. With updates like this one, plus features like conversation mode and instant camera translation, we’re making Translate available anywhere you need it. So when you’re chatting with a new colleague from halfway around the world, conversation mode is perfect. Wondering which subway sign says “exit” on your next global adventure? Instant camera translation has your back. And now, when you’re sending messages or checking out reviews on your phone, you can translate right from within the apps you’re using.

Posted by Barak Turovsky, Product Lead, Google Translate

More than 500 million people use Google Translate every month across web and mobile phones, translating more than 100 billion words every day around the globe. Now, we’re launching Google Translate on all Android Wear watches, too.

Translate is built into the latest Android Wear software update, so you can have bilingual conversations even if you don’t have Google Translate on your phone, or if you’re away from your phone but connected via Wi-Fi.

And it’s easy to use - just speak into your watch to see your conversation translated into any of 44 languages. Flip your wrist to show the translation to a friend. When they respond in their own language, flip your wrist back, and you’ll see in your language what they’ve just said. Google Translate will automatically recognize which of the two languages is being spoken, so once you tap to start the conversation, all you and your buddy need to do is keep talking naturally.
Google Translate covers 90 languages total (for text translation), and we are always working to expand the number of languages that work across various features.

Today we announced that the Google Translate app now does real-time visual translation of 20 more languages. So the next time you’re in Prague and can’t read a menu, we’ve got your back. But how are we able to recognize these new languages?

In short: deep neural nets. When the Word Lens team joined Google, we were excited for the opportunity to work with some of the leading researchers in deep learning. Neural nets have gotten a lot of attention in the last few years because they’ve set all kinds of records in image recognition. Five years ago, if you gave a computer an image of a cat or a dog, it had trouble telling which was which. Thanks to convolutional neural networks, not only can computers tell the difference between cats and dogs, they can even recognize different breeds of dogs. Yes, they’re good for more than just trippy art—if you're translating a foreign menu or sign with the latest version of Google's Translate app, you're now using a deep neural net. And the amazing part is it can all work on your phone, without an Internet connection. Here’s how.

Step by step
First, when a camera image comes in, the Google Translate app has to find the letters in the picture. It needs to weed out background objects like trees or cars, and pick up on the words we want translated. It looks at blobs of pixels that have similar color to each other that are also near other similar blobs of pixels. Those are possibly letters, and if they’re near each other, that makes a continuous line we should read.
Second, Translate has to recognize what each letter actually is. This is where deep learning comes in. We use a convolutional neural network, training it on letters and non-letters so it can learn what different letters look like.

But interestingly, if we train just on very “clean”-looking letters, we risk not understanding what real-life letters look like. Letters out in the real world are marred by reflections, dirt, smudges, and all kinds of weirdness. So we built our letter generator to create all kinds of fake “dirt” to convincingly mimic the noisiness of the real world—fake reflections, fake smudges, fake weirdness all around.

Why not just train on real-life photos of letters? Well, it’s tough to find enough examples in all the languages we need, and it’s harder to maintain the fine control over what examples we use when we’re aiming to train a really efficient, compact neural network. So it’s more effective to simulate the dirt.
Some of the “dirty” letters we use for training. Dirt, highlights, and rotation, but not too much because we don’t want to confuse our neural net.

The third step is to take those recognized letters, and look them up in a dictionary to get translations. Since every previous step could have failed in some way, the dictionary lookup needs to be approximate. That way, if we read an ‘S’ as a ‘5’, we’ll still be able to find the word ‘5uper’.

Finally, we render the translation on top of the original words in the same style as the original. We can do this because we’ve already found and read the letters in the image, so we know exactly where they are. We can look at the colors surrounding the letters and use that to erase the original letters. And then we can draw the translation on top using the original foreground color.

Crunching it down for mobile
Now, if we could do this visual translation in our data centers, it wouldn’t be too hard. But a lot of our users, especially those getting online for the very first time, have slow or intermittent network connections and smartphones starved for computing power. These low-end phones can be about 50 times slower than a good laptop—and a good laptop is already much slower than the data centers that typically run our image recognition systems. So how do we get visual translation on these phones, with no connection to the cloud, translating in real-time as the camera moves around?

We needed to develop a very small neural net, and put severe limits on how much we tried to teach it—in essence, put an upper bound on the density of information it handles. The challenge here was in creating the most effective training data. Since we’re generating our own training data, we put a lot of effort into including just the right data and nothing more. For instance, we want to be able to recognize a letter with a small amount of rotation, but not too much. If we overdo the rotation, the neural network will use too much of its information density on unimportant things. So we put effort into making tools that would give us a fast iteration time and good visualizations. Inside of a few minutes, we can change the algorithms for generating training data, generate it, retrain, and visualize. From there we can look at what kind of letters are failing and why. At one point, we were warping our training data too much, and ‘$’ started to be recognized as ‘S’. We were able to quickly identify that and adjust the warping parameters to fix the problem. It was like trying to paint a picture of letters that you’d see in real life with all their imperfections painted just perfectly.

To achieve real-time, we also heavily optimized and hand-tuned the math operations. That meant using the mobile processor’s SIMD instructions and tuning things like matrix multiplies to fit processing into all levels of cache memory.

In the end, we were able to get our networks to give us significantly better results while running about as fast as our old system—great for translating what you see around you on the fly. Sometimes new technology can seem very abstract, and it's not always obvious what the applications for things like convolutional neural nets could be. We think breaking down language barriers is one great use.

The Google Translate app already lets you instantly visually translate printed text in seven languages. Just open the app, click on the camera, and point it at the text you need to translate—a street sign, ingredient list, instruction manual, dials on a washing machine. You'll see the text transform live on your screen into the other language. No Internet connection or cell phone data needed.

Today, we’re updating the Google Translate app again—expanding instant visual translation to 20 more languages (for a total of 27!), and making real-time voice translations a lot faster and smoother—so even more people can experience the world in their language.
Instantly translate printed text in 27 languages
We started out with seven languages—English, French, German, Italian, Portuguese, Russian and Spanish—and today we're adding 20 more. You can now translate to and from English and Bulgarian, Catalan, Croatian, Czech, Danish, Dutch, Filipino, Finnish, Hungarian, Indonesian, Lithuanian, Norwegian, Polish, Romanian, Slovak, Swedish, Turkish and Ukrainian. You can also do one-way translations from English to Hindi and Thai. (Or, try snapping a pic of the text you’d like translated—we have a total of 37 languages in camera mode.)

To try out the new languages, go to the Google Translate app, set “English” along with the language you’d like to translate, and click the camera button; you'll be prompted to download a small (~2 MB) language pack for each.

Ready to see all of these languages in action?
And how exactly did we get so many new languages running on a device with no data connection? It’s all about convolutional neural networks (whew)—geek out on that over on our Research blog.

Have a natural, smoother conversation—even with a slower mobile network
In many emerging markets, slow mobile networks can make it challenging to access many online tools - so if you live in an area with unreliable mobile networks, our other update today is for you. In addition to instant visual translation, we’ve also improved our voice conversation mode (enabling real-time translation of conversations across 32 languages), so it’s even faster and more natural on slow networks.
These updates are coming to both Android and iOS, rolling out over the next few days.

Translate Community helps us get better every day
On top of today’s updates, we’re also continuously working to improve the quality of the translations themselves and to add new languages. A year ago this week, we launched Translate Community, a place for multilingual people from anywhere in the world to provide and correct translations. Thanks to the millions of language lovers who have already pitched in—more than 100 million words so far!—we've been updating our translations for over 90 language pairs, and plan to update many more as our community grows.

We’ve still got lots of work to do: more than half of the content on the Internet is in English, but only around 20% of the world’s population speaks English. Today’s updates knock down a few more language barriers, helping you communicate better and get the information you need.

People use Google Translate a whole lot—we translate over 100 billion words a day! However, in the past, our translation systems have generally been better at making sense of government and business documents than in helping people casually communicate.

But that’s all changing thanks to people like you and a recent update we rolled out. So the next time you translate informal speech in Google Translate, you might just find a better translation. Here's an example of how it’s improved:
So how exactly are people like you impacting Google Translate? Well, with Translate Community hundreds of thousands of people have generously donated their time in service of cross-language communication. It’s fun and really easy: tell us what languages you speak; choose to either see a phrase and translate it on your own or correct current translations already in the system. Based on translations from the community, we will incorporate corrections and over time learn the language a little better.

There’s a whole lot more work to do, but with more help from everyday people through Translate Community, we can continue to improve the 90 languages we already speak and keep adding more.

During the Translate Community: Google I/O Challenge nearly 2.5 million phrases or about 12 million words were translated and validated by participants during Google I/O and beyond. From Afrikaans to Zulu, we saw approximately 75,000 people representing all of the 117 languages available in Google Translate Community take the lead and improve Google Translate for the languages they speak.
Some of the top contributing languages were what you might expect based on the number of speakers of each language; Spanish, Russian, French and Portuguese led the way through the challenge and languages like Bengali and Vietnamese notably moved their way up the rankings.
Of the languages that are not yet in Google Translate, Kyrgyz speakers shined the brightest with nearly 40,000 phrases translated and validated during the challenge. The Kyrgyz community has continued to plan events and rally to help add their language to Google Translate.

Thank you to the I/O Extended event organizers, Google Developer Groups and everyone who contributed to improving their language in Google Translate throughout this 10-day-challenge. We're excited to continue to work together in improving Google Translate.

For the last 10 months, multilingual users around the world have flocked to the Google Translate Community to help improve their language(s) on Google Translate through translating and validating common phrases.

Since launch, we've seen some amazing contributions, from Kyrgyz speakers who are getting us closer to adding their language to Google Translate, to Bengali speakers who organized 80 translate-a-thon events, significantly increasing translation quality for their language.
Translate Community: Google I/O Challenge (May 26 - June 5, 2015)
Now, we're challenging all Google I/O attendees (onsite and offsite!) to represent your language(s) during the Translate Community: I/O Challenge running from May 26 to June 5, 2015. Our goal is to reach over 5 million total contributions during the challenge.

You can make meaningful contributions in just a few minutes, and remember that all contributions matter—we encourage you to spread the word in your local community and amongst your friends and family to increase the contributions for your language(s). More contributions mean higher quality translations for your language(s), or helping your language(s) become supported on Google Translate, if they aren’t yet.

To get started:
  • Sign up in the new version of Translate Community at
  • Set your language(s) and contribute with as many high-quality translations / validations you have time for
  • Invite others to join the challenge and show support for your language on social with the official #io15 & #loveyourlanguage hashtags
You can follow which languages are getting the most contributions on our Google+ page, where we’ll post updates on who's leading the way throughout the challenge. Besides helping your language rise to the top of our leaderboard, if you’re one of the top high-quality contributors, you’ll get a Google Translate certificate for your linguistic legerdemain and might even get a shoutout on our social channels.

Built with Polymer
In the spirit of Google I/O, we recently released a new version of the Translate Community using Polymer. In addition to supporting your language, be one of the first to try out the new look of Translate Community.
This new version takes advantage of Web Components in Polymer. We're one of the first teams at Google to use Polymer this way—it’s now much easier to add new features like badges, upgrade our design, and ensure it works great on smartphones and tablets, in addition to desktop. We're looking forward to leading the way by offering our community a fun and engaging place to make a positive impact.