Google's Big Photos Jump and Machine Learning

After years of seeming missteps (e.g., Google Plus, Google Glass, Google Wave, Google Keep), Google roared back yesterday with a new Photos service. This seems pedestrian in terms of Google’s usual ambitions but actually it is incredibly on mission; that is, if Google’s mission is to organise the world’s information.

The new service is billed as “Gmail for Photos.” And it really is that. What made Gmail so revolutionary was (a) unlimited (don’t think about it) storage; (b) great search capabilities and (c) filing free design. The first two are obvious while the last one you may not have thought about. File-free design requires users to not worry about where they put things. It is a natural complement to good search. The idea is that you received an email and when you were done with it, it would be archived. You wouldn’t think, for example, which folder to store it in as if it was a filing cabinet. Prior to Gmail, that was a critical thing to do. But even after almost a decade, a few keywords are all I need to find things.

Google finally realised that the email filing problem was precisely the same as the problem we face with photos. We have a ton of photos that we can sometimes choose to share in the moment (i.e., deal with them) and then they get put away. All previous solutions required users to sort through their photos to tag ones they liked. Some would allow places and face recognition but even that had to be taught. But in reality, it was hard to find things. I have 50,000 photos at present, dutifully backed up, for apparently a few decades of sorting after I die.

Well I am glad I never bothered. Google’s new Photos service essentially brings the filing free design to photos. You can store an unlimited amount of photos (just like Gmail), you can search the photos (just like Gmail) and you don’t need to file anything because Google is now able to look at your photos and classify them. I uploaded all 50,000 photos and videos last night — from my iPhone(!) or strictly speaking iCloud — and then awoke to see Google busily at work.

At the moment, I can search for places, times and even backgrounds (like skylines) or activities (apparently I took lots of ice skating photos). Faces will come soon as Google works out who is who and they will — cleverly taking into account age to identify a baby right through to a teenager. But they also seem to group similar photos together so that I don’t look at the same type of image repeated.

Parts of this were present in Apple’s iCloud Photo service but the classification and search is new. And it is almost surely relying on what Google has learned in machine learning — something it has the strongest capabilities on in the world. These are now deep neural networks and one of the main things they have been working on is image recognition. Google has now ported that to the Photos app.

To be sure, some of this existed in the photos area of Google Plus but Google have finally realised that photos, like email and contacts, lies in people’s private space and not in their social place. They want extra control there especially if they are going to use it in a ‘don’t think about it’ manner.

There are two other important things that came from this. First of all, Google are not just classifying your photos but they are improving them. We did not take the photo below or edit it in black and white. Instead there were three other photos taken which got bits of this. Google’s AI worked this out and presented me with this option this morning.

It is simply magical. And then they did it again by creating animated gifs for photos that were similar which created a whole new dimension of art and memory — capturing movement. I can’t describe that here but when you try it you will see. The point here is that Google are doing more than keeping photos, they are making them better — much better.

Second, Google made this available on all platforms. The iOS one worked and uploaded the photos. No strange bugs. They developed it to really work. I actually noticed this alot in Google’s announcements yesterday. Thankfully for all of us they are promoting platforms in a neutral way. I hope others will follow that lead.