Three Stanford graduates trained OpenAI’s CLIP on 500K Google Street View images and it guessed locations from pictures better than a GeoGuessr veteran.
Three Stanford graduates designed a test in their Predicting Image Geolocations or PIGEON program to identify locations on Google Street View. Their project revealed that artificial intelligence is extremely good at geolocating where photos are taken. When fed pictures that AI has never seen before, it was pretty accurate in determining where could it have been taken based on its training alone. The article on NPR calls this a double-edged sword where on the one hand, it can help people identify locations of old pictures or allow field biologists to conduct surveys, it can also be used to expose private location information about individuals that was never meant to be shared.
The test unfolded on the game GeoGuessr where 50 million+ players try to place pins based on Google Street View photos. The team built an AI player using CLIP, OpenAI’s neural network trained to learn visual concepts. They simply trained CLIP using roughly 500,000 Google Street View images (which they say is not that much data) and the performance was “spectacular” with a 95% accuracy in guessing the country and the ability to pick a location within ~25 miles or ~40 km of the actual location.
They put this AI player against one of the best geoguessers, Trevor Rainbolt (video below), and beat him fair and square.
We weren’t the first AI that played against Rainbolt. We’re just the first AI that won against Rainbolt.
Silas Alberti, PIGEON
Their process and findings have been published in the paper on arXiv titled “PIGEON: Predicting Image Geolocations.”
