Tuesday, August 5, 2008

How Image-to-Text Could Be Used in Google Street View

Google Street View’s photo data could be used for more than just plain showing it to users. Internally, Google could do some image analysis – like OCR, optical character recognition, to convert text contained in the images of the houses, shops and so on to text. Such text could be utilized for several things:

• Google could be able to make the streets searchable through a normal text input box. If a club is called “Foobar” and you enter “foobar”, you may find the location even if it’s not available from any existing yellow pages overview service.
• It could be possible to overlay text on the images with translations, should you want to. For instance, when you’re browsing the newly released panorama photos for Tokyo, Japan, you may be curious what this or that street sign means. (In turn, such translated meta data could also be made searchable again... so that if you look for a shoe shop in Tokyo, you don’t need to use the Japanese word for “shoes” to search, but just enter something in English.)
• Depending on Google’s resolution of photo data – the resolution they show off publicly may not be reflecting what they store internally – they may be able to analyze menus of restaurants, price tags of products in shop windows and more. With some tweaking this could possibly be used to calculate the expensiveness of certain neighborhoods, show what type of food is available in a restaurant, show what brands a shop sells and more.

Besides OCR, Google could also analyze non-textual objects in their images. While that may may be tough to do we need to keep in mind they already successfully manage to determine faces (in order to blur them, due to privacy concerns). One thing that should thus be solvable would be, for instance, to count the people in a street to roughly see how busy an area is. What other information do you think could be mined out of Google Street View imagery in the future?

[Image from Google Street View, Tokyo, with contrast adjusted.]

How Image-to-Text Could B ... by Philipp Lenssen | Comments (10)

>> More posts

Advertisement