Google Blogoscoped


Google's Patent Reads Text in Images  (View post) [PersonRank 10]

Friday, January 4, 2008
16 years ago5,093 views

Methods, systems, and apparatus including computer program products for using extracted image text are provided. In one implementation, a computer-implemented method is provided. The method includes receiving an input of one or more image search terms and identifying keywords from the received one or more image search terms. The method also includes searching a collection of keywords including keywords extracted from image text, retrieving an image associated with extracted image text corresponding to one or more of the image search terms, and presenting the image.

A spokesperson for Google said in an e-mail, "...[W]e file patent applications on a variety of ideas that our employees come up with. Some of those ideas later mature into real products or services; some don't. Prospective product announcements should not necessarily be inferred from our patent applications."

It is however worth noting the backgrounds of Luc Vincent and Adrian Ulges, the two computer scientists behind the patent applications. Vincent describes himself on his Web site as "[l]eader of several large geo-related projects, including Street View", as being "[r]esponsible for various engineering aspects of Google Book Search," and as the "[h]ead of Google OCR-related initiatives." Ulges on his Web site notes his involvement in helping to develop "a system that autonomously learns to tag videos with high-level semantic concepts by watching videos from online portals like"

The patent application envisions several possible advantages arising from the technology. "Candidate text regions within images can be enhanced to improve text recognition accuracy," the patent application states. "Extracted image text can also be used to improve image searching. The extracted text can be stored as associated with the particular image for use in generating search results in an image search. Additionally, the extracted image text can be combined with location data and indexed to improve and enhance location-based searching. The extracted text can provide keywords for identifying particular locations and presenting images of the identified locations to a user."

James Xuan [PersonRank 10]

16 years ago #

Cool, I've always imagined this technology being useful, and wishing it existed, but wouldn't it be detrimental to captchas?

Martin Porcheron [PersonRank 10]

16 years ago #

I beg to differ James, on two counts:
a) Google's captcha's are regarded as some of the best, they use mis-formed letters, different colours and curved lines.
b) If Google is leading the way with OCR then that means they can develop CAPTCHAs which they can't read.

Essentially this will lead to even more difficult CAPTCHAs for machines to read.

Above 3 comments were made in the forum before this was blogged,

Kevin [PersonRank 0]

16 years ago #

Well, this looks similar to the one filed by Jacob Stolin, Adobe Systems Incorporated..."Recognizing text in a multicolor image" which can be found at:

Chtitux [PersonRank 1]

16 years ago #

OCR works well in Google : Google Books (with research) is a great example...

Hong Xiaowan [PersonRank 10]

16 years ago #

Barcode system stronger than it. And more simple. The supplier will give the shop the details for per product at the standard format. Ocr will make more mistakes.

And, a shop will refuse the robot running at their place, cause the shop need to keep their price, combines ect as the business secrets.

If robot very small, can fly, will be a good spy tool.

If Google want to catch the target, I advice begin with Barcode System. Then make the Barcode System online. That will be easy to go. and can gain the easy money as adsense. And combined with Gcheckout...

hebbet [PersonRank 10]

16 years ago #

That's sounds very interesting

MapsRus [PersonRank 1]

16 years ago #

What would they do when they find signs incorrectly spelled?

James Xuan [PersonRank 10]

16 years ago #

Good point Martin


I think you mean patent ;-)

Tony Ruscoe [PersonRank 10]

16 years ago #

> I think you mean patent ;-)

Thanks, James. Typo fixed.

Johann [PersonRank 1]

16 years ago #

[put at-character here]Maps:

Maybe this is a start to get retailers to use correct grammar on their signs. Even wipe out Engrish, perhaps. Shame them into learning the local language well enough to spell correctly when they visually assault people with homemade advertising.

Pat [PersonRank 0]

16 years ago #

I second what Kevin says above. This stuff was patented a decade ago. I have read numerous research papers at all kinds of conferences, over the years, talking about this kind of thing. Even if it was filed in 2006, Google is years and years late to the game.

beussery [PersonRank 10]

16 years ago #

Yeah OCR has been around for years. Either way, this seems pretty cool and could be a way to cut down on spam. You know, all those emails with text in an image?

This would also be a good way of checking to see if content in Flash matches content in divs in SWFObject sites.

Lily Grozeva [PersonRank 0]

16 years ago #

Well, good to know things are getting somewhere with the image search, but it still sounds science fiction to me ...

Forum home


Blog  |  Forum     more >> Archive | Feed | Google's blogs | About


This site unofficially covers Google™ and more with some rights reserved. Join our forum!