Wednesday, July 18, 2007

Pattern Matching in Google

I want to brainstorm where in Google’s sites an image comparison/ pattern matching algorithm might make sense:

• Google Images: The user could upload an image, and Google returns similar-looking image results from the web. Also, the user could click on “similar images” on any image in the results. Another more advanced feature: Google could automatically match images with an existing library of things to associate the images with keywords (keywords not present anywhere else on the page, that is).
  Note that some people have reported they already saw a “find similar” link on image results long time ago.
• Google Images Explorer: Starting with an image grid of basic shapes (circle, square, triangle...), you can visually explore all images Google indexed by clicking on any shape, and then always getting similar shapes in return. For instance, you might start clicking on a white-on-black circle. You then get a sun, an orange and so on in return. Clicking on the sun, you’ll find many pictures with a sun on it, and you can continue “browsing” images this way.
• Google Video or YouTube: The user could provide the portrait of a celebrity, and in return get a list of all videos which show this celebrity. E.g. you upload a picture of Harrison Ford, and you’d get the trailer to Indiana Jones 4 in return (looking forward to that movie, by the way).
• Google Maps: In Street View, you could click on any house to find similar houses. This would work with both general architecture, and also chains; click on one McDonald’s, and Google will show you where all the other McDonald’s nearby are (if you’re into their food!). Another potential feature: you upload the photo of an unknown house for research, and Google will show you where this house is located by returning a best match Street View image.
• Google AdSense: Using technology which renders a page as an image file, Google could analyze the dominant page colors to automatically adjust the AdSense colors to something that fits best (just as an option that can be disabled, that is). The web owner could also ask Google to show colors which aren’t similar, but fit well into the color scheme according to certain design formulas Google stores. As another pattern matching application within AdSense, Google could analyze which color combinations are known to yield high click-thru rates, and offer this information to webmasters to help them optimize their ads.
• Google Spam Filter: Again using technology which renders a page as an image file, Google could manually determine a set of spammy seed sites and check if those have something in common visually; then, when a site design is found to be matching popular spam designs, its spam score would be increased (and if it reaches a certain threshold, the site would be banned).
• Google Translator: Google could allow people to upload an image of text which would then be auto-translated into a chosen language, using Optical Character Recognition. This is useful for those circumstances where you only have an image of text, e.g. when I who can’t read Chinese want to translate a Chinese logo (but I don’t know how to enter the Chinese characters the logo shows). On a similar note, the Google Translator could apply OCR to any web URL you’re translating (so that even those sites which make heavy use of graphics could be translated well).
• Google Picasa Web Albums: You could tag parts of an image, and Google would then be able to show you related sections in other images (e.g. you can select an iPod in image 1, tell Picasa it’s an “iPod”, and then let them associate your images 130, 132, and 412 containing this item with the keyword “iPod" for easier finding).
  
  Hard to implement? Perhaps; in 2004, Google’s Sergey Brin said, “I don’t think that in the near future we’re going to have a service that takes a picture, and the computer decides, oh, that’s an elephant, so we search for an elephant. That seems funny to us. We should be able to do it."
• Orkut: A “find your twin” feature would be a fun (and probably, privacy-invading) way to find the single person most resembling yourself, as Orkut already stores user portraits. Another possible (creepy) feature could be “find your ideal partner"; for those of you who are single, your photo portrait would be compared with those Orkut users in a relationship (as determined by their Orkut settings); from the set of best matches, the partner photos would now be used for another comparison search, the results of which (singles only, of course) are returned to you.
• Google “Similar Images” Widget: Bloggers or any other webmasters can put a little Google Gadget onto their page, right below (or wrapping) an image which illustrates their article. Now what Google would do is compare this image with their indexed images, and if they find good matches, display a little linked thumbnail list below the image (or optionally, just a plain link reading “see similar images from other sites”).
• Blogger: Upload an image, and let Google determine a blog template for you (the same could work with Google Page Creator). For instance, if you upload the picture of a sunset over an ocean, Google would then pick colors like red, orange and cyan for your design. (A third-party tool like this already exists actually.)
• AdWords: Google already supports image ad campaigns in AdWords; they could also analyze which kinds of color combinations might yield higher click-thru rates (or in reverse, which kind of color combinations are perhaps determined to be too annoying). These color combinations could then be suggested to AdWords campaign creators.
• Google Sets: Google Sets at this time lets you enter a keyphrase (say, “George Bush”) to then return a set of related keyphrases (like “Ronald Reagan”, “Bill Clinton”, “JFK” etc.). This could also work with images, if the Google algo is smart enough; you upload a pic of Superman and another one of Wonder Woman, and get drawings of Spider-Man, Batman, the Hulk etc. in return.
• Google Web History: It doesn’t happen often, but sometimes I vaguely remember a web page I saw some weeks ago – I can still “see it” in front of me, and remember its color scheme – but I forgot the URL or site name. Now what Google web history could offer is a simple drawing tool which lets you sketch some colored rectangles – a red navigation bar to the left, a yellow logo at the top, a white background – to then return pages in my web history which look similar to the sketch. In fact, such a tool could be expanded to also work on the whole Google web index, and not just the sites you visited before.

• Google Web Design Trends: Google could offer curious web designers a chart of the most popular color design trends, based upon analysis of currently prevalent color combinations on the web. Users could feed this service a seed site they like – say, “TheDesignBlog.net” – to restrict the chart to a specific link neighborhood. This way, you’d be able to find out which color combinations are hot within particular communities (like the search blog community, for instance).

Any other Google places where pattern matching or image comparison might make sense?

Pattern Matching in Googl ... by Philipp Lenssen | Comments (9)

>> More posts

Advertisement