Google Blogoscoped

Thursday, July 19, 2007

Google News’ Cloaking Policy

Google News received a new help center with more details than the old help files on this subject. For instance, you’ll learn that high-quality backlinks affect your site’s ranking in Google News (but that there’s no manual placement of stories onto the News front-page, ever). So far, so good, and much like what we’re used to from web search. But this bit on cloaking is interesting:

[W]e need to be able to circumvent your registration page in order to successfully crawl your site. The easiest way to do this is to configure your webservers to not serve the registration page to our crawlers (when the User-Agent is “Googlebot”). You can verify that the request is actually from our robot by making sure the IP address is within the range of

So with that, Googlebot will never see the publisher’s registration page – but that doesn’t mean the user will necessarily always see it. Google offers two options: “First click free” means the publisher will check the referrer; if the referrer is Google News, then the article page is fully shown, but all other links from that page are “trapped”, as Google calls it (on a side-note, “first click free” is problematic when people copy the URL to link to the page from e.g. their blog). The second option Google calls “subscription designation”, which means that Google will simply add the word “(subscription)” to the News result pointing to the publisher. With the latter solution, a publisher can indeed always hide the actual content from readers, and only show it to the Googlebot.

In comparison, here’s the Google webmaster guideline – and it’s worth noting that Google News sources are also usually “normal” web pages:

Make pages for users, not for search engines. Don’t deceive your users or present different content to search engines than you display to users, which is commonly referred to as “cloaking.”


Blog  |  Forum     more >> Archive | Feed | Google's blogs | About


This site unofficially covers Google™ and more with some rights reserved. Join our forum!