[W]e need to be able to circumvent your registration page in order to successfully crawl your site. The easiest way to do this is to configure your webservers to not serve the registration page to our crawlers (when the User-Agent is “Googlebot”). You can verify that the request is actually from our robot by making sure the IP address is within the range of 18.104.22.168/20.
So with that, Googlebot will never see the publisher’s registration page – but that doesn’t mean the user will necessarily always see it. Google offers two options: “First click free” means the publisher will check the referrer; if the referrer is Google News, then the article page is fully shown, but all other links from that page are “trapped”, as Google calls it (on a side-note, “first click free” is problematic when people copy the URL to link to the page from e.g. their blog). The second option Google calls “subscription designation”, which means that Google will simply add the word “(subscription)” to the News result pointing to the publisher. With the latter solution, a publisher can indeed always hide the actual content from readers, and only show it to the Googlebot.
In comparison, here’s the Google webmaster guideline – and it’s worth noting that Google News sources are also usually “normal” web pages:
Make pages for users, not for search engines. Don’t deceive your users or present different content to search engines than you display to users, which is commonly referred to as “cloaking.”
>> More posts