Then how come Google indexes google.com/reader and displays appropriate title/description for Google Reader's login page: google.com/search?q=google+rea ...
|
|
Ionut – I believe that's because the command in robots.txt is:
Disallow: /reader/
and not
Disallow: /reader
The intention probably was for robots to index www.google.com/reader, but ignore everything inside the /reader/ catalog.
Notice that the result you pointed out uses "www.google.com/reader" URL, while the german result Philipp talks about points to "www.google.de/reader/" (with slash at the end).
|
|
Maybe search engines should be clever and use the data for google.de/reader. I wonder how they decide between google.de/reader and google.de/reader/.
mattcutts.com/blog/seo-advice- ...
|
|
That's the best search engine result page I have seem for a long time.
|
|
I'm guessing "Krake" and the kraken from pirates of the caribbean have the same etymology haha?
|