Along with the Disallow: /urchin_test/ that Philipp post about earlier http://blogoscoped.com/forum/68275.html
I also noticed that
Disallow: /base/search?
is now on the list. This could mean that Google Base results pages cannot be crawled. |
It seems they exclude search results from being crawled, which is always the right thing to do... |
It seems they don't (or they didn't): http://googlesystem.blogspot.com/2006/06/search-google-searches.html |
[Moved from new thread "Robots.txt Update". -Philipp]
Disallow: /base/search? Disallow: /urchin_test/
What's up with the Urchin test...New Google Analytics? |