Does Google support the "noindex" robots.txt directive?
For example:
User-agent: Googlebot Noindex: /page.html |
Within a page's meta tag, it supports "noindex". http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=35303
In robots.txt, I only find the 'disallow' option. http://www.google.com/support/webmasters/bin/answer.py?answer=40364 |