this is an interestin cache by google

http://72.14.207.104/search?q=cache:R2NvTevo1pcJ:www.microsoft.com/robots.txt+&hl=en&gl=us&ct=clnk&cd=1

Just someone who made a link to this file, and google follow the link?

not sure, Tom .. but why would google cache the robot.txt ? if you notice this was crawled under the msn domain too and this brings me to the next question, why would anyone make a link (crawlable) to the rob.txt file ??

Just does not make sense

I agree with TOMHTML.

pd: some people make links, f.e. to Google's robots.txt or Whitehouse.com one.

http://www.google.com/search?num=100&hl=en&lr=&newwindow=1&safe=off&q=inurl%3Arobots.txt&btnG=Search

There seems to be a lot of robots.txt indexed & cached...
'Results 1 – 100 of about 70,000 for inurl:robots.txt. (0.56 seconds)'

LOL, webmasterworld.com has a blog in the working robots.txt file!
http://www.webmasterworld.com/robots.txt

Google also indexes pages that aren't linked from anywhere on the web... if only someone visits the page with the Google Toolbar (PageRank option activated) installed.

Google even indexes its own robots.txt.

http://72.14.209.104/search?q=cache:4V5KG6DE54gJ:www.google.com/robots.txt+%22google.com/robots.txt%22&hl=en&gl=us&ct=clnk&cd=1

> There seems to be a lot of robots.txt indexed & cached...

What happens if you include "Disallow: /robots.txt" into the robots.txt file?

There's an old paradox... there's a barber living on a Greek island, and he's the only barber on that island. He has this rule to only cut the hair of people who don't cut their own hair. Now the barber's hair is growing longer and longer, and he wants to do cut it... but is the barber allowed to cut his own hair?

> but is the barber allowed to cut his own hair?

Yes, but only once ;)