Does anybody know how Google indexes Wikipedia so fast?
Based on this post on the Google Blog (http://googlecustomsearch.blogspot.com/2009/10/contextual-search-experience-for.html), I though that Wikipedia pings Google every time something changes on its pages, similar to how Blogger platform pings Google. Is this true or no?
Or does Google have any special setup of its own to index Wikipedia's pages continuously? |
I think that when Google decides how often to index a certain page it looks at least at pagerank and at the rate the page's content changes. So since Wikipedia's articles have high pagerank and change very often, Google tends to index them often. |
Franta,
I don't know if Google indexes higher ranked websites more often. I always thought that Google only displayed higher ranked websites higher up in search results. Didn't think it affected the way Google indexed the Internet.
I know for a fact that Google's Blogger has the ping feature by which Google gets notified almost as soon as a blog's content changes. I remember reading somewhere a few weeks ago that similar functionality allowed Google to know when other websites such as Wikipedia changed their content.
YC |
Google has a special algorithm to determine how often to crawl a website. The factors include both PageRank and the rate of website changes in the past.
Anyone using Sitemaps standard can ping Google about changes, but I haven't heard about Wikipedia doing so. |
Ludwik & James,
Thanks both of you. That was useful information.
YC |
