Google Blogoscoped

Forum

How does Google index Wikipedia so fast?

YChromosome [PersonRank 1]

Sunday, November 8, 2009
10 years ago3,631 views

Does anybody know how Google indexes Wikipedia so fast?

Based on this post on the Google Blog (googlecustomsearch.blogspot.co ...), I though that Wikipedia pings Google every time something changes on its pages, similar to how Blogger platform pings Google. Is this true or no?

Or does Google have any special setup of its own to index Wikipedia's pages continuously?

Franta H. [PersonRank 6]

10 years ago #

I think that when Google decides how often to index a certain page it looks at least at pagerank and at the rate the page's content changes. So since Wikipedia's articles have high pagerank and change very often, Google tends to index them often.

YChromosome [PersonRank 1]

10 years ago #

Franta,

I don't know if Google indexes higher ranked websites more often. I always thought that Google only displayed higher ranked websites higher up in search results. Didn't think it affected the way Google indexed the Internet.

I know for a fact that Google's Blogger has the ping feature by which Google gets notified almost as soon as a blog's content changes. I remember reading somewhere a few weeks ago that similar functionality allowed Google to know when other websites such as Wikipedia changed their content.

YC

Ludwik Trammer [PersonRank 10]

10 years ago #

Google has a special algorithm to determine how often to crawl a website. The factors include both PageRank and the rate of website changes in the past.

Anyone using Sitemaps standard can ping Google about changes, but I haven't heard about Wikipedia doing so.

James [PersonRank 0]

10 years ago #

Wikipedia does use a sitemap: serverfault.com/questions/3217 ...

YChromosome [PersonRank 1]

10 years ago #

Ludwik & James,

Thanks both of you. That was useful information.

YC

This thread is locked as it's old... but you can create a new thread in the forum. 

Forum home

Advertisement

 
Blog  |  Forum     more >> Archive | Feed | Google's blogs | About
Advertisement

 

This site unofficially covers Google™ and more with some rights reserved. Join our forum!