Google Blogoscoped

Sunday, June 5, 2005

Google’s Categories of Random Query-Evaluation

SearchBistro publishes Google’s General Guidelines on Random Query-Evaluation from 2003 (written by Natasha Doubson, SearchBistro’s Henk van Ess says). In the PDF, Google categorizes queries which are to be evaluated by raters into three groups:


Once a rater is randomly presented the search queries, which are taken from real-life usage examples (only little is filtered out, like pornographic results), there are several different rating categories available. They are:


Also available to raters are more straightforward result comparisons. Here, the rater needs to adjust a slider to say which of two complete results shown is the more relevant. (One can assume that two different ranking algorithm settings are being compared here, e.g. when Google experiments with applying a new spam filter.) The different ratings available are:


Blog  |  Forum     more >> Archive | Feed | Google's blogs | About


This site unofficially covers Google™ and more with some rights reserved. Join our forum!