Google Blogoscoped

Forum

Evolution of a Search Engine  (View post)

Fanfan [PersonRank 1]

Friday, February 2, 2007
17 years ago9,365 views

Fantastic entry!

Art-One [PersonRank 10]

17 years ago #

AI has been a long time dream and it has always been overestimated. Not that there won't be a movement that will be based more on AI technology, but it will be used in a more pragmatic way....

What is the use of Google formulating an answer by its own, when there is something like Wikipedia. Wikipedia, where in fact people are still debating on what's the right definition for something...

So, Philipp, I'm sorry, you've told a nice story, but I don't think that this will be search engines near future... It would already be a great thing that search engines will have a unique interface (like searhmash), that it'll learn from your interests and that some other AI features like show me all images with a red ball or with a face like on this picture....

I do not expect much more than that...

milivella [PersonRank 10]

17 years ago #

This is one of the posts that make this blog be above every other one.

Brock [PersonRank 0]

17 years ago #

The interesting thing about an AI that uses secondary, tertiary, etc. ... conclusions is that it will grow so complex that it cannot understand even itself. Forgot the Google developers not knowing how it works; the AI won't know how the AI works. There are no shortcuts in the algorithmn; the only way to get an answer will be to run the query and "see what happens."

I expect an AI, if problerly pragmatic, will continue to answer search queries. That's its "job." After all, an AI has to eat too (in terms of electricity, bandwidth and hardware replacement), and it will have to earn a paycheck somehow. Not even AI's are an island.

Joe Whyte [PersonRank 1]

17 years ago #

woe philiip this is pretty detailed. I love the idea and I truly hope social search becomes a reality and not a beta like it is now with personalized google search.

The only thing I can see from an seo perspective is that if linking to external sources is out then where does that leave us SEO's? Probably focusing more on SMO rather then SEO.

Aaron [PersonRank 0]

17 years ago #

So, this is all very good, and very interesting. But the one part of the equation that you left out is the advertising. Google is ad-funded, and appears to be making no attempt to move away from that model. If anything, they are increasing the model, reach out into online video, offline print, offline radio, etc.

So the question is: As search continues to improve along the lines that you have described, Philipp, it seems to obviate the need for advertising. If the organic results are so comprehensive and so personal, and if the AI knows you might be interested in purchasing a product or service and can *organically* show you that product or service, without an ad, where does Google's revenue stream go?

In their original PageRank paper from 1998, Larry and Sergei wrote: "In general, it could be argued from the consumer point of view that the better the search engine is, the fewer advertisements will be needed for the consumer to find what they want."

Once these 2nd and 3rd and 4th generation search engines appear, I think that essentially means the number of advertisements will asymptotically approach zero. So where will Google's revenue stream come from?

(And in a related question: If they can't solve the revenue stream question, will they ever build the 4th generation search engine, when to do so means no more advertising income? Matt Cutts has long claimed that advertising is completely independent of the search results, but this is clearly bogus, because there is clearly a connection between how good the results are, and how necessary it is to click an ad. Will Google develop a system so good that no one ever needs to click and ad, again? Or will they purposely "stunt" the quality of the engine, so that people need to click an ad every once in a while? I really am curious about what folks think.)

milivella [PersonRank 10]

17 years ago #

To Art-One: I dont' know which level of AI you don't believe that we'll reach, but automatic inference (Level 4 in Philipp's vision) is something is worked on right now. See, e.g., http://www.cyc.com/cyc/technology/technology/whatiscyc_dir/howdoescycreason

milivella [PersonRank 10]

17 years ago #

To Aron: a possibile scenario is "pay to be included in the Google database, from which it produce the best answers". It would be good because money wouldn't assure visibility: you would need good content, and good reputation. It would be bad if any source that doesn't pay (e.g. academic websites) would be excluded.

Aaron [PersonRank 0]

17 years ago #

Milivella: That does seem possible, but paid inclusion in a database sounds too much like another problem that Larry and Sergei pointed out in their 1998 paper:

"Since it is very difficult even for experts to evaluate search engines, search engine bias is particularly insidious. A good example was OpenText, which was reported to be selling companies the right to be listed at the top of the search results for particular queries [Marchiori 97]. This type of bias is much more insidious than advertising, because it is not clear who "deserves" to be there, and who is willing to pay money to be listed. This business model resulted in an uproar, and OpenText has ceased to be a viable search engine."

While it is true that, in this paid database, Google would probably not let people pay even more to be ranked higher, just the fact that you have to pay to be ranked AT ALL means that those who pay are ranked higher than those who don't pay.. because if you don't pay you don't even appear!

So this solution is too much like the "evil" that the Google guys were trying to avoid in 1998.

Roger Browne [PersonRank 10]

17 years ago #

Great post! – but it assumes that I want an intelligent answer instead of a ranked list of web pages that match my query. For me, that's not usually the case.

Art-One wrote:
> AI has been a long time dream and it has always been overestimated.

What happens is that the "bar" is constantly being raised. If you took Google and showed all that it can do to AI researchers from the 1950s, they would be astounded – it would be beyond many of their wildest expectations.

But teach them how it works – tell them about personal computers, websites, crawlers, ranking algorithms, translation technologies etc and they'll change their minds and say that of course it's not real AI.

I think that's always going to be the case. AI won't be regarded as a solved problem unless AI can be made indistinguishable from human intelligence.

Regarding the ads – it's a valid point that as search results improve, ads will become less useful. Because of that, I think it's inevitable that results and ads will be merged. You have two movie theaters in your area? The search engine will give you the "Rocky LXVII" screening times for the cinema that bids the most.

siggi [PersonRank 1]

17 years ago #

Why did Douglas Lenat present CYC at a Google Tech Talk?

Dev [PersonRank 1]

17 years ago #

omg!
A great post Philipp!
IMHO, I don't think the world is ready for level 3 & 4 yet, It's just plain creepy to see a machine using 'I'..
Just to maintain perspective, neither was the world ready for Gmail, when it came out, so I think people will adapt soon, after it comes out.

Tim [PersonRank 0]

17 years ago #

--"Regarding the ads – it's a valid point that as search results improve, ads will become less useful. Because of that, I think it's inevitable that results and ads will be merged."

..but isn't that "evil"? Hasn't Google said since the beginning that the only reason it is not evil the fact that it keeps ads separate from SERPs?

Frank Taylor [PersonRank 1]

17 years ago #

It's too bad the AI didn't take into consideration the search could have been about the "Rocky Horror Picture Show", or the "Rocky and Bullwinkle Show". :-)

Heebie Sudoku [PersonRank 5]

17 years ago #

Isn't Level 3 kinda already available on Ask.com?

milivella [PersonRank 10]

17 years ago #

To Heebie Sudoku: Philipp has explicitly said "the engine covers the long tail of possible queries, and has an answer to everything imaginable, including stuff that’s not by any means common knowledge." And Ask.com, at its best, has an answer only for some of the objects of the common knowledge.

J. McNair [PersonRank 10]

17 years ago #

Interesting, seeing as Google just made an announcement about "personalized" search results being integrated with organic search results for all Google Accounts. At least on personalized home pages. From the new help pages it seems that they will include results from their properties. Mind you, I haven't noticed much difference yet because I only use search history, but the suggestions from the "Interesting Stuff" home page gadget are pretty good.

Google seems to want to jump to Level 3 (Personalized). Also, Frank Taylor is correct – a proper AI search engin would ask did you mean Rocky Horror Picture Show, Rocky and Bullwinkle, or a documentary about Rocky Road Ice Cream.

Finally, I think that if google's "magical" AI could find the perfect ad for what a user needed, then revenues would increase. Companies would pay tons for the free product research – Google would be finding customers FOR them.
  

Hong Xiaowan [PersonRank 10]

17 years ago #

I think Google have a AI heart, but still have simple homepage well. Now google become some complex.

Hong Xiaowan [PersonRank 10]

17 years ago #

Search should be a free, relax, funny, lazy life. Googler's works is not test our brain's CPU.

Maybe someone's CPU is faster than Googler, but he not like think, he is lazy.

Google is a easy face of life, do not make it like hard work please. How about Google? Good, wonderful, fast, I like it......, It is at 2003 in our office. But now, the answer is complex, perfect, down, tired.

Give the user easy, give you money. It is not the classic business. but users is still human beings.

Matt Cutts [PersonRank 10]

17 years ago #

Philipp, did you do this post before this Google post:
http://googleblog.blogspot.com/2007/02/personally-speaking.html
?

I'm wondering whether you're smart, or smart+psychic. :)

btd [PersonRank 1]

17 years ago #

Why to go to level 3 when we have wikipedia and best of all it is online right now:
http://en.wikipedia.org/wiki/Rocky_%28movie%29

The only true Google killer that I can see now and only competition is "Intelligence of the crowds" . Google exploits it in certain way in its algorithm but that is only first order linear interpolation... based on link, keywords and some filtering.

Anyway wikipedia is evolving and becoming more and more powerful every day.

Philipp Lenssen [PersonRank 10]

17 years ago #

> Philipp, did you do this post before this Google post:

I posted this before reading Google's personalization post, which I had kind of a hard time understanding, by the way...
http://blogoscoped.com/forum/84978.html

Matt Cutts [PersonRank 10]

17 years ago #

Okay, so good psychic call about personalization then. :)

Paul [PersonRank 0]

17 years ago #

The only thing it missed is level 3 taken in a web 2.0 direction. It is very possible for a site such as Wink to have better results if it shows stuff that other people similar to yourself like. (Does that make sense?) The key to a good search engine is the number of clicks/keys pressed until you get the answer you want. This may not be close to the answer someone else wants, and may not even be the correct answer, but once you have it, you are happy. An average search for myself takes about 3 querys, and several clicks on each result page. Google will have to decrease my effort, or I will begin searching for or making a new engine. (I have wanted to do this for some time.)

Chad Okere [PersonRank 0]

17 years ago #

Google is already at stage three, it links to a wikipedia article in the first or second link for almost any query. I'm not sure why "more sources from the internet" would create anything better then a wikipedia entry.

Joey J. [PersonRank 5]

17 years ago #

Level one search engines were not dumb! I still remember some of the holy-crap-obscure ones I used to use. c4.com was one of my favorites, even though it sucked SO MUCH. Also, I still use Altavista in certain cases. Does this make me old?

Milly [PersonRank 10]

17 years ago #

Very nice Philipp, (from one inhabitant of this inhibited planet to another).

Aaron, I wouldn't have thought the correlation between needing to click an ad, and actually clicking it (or indeed merely absorbing its message), is as strong as you suggest.

Google's AI will try to present the best comprehensive and personal results, but advertisers will still pay for the opportunity to present their own wares for consideration (and many people will still surely click them, human nature being as it is, because they're shiny, even while being a less good 'answer').

And most impulse purchases are (by definition I suppose) of things we *aren't* seeking out: so those wouldn't/shouldn't appear in the organic results, and can still be successfully advertised. The AdSense AI might be very clever at knowing what would tickle our fancy, whilst searching for something else entirely.

Similarly, if Google's AI gives us ample results while we're looking for a Ford Galaxy, it may still be that GM and Kia will be happy to pay for the brand association, building for the time we next buy a vehicle. (I guess that would involve some changes to the AdSense CPC/CPM model </shrug>).

I suspect there must be lots more examples too. And if the click-through rate declines, well the page views would surely increase to compensate, as Google (or whichever) becomes everyone's first step to finding everything.

My guess is that better results are little or no threat to the advertising model. And if it is, if the results are *so* good and invaluable, then I guess we'd be happy to pay a little for them.

Andreas [PersonRank 0]

17 years ago #

Very good post, Philipp! But why is nobody getting the whole implication of level 3 and beyond?
1. You ask a question, and google (the AI) knows better than you which answer you expect? It will filter out all information, which it thinks you don't need! So you will get only the information which it thinks you need. This will make censorships (China) much more subtle an efficient.
2. How does the AI know what you need? Well, it collects all your queries, scans your gmail, analyzes the ads you clicked on, and whatever more. It will add all your data into your personal profile, to supply you with seemingly better answers – and of course with better ads.
3. And why not sell the collected data? It's a treasure! Who else can boast with detailed data of millions of potential customers?

Enough of that stuff. But i feel still OK with a level 2 search engine, where I still have the impression of controling the direction of the search.

Andreas

Hong Xiaowan [PersonRank 10]

17 years ago #

Philipp's idea is a perfect dream for Search engine. It will make Google have a super heart but the UI still simple. It is what we want for Google. Users can use Google very very easy, and left all problems for Google to settle.

Redge [PersonRank 1]

17 years ago #

What struck me is how much level 4 and beyond reminded me of Asimov stories...

Chinmay [PersonRank 1]

17 years ago #

I was strikingly impressed by the chronicle of Search traced in this blog post!! The concept of coupling AI with the existing Search Algorithms would indeed make Search Engines' superlative.....and indispensable from our lives!!

The possibilities are endless indeed! Terrific post Philip! (''_'')

milivella [PersonRank 10]

17 years ago #

Level 3 would be relatively easy to realize with the simple method I described in http://blogoscoped.com/forum/83810.html : if Google had computed (or could compute in real time...) all the topics strongly associated with your query, it could easily extract (from the pages it would rank better for that query) the assertions that contain your query + those topics.

deart [PersonRank 1]

17 years ago #

Really, really interesting post! Well done.

Yuku [PersonRank 0]

17 years ago #

This is a very brilliant article. Your imagination power is amazing and realistic. You could write a short book regarding this, I'm sure a lot of people will like it. Don't forget the mock screenshots, too!

Phill Midwinter [PersonRank 0]

17 years ago #

I'm a semantic search engineer, I do agree with a lot of what you've written but some of it is currently a little beyond us I think.

Not that there's anything wrong with that! We'll get there in the end and developers like a challenge ... for the most part.

I've written about some of my development thoughts at http://phillmidwinter.wordpress.com for anyone who's interested in this more.

Adam Jusko [PersonRank 0]

17 years ago #

No matter how sophisticated the technology gets, it's never going to be able to make decisions in the same way a human does---your point about deciding whether a link is good or boring touches on this fact.

It will always be a good thing to have alternatives created by humans instead of solely relying on tweaking algorithms.

Forum home

Advertisement

 
Blog  |  Forum     more >> Archive | Feed | Google's blogs | About
Advertisement

 

This site unofficially covers Google™ and more with some rights reserved. Join our forum!