2010-02-27

Hands off my search engines!

There have been rumblings about the EU looking into allegations of lack of fairness in the Google search results.  Apparently some web sites that offer their own sort of search services feel they have not been displayed as prominently as they would wish in the search results.

Let me explain what Google does:  Google lives and dies by the percieved quality of their search results.  If Google serves up results that are not helpful to their users: that hurts user satisfaction.  Everything Google does on search is measured against what provides the greatest user experience: "can people find what they are looking for?".

Search is a hard problem.  You have billions of web pages, the user enters a few words and in essence, the search engine has to come up with the 10 pages most relevant to those few bits of "intent" that the user provided.  Sure, a given search will usually have lots of hits, but we count on search engines to place what we are looking for at, or near, the top of the result set. 

If you think that search engines have not been innovating since the interface you see has only changed slightly over the years, you should consider this:  the size of the web has been doubling every so many months.  Also, since the top spot in a search result can have very direct and easily measurable financial consequences, a lot more resources are devoted to gaming the search engine ranking algorithms.

In effect this means that a lot of people are working to place their content in front of you -- whether relevant for what you are looking for or not.  These people are working against you, the user, and the search engines.

For some searches I do, search engines unfortunately give me pretty bad results.  For instance, if I look for a web shop that will sell me some gadget and the top 10 results are dominated by price comparison sites or other aggregators of content, that pretty much sucks.  It isn't what I am looking for.  What I am looking for is, preferably, the best place to buy the gadget or background information that will help me make a decision.  If Google observes that I am not too happy about those search results and then decides to change the ranking to give me results I do want, then that is the right thing to do for them and for me the user.

It is that easy.

I find the idea that we would allow government bureaucrats to dictate how search engines rank their results to be a horrible idea.  First and foremost because it is dangerous to let governments dictate what is "truth".

Second, because these people are so obviously not qualified -- they do not even understand the basics of the problems a search engine must solve.  It is dangerous to even pretend that these people are worth listening to.

Third, the interests of Google and the interests of the user are more aligned than that of spammers and bureaucrats and I can't help but feel that spamming is more honorable than the blind application of non-knowledge to the detriment of society by bureaucrats whose main ambition is to stay employed and preferably retain a considerable expense account to be used liberally in a civilized country with good beer.

One can only hope that this process leads to the education of government bureaucrats though I am not holding my breath.

(disclosure: I've worked for a number of search engine companies, including Google, but I do not work in the search engine industry now.  I have no incentive to side with Google, Bing or any other search engine.  I do however have justified reasons to fear stupidity in government.)

2 comments:

  1. The easy solution: Present a list of pages on Teh Interwebs and let a government committee sort it in the proper order.

    Once the committee has finished sorting (and has no backlog) drop the page ranking algorithm and use the government-approved ranking.

    That should keep them busy for a while.

    ReplyDelete
  2. At some point in the future a mandatory course in programming should be taught in grade school from the age of 8. Then at the end of grade school an exam should be given to determine if the student is capable of understanding complexity, optimization and concurrency.

    Those who are incapable of grasping these concepts should not be allowed to hold government office or design inrastructure.

    ReplyDelete