I’ve finally gotten some insight and data regarding a big complaint among domainers regarding search engines. Well, specifically, their complaint is with Google. I’m no search engine historian, but at some point along the line, Google decided to boot pages which the algorithm determined were being parked. The methods by which this happened are of course a secret, but folks can guess. The bot can check IP #’s, your DNS Servers and also analyze page content. It figures your name is parked, and wipes it from the index.

Now as sites, these pages aren’t much to look at. And a dumber robot would look at these pages, with all their links and related links and quickly get run around in a circle. Theoretically, any domain would have an infinite number of pages to index if the robot just kept following all the related links. So I can see why Google would want to get rid of them from a efficiency perspective. I can also see that these pages are competitive to Google. Why direct a searcher to a page with ads that are ultimately provided by a competitor. Even if Google itself is providing the ads on the page, they’d likely rather keep the user around for the prospect of keeping the entire price of a click for themselves.

Here’s the issue: Google has every right to keep sites out of the index that it determines are of low quality (although they’ll make mistakes and punish innocent parties every so often). However, when Google takes sites out of the index, these sites don’t even rank for their domain name. So if type a site like Frank Schilling’s antarctica.com into the Google search bar, I get 691 results, none of them a link to antarctica.com. If my intention was to find out about that particular web site, or to actually use the Google search bar as a proxy for my browser address bar, Google hasn’t really served me very well.

I didn’t used to think this was a particularly big deal. However, the traffic that is currently being sent to bags.com from the weak sister search engines like MSN and Yahoo! comes almost exclusively for the term “bags.com” or “www.bags.com”. Getting a site back into Google’s index can be worth dozens or potentially thousands of type in visitors from Google depending on the name, even if it ranks for no terms other than its own domain name!

A helpful discussion of these issues can be found at Frank’s Blog and Jay Westerdal’s Blog.