One Reason Google is King
Hold on Cowboy
This blog post is pretty old. Be careful with the information you find in here. It's likely dead, dying, or wildly inaccurate.
Almost everyone familiar with the Internet knows that Google is king of search. Even Yahoo and MSN know this, but haven’t figured out how to match Google. Here is a hint for them. Google is king because of the shear amount of crawling that Googlebots perform.
For example, I just launched a website this week dedicated to the Boundary Waters Canoe Area (BWCA), a national forest in north east Minnesota that is accessible only by canoe. It allows maps, forums, photos, and other information for those that are planning a trip in the BWCA.
I’ve been tracking the site with a few analytic programs one being awstats. It sorts through the web logs and tells me what has gone on. To make a long story short, within 12 hours of going live, Google found my new site and index most of the pages. Since then they have been back numerous times, almost hitting the website every day.
Consider that my website is only being promoted on a very small scale. A few links to it here and there on the web, yet Google was smart enough to crawl the entire site. Yahoo and MSN on the other hand have yet to crawl much of my site in its entirety. Look at the screen shot below. The site has been live since Monday evening April 22, 2007. Google has crawled it 2686 times since then and was still crawling it when this snapshot was taken.
Yahoo and MSN have only crawled 13 and 4 pages respectively. My site has over 1500 pages to it. Also note, that I have not submitted my site to any search engine at all (why should I have to).
In summary: Google is king because Google is constantly crawling every site on the planet at once (little bit of an exaggeration). Yahoo and MSN run their spiders about once a month.
