What is the difference between a search engine and a directory?
True search engines such as Google, Yahoo (Web results), MSN, Teoma, Ask Jeeves, use “spiders” or “bots” that crawl the web collecting information from Web sites. The bot is a program that is initiated by a particular search engine. The information is then written back to the search engine database. There is no human intervention involved. A directory on the other hand is a human compiled list of sites that are categorized, typically in alphabetical order. Some top directories charge a fee for inclusion. (Yahoo, Business.com) However, DMOZ http://www.dmoz.org , A volunteer based directory, is still free.
The fundamental difference between search engines and directories is that search engines utilize spiders/crawlers to gather information about websites (e.g. Google), whereas directories are usually human edited and require a human to input this information during the submission process (e.g. Complete Online Directory).
While the average Internet user may not be aware, or care about, the differences between Search Engines (Google) and Directories (Yahoo!), the search results they provide are compiled in very different manners. Major search engines build and maintain huge databases of web pages by utilizing “robots” or “spiders.” These programs “crawl” across the web by following links between sites and pages. Each search engine uses its own set of criteria or algorithms to decide what to index, in what position. Also, some search engines “deep crawl” indexing each page in a web site, while others “shallow crawl” and index only the main “home” page. Yahoo! and LookSmart are the most widely known examples of “directory-based” search portals. Under this model, sites are subjected to an editor’s consideration for inclusion in a search engine’s listings. These directories now require commercial sites to pay a fee to be considered for inclusion in their indexes. While there is no guarantee of favored rankin
