What is the Invisible Web?
The invisible web is the hidden area of the Internet that is not searched automatically by search engines. It is information that is freely available on the Internet, but that will not be retrieved by normal search requests. The information that is hidden on the invisible web takes a little more work to find. The visible web is information you can see or retrieve from search engines. The invisible web is sometimes referred to as the deep web. If you think of the Internet as a giant database, some of the information on the database is easier to find than others. The information held on the visible web is just a fraction of the information available on the Internet. Internet search engine databases already hold huge amounts of information. When you type a question into a search engine, it will bring up a list of possible pages that could answer your question. It is less expensive for the search engine to bring up these answer pages than to store every possible page on the subject you are
The invisible Web, also known as the deep Web or dark matter, is an important problem for Webometrics due to difficulties of conceptualization and measurement. The invisible Web has been defined to be the part of the Web that cannot be indexed by search engines, including databases and dynamically generated pages. Some authors have recognized that this is a quite subjective concept that depends on the point of view of the observer: what is visible for one observer may be invisible for others. In the generally accepted definition of the invisible Web, only the point of view search engines has been taken into account. Search engines are considered to be the eyes of the Web, both for measuring and searching. In addition to commercial search engines, other tools have also been used for quantitative studies of the Web, such as commercial and academic crawlers. Commercial crawlers are programs developed by software companies for other purposes than Webometrics, such as Web sites management,
There are parts of the Internet that search engines simply can’t see. The search engine’s robotic “crawlers” either miss or are locked out of these areas on the Internet. Behind the barriers lie treasure troves of quality information. Collectively this information is called the Invisible Web. There are many categories of invisible information missed by the popular search engines.
