What exactly is a Web robot?
A Web robot is any automated program which requests HTML files from a server. A robot typically processes these files to find links to other Web pages, then processes these pages too. Robots are not quite the same thing as intelligent agents. A true intelligent agent is more sophisticated than the average robot. In theory, an intelligent agent can provide the user with information he/she did not specifically request. In addition, intelligent agents are able to communicate with one another, and are sometimes capable of traveling between host computers.