How long does it take the search engine to recognize changes to indexed documents?
The Penn State Search Engine crawls continuously to accommodate Web pages that change more frequently than those that rarely change. Notably, the search engine will crawl a given site anywhere from every 15 minutes to 90 days, depending on how often a Web page changes. During the search engine’s initial crawl of a given page after the upgrade, another crawl will be scheduled for 3 to 20 days later for pages with high to low PageRank, respectively. Each time a page is recrawled, the search engine will adjust the recrawl interval automatically until it is twice as often as changes are detected (as resources allow). A Penn State Access Account is required for login to see more details regarding the crawling specifications.
