Important Notice: Our web hosting provider recently started charging us for additional visits, which was unexpected. In response, we're seeking donations. Depending on the situation, we may explore different monetization options for our Community and Expert Contributors. It's crucial to provide more returns for their expertise and offer more Expert Validated Answers or AI Validated Answers. Learn more about our hosting issue here.

bin/nutch generate generates empty fetchlist, what can I do?

Bin fetchlist generates nutch
0
Posted

bin/nutch generate generates empty fetchlist, what can I do?

0

The reason for that is that when a page is fetched, it is timestamped in the webdb. So basiclly if its time is not up it will not be included in a fetchlist. So for example if you generated a fetchlist and you deleted the segment dir created. calling generate again will generate an empty fetchlist. So, two choices: 1) Change your system date to be 30 days from today (if you haven’t changed the default settings) and re-run bin/nutch generate… 2) Call bin/nutch generate with the -adddays 30 (if you haven’t changed the default settings) to make generate think the time has come… After generate you can call bin/nutch fetch.

Related Questions

What is your question?

*Sadly, we had to bring back ads too. Hopefully more targeted.

Experts123