Could a robots.txt file be added to avoid crawling and indexing from bots?
• Yes, but note that the file would not be preserved between reboots by default. • First, copy robots.txt to /mnt/store. • Then, you’ll need to edit /mnt/store/knoppix.local.sh, and add the following: • #!/bin/bash cp /mnt/store/robots.txt /usr/local/web • Save that, and make sure that the script is executable.
