Adding a Robots.txt File
ListManager's web interface may be indexed by search engine robots (alternatively called web crawlers, spiders, webants and worms) like any other web site. In some cases, users may not want their Aurea List Manager archives indexed, or they find that indexing overloads their Aurea List Manager web server.
To specify particular directories robots should not visit, you may add a robots.txt file to the Aurea List Manager/tclweb/htdocs directory. For more about robots and using robots.txt to exclude them , see http://www.robotstxt.org/.