This is a basic Django application to manage robots.txt files following the robots exclusion protocol, complementing the Django Sitemap contrib app.
For installation instructions, see the file "INSTALL.txt" in this directory; for instructions on how to use this application, and on what it provides, see the file "overview.txt" in the "docs/" directory or on ReadTheDocs: http://django-robots.readthedocs.org/