What is a robots.txt file?
A robots.txt file tells search engines where they can and cannot go on your site.
First, it tells what content you want to hide from search engines such as Google. You can also tell some search engines (not Google) how they can scan for permitted content.
What does a robots.txt file look like?
Here is the basic format of a robots.txt file:
A robots.txt file can give important information to a hacker to find access points and attack you, e.g. by revealing folders or databases. This is why it is important to delete administrative network paths.
If you are not able to delete them yourself, please contact your IT consultant or contact our support.