Robots.txt Generator
Everything runs in your browser — your text is never uploaded or stored.
Create a robots.txt file that tells search engine crawlers which parts of your site they may visit. Add your allow and disallow rules, include your sitemap address, then copy or download the finished file — free, no signup, nothing uploaded.
Frequently asked questions
What is a robots.txt file?
It is a small text file placed at the root of a website that tells search engine crawlers which parts of the site they may or may not visit.
Where do I put my robots.txt file?
Upload it to the root folder of your site so it is reachable at yourdomain.com/robots.txt. Most hosting control panels let you upload it with their file manager.
Does robots.txt hide a page from search results?
Not reliably. Disallowed pages can still appear in search results if other sites link to them. To keep a page out of search results, use a noindex meta tag or password-protect it.
Do I need a robots.txt file?
No. Sites without one are crawled normally. It is useful when you want to keep crawlers out of admin areas, search pages or duplicate content.