Robots.txt Generator
Build a robots.txt file for search engines. Set user-agent rules, allow/disallow paths, and your sitemap URL.
Paths crawlers should not access. Leave empty if none.
Useful to override a broader Disallow rule for specific folders.
Try it with this example
Keep User-agent: *, Allow: /, add Disallow: /admin/, and set Sitemap to your sitemap URL. Copy the file to your site root.
What is this tool?
How do I create a robots.txt file? Set user-agent, allow/disallow paths, optional crawl-delay, and sitemap URL. Generate copy-ready robots.txt for SEO. Complements Meta Tags Generator. All client-side.