robots.txt Generator
Build your robots.txt file with user-agent rules, allow/disallow paths, and sitemap.
Sitemap
User-Agent Groups
Common User-Agents
Click to add as a new group:
Generated robots.txt
# Add user-agent groups to generate
Build your robots.txt file with user-agent rules, allow/disallow paths, and sitemap.
Click to add as a new group: