Robots.txt Rules Generator

Build crawler rule blocks, manage Allow/Disallow path rules, and attach sitemap index references.

Robots.txt Rules

Sitemap Index URL

Rule Block #1

User-Agent

Disallow Paths

Allow Paths

Rule Block #2

User-Agent

Disallow Paths

Allow Paths

Generated Robots.txt Output

robots.txt

txt

# Created with CloudVantra (https://cloudvantra.com)

User-agent: *
Allow: /
Disallow: /admin
Disallow: /private
Disallow: /api/

User-agent: Googlebot-Image
Allow: /images/
Disallow: /temp/

Sitemap: https://cloudvantra.com/sitemap.xml