Robots.txt Generator

Build a valid robots.txt, including opting out of AI training crawlers.

Site URL (adds a Sitemap line)

Disallow these paths (one per line)

Allow exceptions (one per line)

Crawl delay in seconds (optional — Google ignores this)

Your robots.txt

User-agent: *
Disallow:

Save this as robots.txt in your site’s root, so it is reachable at yoursite.com/robots.txt. Note that robots.txt asks crawlers not to fetch a page — it does not keep a page out of search results if other sites link to it, and it is not a security measure. Use noindex to keep pages out of results, and real authentication to keep them private.