Free SEO Tool

Robots.txt Generator

Name: Atlasia Website Auditor
Rating: 4.9 (1205 reviews)
Author: Atlasia

Take control of your crawl budget. Instantly generate a technically perfect robots.txt directives file that instructs Googlebot exactly what it can and cannot index.

Default User-Agent

Default Crawl Directive

Global Crawl Delay (s)

Absolute Sitemap URL

Specific Directory Rules

Generated robots.txt

User-agent: *
Disallow: /cgi-bin/