Free SEO Tool
Robots.txt Validator
Enter your domain to instantly check if your robots.txt is properly configured, with detailed explanations of any issues found.
Enter a domain above to validate its robots.txt file
What is robots.txt?
A robots.txt file tells search engine crawlers which pages or sections of your site should or shouldn't be crawled. It's placed at the root of your domain (e.g., example.com/robots.txt) and is one of the first files search engines check.
A properly configured robots.txt helps ensure search engines can efficiently crawl your important pages while avoiding duplicate content, admin areas, and other pages you don't want indexed.