Free tool

Robots.txt Tester

Check whether a crawler can access your site by testing its robots.txt rules.

Site URL

We'll fetch /robots.txt automatically and test this URL against it

User Agent