Free Tool
Free Robots.txt Tester
Check whether a specific page is blocked for Googlebot, GPTBot, or any crawler — the exact tool Google retired from Search Console in 2023. No signup required.
— paths tested so far
How this tool works
Fetches your live robots.txt
Not a cached copy — the actual file at your domain root, right now.
Groups by user-agent
Correctly handles multiple consecutive User-agent lines sharing one set of rules — a common pattern many simpler parsers get wrong.
Longest-match wins
Supports wildcards (*) and end-anchors ($), and applies the same specificity rules real crawlers use.
Falls back to the wildcard group
If a crawler has no specific rules, it inherits whatever applies to User-agent: *.
Shows the raw file
So you can see exactly what's driving the verdict, not just trust a black box.
No robots.txt = allowed
If a domain has no robots.txt at all, everything is crawlable by default — this tool reflects that correctly.
Frequently asked questions
Why isn't Google's own robots.txt tester available anymore?
Google removed it from the old Search Console in 2023 with no path-specific replacement in the new version — this tool fills that gap.
What does "longest matching rule wins" mean?
When multiple rules match the same path, the most specific (longest) pattern takes precedence, and Allow wins on a tie — the same logic real crawlers use.
Let's build something
Accidentally blocking your own pages?
We audit and fix crawl configuration as part of every technical SEO engagement.