AI Crawler robots.txt Checker is a free, no-signup tool from AI Ranked, a publication on AI search visibility.
Paste any robots.txt and the checker reports which of 21 AI user agents it allows or blocks: GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User, PerplexityBot, Perplexity-User, Google-Extended, Applebot-Extended, meta-externalagent, CCBot, Bytespider, Amazonbot and more. For every result it shows the rule behind it, so you can see why a crawler is blocked, then copy ready-made rules to allow or block specific bots.
Everything runs in your browser. Nothing you paste is sent to a server.
Built alongside AI Ranked's crawler access study of 261 major websites, which found that 22.6% of them block GPTBot.
