28 AI crawlers, search engines and SEO bots

Check Bot Access

Find out whether ChatGPT, Claude, Perplexity, Google, Bing and other bots can reach your website. Enter a URL and the tool checks real server responses, robots.txt rules and robots meta tags for every bot.

Frequently asked questions

What exactly does the tool check?
For every bot it sends a real request to your URL with the bot's User-Agent, reads the HTTP status, the robots meta tag and the X-Robots-Tag header, and evaluates your domain's robots.txt for that bot. The result is one verdict per bot: Allowed, Blocked, Unverifiable or Error.
What does "Unverifiable" mean?
The server answered with HTTP 403 or 429. Firewalls and CDNs (e.g. Cloudflare) often verify bots by their IP address, not just the User-Agent. Our request comes from a different IP, so the server may block us while still letting the real bot in — or the other way round. Check your firewall or CDN bot settings to be sure.
Why should I care whether AI bots can access my website?
AI assistants like ChatGPT, Claude or Perplexity answer questions using content they can crawl or fetch live. If your website blocks their crawlers, it is less likely to be cited or recommended in AI answers. On the other hand, you may intentionally block training crawlers (e.g. GPTBot) while allowing search and user-triggered bots.
What is the difference between Google-Extended and Googlebot?
Googlebot crawls for Google Search. Google-Extended is not a separate crawler — it is only a robots.txt token that controls whether your content may be used for Gemini model training. That's why it has no HTTP check, only a robots.txt verdict. The same applies to Applebot-Extended.
Do you store the URLs I check?
No. The check runs in real time and the results are only shown in your browser. We do not keep the URLs or results in any database. For the daily limit of 10 checks we only count requests per pseudonymized IP fingerprint until midnight. More in the privacy policy.