WordPress Plugins
GuardPress Royal AI Firewall ForgeCache SiteVault SiteVault Lite SEObolt Royal Links Royal Affiliates FormForge Royal MCP Pro Royal MCP Royal SMTP Royal Access
Free Tools
SERP Preview Schema Validator Meta Tag Checker SSL Checker HTTP Headers Site Scanner Plugin Scanner Royal Links (Chrome) Royal Access (Chrome)
Claude Skills
Skills Library Divi Claude Skill WooCommerce Claude Skill Elementor Claude Skill Submit a Skill
Pricing Blog Switch to Royal Plugin Graveyard Support My Account Cart
Support / GuardPress / Spotting Spoofed AI Crawler Attacks

Spotting Spoofed AI Crawler Attacks: When “GPTBot” Isn’t GPTBot

AI crawlers are everywhere in server logs in 2026, and attackers have noticed. If a firewall alert shows a request from something calling itself GPTBot, ChatGLM-Spider, PerplexityBot, or YouBot, the User-Agent alone doesn’t tell you what it is. Three real GuardPress critical alerts below, each one an attacker in an AI-crawler costume, plus the two independent checks that catch every spoof cold.

TL;DR: two independent checks, both fast

Check the request path first: a real crawler fetches robots.txt, sitemaps, and normal HTML pages, not php://input RCE payloads or /var/run/secrets/ probes. Then cross-check the source IP against the vendor’s published JSON feed (linked table below). If the payload is an exploit or the IP isn’t in the feed, it’s not that bot. It’s an attacker on a rented VPS wearing the bot’s name.

The two tells that catch every spoof

Only one of these is 100% reliable on its own, and it isn’t the reverse DNS. Reverse DNS is a supporting signal at best. Real AI crawlers overwhelmingly run on public cloud infrastructure themselves: OpenAI runs GPTBot on Microsoft Azure and Limelight/Edgio, Perplexity runs PerplexityBot on AWS, and Anthropic runs ClaudeBot on Azure and Google Cloud. So “the reverse DNS ends in a cloud provider’s domain” alone does not prove spoofing, since that’s the everyday shape of real crawler traffic too.

Tell 1: the request path (100% reliable)

Real crawlers fetch robots.txt, sitemap URLs, HTML content pages, and their referenced assets. They do not send ?d+allow_url_include…php://input, they do not request /var/run/secrets/kubernetes.io/serviceaccount/token, they do not probe for /secrets.env or /wp-config.php.bak, and they do not append random-string canary paths. If the requested path is an exploit payload or a credential-harvest probe, it isn’t a crawler, regardless of what its User-Agent says.

Tell 2: IP membership in the vendor’s published feed (authoritative)

Every reputable AI operator publishes a JSON file listing every IP their crawler is allowed to run from. If the source IP isn’t in that file, the request isn’t from that crawler. End of check. Links to all the major operators’ feeds are in the verification table below.

Reverse DNS: useful supporting signal, not proof

The specific format [reversed-ip].bc.googleusercontent.com is the default (unset) PTR record for a Google Compute Engine customer VM, which is what you see when the customer never configured a custom reverse-DNS entry. Legitimate crawler operators typically set custom PTRs (Googlebot’s reverse DNS is crawl-*.googlebot.com, Applebot’s is *.applebot.apple.com) so their traffic can be verified with a forward-confirmed reverse-DNS lookup. A raw bc.googleusercontent.com PTR is much more attacker-shaped than legitimate-crawler-shaped, but it’s a heuristic, not a guarantee. The IP-feed cross-check is what settles it.

Example 1: an attacker wearing GPTBot

GuardPress flagged the request below as a critical threat. The User-Agent claimed to be OpenAI’s crawler:

GuardPress critical alert: attacker with GPTBot User-Agent originating from Google Cloud VPS

Reading the alert top to bottom:

The IP-feed check settles this in one step. OpenAI publishes GPTBot’s authoritative IP ranges as JSON at openai.com/gptbot.json. GPTBot runs on Microsoft Azure and Limelight/Edgio (not on Google Cloud), and 34.95.209.21 is not in the file. That’s the definitive verification: if the IP isn’t in OpenAI’s feed, the request isn’t from GPTBot, regardless of what the User-Agent says.

The requested path is the independent second confirmation. ?d+allow_url_include1+d+auto_prepend_filephp://input is a PHP-CGI argument-injection payload (the shape of CVE-2012-1823) that tries to force PHP to read the request body as executable code, attempting remote code execution on any host still running vulnerable php-cgi. Real crawlers fetch robots.txt and then follow links they discover. They never send that payload to /index.php.

The bc.googleusercontent.com reverse DNS (the default PTR for an unconfigured Google Compute Engine customer VM) is the third signal, consistent with a rented $5–10/month VPS running an off-the-shelf attacker toolkit. But it’s a supporting hint on top of the IP-feed miss and the exploit payload, not proof on its own.

Example 2: an attacker wearing ChatGLM-Spider

Same pattern, different mask. The payload escalates:

GuardPress critical alert: attacker with ChatGLM-Spider User-Agent probing for Kubernetes service account token

Zhipu AI (the operator of the real ChatGLM-Spider) is a Chinese company whose services run from Chinese cloud infrastructure. Google Cloud is not generally reachable from mainland China, so a real ChatGLM crawler request from a US Google Cloud region is already implausible. Regardless, the request paths make the classification trivial.

The payloads are what a real crawler would never send:

None of that is crawler behavior. It’s an automated attacker toolkit running through a target list.

Example 3: an attacker wearing YouBot

GuardPress critical alert: attacker with YouBot User-Agent harvesting credential files

Same rented-GCE-VM origin as the previous two, third different AI-crawler disguise. The paths make it obvious: you.com’s real crawler indexes web content; it does not sweep for exposed credential files.

The payload is a credential-harvest sweep:

The pattern across all three

Signal What was in all three alerts
Payload Exploit-toolkit output: PHP-CGI RCE, K8s service-account token grab, credential-file harvest. Nothing a real crawler would ever request. This is the load-bearing signal; the other rows are supporting.
IP not in any operator’s published feed Cross-checking 34.95.209.21, 35.198.147.41, and 34.6.58.241 against OpenAI’s gptbot.json, Anthropic’s bots.json, and other operator feeds returns no match. Whatever they are, they aren’t any published AI crawler.
Disguise User-Agent set to a real AI-crawler string with the operator’s URL as a plausible callback
Origin All three reverse-resolve to [reversed-ip].bc.googleusercontent.com, the default PTR for an unconfigured Google Compute Engine customer VM. Consistent with a rented VPS, but not a proof-of-spoof signal on its own (some real crawlers do run from cloud IPs).
Volume Dozens of events per IP in the last 30 days, spread across different targets, consistent with automated toolkits running through target lists
UA rotation Two of the three sent the identical PHP-CGI payload: the same toolkit is rotating its UA between AI-crawler brands to defeat naive UA-based rules

The AI-crawler User-Agent is picked for two reasons. First, some sites and firewalls maintain simple UA-based allowlists (“let anything calling itself GPTBot through”) because AI-crawler traffic is desirable for content-visibility. A well-known UA can bypass those rules. Second, admins reading firewall alerts unconsciously downgrade the perceived threat when they see a familiar brand-name bot in the User-Agent, which slows response time.

Verifying a real AI crawler: each vendor’s published IP list

Every reputable AI operator publishes an authoritative list of the IPs their crawler runs from. These are the sources to cross-check against, so bookmark them:

Crawler Operator Authoritative IP list
GPTBot OpenAI openai.com/gptbot.json (JSON with CIDR ranges)
ChatGPT-User OpenAI openai.com/chatgpt-user.json (user-triggered browsing from ChatGPT)
OAI-SearchBot OpenAI openai.com/searchbot.json (ChatGPT Search index crawler)
ClaudeBot / Claude-User / Claude-SearchBot Anthropic claude.com/crawling/bots.json (JSON, refreshed hourly). Anthropic runs their crawler on Microsoft Azure and Google Cloud, so real Claude crawler traffic can legitimately come from cloud-provider IPs. The feed is the authoritative check.
PerplexityBot / Perplexity-User Perplexity perplexity.ai/perplexitybot.json (and perplexity-user.json). Perplexity runs on AWS, so real PerplexityBot traffic has AWS reverse DNS (*.compute.amazonaws.com), which is normal and not suspicious. Verify by feed membership.
Google-Extended Google Shares Googlebot ranges. Verify with googlebot.json. Per Google’s official verification docs, valid Googlebot reverse DNS is googlebot.com, google.com, or googleusercontent.com, then forward-confirm the PTR resolves back to the original IP.
Bingbot (also serves Copilot) Microsoft bing.com/toolbox/bingbot.json
Applebot Apple Reverse DNS must resolve to *.applebot.apple.com and forward-resolve back to the same IP (documented at support.apple.com/119829)
DuckAssistBot DuckDuckGo Documented on duckduckgo.com help pages (search “DuckAssistBot”)
Meta-ExternalAgent / Meta-ExternalFetcher Meta Documented on developers.facebook.com (search “web crawlers”)
Bytespider ByteDance No JSON feed; verify by reverse DNS to *.bytedance.com
YouBot you.com No JSON feed at time of writing; verify by contacting you.com support if you need to whitelist
If the vendor doesn’t publish an IP list, treat any “bot” claiming to be them as unverified

Serious crawler operators publish their ranges so site owners can trust their traffic. If a bot claims to represent an AI company but there’s no published list to check against, err on the side of blocking, since the risk of impersonation outweighs the visibility upside.

What to do when GuardPress flags one of these

Look at the requested paths first

Real crawlers request robots.txt, sitemap URLs, and pages they discover through your internal links. Anything requesting index.php?d+allow_url_include, /var/run/secrets/, /wp-config.php.bak, /.env, /wp-admin/setup-config.php, or a random-string canary path is not a crawler. This is the most reliable signal and it doesn’t depend on any external lookup.

Cross-check the IP against the vendor’s published feed

Grab the source IP from the GuardPress alert. Look it up in the vendor’s JSON feed (see the table above). If the IP is not in the feed, the User-Agent is lying, no matter what brand name it’s wearing.

Click “Block this IP permanently”

The recommendation banner at the bottom of every GuardPress alert already flags cloud-provider IPs with sustained probing, usually with the copy “Sustained probing from a cloud-provider IP. Likely rented VPS attacker. Recommend permanent block.” One click and the IP goes on the manual Blacklist. See IP Blocking & Whitelisting for how that list interacts with the other two IP lists on the same page.

Don’t block whole cloud-provider ranges

Tempting, but the wrong tool. Google Cloud, AWS, and Azure host huge amounts of legitimate traffic you actually want, including real AI crawlers (OpenAI on Azure, Perplexity on AWS, Anthropic on Azure + GCP), plus SaaS backends, monitoring services, payment-gateway webhooks, and uptime checks. Block the specific offending IP, not the whole ASN.

Leave real AI crawlers alone

If a request’s source IP is in the vendor’s published feed and the request is for a normal page (not an exploit path), it’s the real bot, even if the reverse DNS is *.compute.amazonaws.com or *.cloudapp.net or *.googleusercontent.com. Blocking real AI crawlers hurts your visibility in AI-answer surfaces (ChatGPT, Claude, Perplexity, Google AI Overviews). Only block the fakes.

How GuardPress makes the spoof visible

Every critical-threat alert shows five fields in the WHO block: IP, Reverse, Agent, First seen, and Total 30d, plus the requested paths in the WHAT block. That layout is deliberate. The IP, reverse DNS, and requested paths all sit together above the User-Agent, so the signals that actually diagnose a spoof are the first thing your eye lands on before the User-Agent has a chance to bias you.

The firewall itself doesn’t decide these are spoofed. It blocks them on the exploit payload alone (PHP-CGI RCE and Kubernetes-secret probes match the WAF ruleset regardless of what User-Agent they carry). The reverse DNS, User-Agent, and requested paths are all surfaced together so you can classify the alert at a glance and decide whether a permanent block is warranted.

The alerts that reach your inbox are already pre-filtered

One thing that’s not obvious from an individual alert screenshot: GuardPress 1.6.49 and later run a set of pre-dispatch checks on every source IP before sending a security email. Alerts from four low-signal categories are still written to the dashboard and audit log, but the wp_mail() dispatch is skipped for each of them:

High and critical severity alerts (malware detected, admin user created, known-bad signature match) bypass all four filters and email regardless of source. So do “Send test email” previews from the Settings page.

AI crawlers aren’t in the FCrDNS pattern set

The built-in FCrDNS patterns cover the traditional search-engine set only. ClaudeBot, GPTBot, PerplexityBot and other AI-operator crawlers are not allow-listed by name. They flow through the normal alert path. So the auto-allowlist doesn’t handle AI-crawler noise the way it handles Googlebot noise; the primary defense for AI crawlers remains the vendor JSON feed cross-check documented in the verification table above. Developers can extend the pattern set via the guardpress_verified_crawler_patterns filter if a specific AI crawler operator publishes a stable PTR suffix.

What this does mean for the alerts in your inbox: traditional-search noise is already gone. If a security email lands with a Googlebot User-Agent, GuardPress already failed to verify that crawler via FCrDNS, since real Googlebot would have been silently allow-listed. For AI-crawler alerts, the auto-allowlist doesn’t give you the same instant signal, so you fall back to the two independent checks at the top of this article: the exploit request path and the IP-feed cross-check.

The setting lives at GuardPress → Settings → Verified-crawler auto-allowlist. Turn it off if you want to receive email alerts on every crawler hit regardless of verification.

Adding real crawlers to your manual Whitelist

For sites that receive a lot of AI-crawler traffic and want to allow real crawlers explicitly, the manual Whitelist takes CIDR ranges. Drop the ranges from each operator’s JSON feed (linked in the table above) into the whitelist and they’ll bypass every firewall check. Any request wearing that operator’s User-Agent from an IP not in the whitelist still gets evaluated normally.

Frequently asked

Is it possible to fake the reverse DNS too?

An attacker can set the PTR of their own IP to any hostname they want. What they cannot do is control the forward DNS for a domain they don’t own. The definitive check is forward-confirmed reverse DNS (FCrDNS): reverse-resolve the source IP to a hostname, then forward-resolve that hostname back to an IP, and confirm it matches. If an attacker sets the PTR to crawl-1-2-3-4.googlebot.com, the forward lookup on that hostname will not return their IP, since Google owns the googlebot.com forward zone. Google, Apple, and Bing all document FCrDNS as the canonical way to verify their crawlers.

Why didn’t I get an email when real Googlebot crawled my site during an attack window?

You wouldn’t. GuardPress 1.6.49+ auto-suppresses email alerts from FCrDNS-verified traditional-search crawlers (Googlebot, Bingbot, Yahoo Slurp, Yandex, DuckDuckBot, Applebot, Baiduspider) so the noise of legitimate bot traffic doesn’t crowd out the alerts you actually need to act on. The events are still written to the dashboard and audit log; only the email dispatch is skipped. When a Googlebot- or Bingbot-branded alert does arrive in your inbox, that’s the useful signal: the sender failed FCrDNS verification, which is itself suggestive of a spoof. Note that AI-operator crawlers (ClaudeBot, GPTBot, PerplexityBot) are not in the default FCrDNS pattern set, so alerts about them are not auto-suppressed. Verify AI crawlers against the vendor JSON feed instead.

Why don’t attackers just rent from the same cloud regions the real crawlers use?

They’d still be in a different sub-range. OpenAI, Anthropic, and Perplexity each publish the specific CIDR blocks their crawler uses within Azure/AWS/GCP, and those blocks are dedicated allocations, not general customer VM space. An attacker on a random Azure VM is in Azure IP space but not in openai.com/gptbot.json’s specific ranges, so the IP-feed check still catches them. This is exactly why the published IP feeds are load-bearing as a trust signal: they identify specific allocations that only the operator can request traffic from.

What about a new AI crawler with no published IP list?

Common in 2026: a new crawler shows up, sets a plausible User-Agent, and there’s no operator-published feed to check against. Without that feed, there’s no way to distinguish a legitimate new crawler from an impersonator, so you’re relying on the request-path check alone. If it’s only fetching robots.txt and content pages, it’s crawler-shaped and probably fine. If it’s probing exploit paths, block it regardless of the branded UA. Until the operator publishes an authoritative IP list, treat them as unverified, not automatically trusted.

Should I set a rate limit on real AI crawlers instead of a full allow?

Yes, if you have the traffic to justify it. AI crawlers can be aggressive when they’re re-indexing a large site, and even legitimate crawler load can crowd out user requests on a small host. GuardPress’s rate-limit rules apply to any IP not on the manual Whitelist, so leaving the AI-crawler ranges off the Whitelist and letting the normal rate limiter smooth their traffic is a defensible middle ground.

Related GuardPress topics

If this article was useful, these are the natural next reads:

Back to GuardPress Support GuardPress Plugin Page