What is user-agent spoofing?

By Nik Andreev, Founder & SEO specialist · Last reviewed · Editorial policy

User-agent spoofing is when a client sends a User-Agent header that belongs to someone else, most often a trusted crawler such as Googlebot or GPTBot. It works because the header is plain text the client chooses. You catch it by checking the request against something the real operator controls: its DNS, its published IP ranges or its signing key.

At a glance

What is faked
The User-Agent request header (and sometimes other headers)
Why it works
The header is self-reported text with nothing backing it
Common targets
Search crawlers (to bypass blocks) and normal browsers (to blend in)
How to catch it
FCrDNS, the operator's published IP ranges, or a Web Bot Auth signature
What it is not
Proof of a bad actor in every case. Some tools spoof a browser just to get a normal page.

How to spot it

  • Start from the claim: filter your logs for well-known tokens (Googlebot, GPTBot, ChatGPT-User, bingbot) and group by source IP.
  • Run the operator’s check: for Google and Bing, forward-confirmed reverse DNS. For OpenAI, the IP must sit inside its published ranges, such as openai.com/gptbot.json. Anything that fails is spoofing the name.
  • Match exactly: use the exact published CIDR blocks and the exact hostname suffix. A nearby address on the same cloud network, or a hostname like googlebot.com.example.net, is still a fail.
host 203.0.113.7
# 7.113.0.203.in-addr.arpa has no PTR record
# User agent said Googlebot. Not in common-crawlers.json either: spoofed.

The other direction

Spoofing also runs the other way: bots that send an ordinary Chrome user agent to look human. No user-agent rule catches those. That traffic is found by behaviour, or, for agents that sign their requests, by their HTTP message signature, which names the operator whatever the user agent says.

The checks that catch it

FCrDNS Reverse lookup, then forward lookup. The round trip has to land on the same IP. Published IP ranges The operator's own list of addresses. A claimed bot outside it fails. Signed agent An agent that signs its requests, so its name can't be copied without its private key.

Quick answers

Is a request with a Googlebot user agent always Googlebot?

No. Google documents reverse DNS and its published IP ranges as the way to check. A Googlebot string that fails both is a spoof.

Why would someone fake a crawler?

To get past rules that let search crawlers through, or to scrape content a site serves to bots but not to anonymous visitors.

Does blocking spoofed bots hurt my SEO?

No, as long as you verify first. Blocking requests that fail the operator's check leaves real crawlers untouched.

Sources