What is a user agent?

By Nik Andreev, Founder & SEO specialist · Last reviewed · Editorial policy

A user agent is the software making an HTTP request, and the User-Agent header is the string it sends to name itself. Browsers send a long Mozilla/5.0 string with platform and engine details. Crawlers usually add a product token such as Googlebot or GPTBot plus a link to their docs. The client chooses the value, so anything can send anything.

At a glance

Type
HTTP request header
Defined in
RFC 9110 (HTTP Semantics), section 10.1.5
Syntax
product/version, optionally followed by more products and (comments)
Set by
The client. Any script can send any value.
Bot convention
A product token like Googlebot/2.1 and a +URL to the operator's docs
Browser trend
User-agent reduction: fixed platform and device values, minor version shown as 0.0.0

How it works

  • Structure: a list of product/version tokens with optional comments in brackets. Browsers follow the pattern Mozilla/5.0 (<system>) <platform> (<details>) <extensions>.
  • Bots: most well-behaved crawlers wrap their token in a compatible comment and add a link to their documentation, so the name you match on is the token, not the whole string.
  • Browsers are saying less: with user-agent reduction, Chrome freezes the platform version, device model and minor version numbers. Sites that need more detail ask for it with User-Agent Client Hints (Accept-CH and Sec-CH-UA-* headers).
# Browser (Chrome on Windows)
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Safari/537.36

# Crawler
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Why it is not an identity

The header is written by the client, so a scraper can paste in a Googlebot string with one line of code. Use the user agent to decide which check to run (a request claiming Googlebot gets a reverse DNS lookup), then trust the check, not the string. Faking it on purpose is user-agent spoofing.

What to do with a user agent

User-agent spoofing Sending someone else's user agent, usually a famous crawler's, to get treated like them. Verified bot A bot whose claimed name has been checked against something the operator controls. Server log Where user agents get recorded, for every request, including the ones JavaScript analytics never sees.

Quick answers

Why does every browser user agent start with Mozilla/5.0?

Historical compatibility. Sites once served better pages to Mozilla, so other browsers copied the prefix and it stuck. It no longer means Firefox.

Should I match the full user-agent string?

No. Match the bot's product token, such as Googlebot or ChatGPT-User. Operators run several variants and change version numbers.

Can a user agent prove who sent a request?

No. It is text the client chooses. Proof comes from reverse DNS, published IP ranges or a cryptographic signature.

Sources