What is OAI-SearchBot?

By Nik Andreev, Founder & SEO specialist · Last reviewed · Editorial policy

OAI-SearchBot is the crawler OpenAI uses to surface websites in ChatGPT's search features. It follows robots.txt and has its own token, separate from GPTBot. Blocking it means your pages will not be shown in ChatGPT search answers, though OpenAI says they can still appear as navigational links. Blocking GPTBot does not affect it.

At a glance

Operator
OpenAI
Type
Search crawler for ChatGPT search
User-agent token
OAI-SearchBot (the version number may change)
Follows robots.txt
Yes. OpenAI says changes can take about 24 hours to apply
How to verify
Source IP in OpenAI's published list, openai.com/searchbot.json
Blocking it means
Not shown in ChatGPT search answers. Can still appear as navigational links

How to spot it

  • In your logs: the user agent contains OAI-SearchBot. Match the token, not the full string, because OpenAI says the version number may change. Its robots.txt fetches can carry a robots.txt marker in the string.
  • To verify it: the source IP must be in the ranges OpenAI publishes at openai.com/searchbot.json. An OAI-SearchBot user agent from any other IP is user-agent spoofing.
  • If it is missing: check that robots.txt, your CDN or your WAF is not blocking it. OpenAI recommends allowing it in robots.txt.
User-agent: OAI-SearchBot
Allow: /

User-agent: GPTBot
Disallow: /
From Lume's data

OAI-SearchBot vs GPTBot

More
requests than GPTBot, on every opted-in site

Method: Requests to sites that opted in to this anonymous analysis, August, September, October and November 2026 (to 4 November). Each site is summarized separately, then figures are rounded. Counts are requests claiming each bot name.

OpenAI's other bots

GPTBot Collects content that may be used to train OpenAI's models. A separate setting. ChatGPT-User Fetches a page when a ChatGPT user asks. OpenAI says robots.txt rules may not apply.

Quick answers

Can I block GPTBot but allow OAI-SearchBot?

Yes. OpenAI says each setting is independent, so you can opt out of training and still appear in ChatGPT search.

Does OpenAI crawl twice if I allow both bots?

OpenAI says that if both are allowed, it may use the results of one crawl for both uses to avoid duplicate crawling.

How long does a robots.txt change take?

OpenAI says it can take about 24 hours for its systems to adjust after you update robots.txt.

Sources