Block GPTBot robots.txt Template

Copy the minimal GPTBot block

Use this group when you want to tell OpenAI that your site's content should not be collected by GPTBot for possible use in training generative AI foundation models:

User-agent: GPTBot
Disallow: /

Publish the rule in the robots.txt file at the root of the exact host, such as https://example.com/robots.txt. Add the same policy separately to any subdomain that serves content.

What blocking GPTBot changes

OpenAI documents GPTBot as its training-oriented crawler. Disallowing it expresses that future crawls of the affected URLs should not be used for training OpenAI's generative AI foundation models. The rule applies to compliant GPTBot requests after OpenAI has refreshed its robots.txt policy.

This setting does not remove material that was collected previously, resolve contractual questions, or stop a requester that ignores robots.txt. It also does not protect confidential content. Keep anything private behind authentication and server-side authorization.

GPTBot is independent from ChatGPT search

OAI-SearchBot is a separate OpenAI token used to surface websites in ChatGPT search features. Blocking GPTBot does not require blocking OAI-SearchBot. If your goal is “allow search, block training,” you can make that intent explicit:

User-agent: GPTBot
Disallow: /

User-agent: OAI-SearchBot
Allow: /

ChatGPT-User is different again. It can fetch a page because a ChatGPT user or Custom GPT requested it, and OpenAI states that robots.txt rules may not apply to those user-initiated actions.

Block only part of the site

Replace the root slash with stable path rules when only certain areas should be excluded:

User-agent: GPTBot
Disallow: /members/
Disallow: /licensed-archive/
Allow: /public/

Do not list a sensitive path merely to hide it. Robots.txt is public and can reveal the path name. Enforce access restrictions at the server before relying on crawler instructions.

Avoid common configuration mistakes

Do not combine GPTBot and OAI-SearchBot on one User-agent: line or assume that a company-wide name will match every OpenAI product. Keep each token in its own group so the intent remains auditable. Also check whether a CDN, hosting panel, or framework generates a virtual robots.txt response that overrides the file you edited. The public response—not the local filename—is what the crawler reads.

Check the published result

1. Open the public robots.txt URL and confirm it returns plain text successfully.
2. Test both the homepage and a representative restricted URL with the checker.
3. Make sure an existing User-agent: * group has not accidentally blocked ordinary search crawlers.
4. Review logs using the full user-agent string and OpenAI's published IP ranges when you need to verify real GPTBot traffic.

For a full policy rather than a one-bot rule, see the GPTBot vs OAI-SearchBot guide and the selective template.