Applebot-Extended robots.txt: Opt Out of Apple AI Training

What Applebot-Extended controls

Applebot-Extended is a secondary robots.txt user-agent token that gives publishers a separate choice about model-training use. Apple says it can be disallowed when you do not want website content used to train the general-purpose foundation models powering generative AI features across Apple products, including Apple Intelligence, Services, and Developer Tools.

Applebot-Extended does not crawl webpages itself. The actual web crawl is performed by Applebot. The Extended token is consulted later to determine how Apple may use the data Applebot collected.

Keep Apple search open while opting out of training

The most common selective policy is to allow Applebot and disallow Applebot-Extended. This keeps eligible pages discoverable through Apple search experiences such as Siri, Spotlight, and Safari while expressing an opt-out from the foundation-model training use covered by Applebot-Extended.

Apple explicitly states that pages disallowing Applebot-Extended can still appear in search results. Therefore, do not block Applebot itself unless you also want to restrict Apple search crawling.

Do not confuse training with current-answer context

Apple documents another, separate use of crawled content: providing additional context and up-to-date information when models generate answers for display in Apple products and services. Apple says publishers can use the nosnippet meta tag on specific content to opt out of use in broad world-knowledge answers. That control is different from Applebot-Extended, which covers training of general-purpose foundation models.

For paywalled pages, Apple also supports the isAccessibleForFree: false structured-data property. These controls solve different problems and should not be treated as interchangeable.

Whole-site and path-level policies

Use Disallow: / to opt the entire host out of the covered training use. Use directory-specific rules when only selected sections should be excluded. Robots.txt is scoped per host, protocol, and port, so each relevant subdomain needs its own file.

Implementation checks

Review the live file after every deployment or platform migration. A robots.txt rule stored in a CMS field has no effect unless the public host actually serves it from the expected root URL with the intended content. Also confirm that a CDN, security plugin, or stale cache is not serving an older version. When only selected directories are excluded, test both an allowed public URL and a disallowed URL so the policy is not broader than intended.

1. Keep an explicit Applebot group if search discovery matters.
2. Add Applebot-Extended as a separate group.
3. Avoid a broad wildcard rule that accidentally blocks Applebot.
4. Publish the file at the host root.
5. Verify the final groups with the checker.

For a broader multi-company policy, start with the allow search, block AI training template and adapt each user agent separately.