Shopify robots.txt Template for AI Crawlers

Preserve Shopify’s default rules

Shopify generates a robots.txt file automatically, and its default configuration works for most stores. When you need AI-specific controls, create a robots.txt.liquid template and output Shopify’s existing groups before appending your custom crawler rules. This avoids discarding platform rules that protect crawl efficiency for carts, searches, account flows, filters, and other store URLs.

{% for group in robots.default_groups %}
  {{- group.user_agent }}
  {% for rule in group.rules %}
    {{- rule }}
  {% endfor %}
  {% if group.sitemap != blank %}
    {{- group.sitemap }}
  {% endif %}
{% endfor %}

User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: CCBot
Disallow: /

User-agent: Google-Extended
Disallow: /

The exact whitespace can be refined with Liquid whitespace control, but the important design is to keep the default groups and add only the extra policy you need.

Create robots.txt.liquid in the active theme

In Shopify’s theme code editor, open the Templates folder, create a new file named robots.txt.liquid, and add the reviewed template. Shopify supports a limited set of Liquid objects in this file, including robots, groups, rules, user agents, sitemap, and request data.

A theme change can alter which template is active, so document the customization and verify it after publishing or switching themes. Work in a duplicate theme first when the store is live and revenue-sensitive.

Choose crawler rules according to store goals

The example blocks several training or AI-use controls while preserving Shopify’s normal search behavior. Do not add User-agent: * with Disallow: / unless the entire storefront should disappear from compliant crawling. If AI search visibility is valuable, keep search-oriented agents separate from training-oriented agents rather than copying a blanket “block all AI” list.

Robots.txt does not protect customer accounts, order information, unpublished products, wholesale pricing, or paid resources. Shopify’s authentication and application permissions must enforce those restrictions.

Avoid common Shopify mistakes

• Replacing all default output with a few static AI rules.
• Removing the sitemap generated through Shopify’s default groups.
• Blocking product, collection, image, CSS, or JavaScript URLs without testing storefront rendering.
• Editing an unpublished theme while assuming the live store has changed.
• Using robots.txt as a substitute for store privacy or access controls.

Verify the published store

1. Open https://your-store-domain.com/robots.txt after publishing the theme.
2. Confirm that Shopify’s default groups, sitemap, and custom AI groups are all present.
3. Use the robots.txt checker to test Googlebot and every custom token.
4. Recheck the file after a theme switch or major storefront change.