How to Edit Robots.txt in Shopify (2026 Guide + Fix Indexing Issues)

How to view, understand, customise, and get more out of robots.txt on every Shopify plan, including advanced SEO strategies, common mistakes, and full code examples.

Applies to: Shopify Starter · Basic · Shopify · Advanced · Plus  |  Reading time: ~15 min  |  Last updated: 2025  |  By Hamza Taj

1. What Is robots.txt? (And Why Should Shopify Owners Care?)

Every time a search engine crawler, such as Googlebot, Bingbot, or any other spider, visits your website, the very first file it reads is robots.txt. This file, located at the root of your domain, acts as a set of standing instructions that tells crawlers which pages they are welcome to crawl and which they should skip.

For a Shopify store, this matters more than most site owners realise. If search engines waste their crawl budget on low-value pages like cart pages, checkout flows, or filtered collection URLs, they have less capacity to discover and index your high-value product and collection pages. The result is slower ranking, missed indexing, , and in some cases duplicate content penalties that pull your entire store down. (For the full picture of how technical SEO ties into store performance, see our Shopify Conversion Rate Optimization guide.)

The anatomy of a robots.txt file

A robots.txt file is a plain text file made up of groups called “records.” Each record addresses one or more user agents (crawlers) and gives them a set of rules. The key directives you will encounter are:

How robots.txt affects your SEO

Search engines have a finite amount of time and resources to spend on any given site. This is called your crawl budget. For small stores (under 500 SKUs) Google rarely has trouble crawling everything, but as your catalogue grows, crawl budget becomes a real concern. robots.txt lets you direct crawlers away from wasteful URLs so they concentrate on the pages that actually drive revenue.

The three SEO outcomes most directly connected to robots.txt configuration are:

• Crawl budget efficiency: Blocking thin, duplicate, or parameterised URLs ensures Google spends its time on important pages.
• Duplicate content control: Shopify creates multiple URL paths for the same product. robots.txt (or canonicals) prevents Google from indexing all of them.
• Indexing accuracy: Pages that should never appear in search (checkout, orders, admin) stay out when properly disallowed.

2. Shopify’s Default robots.txt: What It Looks Like and What It Does

Unlike a traditional website where you create and manage the robots.txt file yourself, Shopify auto-generates this file dynamically. You cannot access it through the file manager the way you would on WordPress or a custom server. To see your current robots.txt, simply visit:

https://your-store.myshopify.com/robots.txt

The file is regenerated automatically based on your store structure. Here is what a standard Shopify-generated robots.txt typically includes:

User-agent: *
Disallow: /admin
Disallow: /cart
Disallow: /orders
Disallow: /checkout
Disallow: /carts
Disallow: /account
Disallow: /56789/checkouts
Disallow: /56789/orders
Disallow: *?*oseid=*
Disallow: *?*preview_theme_id=*
Disallow: *?*preview_script_id=*
Allow: /
Sitemap: https://your-store.myshopify.com/sitemap.xml

Breaking down each default rule

3. Can You Edit robots.txt on Shopify? (Plan-by-Plan Breakdown)

This is the question most store owners search for and the answer depends entirely on which Shopify plan you are on. Here is the definitive breakdown:

The key limitation for non-Plus merchants is that you cannot remove or override Shopify’s default rules. You can only add new Disallow or Allow directives on top of them. This is a deliberate restriction to prevent store owners from accidentally blocking important pages. For the vast majority of stores, this partial control is sufficient. If you genuinely need full control, Shopify Plus is the only native path, though most merchants will find that a properly configured robots.txt.liquid template through expert Shopify development covers 95% of real-world SEO requirements.

What is the robots.txt.liquid template?

Introduced in 2021 as part of Shopify’s Online Store 2.0 architecture, the robots.txt.liquid template is a Liquid-powered file that controls how the robots.txt output is generated for your store. It merges Shopify’s default rules with any custom rules you add, and is available in any theme that supports OS 2.0 (Dawn, Debut updated versions, most major premium themes).

If your theme does not have this template, you can add it manually via the code editor, but the theme itself must be OS 2.0 compatible for the customisation to take effect.

4. How to Edit robots.txt on Standard Shopify (Step-by-Step)

The following walkthrough applies to Shopify Basic, Shopify, and Advanced plans using an Online Store 2.0 compatible theme. You will be working inside the Liquid template system. No external tools or apps are required.

Step 1: Access the Theme Code Editor

• Log in to your Shopify Admin and navigate to Online Store in the left sidebar.
• Click Themes. Find your active theme at the top of the page.
• Click the three-dot menu (…) next to your active theme, then select Edit code.

Step 2: Locate or Create the robots.txt.liquid File

• In the code editor’s left panel, open the Templates folder.
• Look for a file called robots.txt.liquid. If it exists, click to open it.
• If it does not exist, click Add a new template, select robots.txt from the dropdown, and click Create template.

Step 3: Understand the Default Template Code

When the file first loads, you will typically see something like this:

{% for group in robots.default_groups %}
  User-agent: {{ group.user_agent }}
  {% for rule in group.rules %}
    {{ rule }}
  {% endfor %}
{% endfor %}

Sitemap: {{ routes.root_url }}sitemap.xml

This Liquid loop iterates through Shopify’s default rule groups and outputs them exactly as Shopify defines them. Your job is to add custom rules inside or after this loop.

Step 4: Add Your Custom Rules

To add a Disallow for a specific path, for example, blocking your internal search results page and all filter parameter URLs, modify the template like this:

{% for group in robots.default_groups %}
  User-agent: {{ group.user_agent }}
  {% for rule in group.rules %}
    {{ rule }}
  {% endfor %}

  Disallow: /search
  Disallow: /*?sort_by=
  Disallow: /*?filter.
{% endfor %}

Sitemap: {{ routes.root_url }}sitemap.xml

By placing your custom Disallow lines inside the for group loop but outside the inner for rule loop, you ensure they are applied to every user-agent group, including Googlebot, Bingbot, and any other bot that Shopify addresses in its defaults.

Step 5: Save and Verify

• Click Save in the top-right corner of the code editor.
• Open a new browser tab and visit: yourstore.com/robots.txt
• Confirm your new rules appear in the output alongside the default Shopify rules.
• Go to Google Search Console > URL Inspection > enter your store URL > check the “Crawl allowed?” status.

5. How to Edit robots.txt on Shopify Plus (Full Control)

Shopify Plus merchants have complete control over the robots.txt.liquid file. You can discard Shopify’s default rule groups entirely and write a custom robots.txt from scratch, exactly as you would on any other platform. This is powerful, but it also means you bear full responsibility for any crawling or indexing issues that result. If you are not comfortable making these changes yourself, it is worth hiring a Shopify developer who has done it before, a single misplaced Disallow can cost weeks of organic traffic.

Accessing the file: same path, different capability

The access route is identical to standard Shopify: Admin > Online Store > Themes > Edit code > Templates > robots.txt.liquid. The difference is that on Plus, you can delete the default Liquid loop and replace it with raw robots.txt directives.

A production-ready Shopify Plus robots.txt template

The following template is a solid starting point for a Shopify Plus store. It preserves all the important default blocks, adds best-practice SEO rules, and sets a separate instruction for Googlebot:

# ─── Global rules ───────────────────────────────────────────
User-agent: *

# Block operational and private pages
Disallow: /admin
Disallow: /cart
Disallow: /orders
Disallow: /checkout
Disallow: /carts
Disallow: /account

# Block thin / duplicate content
Disallow: /search
Disallow: /*?sort_by=
Disallow: /*?filter.
Disallow: /*?q=
Disallow: /collections/*+*

# Block internal Shopify parameters
Disallow: *?*oseid=*
Disallow: *?*preview_theme_id=*
Disallow: *?*preview_script_id=*

# Allow all other paths
Allow: /

# ─── Google-specific rules ──────────────────────────────────
User-agent: Googlebot
Crawl-delay: 1

# ─── Sitemap ────────────────────────────────────────────────
Sitemap: https://yourstore.com/sitemap.xml

6. Six Real-World robots.txt Use Cases for Shopify Stores

Understanding the syntax is only half the battle. The real value of robots.txt customisation comes from knowing when and why to apply it. Below are the six scenarios where Shopify store owners most frequently need to act.

Use Case 1: Blocking parameterised filter and sort URLs

Shopify’s native filtering and sorting system generates a separate URL for every combination of filters a shopper applies. A single collection page can spawn hundreds of parameterised variants such as:

/collections/shoes?sort_by=price-ascending
/collections/shoes?filter.p.tag=running&sort_by=best-selling
/collections/shoes?filter.v.option.color=red&filter.p.price.gte=50

None of these pages has unique content. They are simply different views of the same product set. Allowing Google to crawl and index all of them wastes crawl budget, dilutes your collection pages’ authority, and can create duplicate content issues. The fix:

• Disallow: /*?sort_by=, blocks all sort parameter variants
• Disallow: /*?filter., blocks all Shopify filter parameter variants

Use Case 2: Controlling collection-based duplicate product URLs

Shopify automatically creates two canonical URL formats for every product: the root product URL and a collection-scoped URL. For example, a single product called “Red Running Shoe” can be reached at:

/products/red-running-shoe
/collections/running/products/red-running-shoe
/collections/shoes/products/red-running-shoe

Google will eventually treat one as canonical and ignore the others, but this process is slow and error-prone. The recommended approach is to use canonical tags rather than robots.txt here (covered in Section 8), because blocking collection-scoped product URLs with robots.txt can sometimes interfere with internal linking signals. However, if you are on Plus and managing a very large catalogue, a targeted Disallow can be effective when combined with canonical tags.

Use Case 3: Blocking internal site search result pages

Your store’s internal search results, at /search?q=keyword have no value in Google’s index. They are thin, duplicative, and provide a poor experience for anyone who lands on them from a SERP. Blocking them is straightforward and nearly always the right call:

Disallow: /search

Use Case 4: Protecting preview and development URLs

When you preview a theme or use Shopify’s theme editor, special parameter strings are appended to URLs (such as preview_theme_id and preview_script_id). Shopify blocks these by default, but if you have modified the robots.txt.liquid template and inadvertently removed these rules, these preview URLs can end up indexed. Always verify they remain in your output.

Use Case 5: Blocking thank-you and post-purchase pages

Thank-you pages generated after a purchase (/thank_you, /orders/[id]/thank_you) contain personalised order data and have zero value in search results. These are blocked by default via the /orders Disallow, but if your theme uses custom post-purchase landing pages at different paths, those should be explicitly added to your robots.txt.

Use Case 6: Adjusting crawl speed for large catalogues

If your Shopify Plus store has tens of thousands of product pages and you are experiencing server load spikes during Google’s crawls, a Crawl-delay directive on a per-bot basis can smooth this out:

User-agent: Googlebot
Crawl-delay: 2

Note that Google has publicly stated that Googlebot does not always respect Crawl-delay and prefers you use the crawl rate setting in Google Search Console instead. However, Bing and other crawlers do respect this directive, making it useful for broader crawl management.

7. Six robots.txt Mistakes That Kill Shopify SEO

robots.txt errors can cause significant, sometimes catastrophic damage to your store’s search visibility. The following mistakes are not hypothetical; they appear regularly in SEO audits of Shopify stores at every level.

Mistake 1: Disallowing the entire site

The single most destructive robots.txt configuration possible is a blanket Disallow that blocks all crawling. It looks harmless enough:

User-agent: *
Disallow: /

This one-line change can wipe your entire store from Google’s index within days. It is often introduced by accident during theme testing or when a developer copies a “maintenance mode” robots.txt configuration from another project. Always confirm your live robots.txt is not set to this before and after any theme changes. (If your store has suddenly stopped converting and you are not sure why, a stray Disallow is one of several silent killers. See our deeper diagnostic on why a Shopify store stops converting.)

Mistake 2: Blocking CSS, JavaScript, and font files

An older SEO myth, still circulating in outdated guides, holds that blocking CSS and JS files from crawlers saves crawl budget. The opposite is true. Google uses a rendering engine to understand your pages’ visual layout and content. If it cannot access your stylesheets and scripts, it cannot fully render your pages and may rank them lower as a result, or fail to extract structured data correctly. Never block:

• /assets/: Shopify’s primary folder for CSS, JS, fonts, and images
• Any .css, .js, or .woff file paths

Mistake 3: Confusing robots.txt with security

robots.txt is a publicly readable plain text file. Anyone can view it, including bad actors. If a page contains sensitive information, blocking it in robots.txt does not hide it; it actually advertises its existence. Sensitive pages must be protected with authentication, session management, or proper access controls. robots.txt is not a substitute for any of these.

Mistake 4: Disallowing pages you want indexed but haven’t submitted

A surprisingly common scenario: a store owner blocks a collection or product page with robots.txt because it is “not ready yet,” then forgets to remove the Disallow rule after publishing. The page is live, the meta robots tag says “index,” but Googlebot never sees it because of the robots.txt block. The rule of thumb is simple: if a page should eventually rank, do not block it in robots.txt, even temporarily. Use a password or draft status instead.

Mistake 5: Omitting the Sitemap directive

Adding your sitemap URL to robots.txt is not mandatory, but it is one of the easiest wins available. Googlebot reads robots.txt early in its discovery process, meaning a sitemap declaration here can accelerate the discovery of new pages significantly. Every robots.txt file should end with:

Sitemap: https://yourstore.com/sitemap.xml

Shopify automatically generates a full sitemap for your store, so this URL will always exist and always be up to date.

Mistake 6: Blocking pages already indexed, then expecting them to disappear

Adding a Disallow rule does not remove a page from Google’s index. If a page is already indexed, robots.txt can only prevent Googlebot from re-crawling it, the existing indexed version stays in search results until Google naturally refreshes or until you actively request removal. For pages that need to be urgently de-indexed, use Google Search Console’s URL Removal tool or add a noindex meta tag to the page.

8. robots.txt Alternatives: Choosing the Right Tool for the Job

robots.txt is one of several crawling and indexing control mechanisms available to Shopify store owners. Choosing the right one for each situation matters. Using robots.txt when a canonical tag is needed, for example, can actually make things worse. Here is how the main options compare:

Using meta robots noindex tags as an alternative

When you want a page to remain publicly accessible but absent from search results, the meta robots noindex tag is the right tool. This is a HTML tag added inside the <head> section of a page:

<meta name="robots" content="noindex, nofollow" />

In Shopify, you can add conditional noindex tags to specific page types through your theme’s Liquid templates. Common use cases include policy pages, login-gated content, and thin tag pages that you are not ready to optimise. If you are not familiar with editing Liquid, this is the kind of small theme tweak a Shopify development specialist can usually finish in under an hour.

9. How to Test and Monitor Your Shopify robots.txt

Making changes to robots.txt without a verification process is how accidental de-indexing happens. Every edit, however small, should be followed by a structured check. Here is the process:

Immediate verification (< 5 minutes)

• View the raw file: Open yourstore.com/robots.txt in a browser. Confirm your new rules appear. Confirm no unintended rules have been added or removed.
• Check for Disallow: / : Scan the file for this string. If it is present, investigate immediately.
• Confirm the Sitemap line is present at the bottom of the file.

Google Search Console testing

• Open Google Search Console for your property.
• Navigate to Settings > Crawling > robots.txt.
• Use the robots.txt tester to enter specific URLs and confirm whether Googlebot can or cannot crawl them.
• Run the URL Inspection tool on your most important product and collection pages to confirm “Crawl allowed: Yes.”

10. Frequently Asked Questions

The questions below represent the most common searches around Shopify robots.txt. Each answer is designed to be complete and actionable.

How do I view my Shopify store’s robots.txt?

Simply visit your store’s domain followed by /robots.txt in any browser, for example, yourstore.com/robots.txt. No login is required. This file is always publicly accessible, and what you see there is exactly what Google sees when it visits your store.

Can I completely rewrite robots.txt on a non-Plus plan?

No. On Shopify Basic, Shopify, and Advanced, the robots.txt.liquid template allows you to append custom Disallow and Allow rules to Shopify’s defaults, but you cannot remove or override the default rules. Full rewriting capability requires Shopify Plus.

Will adding a Disallow rule immediately stop Google from crawling that page?

Not instantly. Google caches your robots.txt for up to 24 hours. After that, Googlebot will respect the new rule on its next crawl. If a crawl is already in progress when you save the change, it may continue crawling disallowed URLs during that session.

My robots.txt.liquid template does not appear in my theme, what do I do?

This usually means your theme is not Online Store 2.0 compatible. You have two options: upgrade to an OS 2.0 theme such as Dawn (free from Shopify), or consider upgrading to Shopify Plus for direct robots.txt access. You can check if your theme is OS 2.0 compatible by looking for JSON templates in the Templates folder of your theme code editor.

Does blocking URLs in robots.txt remove them from Google’s index?

No. This is one of the most important distinctions in SEO. Disallowing a URL prevents Googlebot from re-crawling it, but any page that was indexed before the Disallow was added remains in the index. To remove an already-indexed page, use the URL Removal tool in Google Search Console or add a noindex meta tag to the page.

Should I block my /collections/ pages in robots.txt?

Generally, no. Collection pages are high-value SEO targets that target category keywords and pass authority to product pages. The exception is filtered or parameterised collection URLs (those with query strings like ?sort_by= or ?filter.), which should typically be blocked. Blocking clean collection URLs like /collections/shoes is almost always an SEO mistake.

I am on Shopify Plus: how do I verify my robots.txt changes went live?

Visit yourstore.com/robots.txt directly after saving. For a more authoritative test, use the robots.txt tester in Google Search Console or use a third-party tool such as the Google robots.txt tester at search.google.com/search-console/tools. Also use Google Search Console’s URL Inspection on your key pages to confirm “Crawl allowed: Yes.”

Can robots.txt fix my duplicate content problem?

Partially. robots.txt can prevent Google from crawling duplicate URL variants, which stops new duplicates from entering the index. However, it cannot consolidate ranking signals the way canonical tags do. For Shopify’s specific duplicate content issues, particularly collection-scoped product URLs, canonical tags are the more effective and commonly recommended solution. Use robots.txt for query string variants, and canonical tags for structural URL duplication.

11. Quick Reference: robots.txt Cheat Sheet for Shopify