llms.txt is a plain Markdown file at yourdomain.com/llms.txt that gives AI agents a short summary of your site and a curated list of links to your most useful pages. It was proposed by Jeremy Howard in September 2024 and is easy to create in under an hour. But does llms.txt work for AI search visibility? As of October 2026 there is no public evidence that it does. Google says its Search ignores the file, no major AI company has announced that its search or answer engine uses it, and independent log data shows most llms.txt files are rarely fetched. It is a low-cost, low-risk file worth having, especially since platforms like Shopify now publish one for you, but it is not a ranking lever.
Below: the format, platform support, what Google and AI companies have said, a copy-ready ecommerce template, and the robots.txt rules that matter more.
What is llms.txt?
The llms.txt proposal was published by Jeremy Howard (co-founder of Answer.AI and fast.ai) on September 3, 2024. The idea: web pages are built for people, while an AI agent with a limited context window does better with a concise summary and direct links to clean content.
The proposal was updated to a v2 in 2026. Key points from the current spec:
- The file lives at /llms.txt at the site root, or at a subpath such as /docs/llms.txt to cover only that section.
- It is written in Markdown, so both people and language models can read it, and simple parsers can process it.
- Pages with useful information can also offer a clean Markdown version at the same URL with .md added (for example page.html.md).
- v2 recommends standard link relations so agents can find these files: rel="alternate" type="text/markdown" for a page's Markdown version and rel="describedby" for the llms.txt that covers it.
llms.txt is not robots.txt: it does not allow or block anything. It is also not a sitemap. A sitemap lists every URL, while llms.txt is a short, hand-picked map for agents.
The llms.txt format, section by section
The spec defines these sections, in this order:
- An H1 with the site or project name. This is the only required section.
- A blockquote with a short summary containing the key facts needed to understand the rest of the file.
- Optional free text (paragraphs or lists, but no headings) with more detail on how to interpret the site.
- Zero or more H2 sections with "file lists". Each item is a Markdown list entry with a required [name](url) link, optionally followed by a colon and a short note.
- An H2 named "Optional" for secondary links an agent can skip when it needs a shorter context.
Some sites also publish llms-full.txt, a single file with the full text of key pages. It is not part of the core spec, but it is a common convention, and Shopify serves one by default.
Does llms.txt work? What Google and AI companies have said
This is where honesty matters. Here is what has been said on the record, and what has not.
Google: Search ignores it
Google's official guide to optimizing for generative AI in Search (last updated July 10, 2026) is direct. It says you do not need to create new machine-readable files, AI text files, markup or Markdown to appear in Google Search, because Google Search itself doesn't use them. It adds that creating and maintaining llms.txt files is completely fine, but doing so "will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them." The same guide lists creating unnecessary AI text files like llms.txt among tactics to skip.
Google's Search team has been consistent on this:
- John Mueller, in an April 2025 Reddit r/TechSEO thread, wrote: "AFAIK none of the AI services have said they're using LLMs.TXT (and you can tell when you look at your server logs that they don't even check for it). To me, it's comparable to the keywords meta tag."
- Gary Illyes, at Google's Search Central Deep Dive event in Asia Pacific in July 2025, was reported by attendee Kenichi Suzuki as stating that Google doesn't support llms.txt and isn't planning to. That is a secondhand account, not a published transcript.
Google's other teams: mixed signals
Google is not a single voice. In 2026, Chrome's Lighthouse added an experimental Agentic Browsing category that includes a check for an llms.txt file at the domain root. Chrome's documentation calls this experimental and based on proposed standards, and it sits outside the SEO score. Lighthouse checking for a file is not Search using it.
AI companies: they publish it, but do not confirm reading it
OpenAI, Anthropic and Google's Gemini API all publish llms.txt files for their own developer documentation. That suggests the format helps coding agents navigate docs. However, as of October 2026 we found no official statement from OpenAI, Anthropic, Perplexity or Google confirming that ChatGPT search, Claude, Perplexity or AI Overviews use a site's llms.txt to decide what to retrieve, cite or recommend. OpenAI's and Anthropic's crawler documentation explains robots.txt controls, not llms.txt.
What the log data shows
An Ahrefs study published in June 2026 checked 137,210 domains in its web analytics product. About 28% published an llms.txt file, and 97% of those files received zero requests in May 2026. Of the requests that did occur, AI bots made about 19.5%, with GPTBot the largest single AI requester, while AI retrieval bots (the ones that fetch pages to answer live questions) made only about 1.1%. Ahrefs notes its customer base skews technical, so adoption is likely overstated. The study's conclusion: for AI search visibility, llms.txt is largely decoration today, and its clearest audience is coding and agentic tools.
Bottom line: the impact of llms.txt on AI search visibility is unproven. Treat it as cheap hygiene and future-proofing for agents, not as a growth tactic. What actually drives AI citations is the same foundation covered in our guide to generative engine optimization: crawlable pages, clear product facts, strong reviews and genuine authority.
Which platforms and tools support llms.txt?
| Platform or tool | What it does | Can you edit it? |
|---|---|---|
| Shopify | Every store serves a default /agents.md; /llms.txt and /llms-full.txt mirror it automatically | Yes, via theme templates (agents.md.liquid or llms.txt.liquid) |
| Wix | Automatically generates and maintains llms.txt for upgraded sites with a custom domain | Yes, in SEO tools and settings; editing stops auto-updates |
| WordPress with Yoast SEO | Toggle under Site features, AI tools; generates the file automatically | Yes, manual page selection |
| Yoast SEO for Shopify | llms.txt toggle under Advanced settings | Yes, manual page selection |
| WordPress with AIOSEO | Includes an llms.txt generator feature | Yes |
| Docs platforms (Mintlify, GitBook) and plugins for VitePress, Docusaurus, Drupal | Generate llms.txt for documentation sites | Varies by tool |
Shopify stores: you already have one
Shopify's May 28, 2026 developer changelog confirmed that every store includes a default agent-discovery file at /agents.md, and that /llms.txt and /llms-full.txt point to that content by default. The Shopify-generated file is written for shopping agents. It describes the store's Universal Commerce Protocol (UCP) endpoints, checkout rules and store policy links, not your brand story.
If you want custom content, Shopify's llms.txt.liquid documentation explains the lookup order: llms.txt.liquid first, then agents.md.liquid, then Shopify's default. Two practical notes:
- Shopify recommends most stores keep the managed default. If you customize, you take responsibility for keeping the file current.
- These templates render with a restricted Liquid context. Only the request and agents objects are available, so you cannot loop over collections or products. Write your curated links as static Markdown, and consider keeping the agents object's UCP and MCP lines so shopping agents still find them.
If you previously served a custom llms.txt through a redirect, app or CDN worker, check what yourstore.com/llms.txt returns today. It may now show Shopify's version.
Do you need an llms.txt generator?
For most stores, no. A generator that dumps every URL defeats the purpose, which is curation. Use your platform's built-in option and edit it by hand, or write the file yourself from the template below.
How to create an llms.txt file in 6 steps
- Check what you already serve. Visit yourdomain.com/llms.txt. Shopify, Wix and some SEO plugins may already publish one.
- Pick 15 to 40 pages that answer buyer questions. Top categories, best sellers, size guides, shipping and returns, warranty, about, contact and FAQ. Skip cart, account, search results and filter URLs.
- Write a factual summary. One blockquote stating what you sell, who for, where you ship and what makes you different. No slogans or superlatives you cannot back up.
- Add one-line notes to each link. Tell the agent what the page answers, for example "free US shipping over $75, 30-day returns".
- Publish at the root as plain text. Serve it with a 200 status and a text/plain or text/markdown content type, not an HTML page or soft 404.
- Set an update rhythm. Review it whenever policies, prices or top categories change, and at least quarterly. Keep it in version control and limit who can edit it, since anything an AI agent reads is a potential prompt-injection surface.
llms.txt example: copy-ready template for an ecommerce store
Replace the bracketed placeholders. Keep notes factual and current. If your policies differ by country (for example US and India), say so explicitly.
# [Brand Name]
> [Brand Name] is an online store selling [main product category, e.g. handmade leather bags and wallets] for [target customer]. We ship to [countries/regions] from [warehouse location]. Founded in [year]. All prices on product pages are in [currency] and include/exclude [tax].
Key facts for assistants:
- Shipping: [free shipping threshold and typical delivery times by region]
- Returns: [return window, condition rules, who pays return shipping]
- Payments: [accepted methods, e.g. cards, PayPal, UPI, cash on delivery]
- Support: [email] and [phone or chat hours with time zone]
- Product details, prices and stock change often. Always confirm on the linked product page.
## Shop by category
- [Category One](https://www.example.com/collections/category-one): [what is in it, price range]
- [Category Two](https://www.example.com/collections/category-two): [what is in it, price range]
- [Category Three](https://www.example.com/collections/category-three): [what is in it, price range]
## Best sellers
- [Product Name A](https://www.example.com/products/product-a): [one-line description, key spec or material]
- [Product Name B](https://www.example.com/products/product-b): [one-line description, key spec or material]
- [Product Name C](https://www.example.com/products/product-c): [one-line description, key spec or material]
## Buying help
- [Size and Fit Guide](https://www.example.com/pages/size-guide): [measurements and how to choose]
- [Product Care](https://www.example.com/pages/care): [cleaning and maintenance]
- [FAQ](https://www.example.com/pages/faq): [top pre-purchase questions answered]
## Policies
- [Shipping Policy](https://www.example.com/policies/shipping-policy): [regions, costs, timelines]
- [Refund Policy](https://www.example.com/policies/refund-policy): [window and process]
- [Warranty](https://www.example.com/pages/warranty): [coverage and how to claim]
- [Privacy Policy](https://www.example.com/policies/privacy-policy): [data handling summary]
## About
- [Our Story](https://www.example.com/pages/about): [founding, materials, sourcing, certifications]
- [Contact Us](https://www.example.com/pages/contact): [support channels and hours]
## Optional
- [Blog](https://www.example.com/blogs/news): [buying guides and how-to articles]
- [Gift Cards](https://www.example.com/products/gift-card): [denominations and validity]
- [Wholesale](https://www.example.com/pages/wholesale): [B2B inquiries]
- [Sitemap](https://www.example.com/sitemap.xml): [full list of public URLs]
Link to canonical URLs only, and never put claims in llms.txt that do not appear on the linked page.
robots.txt matters more: AI crawler user agents in 2026
If you want visibility in AI answers, your robots.txt decides whether AI systems can read your pages at all. Our robots.txt guide covers the basics. These are the AI user agent tokens confirmed in each company's official documentation:
- OAI-SearchBot: surfaces sites in ChatGPT search. OpenAI's crawler documentation says sites opted out will not be shown in ChatGPT search answers, though they can still appear as navigational links. Changes can take about 24 hours to apply.
- GPTBot: crawls content that may be used to train OpenAI's foundation models. It is independent of OAI-SearchBot, so you can block training and still allow search.
- ChatGPT-User: fetches pages when a user asks ChatGPT something. OpenAI says robots.txt rules may not apply because these actions are user-initiated.
- ClaudeBot: collects content that could contribute to Anthropic's model training. Anthropic's help article also lists Claude-SearchBot (search indexing) and Claude-User (fetching for user questions), says its bots honor robots.txt, and supports Crawl-delay.
- PerplexityBot: surfaces and links sites in Perplexity search results and, per Perplexity's documentation, is not used to train foundation models. Perplexity-User fetches pages for live user questions and generally ignores robots.txt.
- Google-Extended: per Google's crawler documentation, a robots.txt token only, not a separate crawler. It controls whether Google may use your content to train future Gemini models and for grounding in Gemini Apps and Vertex AI. Google states it does not affect inclusion or ranking in Google Search. To control AI Overviews and AI Mode, Google points to Googlebot rules and snippet controls such as nosnippet, not Google-Extended.
A common setup for an ecommerce store that wants AI search visibility but prefers to opt out of model training:
# Allow AI search and answer engines
User-agent: OAI-SearchBot
Allow: /
User-agent: Claude-SearchBot
Allow: /
User-agent: PerplexityBot
Allow: /
# Opt out of model training
User-agent: GPTBot
Disallow: /
User-agent: ClaudeBot
Disallow: /
User-agent: Google-Extended
Disallow: /
# Everyone else, keep private paths out
User-agent: *
Disallow: /cart
Disallow: /checkout
Disallow: /account
Sitemap: https://www.example.com/sitemap.xml
This is a business decision, not a default. Blocking training bots does not remove you from AI search, but some brands allow everything for maximum exposure. On Shopify, edit robots.txt through the robots.txt.liquid theme template. Also check that your CDN or firewall is not silently blocking these bots.
Common mistakes
- Expecting rankings or citations from llms.txt alone. Google says Search ignores it, and no AI search engine has confirmed using it.
- Dumping your whole sitemap into it. The format exists for curation. Hundreds of URLs make it less useful, not more.
- Using llms.txt to block AI. It has no access-control function. Use robots.txt for that.
- Letting it go stale. An outdated return window or shipping price in llms.txt can mislead an agent. Update it with every policy change.
- Overwriting Shopify's agent data without a reason. The default file includes UCP and checkout details that shopping agents rely on.
- Serving it as HTML or a soft 404. It should return a 200 status with plain text or Markdown.
- Accidentally blocking OAI-SearchBot or PerplexityBot while trying to block training bots. Check each token separately.
llms.txt checklist
- Checked whether your platform already serves /llms.txt
- H1 with brand name and a factual blockquote summary
- 15 to 40 curated links with one-line notes, grouped under H2s
- Policies (shipping, returns, warranty) linked and current
- "Optional" section for secondary pages
- Served at root with a 200 status and text content type
- robots.txt reviewed for OAI-SearchBot, GPTBot, ClaudeBot, Claude-SearchBot, PerplexityBot and Google-Extended
- Server logs or bot analytics checked to see which agents actually request the file
- Quarterly review on the calendar
Where llms.txt fits in your AI visibility plan
Think of llms.txt as the last 5% of an AI-ready site, not the first. The work that moves AI visibility is the work that also moves search: clean technical SEO, complete product information, structured data that matches your pages, reviews and real third-party mentions. Our walkthrough on how to rank in AI search covers those priorities step by step, and the free GEO readiness score tool shows where your site stands today.
If you want a team to handle it end to end, from robots.txt and crawler access to product content that AI assistants can confidently cite, our GEO and AEO service is built for ecommerce brands selling in the US and India. We will tell you honestly which tactics are proven and which, like llms.txt, are simply good housekeeping.
Sources
- llmstxt.org: Jeremy Howard's llms.txt proposal (Sept 2024, v2 2026), file format and Markdown page conventions.
- Google Search Central AI optimization guide: Google Search ignores llms.txt; it neither helps nor harms rankings.
- Shopify llms.txt.liquid documentation: Shopify-managed /llms.txt mirrors /agents.md; template lookup order and restricted Liquid context.
- OpenAI crawler overview: OAI-SearchBot, GPTBot and ChatGPT-User user agents and robots.txt behavior.
- Anthropic crawler help article: ClaudeBot, Claude-SearchBot and Claude-User purposes, robots.txt and Crawl-delay support.
- Google common crawlers: Google-Extended is a robots.txt token for Gemini training and grounding, with no effect on Search ranking.
Frequently asked questions
Is llms.txt a ranking factor in Google?
No. Google's AI optimization guide says Google Search ignores llms.txt files, so they neither help nor harm visibility or rankings in Search, including its AI features.
Do ChatGPT, Claude or Perplexity read llms.txt?
None of these companies has officially confirmed that their search or answer products use a site's llms.txt. OpenAI and Anthropic publish llms.txt files for their own developer docs, and some AI bots do request the file occasionally, but impact on citations is unproven.
Where should I put my llms.txt file?
Place it at the root of your domain so it loads at yourdomain.com/llms.txt, served with a 200 status as plain text or Markdown. The spec also allows subpath files such as /docs/llms.txt that cover only that section.
Does Shopify create llms.txt automatically?
Yes. Since Shopify's May 2026 update, every store serves a default /agents.md, and /llms.txt and /llms-full.txt mirror it. You can override it with an llms.txt.liquid or agents.md.liquid theme template.
Can llms.txt block AI crawlers from using my content?
No. llms.txt is only a reading guide and has no access-control function. To allow or block AI crawlers such as GPTBot, ClaudeBot or Google-Extended, use robots.txt rules.